Sovran
Sovereign environments. AI agents that never need the open internet—or someone else’s model API.
It’s been almost a year since I launched my previous experiment. It was hard to pick a direction. As my mate Stu told me:
the J curve on AI is so startling at the moment it’s hard to pick which direction one might head.
But a month ago, something started clicking. First, local models. As I posted then
Then, not only local models.
Privacy is a concern
Regulated companies are very picky about AI and privacy. Will my data be used for training?
AWS Bedrock is a very good solution for many companies. For those who don’t know it, when you use a model (like Opus 4.8) through Bedrock, the model runs in AWS’s own infrastructure, and Amazon promises that neither they nor the model provider will train on your prompts.
So Bedrock with the latest models sounds perfect for many companies. It is powerful, and it is private.
But it is not for everyone. Some companies want to go further.
NAT-gateway detector
More recently, we started hearing about agents under training being able to escape their sandboxes.
The incident with OpenAI caught my attention. The escape went through because, among other things, the agent hacked Artifactory..
A proxy with a real internet connection to get some packages?
Downloading packages “requiring” an internet connection triggers my “NAT-gateway detector”.
“What is your NAT-gateway detector?”
I’m glad you asked.
NAT-gateway is an outbound service in AWS. It is super expensive, and many times there are cheaper and more secure ways of downloading packages and even Docker images. S3 is usually a great solution for those “downloading packages” requirements.
Because NAT-gateway is so expensive, when I hear about “I need Internet to download the packages” I immediately think: maybe you need an internet connection, maybe you don’t. But not to download packages.
So my “NAT-gateway detector” got me thinking.
Privacy as the product
I’m sure that frontier labs require outbound internet connectivity when training agents. Not everything can be S3-mediated.
I’m also convinced many companies (even regulated companies) require frontier models when integrating AI into their products. Some of them will require internet connectivity. That is a perfectly valid use case.
Companies with those requirements might rely on the labs contracts, or alternatively use Bedrock to further insulate data from training.
But I do think there is a niche for companies that on the functionality–privacy trade-off prefer to go all the way to privacy.
So let’s imagine we could create a product that doesn’t have internet connectivity and it doesn’t have a connection to a model provider. A product that runs a not very powerful but local model internally. That runs the harness/agent internally. That we can only connect to the agent through a pre-defined interface. That the agent can only download a defined set of packages. That you can host it yourself on the cloud, or even eventually on-premise.
Well. I imagined, and I decided to give it a go.
The result is Sovran.
Gory details
As a reference architecture, three accounts. Two as clients, one as a provider. The provider and the clients only communicate through PrivateLink. The provider runs a model and some agents. The model is Gemma 4 E2B IT (a model from Google that allows tool calling), using vLLM on an ASG listening through a load balancer and cannot dial home. The agents can listen for connections (each set of agents from its own PrivateLink only), and communicate with the model. It can also download packages from S3. And write to S3 for persistence only (each client to its own bucket).
I used a g5.xlarge (roughly 1.12$ an hour), but I’ve been told (Grok and Claude helped a lot) it supports 10 users concurrently on 128k context. I’m also willing to test g6.xlarge, but it is not supported in eu-west-1, and I’m waiting for my quota in eu-west-2 to be increased (updated: Now I’ve tested with g6.xlarge in eu-west-2)
At this point it is just a bunch of Terraform code using ECS instead of the planned Firecracker. Multi-tenant agent hosts was on the plan but they already exist but it would be more complicated for a PoC.
Conclusion
Beyond all these details, this proves it is possible to have agent and inference with privacy in mind. More costly, less powerful. But private. For the very few.
And of course a domain and a landing page in case it can be expanded into something.