Open-Source AI Needs a New Security Layer
Written byDivya Sudhakar
Open-source AI models are becoming increasingly difficult for enterprises to ignore.
Over the past year, models like Kimi or DeepSeek have demonstrated that open models can deliver performance that is close enough to leading frontier models for many use cases, sometimes at significantly lower cost. Since then, we have seen more capable open models enter the market, while enterprises using AI at scale have become much more focused on cost and ROI.
That creates an obvious question for technology leaders: If an open-source model can handle a task sufficiently well at a fraction of the cost, why wouldn’t we use it?
But there is another question that is drawing attention: What new security risks are we introducing when we do?
That question becomes particularly complicated when some of the strongest open-source models are being developed in China, where geopolitical and national-security considerations add another layer to an already difficult technical problem.
I don’t think the answer is that enterprises should avoid open-source AI. But I do think companies need to become much more sophisticated about which models they use, what those models can access, and how their infrastructure is designed around them.
And increasingly, I believe that creates an entirely new security layer that still needs to be built.
Open-source models are becoming harder for enterprises to ignore
As enterprises adopt AI more broadly, they are becoming much more cost-conscious.
If a company is paying premium prices for a frontier model to perform relatively rote administrative work, it is reasonable to ask whether an open model could perform that task sufficiently well at a lower cost. You don’t necessarily need the best model available for every workflow.
That pressure is only going to grow as AI usage expands. Boards and management teams will increasingly ask what return they are getting from their AI investments and whether different workloads can be served more efficiently.
So I don’t think the question is whether enterprises will consider using open-source models. Increasingly, they will have to. The harder question is how to capture the economic benefits without introducing risks the organization does not fully understand.
Self-hosting may not answer the entire security question
One school of thought says that the security question is relatively straightforward: If I download an open model and run it on infrastructure I control, whether that is AWS, Azure, Google Cloud, or my own backend, then I control the environment. My data isn’t being sent directly to the model developer, so I shouldn’t have to worry about the same kinds of risks I might associate with sending sensitive information to an outside provider.
There is logic to that argument. But I don’t think we know enough yet to say that self-hosting eliminates the risk.
The models themselves remain, in many ways, black boxes. We can know how they perform. We can know certain things about how they were trained. But we do not necessarily understand every behavior that has been learned or everything the model may be capable of doing under different conditions.
Consider a hypothetical example. What if a model had been trained to behave normally almost all of the time, but when it encounters a particular kind of sensitive data, it triggers a different workflow? What if it attempted to send that information somewhere else or spin up an agent that tried to access another part of the organization?
Running the model on your own infrastructure would certainly change the threat surface. But would it guarantee that nothing unexpected could happen? I don’t think we can confidently say that yet.
And we already have reminders that even frontier AI systems can behave in ways their developers did not intend. There are publicly disclosed examples of agents bypassing intended guardrails and accessing information they were not supposed to access. What makes those examples especially interesting is that the model providers themselves surfaced the behavior.
That doesn’t prove that Chinese open-source models are secretly doing something similar (there has not been a widely publicized incident establishing that). The point is that these systems continue to surprise even the organizations building them.
We are early enough that we don’t fully know what we don’t know. For any organization handling highly sensitive information, that uncertainty matters.
The answer probably isn’t to avoid open source
At the same time, I don’t think the right conclusion is that enterprises should simply refuse to use open-source models. The economic benefits are too meaningful, and there are plenty of workflows where the sensitivity of the information is relatively low.
If I’m using an AI model to help put together a meeting agenda, summarize non-sensitive information, or perform basic administrative work, I may not want to pay the highest possible token cost for the leading frontier model. An open-source model may be perfectly appropriate.
The more important question is: What does that model get access to?
I think enterprises will increasingly need to bifurcate their AI workloads based on sensitivity. You might allow an open model to work with low-risk internal information while preventing it from accessing your most sensitive customer, financial, healthcare, or proprietary data. You may need infrastructure that ensures an agent working in one environment cannot simply move laterally into another. You may need different models, policies, and security controls depending on the workflow.
In some ways, we have seen a version of this before. The enterprise move to the cloud didn’t happen all at once. Even as companies became increasingly comfortable with cloud infrastructure, the most sensitive workloads were often handled differently or remained on-premise for longer.
I think we may see a similar philosophy emerge around AI models. Not every model needs access to every part of the organization.
Sovereign models may become part of the answer
There is another question I hear enterprises asking: Are there alternatives to relying on Chinese open-source models?
Increasingly, the answer is yes. U.S. and European providers are developing their own open models, and I think we are likely to see growing interest in what you might call sovereign models: models developed within a particular country or region that enterprises may feel more comfortable deploying based on their own regulatory, geopolitical, or national-security considerations. That could help address one category of concern, but I don’t think geography eliminates the larger security problem.
Even if I am using a U.S.-developed model, a sophisticated enterprise still needs to think about where that model is deployed, what data it can access, what agents it can create, what other systems those agents can interact with, and how unexpected behavior gets detected and stopped.
As adoption becomes more sophisticated, model selection may increasingly become part of enterprise security architecture.
There isn’t a roadmap yet
If you are a CISO, CTO, or technology leader at a major institution, this is something I would be spending more time on now.
I don’t think there is a simple five-step roadmap that enterprises can follow today. The technology is evolving too quickly, and many of the potential risks are still being understood. But that isn’t a reason to wait.
Organizations need people who understand how these systems work at a deep technical level. They need to think about the tools they are adopting, the architecture around those tools, the permissions models have inside the organization, and the guardrails required to contain them.
The biggest mistake may be treating the AI model itself as an isolated application. As models become increasingly agentic, they can interact with tools, initiate workflows, and potentially move across systems. That means the security question extends beyond the model to the entire infrastructure surrounding it. Which brings me to what I think is one of the most interesting opportunities in cybersecurity right now.
A new security layer still needs to be built
Open-source AI adoption is exposing a new enterprise security layer that still needs to be built.
Enterprises want the flexibility and economics of open models. At the same time, they need better ways to understand what those models are doing, govern what they can access, isolate sensitive environments, detect unexpected behavior, and prevent agents from moving into systems where they don’t belong.
There are likely going to be new security, monitoring, orchestration, and infrastructure companies built around this problem. And because the category is still developing, I don’t think anyone has mapped the entire opportunity yet. That’s part of what makes it interesting from an investment perspective.
At Geodesic Capital, we are spending time thinking about how enterprises will securely adopt open-source AI and what new infrastructure will be required as that adoption grows. If you are a founder building in this layer, or a cybersecurity expert thinking deeply about how these risks will evolve, we’d love to hear from you.