How the Pentagon Uses AI Without Leaking Secrets
The U.S. military and intelligence agencies face a problem that most large companies will soon encounter: how to use powerful AI systems on sensitive information without exposing that information to the world.
A small group of infrastructure companies has quietly built the technical scaffolding to solve this. These firms-largely unknown outside defense circles-enable the Pentagon, CIA, and other agencies to deploy commercial AI models on classified networks while keeping secrets compartmentalized. The market is worth roughly $2 billion today, according to Nicolas Chaillan, founder of Ask Sage, a platform used by thousands of Defense Department teams.
The secrecy dilemma
Commercial large language models like Claude and ChatGPT work by learning patterns from training data. Feed one all your company's internal documents, and it becomes fluent in your business-but it also remembers everything. A competitor or adversary with access to that AI could extract your proprietary information through careful prompting.
For the CIA or military, the stakes are higher. Intelligence agencies compartmentalize information strictly. An analyst working on Iraq operations has no business reading classified reports about China. If that analyst's AI assistant suddenly knows all agency secrets, compartmentalization collapses.
"Let's say I'm an Iraq analyst," explains Brian Raymond, CEO of Unstructured, an AI infrastructure company. "If I could suddenly ask, 'Tell me all the assets we have in some county in Asia and tell me all their real names'-those are our most closely guarded secrets."
The technical workaround: Secure RAG
The solution centers on a technique called Retrieval Augmented Generation, or RAG. Instead of training an AI model on sensitive documents, RAG keeps those documents in a separate, secure library. When a user asks a question, the system retrieves relevant classified information from that library, feeds it to a commercial LLM running on secure servers, and generates an answer-without the model ever "learning" the secrets.
Think of it as the AI stepping into a secure briefing room on a need-to-know basis, getting the information it needs, and then forgetting it when the session ends.
Unstructured handles the foundation of this scaffolding, converting messy internal files into formats that can be safely searched in secure databases. Companies like Arize AI monitor these systems for errors and hallucinations. At the top sits Ask Sage, which provides the user interface where Pentagon employees can query approved commercial models while drawing from restricted data.
The Pentagon's in-house alternative
In December, the Defense Department launched GenAI.mil, its own internal AI platform. Defense Secretary Pete Hegseth told the department to "login, learn it, and incorporate it into your workflows immediately." Over a million users signed on.
But GenAI.mil currently handles only unclassified work. It cannot perform secure RAG on top-secret databases. Pentagon officials said the department plans to deploy AI across "all classification levels" eventually, but offered no timeline or technical details.
That limitation may protect companies like Ask Sage. The platform remains the only commercial tool approved for classified Pentagon networks. Ask Sage was acquired by defense analytics firm BigBear.ai in November for $250 million, and 14,000 teams across 27 U.S. government agencies still use it.
A broader business opportunity
The secrecy problem extends far beyond government. Any organization sitting on proprietary data-law firms with case documents, pharmaceutical companies with research, investment banks with due diligence memos-faces the same dilemma.
Emily Harding, a former CIA analyst now at the Center for Strategic and International Studies, said roughly 100 to 200 people in the intelligence community deeply understand this problem. "I think there's millions and millions of business people who are going to face this same problem, not with as high stakes."
Arize CEO Jason Loepatecki sees the infrastructure space as one of the fastest-growing sectors in AI. "The world's data is infinite, and the pockets of data that you don't want to be trained publicly are large," he said.
For government workers drowning in compliance tasks and document summaries, these platforms offer practical relief. They reduce manual burden while keeping secrets where they belong-compartmentalized and secure.
Learn more about AI for Government and Generative AI and LLM technologies.
Your membership also unlocks:
