A new survey of 111 national security practitioners reveals that speed and outdated legal frameworks are the biggest obstacles to countering AI-driven threats, while chronic underinvestment in testing leaves the U.S. government unable to independently verify what frontier AI systems can actually do. The Institute for Security and Technology (IST) report, published Wednesday and conducted between late April and mid-July, captures expert fears that dependence on AI industry expertise makes the government vulnerable to regulatory capture.
The verification gap
One of the report's sharpest findings centers on the government's inability to assess AI claims from private labs. "Respondents pointed to chronic underinvestment in test and evaluation, the absence of independent evaluation processes, and no reliable way to benchmark frontier capabilities," the report said. The consequence is straightforward: government leaders cannot confirm how AI systems will behave before integrating them into military technology and other critical platforms. "Without its own benchmarks, the [government] is left to take capability claims on faith rather than measurement."
Experts also expressed deep skepticism about trusting current AI systems to operate independently. "Many see hallucination as intrinsic to AI models, making systems highly problematic to rely on without significant, detailed human checking," IST observed. As one respondent put it, "AI can make a good analysis better and a bad one worse."
Cyber offense versus defense
More than 80% of respondents said AI would "very likely" improve cyber defense capabilities across the board - from vulnerability discovery to malware analysis to incident response. Yet 57% also said AI was helping attackers more than defenders in the near term. The top cybersecurity risk associated with AI is its ability to find and exploit vulnerabilities, which respondents ranked first by a wide margin.
Professionals pursuing an AI for Cybersecurity Analysts learning path will recognize this asymmetry. The bottleneck, IST noted, is not the technology itself but deploying cyber defense at scale. Weak cybersecurity "will enable many of these other risks," including loss-of-control scenarios, adversarial manipulation, and the theft of model weights and weapons designs.
Regulatory fragmentation
Legal and regulatory frameworks ranked second on the list of hurdles to combating AI threats, but respondents showed no consensus on the right approach. Roughly one-quarter advocated for cross-sector federal requirements, 31% supported sector-specific requirements, and 36% favored a hybrid model. The split underscores the difficulty facing lawmakers trying to craft rules that satisfy security needs without stifling innovation.
For those shaping these decisions, the AI for Policy Makers course addresses the exact tension between rapid technological change and deliberate governance that the survey highlights.
Why this matters for government professionals
The survey makes clear that basic cybersecurity failures - not exotic AI capabilities - are likely to trigger the most damaging incidents. Experts encouraged policymakers to resist focusing too narrowly on AI's unique features when evaluating risk. The more immediate priority is closing the gaps in cyber defense at scale, building independent testing capacity, and resolving the regulatory patchwork before adversaries exploit the lag. For agencies integrating AI into critical systems, the report is a blunt reminder: if you cannot benchmark it, you cannot trust it.
Your membership also unlocks:
