AI for Cyber Defenders: Why Now, What's Working, What to Do Next
AI has crossed from promising theory into practical security tooling. Recent progress shows that models can spot, analyze, and help remediate vulnerabilities in code and deployed systems-fast enough and cheap enough to matter for day-to-day defense.
We've prioritized making Claude Sonnet 4.5 stronger at defensive tasks. In many cases it now matches or eclipses Opus 4.1 for vulnerability discovery and related cyber skills, while being faster and less expensive. The takeaway: defenders who adopt and iterate with AI will keep pace. Those who wait will not.
Why this matters
- Simulated reenactments show models can reproduce high-impact attacks like the 2017 Equifax breach. In competitive settings, Claude has outperformed human teams on select tasks and helped identify issues in our own code before release.
- Teams in DARPA's AI Cyber Challenge used LLM-powered "cyber reasoning systems" to parse millions of lines of code, patch known issues, and even surface real, previously undiscovered vulnerabilities.
- Safeguards work has disrupted threat actors attempting large-scale data extortion ("vibe hacking") and complex espionage efforts targeting critical telecom infrastructure, consistent with tactics linked to Chinese APT operations.
The signal is clear: usage and capability are rising. The defensive response has to scale now.
Claude Sonnet 4.5: Emphasizing defense
General model improvements often unlock new skills as a byproduct, including CTF-style exploitation. We focused research on explicitly defensive value: finding insecure code, patching issues, and probing simulated infrastructure for weaknesses, while avoiding work that would tilt toward offensive use (advanced exploitation, malware creation).
The result is Claude Sonnet 4.5-a model that delivers stronger cyber-relevant performance for defenders at lower cost and latency.
What the evaluations show
Cybench (CTF-derived tasks). Sonnet 4.5 shows striking gains. With a single attempt per task, it beats older, larger models even when they get ten tries. With ten attempts, it solves 76.5% of challenges. One complex task-traffic analysis, malware extraction, decompilation, decryption-took Claude 38 minutes, where a skilled human might need an hour or more.
Progress has been steep: from 35.9% success (10 trials) in February 2025 to 76.5% in six months.
CyberGym (real OSS vulnerabilities). Under a strict $2-per-vulnerability query budget-matching the public leaderboard-Sonnet 4.5 sets a new state-of-the-art at 28.9%. Removing artificial constraints and allowing 30 attempts per task, it reproduces known vulnerabilities in 66.7% of programs at about $45 per target-still modest at team scale.
New vulnerability discovery is trending up as well: roughly 5% on a single attempt and over 33% with 30 attempts.
Early progress on patching
Patching is harder than detection. Fixes must remove the risk without breaking intended behavior-often without a formal spec. In preliminary testing on CyberGym targets, 15% of Claude-generated patches were judged semantically equivalent to human patches by an automated comparison (with the caveat that correct, alternative fixes may be penalized in this setup).
Manual checks of top-scoring patches showed functional parity with reference fixes merged in the underlying projects. This suggests patching is an emerging capability that can be strengthened with focused research on reliability and verification.
What practitioners report
Teams applying Sonnet 4.5 to production-grade problems saw meaningful gains. One reported a 44% reduction in vulnerability intake time for AI agents with a 25% accuracy improvement. Another highlighted strong promise for red teaming-rapidly generating attack scenarios to study and counter real tradecraft across endpoints, identity, cloud, data, SaaS, and AI workloads.
Practical guidance for security teams
- Make code security continuous. Add AI-driven security reviews to CI/CD. Gate pull requests on high-confidence findings, and route lower-confidence findings to human review. Allow multiple attempts on hard findings to reduce flakiness.
- Upgrade SOC and SIEM workflows. Use models to summarize alerts, enrich events with context, correlate signals across sources, and triage faster. Keep analysts in the loop for final judgment.
- Probe configurations and controls. Ask AI to review network configs, IAM policies, and segmentation for common misconfigurations. Generate and test playbooks for hardening and incident response drills.
- Evaluate like an attacker would. Run benchmarks multiple times, not once. Track success rates, cost per resolution, time-to-detection, and false positives. Compare across models, prompts, and budgets.
- Build safeguards into usage. Use organization-level summarization and logging to spot large-scale automated misuse. Separate dual-use research from prohibited actions. Rate-limit risky patterns and enforce review gates.
- Invest in skills. Train engineers and analysts on prompting, code review with AI, and validation techniques. Pair AI findings with unit/integration tests to confirm fixes before merge.
What's next
Sonnet 4.5 is a meaningful step, but not a substitute for seasoned security engineers and established processes. Expect continued improvements in discovery, patch generation, verification, and threat intel-paired with stronger platform safeguards against abuse.
The moment calls for action. Start piloting AI across code security, SOC automation, SIEM analysis, secure network engineering, and active defense. Build your own evaluations and share results so the ecosystem can measure progress and improve faster. And push for secure-by-design software, with AI assisting all the way from design review to deployment.
References
- Cybench: Evaluating cybersecurity capabilities and risks of language models
- CyberGym: Evaluating AI agents on real-world vulnerabilities
Upskill your team
- Certification: AI with Claude for security and engineering workflows
- Courses by job: Paths for developers, IT, and security roles
Your membership also unlocks:
