State AI Procurement Is Broken. Here's How to Fix It
State and local governments are buying AI systems at scale, but their contracts almost never include provisions for transparency, fairness, or accountability. An analysis of over 1,000 AI contracts across California, Utah, and Florida found that 77% of contract language is standard boilerplate. Only 5.3% addresses transparency. Just 2.4% addresses fairness and accountability.
These procurement decisions lock in governance choices for a decade or longer. Once signed, the window to negotiate stronger terms largely closes.
The consequence is real. Michigan's botched unemployment fraud detection system wrongly accused over 34,000 people between 2013 and 2015. The state's original $52 million contract with Fast Enterprises included no provisions for algorithmic transparency, bias testing, or independent auditing. When Michigan replaced the system, it signed a $78 million contract with Deloitte-again without meaningful oversight clauses. The total cost: over $125 million across two contracts, plus a $20 million class-action settlement.
Procurement Is Government's First Line of Defense
Most AI policy debate focuses on regulation and legislation. But the more consequential decisions happen earlier, in contract negotiations. Procurement is how technology enters government.
Standard government procurement evaluates cost, vendor qualifications, and regulatory compliance. It rarely assesses algorithmic risk. Agencies typically don't ask about bias testing, access to training data, or how a model makes decisions. Procurement capacity has not kept pace with technical complexity.
The problem compounds through cooperative purchasing agreements. More than 4 out of 5 state AI contracts were negotiated through the NASPO ValuePoint platform, which allows one state to negotiate terms that dozens of others adopt without rerunning the procurement process. This concentrates risk. Boilerplate language from a single contract becomes the template for many jurisdictions.
Long Contracts Trap Governments in Bad Decisions
The median state AI contract runs seven years. Some span a decade. Governments often respond to failed AI implementations by signing longer contracts with more established vendors, not shorter ones.
Once deployed, vendor lock-in sets in. Governments lose meaningful control over the data a system processes. Vendors retain proprietary rights over training data, model architectures, and performance analytics. Switching providers becomes costly and legally complex. Institutional knowledge becomes embedded in vendor systems, making transition nearly impossible.
Arkansas could not explain details of a model used to determine Medicaid benefits. Idaho refused to disclose a benefits allocation formula, claiming it was a vendor trade secret. Public decision-making systems remained shielded from public accountability.
Three Reforms States Can Implement Now
State governments have the authority to strengthen AI procurement without new legislation. Three reforms can be implemented immediately.
1. Adopt Standardized Contract Clauses Aligned With NIST Framework
State procurement offices should develop standardized responsible AI contract language aligned with the National Institute of Standards and Technology's AI Risk Management Framework. The EU's AI Act distinguishes between high-risk and low-risk systems with model contractual clauses tailored to each category. The U.S. lacks equivalent standards.
IEEE Standard 3119-2025 provides a ready-made procurement framework covering problem definition, solicitation, vendor evaluation, and contract monitoring. A multi-state working group convened through NASPO could adapt these standards into model contract clauses within 12 months.
At minimum, clauses should address:
- Data governance and retention
- Algorithmic bias testing
- Explainability requirements for high-risk decisions
- Breach notification procedures specific to AI systems
- Performance benchmarks with renewal contingencies
Standardized language reduces administrative burden and lowers barriers for smaller vendors unfamiliar with government procurement.
2. Implement Risk-Tiered Procurement Review
San José's Digital Privacy Office already uses this approach. When a city department submits a procurement request, the office assesses risk level. Low-risk systems are approved without delay. High-risk systems trigger impact assessment and vendor review.
Colorado's Office of Information Technology uses a NIST-based risk assessment framework to evaluate all generative AI use cases. This provides a state-level proof of concept.
States with existing AI governance infrastructure are natural pilots. California's Governor issued an executive order in 2023 directing AI procurement guidelines. Colorado's AI Act already requires reasonable care for high-risk systems. San José's Digital Privacy Office operates within the city's IT department without a dedicated budget line, showing this model requires designating existing staff rather than creating new offices.
3. Require Structured AI Fact Sheets for Contract Award and Renewal
Vendors should complete structured AI fact sheets as a condition of contract award and renewal. These function as "nutrition labels" for government AI systems, modeled on San José's template and inspired by IBM Research's AI FactSheets 360.
Fact sheets should capture:
- Training data provenance and representativeness
- Model performance metrics and known limitations
- Bias audit results across protected classes
- Update and versioning procedures
- Data retention and deletion policies
- Human oversight mechanisms
Fact sheets must be updated whenever a model is retrained or its scope changes. They should be required for both initial contract award and each renewal cycle.
Shorter contract terms with performance-contingent renewal preserve government's ability to reassess and change course. The principle is simple: no test, no renewal.
Federal Role: Don't Block State Innovation
The federal government can reinforce state efforts without preempting them. The AI in Government Act of 2020 and OMB Memorandum M-25-21 offer templates states can adapt. Recent federal proposals to create regulatory ceilings on state AI rulemaking risk stunting useful state innovation.
Implementation Is Feasible
A NASPO-convened working group could develop model contract clauses once for shared use across all member states, amortizing development costs. IEEE 3119-2025 provides a ready-made framework reducing the need for states to develop standards independently.
The cost of inaction exceeds the cost of reform. AI initiative failure rates in government reach 70 to 85%. The federal government spends 80% of its $100 billion IT budget maintaining legacy systems.
Strengthening procurement today will shape AI outcomes for decades. Procurement is the mechanism through which governments quietly govern AI at scale.
Learn more about AI for Government or explore the AI Learning Path for Policy Makers to understand governance frameworks and procurement standards.
Your membership also unlocks:
