Deloitte to repay part of $440k report after AI-related citation errors
Deloitte will return the final instalment of a $440,000 contract to the Albanese government after errors were found in a departmental review that used generative AI. The Department of Employment and Workplace Relations (DEWR) says the repayment will be made public once finalised. Deloitte and the department maintain the report's findings and recommendations are unchanged.
The review examined DEWR's targeted compliance framework and its IT system, which automates penalties when jobseekers miss mutual obligations. The July report flagged weak "traceability" to legislation, "system defects," and a system "driven by punitive assumptions of participant non-compliance." It was re-uploaded after multiple incorrect or nonexistent references were identified.
What happened
Deloitte acknowledged incorrect footnotes and references and updated the final report. The appendix now cites use of a generative AI toolchain based on Azure OpenAI GPT-4o hosted in DEWR's Azure tenancy. Deloitte did not say AI caused the errors and stands by the substance of the review.
University of Sydney academic Dr Christopher Rudge said the report showed signs of AI "hallucinations," where sources are fabricated or misread. Labor senator Deborah O'Neill said it appeared "AI is being left to do the heavy lifting," adding: "Deloitte has a human intelligence problem… A partial refund looks like a partial apology for substandard work."
The Australian Financial Review reported nonexistent references, including to academics and a robodebt case, Deanna Amato v Commonwealth. Deloitte's final report states it amended the summary of the Amato proceeding, which previously contained errors.
Why this matters for government
AI-assisted drafting can reduce cost and time, but it shifts risk to verification and accountability. In compliance settings that impact payments and penalties, weak evidence chains can mislead decision-makers and entrench faulty automation. Refunds recoup money, but they don't recover time or trust.
Immediate steps for procurement and assurance
- Demand disclosure: Vendors must declare any AI use (model name, version, host, prompts, datasets) and where it was used in the deliverable.
- Enforce verifiable evidence: Require primary sources or authoritative citations with working links or document IDs; set a 100% check for legal and legislative references.
- Tie payment to quality gates: No final payment until citations are validated, facts are traceable, and recommendations map to legislation, policy, and business rules.
- Add auditability: Keep logs of AI interactions and human reviews; retain for internal audit and ANAO access.
- Protect data: Limit tools to agency-controlled environments; ban training on agency data; align with PSPF/ISM controls.
- Assign human accountability: Name responsible reviewers for legal, policy, and technical content; define liability for defects and misrepresentation.
- Stress-test automated decisions: Require impact assessments, documented error rates, and human-in-the-loop escalation for penalties and edge cases.
- Use procurement levers: Bake in withhold/repayment clauses for quality failures and undisclosed AI use under the Commonwealth Procurement Rules.
Key facts from the DEWR review
- Commissioned: December 2024. First published: 4 July. Re-uploaded after errors were identified.
- Findings retained: Department and Deloitte say recommendations are unchanged despite citation fixes.
- AI involvement: Part of the report used Azure OpenAI GPT-4o in DEWR's Azure tenancy.
- Corrections: Nonexistent citations removed; summary of the Amato proceeding amended.
- Repayment: Deloitte will return the final instalment; the department will publish details once complete.
Questions to ask your next consulting vendor
- Which sections were produced or assisted by AI? Which model and environment?
- Who verified each claim and citation, and can you show the evidence chain?
- What hallucination tests and citation checks were run, and by whom?
- What are the repayment/penalty triggers if errors surface after delivery?
Tools and references
Upskill your team
If you're updating procurement or assurance for AI-assisted consulting, targeted training helps reduce these risks. See AI training for public sector teams for practical courses on AI governance, assurance, and audit-ready documentation.
