Deloitte refunds A$97K after AI errors in Australian government report
Deloitte has refunded over A$97,000 (about US$63,000) to Australia's Department of Employment and Workplace Relations (DEWR) after AI-generated errors were found in an assurance review the firm delivered. The original December 2024 contract was valued at about A$440,000, making the refund less than a quarter of the total. DEWR said AusTender will be updated to reflect the refund.
A revised version of the report was published this month after the original included a fabricated quote from a federal court judgment and citations to non-existent academic papers, according to the Associated Press. The review examined the department's IT system that automates penalties in Australia's welfare program. DEWR said some footnotes and references were incorrect, but the substance of the review remains.
Why this matters for government finance and procurement
This is a clear signal: AI can speed up analysis, but it can also introduce confident falsehoods. Accountability still sits with agencies and vendors. Refunds help, but the real cost is time, trust, and rework.
As one accounting professor put it, firms need to train staff on effective and ethical AI use and review AI output as if it came from a new hire. That's the right mindset for public sector buyers too.
Immediate actions for agencies
- Require vendors to disclose if, where, and how AI tools are used in deliverables.
- Make human-in-the-loop review mandatory for all legal citations, statistics, and references.
- Build source verification into acceptance criteria: every quote, footnote, and dataset must be independently checkable.
- Add audit trails: require vendors to keep drafts, prompts, tool logs, and source lists for spot checks.
- Include remedies: partial refunds, rework at no cost, and termination rights for fabricated or unverifiable content.
- Use structured sampling: randomly select sections to deep-audit before approving final payment.
- Protect data: prohibit training on agency data and require model and tool disclosure.
- Train internal reviewers on AI failure modes (hallucinations, sourcing gaps, mis-citations).
Procurement clauses to consider
- AI usage declaration: Vendor must declare all AI tools used and their role in drafting, analysis, or citation.
- Source transparency: Full reference list with accessible URLs or documents; no unverifiable sources.
- Citation accuracy warranty: Legal and academic citations must be exact; fabrications trigger predefined remedies.
- Model governance: Identify models, versions, and guardrails; confirm no sensitive data is retained by tools.
- Quality gates: Named senior reviewer sign-off; checklist for citations, quotes, and numerical consistency.
- Payment holdback: Tie a portion of fees to passing agency verification checks.
Operational checks before accepting any AI-assisted report
- Cross-check every legal quote against the official judgment text.
- Verify academic references: journal, author, year, DOI or working link; confirm the cited claim actually appears.
- Re-run key calculations with agency or third-party data to confirm results match.
- Scan for phrasing patterns common to AI output and request underlying sources where style looks generic.
Governance touchpoints
- Risk register: Flag AI-assisted deliverables as a distinct risk category with owners and controls.
- Supplier performance: Track defects tied to AI use and factor them into future award decisions.
- Transparency: Update public notices (e.g., AusTender) when refunds or corrections occur.
If your team needs structured upskilling on safe, effective AI use and review practices, here's a practical collection of role-based programs: AI courses by job role.
References and resources: Department of Employment and Workplace Relations (DEWR) | AusTender
Bottom line: treat AI-assisted work like an intern's draft-useful, but only after a rigorous human review. Build the checks into contracts, confirm the sources, and keep a real remedy on the table.
Your membership also unlocks:
