Universities have largely treated generative AI as an integrity problem, banning it or requiring disclosure statements. Employers see it differently. They need graduates who can decide when to trust a model's output and when to override it. That gap is driving new assessment designs that force students to argue against AI rather than simply use it.
In a business information systems course at a major Australian university, students now complete two linked tasks. First, they participate in a live debate on how AI is reshaping work, defending positions on individual, organisational, and future-of-work impacts without notes while fielding peer questions. The second task follows immediately: each student takes a position they defended, asks an AI tool to produce the strongest possible counterargument, and then evaluates it in writing.
"The task does not ask whether the AI was right; it asks whether the student can tell," the course design states. Students must identify well-reasoned points, where the AI overreaches, and where it fails to engage with disciplinary evidence, connecting their judgment to course concepts.
Design principles for developing AI judgment
Create a stake in the question. Students need a defended position before they can critically evaluate an AI output. Without that intellectual commitment, they have nothing to test the model against. Public debate and oral defense create the necessary stake.
Be adversarial. The standard use case positions AI as an assistant, producing summaries or refinements. Instead, students should ask the AI for the best counterargument to their position. This forces them to assess where the model is overconfident, poorly evidenced, or misses disciplinary reasoning.
Emphasise reflection as a metacognitive act. The highest marks go to students who can reconstruct their reasoning, acknowledge when the AI made a fair point, and explain why it was or wasn't sufficient to change their view. Documenting every AI interaction isn't enough; the quality of judgment matters.
Capabilities employers need
Student reflections and performance point to three capabilities that develop through this adversarial approach. Evidence-based argumentation grows from the public accountability of a live debate, pushing students beyond surface familiarity. Metacognitive self-assessment requires them to spot gaps in their own reasoning rather than rationalise them away. Calibrated judgment about AI emerges as students move from uncritical assumptions to a grounded sense of where human judgment remains indispensable.
These capabilities align with what workplaces increasingly demand: graduates who can interrogate a model's reasoning, identify where it is confidently wrong, and take responsibility for the decision that follows. The bottleneck in AI-supported work is not generation but evaluation.
Educators looking to implement similar assessment designs can explore an AI Learning Path for Teachers that focuses on building these evaluative skills in the classroom.
Why this matters for education professionals
Treating AI as an integrity problem leaves students without the judgment skills employers value. Assessment that forces students to challenge AI, weigh its claims, and articulate why they trust or override it produces accountable thinkers rather than fluent users. Until evaluation becomes the core of AI literacy pedagogy, graduates will remain unprepared for workplaces where the critical question is not what AI can generate, but whether the human in the loop knows when to say no.
Your membership also unlocks:
