When to Ask AI for Help: Timing Matters for Critical Thinking
Waiting to consult an AI chatbot until you've partially worked through a problem produces better critical thinking results than using the tool from the start, according to research presented April 14 at the CHI conference on Human Factors in Computing Systems in Barcelona.
Researchers at the University of Chicago assigned 393 people to eight groups based on two variables: available time (30 minutes or 10 minutes) and when they could access ChatGPT (early, continuous, late, or never). Participants played city council members reviewing documents about a water contamination proposal and wrote essays explaining their decisions.
The strongest essays came from participants who had 30 minutes and used the chatbot only late in the process. This group also incorporated the most perspectives in their arguments, a measure of reduced bias in reasoning.
The Time Pressure Problem
When time was tight-just 10 minutes-the pattern flipped. Early chatbot access produced the best essay scores among the rushed groups. But this came with a cost.
Using AI under time pressure risks adopting the chatbot's framing rather than developing independent arguments, said Mina Lee, the computer scientist who led the study. "When you are under time pressure and use AI to boost your performance, then you are basically risking the AI's framing, and that reduces the kinds of arguments that you make and your engagement with the documents," Lee said.
Participants without chatbot access but with sufficient time performed best at recalling information from the source documents. This suggests that independent reasoning builds stronger knowledge retention.
Two Types of Learning
The findings align with established research on how people learn, according to Barbara Oakley, a systems engineer and education expert at Oakland University. Slow learning involves careful reasoning and weighing options. Fast learning relies on quick judgments with little reflection.
Participants who reasoned through material independently before using AI benefited from that deliberate, slower approach. They had built understanding before consulting the tool.
What This Means for Your Work
The research highlights a trade-off between speed and independent reasoning. In real-world scenarios with tight deadlines, people often must choose between using AI early for faster results or investing time upfront for deeper understanding.
Effective use of chatbots requires what Lee calls "AI literacy"-understanding your own thinking patterns and weighing the risks and benefits of tool use at different problem-solving stages. Consider when you reach for prompt engineering techniques or chatbot assistance. Are you under genuine time pressure, or could you benefit from independent reasoning first?
"I think our work targets time constraints as the first step towards understanding," Lee said. The takeaway: be aware of what you're trading when you use AI, and when.
Your membership also unlocks:
