AI help can blunt skill growth for novice devs. Here's what the data says-and what to do about it
Researchers from Anthropic, including Judy Hanwen Shen and Alex Tamkin, ran randomized experiments on developers learning the Trio async library. The outcome is clear: heavy reliance on AI assistance lowered conceptual understanding, code reading, and debugging skill-without meaningful speed gains.
AI can boost output in the moment, but competence comes from struggle, reading code, and resolving errors. If you're training juniors or onboarding to a new stack, how you use AI matters more than whether you use it.
The Trio experiment, in plain terms
Participants tackled asynchronous programming challenges with the Trio library, chosen for its relative unfamiliarity compared to asyncio (based on StackOverflow activity). One group worked solo. The other used an AI assistant inside an online interview platform.
Skills were measured across three areas: conceptual understanding, code reading, and debugging. Assessments included multiple-choice questions, code analysis, and targeted debugging, all specific to Trio.
The numbers worth caring about
- Library-specific skills dropped by 17% (about two grade points) with AI assistance.
- No statistically significant acceleration in completion time with AI.
- Interaction overhead was real: some participants asked up to 15 questions or spent 30%+ of their time composing prompts.
- Generation-Then-Comprehension: 24 minutes, 86% quiz.
- AI Delegation: 22 minutes, 65% quiz.
- Iterative AI Debugging: 31 minutes, 24% quiz.
- Hybrid Code-Explanation: 22 minutes, 35% quiz.
- Progressive AI Reliance: 19.5 minutes, 39% quiz.
The control group improved by confronting errors directly. Full delegation sometimes looked faster in the short run, but it taxed future performance by skipping the learning.
Why skills drop with AI "on"
When the assistant writes or fixes everything, you skip the parts that build intuition: tracing flow, reading unfamiliar code, and forming mental models. The assistant reduces friction-and with it, the cognitive work that creates durable knowledge.
In contrast, patterns that keep you thinking-asking for explanations, probing concepts, verifying with docs-preserve learning even with AI in the loop.
Six interaction patterns-and the ones that protect learning
Researchers identified six usage patterns. Three protected learning by keeping cognitive engagement high. The common thread: they pushed for explanations and conceptual clarity instead of copy-pasting solutions.
- Good signs: "Explain this API before generating," "What's the right concurrency model in Trio for X?", "Show me how this differs from asyncio," "Give a minimal example; I'll extend it."
- Red flags: "Write the whole function," "Fix my code" on repeat, bouncing between AI-generated patches ("iterative AI debugging") that ballooned time and crushed scores.
Use AI without burning your skill compounding
- Set a first-principles window: spend 10-15 minutes sketching the approach and writing a minimal attempt before asking the AI for anything.
- Ask for concepts first, code second: "Explain nursery and cancel scopes in Trio with a 10-line example," then implement yourself.
- Force comparison: generate an AI snippet, then annotate each line in comments. If you can't explain it, you can't ship it.
- Timebox assistant use: cap it to 20-30% of a task. Track prompts-per-task to avoid the 15+ question trap.
- Read errors before prompting: summarize the error and your hypothesis; then ask a targeted question.
- Write tests early: use AI to suggest edge cases, not final solutions. Validate your own fixes against those tests.
- Pull the docs: link to the exact section in docs when accepting AI code. If it's not in the docs, treat it as suspect.
Team-level guardrails for leads and reviewers
- PR template additions: "What concept did you verify?", "Which doc sections support this approach?", "What failed first, and what did you learn?"
- Usage metrics: track ratio of "why/how" prompts to "do it for me" prompts. Reward explanation quality in reviews.
- Pairing policy for juniors: AI can assist, but the human partner must narrate the reasoning path before accepting code.
- Error-first onboarding: require newcomers to reproduce, isolate, and fix at least one non-trivial error per module without AI-generated patches.
Where AI still fits
Use it to explain APIs, generate minimal examples, draft tests, and compare approaches. Treat it like a senior who answers questions, not a ghost coder who replaces your thinking.
Speed matters, but retained skill compounds. Optimize for both by keeping the hard parts-reasoning, reading, and debugging-in your hands.
Limitations to keep in mind
The study ran in a controlled setting, on a specific library (Trio), with novices learning new concepts. Results may shift with experienced engineers, different tooling, or production constraints-but the core signal is strong: cognitive engagement is the variable that moves learning.
Resources
- Trio documentation: trio.readthedocs.io
- If you're formalizing dev upskilling with AI in your org: AI Certification for Coding
Bottom line
AI can help you ship. But if you let it think for you, your skills will stall. Keep the assistant in the passenger seat and your brain in the driver's seat.
Your membership also unlocks:
