A preregistered study from ETH Zurich involving 100 tertiary-level students found that computer-science knowledge, not daily AI use, predicts success at vibe coding. The paper, by Sverrir Thorgeirsson, Theo B. Weidmann and Zhendong Su, appears in the Proceedings of the 2026 CHI Conference on Human Factors in Computing Systems. The finding challenges the widespread pitch that AI tools let anyone build software without understanding the fundamentals.
Computer-science achievement correlated with vibe-coding performance at r=.39, while writing skill came in at r=.29. CS achievement remained significant even after the researchers controlled for domain-general cognitive ability. In the joint model, it carried roughly twice the unique variance of writing skill.
The no-code pitch hits a wall
Students who reported more exposure to large language models in daily life performed worse on the vibe-coding task, not better. Familiarity with the tool did not translate into skill using it for this job. That lands directly on the claim from AI coding vendors and bootcamp marketers that you no longer need to understand loops, state, data structures, or failure modes to ship a product.
The researchers built the writing test around something concrete. Each participant wrote a 300-to-450-word explanation of a technical concept from their own coursework. That is not a creative-writing exercise. It tests whether you can describe a system precisely enough that a model can act on it correctly. A vague prompt produces vague code. A prompt that names edge cases, data shapes, and failure conditions gives the model a better target. Vibe coding did not remove the need for precision - it moved some of that precision from source code into English.
The skill didn't disappear - it moved
For solo, non-technical founders building an MVP entirely through prompts with tools like Cursor, Replit, and Lovable, the study points to the missing skill. You can get further than you could five years ago. The ceiling, however, appears to be set by the same knowledge AI was supposed to replace. The tool amplifies what you already know. It does not hand you what you do not.
A second data point reinforces the warning. METR, a nonprofit research group, published a randomized controlled trial in July 2025 on 16 experienced open-source developers working across 246 real tasks in repositories they knew well. The developers expected AI tools to cut completion time by 24%. Instead, they took 19% longer. Confidence keeps outrunning the measured result, whether the user is a novice or a veteran.
For teams hiring or training people to use AI coding agents, the ETH Zurich findings argue against treating vibe coding as a shortcut around fundamentals. Hire for CS knowledge, not prompt count. Someone who understands data structures and failure modes will write a sharper prompt than someone who simply opens ChatGPT every day. This aligns with what structured AI Coding Courses teach - the human steering the model still matters, and effective steering depends on old-fashioned domain knowledge. Similarly, Generative Code Courses emphasize that precision in specification, not frequency of tool use, drives output quality.
Why this matters for science and research professionals
For researchers who evaluate tool claims, design experiments, or train teams, the ETH Zurich paper offers a template for cutting through vendor narratives. The study preregistered its design, used concrete performance tasks rather than self-reported satisfaction, and tested the specific mechanism - daily AI habit versus domain knowledge - that the industry's marketing relies on. When a vendor says a tool replaces the need for expertise, the right question is not whether the demo looks smooth. The question is whether the measured performance data shows the same thing. Here, it did not.
Your membership also unlocks:
