AI prose still reads like AI - new study maps the stylistic gap
Researchers at University College Cork (UCC) used literary stylometry to compare human short stories with text from large language models. The verdict: AI writes polished, fluent prose, but it follows a narrow, uniform groove. Human authors show wider stylistic range shaped by voice, intent, and lived experience.
The study, led by Dr James O'Sullivan at UCC's School of English and Digital Humanities, analysed hundreds of stories from people alongside outputs from GPT-3.5, GPT-4, and Llama 70B. Subtle linguistic markers - especially the frequency of common words - revealed clear, consistent differences that hold across models and prompts.
How the analysis worked
Stylometry clusters texts by shared stylistic fingerprints. AI systems formed tight clusters, each model grouped by its own uniform patterns. Human writing scattered widely, reflecting individual habits and creative choices.
GPT-4 wrote with even more consistency than GPT-3.5. GPT-3.5 occasionally drifted closer to human style, but those moments were rare. Llama 70B showed the same clustering effect: compact, predictable, and distinct from human work.
"While AI writing is often polished and coherent, it tends to show more uniformity in word choice and rhythm. In contrast, human writing remains more varied and idiosyncratic, reflecting individual habits, preferences and creative choices. Even when ChatGPT tries to sound human, its writing still carries a detectable fingerprint, which suggests that computers and people don't yet write in quite the same style."
What this means for creatives
If you lean on AI for drafts, expect sameness. Your advantage is voice - rhythm, risk, and the strange turns machines avoid. Use AI for scaffolding and speed; make the final cut unmistakably yours.
- Build a voice board: 5 passages that feel like you. Compare every draft against it.
- Impose constraints: sentence fragments, unexpected verbs, sensory details from a lived scene.
- Vary rhythm: short bursts next to longer runs; break patterns AI tends to repeat.
- Seed specifics: names, places, niche references, and contradictions only you would write.
- Run a sameness pass: highlight repeated phrasing and swap clichés for sharper choices.
Don't use stylometry to police students
The researchers caution against stylometry as an AI detector in education. Students shift style by task, context, and support, which makes detection unreliable and ethically questionable. Use stylometry to study broad patterns, not to judge authorship.
The bigger questions for art and authorship
LLMs can churn out emails and summaries on cue. Automating literature is a different thing - it raises hard questions about authenticity, originality, and what authorship means when style can be mimicked but not lived.
The team points to the need for broader datasets, better prompts, and testing across new models. For creatives, the takeaway is simple: double down on the parts of your craft that resist compression - your history, your taste, your point of view.
Go deeper
- Nature: Humanities and Social Sciences Communications - publication venue for the study.
- AI tools for copywriting - practical options to draft faster, then revise for voice.
Your membership also unlocks:
