A writer who tested their own work across multiple AI detectors got back contradictory results for the same piece of writing - and the problem extends to content created years before generative AI existed. The experiment raises questions about whether businesses and editors should rely on tools that cannot consistently identify human writing.
The writer, who has used AI for content work for the past decade, decided to write one article without any AI assistance. After finishing, they ran it through several well-known AI detectors. The results: 100% AI-generated from one tool. 78% AI from another. 42% from a third. One tool said a human wrote it. A couple required payment before showing results.
"Same article. Same words. Different answers every time," the writer noted. "Which left me with a simple question: If these tools cannot agree on something I know is entirely human, what exactly are they detecting?"
Detectors flag writing from before AI writing tools existed
The writer tested an article from 2014, eight years before ChatGPT launched. Most tools cleared it, but one gave it only a 66% probability that a person wrote it. Another article published in 2019 for Search Engine Journal produced similar confusion. One tool said it was 16% AI. Another "claimed it had found traces of GPT, a product that didn't even exist yet."
Stanford researchers have documented this problem at scale. They found that AI detectors incorrectly flagged much of the writing from non-native English speakers as machine-generated. Meanwhile, a 2026 study published in ScienceDirect found that small human edits allowed genuine AI-generated content to bypass detection entirely.
False positives create a market for "humanizers"
Several companies sell AI detectors on one side and "humanizer" tools on the other. The writer tested this directly: an article they wrote in 2014 was scored 100% human by one detector, which then offered to "humanize" it - for a fee. "Humanize what exactly?" they asked. "The human? Irony doesn't get much better."
The dynamic amounts to what the writer described as a "false economy." Writers who worry about being wrongly flagged feel pressure to buy a solution. The article's author described the pattern as "fear of writing" - or FOW - where content is judged first by detector score, before any consideration of accuracy, originality or usefulness.
LinkedIn's slop button and the limits of detection
LinkedIn recently added a button allowing users to flag content that "seems like AI slop." Dan Roth, LinkedIn's editor in chief and vice president of content, told The Wall Street Journal: "Obviously, a big share of LinkedIn's content is AI-assisted, but nobody can know exactly how much and including LinkedIn, and anyone claiming they do is selling a detection tool." A LinkedIn spokesperson added: "AI can be a great support tool in helping people in articulating ideas, refining language, or making language more concise."
Indiana University's Kelley School of Business bans AI detection tools outright in its faculty AI playbook. The school calls them unreliable and tells staff not to upload student work to them.
Why this matters for writers
Writers lose work and second-guess their craft when editorial teams treat detector scores as gospel. The original author's final point is direct: "Check the facts and back human judgment ahead of tools, push writers for original thinking and real expertise. Just don't confuse any of that with a detector score." For professionals navigating this, AI for Writers covers how to use writing assistants without running into detection issues. Dedicated AI Learning Path for Technical Writers resources can also help writers build skills around clear policy rather than chasing unreliable numbers.
Your membership also unlocks:
