AI Detectors Flagged 61% of Real Student Essays as AI — Can You Trust the Score?
Did a detector just flag something you wrote yourself — an essay, a cover letter, a report — as AI-generated?
A detector score is a probability, not proof. And the research shows it can be badly wrong in exactly the cases where it matters most.
What the Studies Actually Found
The story changed twice: the 2023 tools were unreliable, the best 2026 tools are far better — and the same three weak spots still break them.
When ChatGPT went mainstream in early 2023, detection tools rushed to answer one question: did a machine write this? Three years of independent tests now give a clear answer — and it depends heavily on which tool, which text, and which year you mean.
2023: The Tools Were Unreliable
OpenAI's own classifier is the most honest data point from that year. On its own English test set, it caught only 26% of AI-written text and wrongly labeled 9% of human writing as AI. OpenAI shut it down on July 20, 2023, citing low accuracy.
The Stanford study from the same year is the one that still gets quoted — for good reason. Seven popular detectors were run on 91 essays written by non-native English speakers for the TOEFL exam. On average, the detectors flagged 61.22% of those human-written essays as AI-generated, and 89 of the 91 were flagged by at least one detector. Essays by US eighth-graders, written in native English, were classified almost perfectly. The tools were not detecting AI. They were detecting writing that looked different from native-speaker norms.
A separate test of 14 tools by Weber-Wulff and colleagues reached the same verdict from another angle: every tool scored below 80% accuracy, and AI text that had been paraphrased was caught only 26% of the time.
2025–2026: The Best Tools Got Much Better
The newer tests tell a different story — for the best tools, on the easiest texts. A 2025 University of Chicago working paper tested Pangram, GPTZero, and Originality.ai on 1,992 human passages in six genres and found false positive rates at or below 1% on medium and long passages. Epoch AI's July 2026 test went further: on 495 human passages of about 500 words, Pangram and GPTZero flagged none, and Originality.ai flagged 19, or 3.8%. Plain AI text from current models was caught nearly every time.
But the average tool still lags the best. A February 2026 study of 192 texts put Turnitin at 61% and Originality.ai at 69% macro-averaged accuracy. And even the best tools slipped when AI text was written to imitate a specific human author — Epoch AI found they missed 10% to 18% of those passages, and about a quarter of the imitations in scientific writing.
Why Detectors Get It Wrong
A detector measures how predictable your writing looks. Careful, formal, or polished writing looks predictable.
Most detectors work on two statistical signals: perplexity, how predictable each word is given the words before it, and burstiness, how much sentence length and structure vary. Human writing is usually less predictable and more uneven than model output. The problem is that plenty of human writing is predictable too — and plenty of AI writing is not.
- Non-native English: Writers working in a second language often use simpler, more standard phrasing — exactly the pattern detectors read as machine-made. The Stanford result, 61% of TOEFL essays flagged, is the clearest proof.
- Short texts: A detector needs length to judge. OpenAI said its own classifier was very unreliable on texts under 1,000 characters. A 200-word answer and a 2,000-word essay are not the same test.
- Formal and formulaic writing: Academic and business prose follows set patterns — introductions, transitions, standard phrasing. Those are the same patterns language models produce, so the two look alike to a statistical test.
- Edited or polished text: Every round of editing smooths out the quirks that mark writing as human. The more polished the draft, the more it resembles model output — which leads to the trap in the next section.
None of this is a secret inside the field. Detectors judge the text itself, not where it came from. They cannot see your screen, your drafts, or your chat history. That is why their output is always an educated guess about authorship, not a record of it. The same statistical guessing is behind a different failure: AI summaries invent details for a similar reason — the model predicts plausible text, and plausibility is not the same as truth.
The Polish Trap: Fixing Your Own Draft Can Flip the Verdict
Running your own draft through AI for a grammar fix can be the thing that gets it flagged.
This is the finding most top guides leave out. A 2025 University of Maryland study by Saha and Feizi tested what happens when human-written text gets a light AI polish — a grammar pass, a clarity edit, nothing more. Minimal polishing with GPT-4o pushed detection rates to between 10% and 75%, depending on the detector. GPTZero flagged about 64% of human-written texts as AI after extremely minor AI edits.
The detectors could not tell a lightly polished draft from a heavily AI-written one. They struggle to grade degrees of AI involvement at all — a text either looks machine-like or it does not, and polishing moves it across that line.
The practical lesson is simple. If you use AI to edit your own writing, keep the original draft. The unpolished version is your proof that the ideas and the words started with you — and it is the one thing a detector can never produce.
What to Do If a Detector Flags Your Work
Do not argue with the score. Challenge what it can and cannot show — and bring evidence the detector never sees.
1. Ask what was actually tested
Which tool, which version, and what threshold turns a score into a flag? How long was the text? A single percentage with no tool name and no threshold is not a finding. Ask for the report, not the number.
2. Check the three weak spots
Was the text short? Was it formal or written in non-native English? Was it polished with an AI editor? If any of those apply, say so — the peer-reviewed research is on your side, and the Stanford and Maryland numbers above are the citations to use.
3. Show your process, not just your text
Drafts, outlines, notes, version history, search history. A detector reads only the final text; your process is the evidence it cannot see. An early draft with typos and half-finished sentences is worth more than any argument about the score.
4. Ask for a human review — in writing
Keep it calm and factual, and attach the drafts. Something like this works: "This score measures how predictable my writing looks, not whether I wrote it. Here are my drafts and revision history — could a person review the work?" A written request creates a record, and a record is what turns a score back into a conversation.
One thing not to do: do not try to beat the detector with a paraphraser or a "humanizer" tool. It only adds another layer the next check can flag, and in a school or work setting it reads as an admission that you had something to hide.
When a Detector Score Is Actually Useful
A score is a reason to look closer — never the verdict.
Used that way, detectors have a real job. A teacher reviewing a hundred essays can use a high score as a prompt to ask for drafts — a conversation starter, not an accusation. A hiring manager can treat a flag as a reason to ask a candidate to walk through their work. The score points at where to look. A person still has to look.
The technology is also shifting under the question. On October 5, 2026, OpenAI announced textGrain, an invisible statistical watermark built into ChatGPT and Codex text for users in the EU, with an opt-in for API customers elsewhere. Detection would move from guessing at style to checking a signal — but the detector is available only to approved researchers and expert organizations for now, and even watermarks weaken under editing: in OpenAI's own test, replacing 10% of words with synonyms cut detection from about 92% to 66%. A missing watermark proves nothing, and for most people, the guesswork tools are still the only ones in use.
If you want the other side of this problem — how chatbots build their answers in the first place, and why that makes them invent things — see how AI chatbots generate answers.
Key Takeaway
AI detectors are better than they were in 2023. The best ones now clear long, ordinary human writing almost every time. But they still misfire on short texts, formal and non-native writing, and anything AI-polished — and a single score cannot tell you who wrote a passage.
If a detector flags your work, do not defend the score. Show your drafts and ask a person to look.
Sources
• Liang, W., et al. (2023). "GPT detectors are biased against non-native English writers." Patterns, 4(7). doi.org/10.1016/j.patter.2023.100779
• OpenAI (2023). AI text classifier — caught 26% of AI text, 9% false positives; discontinued July 20, 2023.
• Weber-Wulff, D., et al. (2023). Detection-tool testing. International Journal for Educational Integrity.
• Saha, S., & Feizi, S. (2025). "Almost AI, almost human: The challenge of detecting AI-polished writing." arXiv:2502.15666. doi.org/10.48550/arXiv.2502.15666
• Jabarian, B., & Imas, A. (2025). "Artificial writing and automated detection." NBER Working Paper 34223. doi.org/10.2139/ssrn.5407424
• Epoch AI (July 2026). "AI detectors' false negatives." Test of Pangram, GPTZero, and Originality.ai on 495 human passages.
• OpenAI (October 5, 2026). "Our approach to EU text provenance rules" (textGrain watermark).

Comments
Post a Comment