Every published AI detection accuracy figure is true and incomplete at the same time. Turnitin's sub-1 percent document rate holds only for documents already 20 percent machine-written. GPTZero's 95.7 percent comes from a benchmark GPTZero says contains only fully human or fully AI text. The qualifier is the story.
Why This Matters
A detector's headline number gets quoted in academic misconduct hearings, in hiring decisions, and in news coverage. The condition attached to that number gets dropped somewhere between the vendor's blog post and the second paragraph of a summary. What reaches the person accused of using AI is a percentage with no unit and no scope.
A false positive rate describes a population, not a document. A tool can carry a 1 percent false positive rate and still flag your specific essay, because that rate is a property of how the tool performs across thousands of texts, not a promise about the one in front of you.
This page ranks no tool and declares no winner. It contains no first-party detector measurement, because we hold no API or institutional access to these products. What it does contain is the condition attached to each published figure, and a record of what happened when we turned the same scrutiny on our own scoring tool.
What Accuracy Figures Do Turnitin and GPTZero Publish?
Turnitin states a document-level false positive rate under 1 percent for documents with 20 percent or more AI writing. That condition matters, because the headline figure describes documents already at least a fifth machine-written. Turnitin separately states a sentence-level false positive rate around 4 percent, which counts a different unit than the document-level figure.
GPTZero reports detecting 95.7 percent of AI texts while incorrectly predicting 1 percent of human texts as AI on RAID, rising above 99 percent when discontinued models such as GPT-3.5 are excluded. GPTZero itself states that RAID and most detection benchmarks consider only fully human or fully AI texts, so the benchmark excludes the mixed and lightly edited drafts where false positives occur.
| Tool | Published Figure | Unit | Condition |
|---|---|---|---|
| Turnitin | Under 1 percent | Document-level false positive rate | Documents with 20 percent or more AI writing |
| Turnitin | Around 4 percent | Sentence-level false positive rate | Different unit than the document figure |
| GPTZero | 95.7 percent detected, 1 percent human flagged | Text-level on RAID | Fully human or fully AI texts only |
| GPTZero | Above 99 percent | Text-level on RAID | Excludes discontinued models such as GPT-3.5 |
| OpenAI (retired July 2023) | 26 percent detected, 9 percent human flagged | Text-level | Classifier withdrawn after these results |
OpenAI retired its own AI classifier in July 2023 after reporting it identified only 26 percent of AI-written text while flagging 9 percent of human-written text as AI. We could not fetch the original page because openai.com returns HTTP 403 to our fetcher, so we record those figures as third-party corroborated rather than taken from the primary source.
Why Can't You Compare a Document-Level Rate to a Sentence-Level Rate?
Because the two rates divide different things by different denominators. A document-level rate asks how many whole essays got flagged, and a sentence-level rate asks how many individual sentences got flagged. A single misjudged sentence inside a 40-sentence essay produces a sentence-level hit without producing a document-level hit.
Turnitin's under 1 percent figure counts documents. Its around 4 percent figure counts sentences. Quoting one against the other, or against GPTZero's text-level RAID numbers, compares numerators drawn from different populations.
The unit column is what makes the comparison table readable. Turnitin's 800,000 academic writing samples, all written before ChatGPT's release, produced a higher-than-comfortable false positive rate on submissions under 300 words, which is why the tool now refuses to score anything shorter. That 300-word minimum is a published admission that the unit and the length of the text change the error rate. A short submission is a different measurement problem than a long one.
Why Can't a Clean-Endpoint Benchmark Estimate Error on Mixed Text?
Because a benchmark built from fully human and fully AI texts has no mixed drafts in it, so it cannot measure how the tool behaves on the drafts that sit between. GPTZero states this about RAID directly. Most detection benchmarks consider only fully human or fully AI texts.
Real submissions are not clean endpoints. A student drafts an outline, writes the body, runs a paragraph through a grammar tool, and pastes in two sentences from a source. That document is neither fully human nor fully AI, and it is the document type most likely to land in a disciplinary file.
The 61 percent figure from Liang et al. (arXiv 2304.02819, published in Patterns) shows what happens when the benchmark population shifts. Roughly 61 percent of TOEFL essays written by non-native English speakers were flagged as AI-generated, though the writers used no model. Turnitin publishes its own research stating no statistically significant bias against English Language Learner writers, a direct response to that finding. Two published claims, two different test populations, and the disagreement lives entirely in the condition column.
Why Does Evasion Stay a Moving Problem Rather Than a Solved One?
Because machine text sits near a local optimum of the model's own probability, so masking a handful of tokens lowers its likelihood under the model while human text stays roughly flat under the same perturbation. Mitchell et al. (arXiv 2301.11305, ICML 2023) named this property in DetectGPT.
Krishna et al. (arXiv 2303.13408) showed that paraphrase attacks dropped detector accuracy hard, and that the defense which restored it was retrieving against a database of known machine-generated outputs rather than training a stronger classifier. The fix was a retrieval problem, not a classification problem.
GPTZero states that under adversarial attacks every method suffers a decrease in detection rate at a 1 percent false positive rate. Evasion is a shared property of this class of tool, not a defect in one product. A detector tuned against today's paraphrase attack gets retuned against tomorrow's, and the target keeps moving.
What We Found in Our Own Scorer
We applied the same scrutiny to our own scoring tool and found seven false positive classes in it. One class had an aphorism rule reading list numbering as a rhetorical pattern, matching fragments such as a phrase followed by the numeral five, fixed by requiring the sentence-final word to begin with a letter. Another class had a title-style pattern firing on ordinary headings and systematically overstating the family count on long articles, fixed by matching only fully capitalised forms, after which zero of the 32 corpus pages were affected.
Our scorer masks genuine step lists before judging, and exempts numeric ranges inside table cells from em dash counting, because both were false positive sources. For the highest-risk rule, the not-X-but-Y construction, our scorer records the hit without scoring it, because pure regex detection of rhetorical parallelism has an unverified false positive rate and scoring it before sampling would mark human narrative as machine tone.
Our lint regression suite reports 61 passed and 0 failed, measured 2026-09-16. The same passage scored twice can differ by 0.3 to 0.5 points, so a before-and-after comparison smaller than about half a point cannot be attributed to the edit at all. One published measurement in our own records moved 7.8 to 8.0 across runs of identical text.
That reproducibility finding is the one worth carrying to any vendor's claim. If identical text moves 0.3 to 0.5 points between runs, a single-run improvement claim under half a point is unmeasurable on this class of tool. The structured claims dataset for this benchmark is downloadable, and it records every figure above with its unit, the condition that makes it true, and a link to the page it came from. The scoring runs behind the reproducibility figure are in the ForgeRank published dataset, and the ForgeRank methodology page sets out how each of its four dimensions is scored. If the conclusion you're drawing from this page is that a detector percentage can't tell you whether a draft is publishable, the practical replacement is a scored pass over the draft itself: how to check AI content quality before publishing walks through the same six steps with the thresholds we use.
Key Takeaways
- Turnitin's under 1 percent document-level rate applies only to documents with 20 percent or more AI writing, and its around 4 percent sentence-level rate counts a different unit.
- GPTZero's 95.7 percent detection at 1 percent false positive comes from RAID, which GPTZero states contains only fully human or fully AI texts.
- OpenAI's retired classifier reported 26 percent detection with 9 percent of human text flagged, and we record those figures as third-party corroborated because openai.com returns HTTP 403 to our fetcher.
- Liang et al. (arXiv 2304.02819, published in Patterns) found roughly 61 percent of TOEFL essays by non-native English speakers flagged as AI-generated, though no model was used.
- Identical text scored twice can differ by 0.3 to 0.5 points, which makes any single-run before-and-after claim under half a point unmeasurable.
Frequently Asked Questions
What is a false positive rate in AI detection?
A false positive rate is the share of human-written texts a detector labels as AI-generated across a test population. It describes how the tool performs across many documents. It does not predict whether your specific document will be flagged.
Why does Turnitin refuse to score short submissions?
Turnitin validated against 800,000 academic writing samples written before ChatGPT's release and found a higher-than-comfortable false positive rate on submissions under 300 words. The tool now declines to score anything shorter, which removes the length band where its error rate was worst.
Is GPTZero's 95.7 percent accuracy figure comparable to Turnitin's 1 percent?
No. GPTZero's 95.7 percent is a detection rate measured on RAID, a benchmark of clean human and AI texts. Turnitin's under 1 percent is a document-level false positive rate on documents already 20 percent machine-written. Different units, different populations, different conditions.
Can paraphrasing a document defeat AI detection?
Krishna et al. (arXiv 2303.13408) found paraphrase attacks dropped detector accuracy hard. The defense that restored accuracy retrieved against a database of known machine-generated outputs rather than training a stronger classifier. GPTZero states every method loses detection rate under adversarial attack at a 1 percent false positive rate.
Does a low false positive rate protect an individual writer?
No. A false positive rate is a population statistic. A tool with a 1 percent rate still flags some individual human documents. The 61 percent TOEFL finding from Liang et al. shows how far a population-level error can drift when the test population changes.
Why does the same text score differently on repeat runs?
Our own records show identical text moving 7.8 to 8.0 across runs, a spread of 0.3 to 0.5 points. Any before-and-after comparison smaller than about half a point falls inside that noise band and cannot be credited to the edit.
This page ranks no tool and declares no winner, and it carries no first-party detector measurement because we hold no API or institutional access to these products. What it carries is the condition attached to each published number, which is the part that survives when a headline gets quoted. Open the ForgeRank methodology page tonight and check whether the accuracy figure you last cited came with its unit and its condition attached.