People ask whether a particular detector is accurate because they want to know whether the flag on their own draft can be trusted. A useful answer needs the question split in two. A detector can be accurate at the job it was built for and still be useless as evidence about authorship, because the job it was built for is measuring how typical a passage looks to a language model.
What an accuracy number is actually reporting
Detector accuracy is reported as a rate, and a rate hides the threshold it was measured at. Any classifier trades false positives against false negatives, so a detector tuned to catch machine text will flag human prose that shares the same surface statistics. Raising the catch rate raises the misclassification rate with it. A published figure of ninety-something percent accuracy describes one chosen operating point on that curve, which is why two studies of the same tool can report different numbers without either being wrong.
The rate also varies by population of writer. Liang et al. (arXiv 2304.02819, published in Patterns) tested detectors against essays written by humans and found that roughly 61 percent of TOEFL essays written by non-native English speakers were flagged as AI-generated. Those writers used no model. They wrote in the controlled, textbook register that an exam rewards, and the register is what the tool read.
Is GPTZero accurate?
GPTZero's method computes perplexity and burstiness. Perplexity asks how surprising each token is to a language model. Burstiness is the variance in that surprise across the document, so a page of evenly sized, evenly complex sentences scores as machine-like even when the vocabulary is rich.
DetectGPT (Mitchell et al., arXiv 2301.11305, ICML 2023) tests a related property of the same model. It perturbs a passage, masking a handful of tokens at a time, and measures whether the text's likelihood moves. Machine-written text sits near a local optimum of the model's own probability distribution, so perturbation lowers the score. Human text sits away from that optimum, and perturbation leaves the number roughly flat.
Read those mechanisms closely and the limitation becomes visible. Both describe the statistical shape of the passage in front of them. Neither one has access to your revision history, your notes, your browser tabs, or the fact that you typed the sentences yourself. A high accuracy rate for the model's purpose and a reliable verdict about authorship are different claims, and only the first one is being measured.
Is Turnitin's AI detection accurate?
The academic case carries the highest stakes and the clearest mechanism. A flagged submission triggers a conversation with a person who has authority over your grade, and the flag arrives with no accompanying explanation of what produced it.
Turnitin's AI writing detection reports a percentage for the submission and marks the segments behind that number. The percentage summarises many segment-level decisions, which is where the accuracy question gets slippery. A setting that catches more machine text also catches more human text, so a reported figure describes one position on that trade-off rather than a property of your essay.
Length changes that reading as well, because the figure is a share of measured segments. A forty-page thesis and a two-page response under the same threshold are not the same measurement: the longer document gives the detector more segments to read, more chances for one of them to look typical, and therefore more weight behind a single flagged passage than the same passage would carry inside a shorter submission.
Turnitin itself publishes guidance on false positives in its AI writing detection, which is a useful admission from the vendor: the company that sells the check tells readers that a flag is a signal to investigate rather than a finding of fact. Long, evenly paced, formulaic prose is the genre academic writing rewards, and it is also the genre that reads as typical to a language model. A student following the taught structure is producing the property the detector measures.
The similarity score and the AI score are two different numbers
Students and instructors compare the wrong number routinely. Turnitin reports a similarity percentage from its plagiarism matching, and that figure comes from a different signal than its AI writing detection. The two land in different places in the interface, which is part of why they get read as one number: the similarity figure appears as the headline result while the AI indicator sits further in, so a reader who stops at the front page has seen only one of them. A submission can carry a low similarity figure alongside a high AI figure, or the reverse, and neither number is derived from the other.
The distinction matters because the two failures have different remedies. A high similarity score calls for attribution: quoted material and citations that credit the matching source. A high AI score calls for provenance, and provenance is made of ordinary artifacts rather than arguments: the document's own version history with its edit timestamps, earlier drafts saved under working titles, the notes file where the structure was sketched, and the reading list the sources came from. Working on one does nothing for the other, and a student who answers an AI flag by hunting for matching sources is working on the wrong problem.
Why rewriting the draft does not clear a flag
The intuitive fix is to run the flagged text through a paraphrasing tool and resubmit. Krishna et al. (arXiv 2303.13408) measured how well that works against detectors and found that paraphrase attacks dropped detector accuracy hard, so the instinct is not imaginary. Their defense was retrieval: matching the incoming text against a database of known machine-generated outputs and looking for near-matches, which restored detection.
The route also fails against the specific signals a scorer reads. A paraphraser swaps content words and preserves syntax. Function words land in the same positions in the same proportions, because they carry the sentence's skeleton. Sentence-length variance stays wherever the sentence boundaries were, because paraphrasing keeps them. Punctuation habits survive untouched.
We measured how much surface editing is worth on real drafts. Across 29 drafts in the ForgeRank published dataset, cleaning AI-sounding phrasing moved no draft up a score band. Eight of the nine drafts scoring 7.0 or above stayed flagged as heavy on AI phrasing after cleanup. The phrasing pass recorded 594 hits across 23 distinct families in that set, with em-dash density alone firing 182 times across 19 drafts. Removing those marks changed nothing the measurement could see.
There is a further trap, because the obvious workaround is to have a language model do the rewriting. That model samples from the same distribution that produced the original text, so it re-imposes the pattern rather than removing it. A paraphrase from the same model family comes out flatter than the draft, since paraphrasing strips the local quirks and leaves the model's default cadence.
What to check instead of the detector score
If the goal is to establish that a piece of writing is sound rather than to move a number, the useful checks are the ones a detector cannot perform at all.
Attach a source to every number. A figure that traces to a source you personally opened can be verified. A figure that traces nowhere is a liability regardless of what any detector says.
The clearest case in our data sits at the bottom of the distribution. Ten drafts scored 4.0 on all four dimensions: information gain, specificity, source compliance, and argument depth. Ten others scored 4.0 on information gain, 4.0 on specificity, and 4.0 on argument depth, with source compliance at 7.0, and their headline came out at 4.8. The two groups are identical everywhere except whether their numbers traced to a source. Source compliance was the only dimension in the whole 29-draft set that ever moved a score on its own, and it moved it twice.
Add the detail only you could know. A specific that comes from having been there, having run the test, or having had the conversation cannot be produced by a model that was not present, and it reads that way.
Check whether the claim survives without its number. A mechanism explanation, meaning how a process works, in what order, and at what cost, carries itself. A statistic does not. When a claim is a number, either attach the source or cut the number and keep the insight in words.
If you are facing an accusation
Detector output is one measurement among many, and it is a weak one. The evidence that resolves an authorship question is provenance: drafts, revision history, the sources you worked from, and notes with timestamps. Produce those. Re-running the text through a different detector produces a second weak measurement, and it can come back the other way, which helps nobody.
If an institution is treating a detector score as proof, the honest position is that the tool is measuring something other than authorship. The published evidence says it flags humans at rates high enough to disqualify it as a sole determinant, and the mechanism explains why: typicality is not authorship, and a great deal of good writing is typical.
The boundary
Nothing here establishes that any particular flag was wrong. A false-positive rate describes a population, not your document, and there is no test that settles an individual case by re-running the same class of instrument. What the mechanism establishes is narrower and more useful: a detector score answers a question about the statistical shape of your prose, and the question people ask when they see that score is about who wrote it. Those are different questions, and only one of them is being measured.
The same logic holds when the setting is publishing rather than a grade. Score the draft for information gain, specificity, source compliance, and argument depth, and put a source behind every number. Our four-dimension diagnostic is free to run on any draft, and the ForgeRank methodology page sets out how each of the four is scored and where the pass line sits.