An AI writing checker scores the text in front of it: sentence construction, phrasing families, how claims attach to evidence. It does not score the person who typed the words. A detector makes a different measurement entirely, estimating origin from token probability. Confusing the two outputs is the most expensive mistake in this category.
Why this matters
A clean phrasing report gets read as a character reference. A writer runs a draft through a checker, sees no flags, and concludes the piece is safe. What the tool confirmed is narrower: the surface of the text looks like prose a person would build.
The ForgeRank published dataset shows the gap. Across 29 scored drafts, a phrasing pass recorded 594 hits across 23 distinct families, and em-dash density alone fired 182 times across 19 drafts. That is a measurement of text, not of authorship.
The cost lands later. Someone submits a draft on the strength of a clean report, a client runs a detector, and the two results disagree in public.
What does an AI writing checker score?
It scores observable features of the artifact: sentence construction, phrasing families, and how claims attach to evidence. A checker reads what is on the page. Because phrasing families cluster in recognizable patterns, the tool can count them and report density. The consequence for you is a repair list, not a verdict. The boundary sits at authorship: a phrasing report describes the surface of the text and cannot establish who wrote it.
The ForgeRank methodology page draws the same line. A phrasing pass reports which constructions appear and how often. It does not estimate which model produced them, and it does not certify a human.
| What the tool reads | What it reports | What it cannot report |
|---|---|---|
| Sentence construction | Pattern density per family | Who wrote the sentence |
| Phrasing families | Hit counts, 23 families | Which model, if any |
| Claim attachment | Whether evidence is named | Whether the claim is true |
| Em-dash density | 182 hits across 19 drafts | Intent behind the punctuation |
How is a detector different from a writing checker?
A detector estimates origin from token probability, not from writing quality. Machine-written text sits near a local optimum of the model's own probability, so masking a handful of tokens lowers its likelihood under the model, while human text stays roughly flat under the same perturbation, per Mitchell et al. (arXiv 2301.11305, ICML 2023). Because the measurement is probabilistic, the output is a guess with a confidence number attached. The consequence for you is that a detector verdict moves when the text is perturbed, even when nothing about the writing improved.
OpenAI retired its own AI classifier in July 2023 after reporting it identified only 26 percent of AI-written text while flagging 9 percent of human-written text as AI. That pair is the clearest public statement of the ceiling. A tool that misses roughly three in four machine-written samples and mislabels human writing is not a gatekeeper.
The boundary is severe. Liang et al. (arXiv 2304.02819, published in Patterns) found that roughly 61 percent of TOEFL essays written by non-native English speakers were flagged as AI-generated, though the writers used no model. The essays were scored as machine output because the writing was simple, not because a model produced it.
Does cleaning AI phrasing improve the score?
No. Cleaning AI-sounding phrasing moved no draft up a score band, and eight of the nine drafts scoring 7.0 or above stayed flagged as heavy on AI phrasing after cleanup. That is the ForgeRank published dataset result, and it resolves the assumption directly. A phrasing pass removes surface markers. It does not change the underlying construction that produced them.
Because the flagged families come from how sentences are built, deleting a dash or swapping a connector leaves the structure intact. The consequence for you is a false sense of progress: the hit count drops, the draft reads the same, and the score band does not move.
The boundary is worth stating plainly. A clean phrasing report is not an upgrade and not proof of human writing. It describes the surface of the text in both directions.
Why do paraphrase attacks break detectors?
Paraphrase attacks dropped detector accuracy hard, and the defense that restored it was retrieving against a database of known machine-generated outputs rather than training a stronger classifier, per Krishna et al. (arXiv 2303.13408). Because the detector was matching surface patterns, rewording the text removed the patterns it depended on. The consequence for you is that any detector result is unstable under editing that changes nothing about meaning.
The boundary matters here. Retrieval against known outputs restores accuracy for text that resembles what is already in the database. Text that does not resemble anything stored gets no such help.
Do not read this as a method for defeating detection. It is evidence that origin estimation is fragile, which is a reason to stop treating it as a verdict.
What most people get wrong about AI writing checkers
The widespread error is treating a phrasing report and a detector verdict as the same measurement reported twice. They are not. One reads the artifact's construction. The other estimates origin from token probability. A draft can score clean on phrasing and flag as machine-written, and the reverse happens too.
The misconception persists because both tools return a number, and numbers invite comparison. A density count and a confidence percentage look like two readings of one thing. They are two different questions: how is this written, and where did it come from.
The correction is procedural. Use a phrasing pass to find constructions worth rewriting. Use nothing to certify authorship, because no available tool does that reliably. When you need to know who wrote something, ask for the draft history and the notes behind it.
Key takeaways
- A writing checker scores sentence construction, phrasing families, and claim attachment. It cannot establish who wrote the text.
- A detector estimates origin from token probability. Machine text sits near a local optimum, so small perturbations move the result, per Mitchell et al.
- OpenAI's July 2023 classifier withdrawal reported 26 percent detection of AI text and 9 percent false flags on human text.
- Liang et al. found roughly 61 percent of TOEFL essays by non-native English speakers flagged as AI-generated despite no model use.
- Cleaning AI phrasing moved no draft up a score band, and eight of nine drafts at 7.0 or above stayed flagged, per the ForgeRank published dataset.
Frequently asked questions
Can an AI writing checker tell if a human wrote something?
No. A checker reads phrasing density and sentence construction, which describe the text's surface. It cannot establish authorship in either direction. A clean report is not proof of human writing, and a flagged report is not proof of machine writing.
Why did 61 percent of TOEFL essays get flagged as AI?
Liang et al. (arXiv 2304.02819, published in Patterns) found roughly 61 percent of essays by non-native English speakers were flagged, though the writers used no model. Simple sentence construction resembles machine output to a detector, which is why the flags appeared.
Does rewriting text defeat AI detection?
Paraphrase attacks dropped detector accuracy hard, per Krishna et al. (arXiv 2303.13408). That shows origin estimation is fragile, not that rewriting is a reliable method. Accuracy was restored by retrieving against known machine outputs, not by a stronger classifier.
What does a phrasing report measure?
It measures the artifact: sentence construction, phrasing families, and how claims attach to evidence. The ForgeRank published dataset recorded 594 hits across 23 families in 29 scored drafts. The report describes the text and stops at authorship.
Why did OpenAI retire its AI classifier?
OpenAI retired it in July 2023 after reporting it identified only 26 percent of AI-written text while flagging 9 percent of human-written text as AI. The error rate on human writing made the tool unusable as a gatekeeper.
Conclusion
A writing checker and a detector answer two different questions, and only one of them concerns the text you can control. Score the writing for what it is: construction, phrasing, claim attachment. Stop reading a clean report as proof of anything about the author. Open your last draft tonight and count how many claims carry a named source.