Research
The best AI detector? Build a fair comparison first
A practical comparison worksheet for AI detection, human false positives, highlights and mixed writing—not an invented leaderboard.
“Which AI detector is best?” sounds like a request for one number. A useful answer starts with a narrower question: best for which writing, in which language, and for what decision? An educator reviewing essays and an editor screening news copy may need different evidence.
This is a testing guide, not a benchmark ranking. Certaffy has not established superiority over Turnitin, Grammarly, Originality.ai or GPTZero in a shared independent test. The worksheet below helps you evaluate those products without treating unrelated marketing figures as a head-to-head result.
Start with each product's own explanation
| Product | Useful starting point | What to clarify before comparing |
|---|---|---|
| Turnitin | AI Writing Report guide | Eligible text, report interpretation and score-display rules |
| Grammarly | AI detector overview | Detection output versus writing assistance and authorship tracking |
| Originality.ai | Product and research articles | Which detector setting and published test conditions apply |
| GPTZero | Result-interpretation guidance | Document probabilities versus sentence-level signals |
| Certaffy | Score and highlight guide | Document verdict, estimated AI share and displayed coverage |
Read current definitions at the time of your test. Vendor explanations are useful documentation, but a vendor's own comparison is not an independent evaluation of Certaffy.
Use the same documents, not just the same topics
Create a small pilot set from texts you are authorised to use. Keep human examples with credible provenance, recorded AI outputs and, if relevant, mixtures whose construction is documented. Separate essays, reports, blogs, news and emails. A human-source article with a new AI introduction is not a pure-human example.
Save the exact submitted text, its length, the tool version or test date, settings, extraction warnings and result. Check PDF extraction before calling differences model errors. Do not shorten a document for one tool and silently compare it with a full-document result from another.
Keep an untouched confirmation set. If you inspect its errors and adjust thresholds or choose a model because of them, it becomes development data. Repeated testing on familiar examples can make a result look more general than it is.
A comparison worksheet you can copy
| Field | What to record |
|---|---|
| Input | Document ID, genre, language, word count and exact saved text |
| Provenance | Human, AI, traceable mixture or unknown; supporting record kept separately |
| Conditions | Date, detector version if available, settings and extraction status |
| Document result | Verdict and its defined score; any refusal or uncertainty |
| Highlights | Visible marked passages, including wrongly marked human sections |
| Mixed estimate | Estimated AI share, only if the tool defines one |
| Limitations | Unsupported input, missing output or insufficient sample size |
Do not convert a tool's “80% probability of AI” into “80% of its words are AI.” Compare like with like, and report an unavailable metric as unavailable.
Report both kinds of mistakes
For pure documents, report the fraction of known AI documents flagged and the fraction of known human documents wrongly flagged. Include the denominators: “2 of 40 human documents” is more informative than “5%” alone. Small samples cannot establish a reliable population error rate.
For traceable mixtures, use separate measures: percentage-estimation error, highlight precision, AI-text recall and false highlights on human text. An aggressive mask can catch more AI and wrongly colour more human writing at the same time. Our worked highlight example shows the arithmetic.
The RAID research paper found generalisation challenges across domains, generators and altered text in its benchmark. That motivates broader testing; it does not prove any product will win your current comparison.
Where Certaffy belongs on your shortlist
Try its academic or business workflow if you want to review a score alongside passages and document context. Evaluate the actual outputs on your own permitted samples. Choose on demonstrated fit, human protection and review usefulness—not on a promise of perfect detection.
Prepared with AI assistance for Certaffy. Sources checked on September 29, 2026. No comparative accuracy experiment was performed for this article.