The question comes up constantly: "Can ChatGPT be detected?" Teachers want to know before setting essay assignments. Editors want to know before publishing submissions. HR teams want to know before trusting cover letters.
The honest answer is nuanced — and anyone who tells you otherwise is either selling something or oversimplifying.
What Detection Actually Measures
AI text detectors don't identify specific models. They don't recognize "ChatGPT handwriting" the way you might recognize a friend's handwriting. Instead, they measure statistical properties of text that correlate with AI generation.
The two primary signals:
Perplexity measures how surprising each word choice is given the preceding context. AI language models are trained to produce high-probability (low-surprise) continuations. Human writers are messier — they use unexpected words, deviate from the most probable path, and make idiosyncratic choices that feel surprising to a model.
Burstiness measures variation in sentence length and complexity. Humans write in bursts: short punchy sentences followed by longer, more complex ones. AI text tends toward uniform sentence length — not because of any design decision, but because the training objective rewards consistency.
Detectors combine these signals (and others) to produce a probability score.
Accuracy: The Realistic Picture
Independent benchmarks put current AI text detectors at roughly 80–90% accuracy on unmodified ChatGPT output. But that headline number hides important nuances.
False positive rates matter enormously. A 90% accurate detector with a 10% false positive rate will incorrectly accuse 1 in 10 genuine human writers. For a teacher with 30 students, that's 3 wrongful accusations per assignment. This is why reputable detectors — including Aiscern — explicitly warn that results should not be used as sole evidence of AI use.
Accuracy degrades significantly when text is modified. Light paraphrasing, style adjustments, or even just asking ChatGPT to "write more conversationally" can drop detection accuracy to 60–70%.
Non-native English writers are flagged more often. This is a known bias in current detectors. Non-native speakers often write with lower perplexity (more predictable word choices) because they stick to common vocabulary and standard sentence structures — the same statistical signature as AI text.
What Makes Text Harder to Detect
In descending order of effectiveness:
- Extensive human editing — rewriting paragraphs in your own voice, adding personal anecdotes
- Style prompting — asking the AI to write with more sentence variety, unusual vocabulary, and deliberate imperfections
- Domain-specific jargon — highly technical text with specialized vocabulary reduces the effectiveness of general-purpose detectors
- Short text — under 250 words, statistical signals are too weak for reliable conclusions
- Mixing AI and human content — interleaving AI-drafted paragraphs with human-written ones produces inconsistent signals
What Detection Is Actually Good For
Despite the limitations, AI text detection provides genuine value in specific contexts:
Bulk screening — detecting AI use in large volumes of content (thousands of student submissions, or a content farm's output) is statistically reliable even if individual results have error rates. Patterns across a corpus are more trustworthy than any single verdict.
Establishing reasonable suspicion — a high-confidence AI verdict, combined with other contextual evidence (student's demonstrated writing level, submission timing, lack of personal voice), contributes to a holistic assessment.
Self-auditing — publishers and platforms can use detection to maintain editorial standards, not as a binary gatekeeping tool but as a quality signal that triggers human review.
The Right Way to Use Detection Results
Treat AI detection scores the same way you'd treat a plagiarism score: as an input to human judgment, not a verdict.
A 95% AI confidence score means the text has strong statistical similarities to AI-generated content. It doesn't prove AI was used. A 30% score doesn't prove it wasn't. In both cases, the appropriate response is to look for additional signals and apply contextual reasoning.
Aiscern's detection provides a verdict, confidence score, and a list of the specific signals that contributed to that verdict. This transparency lets you understand why the system flagged text — not just whether it did.