Buy me a coffee · $1
×

Support us with a small donation 💝

Every coffee keeps the site running. Thank you!

Detecting AI Text Does Not Work, and That Is the Point

 ·  1 min read

Tools claiming to identify machine-written text are widely sold and widely wrong. The error rate is high in both directions, and the errors are not randomly distributed.

Why it fails

These tools measure statistical properties – how predictable each word is given the previous ones. Machine-generated text tends to be more predictable than human writing.

Tends to. A careful human writer producing clear, conventional prose scores as machine-written. So does anyone writing in a second language, because non-native writing uses more common constructions.

That last point matters enormously. False accusations land disproportionately on people writing in a language they learned later, which in education means a specific and identifiable group of students.

The arms race that cannot be won

Every detection method can be defeated by asking a model to write less predictably, or by editing the output. Detection improves, evasion improves, and the tools remain unreliable throughout.

What is being tried instead

Provenance rather than detection. Cryptographic signatures attached at creation, recording what tool produced a file and what modifications followed.

That works in the sense that a signature is verifiable. It does not work for text that is copied and pasted, which is how text moves.

What institutions actually do

The schools handling this sensibly have stopped trying to detect and started changing assessment. Work produced under observation, oral defence of written work, and assignments where the process matters more than the artifact.

That is more effort and it does not produce false accusations, which is a trade most people would take.

The practical position

If someone accuses you based on a detector score, the score is not evidence. If you are considering using one to accuse someone, it is not evidence then either.