The Economist (‘twas a gift link, but alas, I guess gift views have been used up — here’s an archive link in case the gift link is vexing you):
You can discover AI’s hallmarks by comparing the writing of man
and machine. To do this you need a baseline that is distinctive
and familiar. The Economist turned to prose that we’re sure is
human and that readers will recognise: our own. We designed a
study to ask top LLMs — OpenAI’s ChatGPT, Anthropic’s Claude,
Google’s Gemini and xAI’s Grok — to write versions of our articles
without consulting the web. (As a prompt, we gave them the
AI-generated summaries that we have experimentally added to some
of our articles.)
This gave us a corpus of human and AI creations and we compared
them across 55,940 sentences and 1.2m words. To make sure we were
detecting AI quirks rather than our own, we also checked the AI
texts against journalism from CNN, the New York Times and the
Washington Post. Excerpts from hit novels published between 1950
and 2022 offered another test.
Our findings are surprising. AI prose is distinguishable by word
and punctuation choice as well as sentence and paragraph
structure. But its hallmarks are not what you might expect, partly
because its writing style has changed with software updates. That
does not mean that LLMs are great writers: their prose lacks
lucidity and elegance and is often formulaic. So those aspiring to
be impressive (human) storytellers should avoid the following
peculiarities in their own prose.
On point for today.
Some specific findings:
Much of this language could be described as what George Orwell
called “pretentious diction”. He railed against writers who “dress
up simple statements” with complicated words and jargon to sound
clever. Such pontificating penmen, Orwell observed, also believe
that “Latin or Greek words are grander than Saxon ones”. (Bots
agree: more Latinate suffixes crop up in their writing than in
human texts.)
Then look at punctuation. Many believe LLMs stuff their prose with
em-dashes, but that is not true after the most recent updates.
Today only Claude uses more em-dashes than human writers, with
ChatGPT using markedly fewer than any other writer in our study.
Humans rejoice — and start using dashes again.
A better way to spot AI-generated writing would be to look for
texts without much punctuation at all. LLMs are very Joycean about
it: they use fewer commas and semicolons than humans (and hardly
any parentheses). They use less punctuation in part because they
write longer sentences — “and” is their most overused word — and
in part because they do not quote experts.