AI Writing Is Getting Harder to Catch #ai #aiwriting #ainews
Read the Portuguese translation →
Author: Better Stack · Watch on YouTube
Video authored by the channel shown, not by IANIA; all rights belong to the creator. Transcript and summary generated automatically; the original-language content is preserved.
Original language: English
Original video: AI Writing Is Getting Harder to Catch #ai #aiwriting #ainews · Channel: Better Stack · 02:35 · Watch on YouTube
Topics: AI evaluation and benchmarks, AI content generation, AI models
Summary
Section titled “Summary”A study compared 10,000 human-written articles published before ChatGPT with 90,000 AI articles generated from summaries of the same topics. It found that AI writing has identifiable patterns, but those patterns change across model versions, and newer models do not necessarily sound more like humans. The models are increasingly similar to one another, while human writing still stands out through features such as exclamation marks, parenthetical asides, and personal anecdotes. The researchers caution against judging authorship based on individual AI tells and have released the study’s data set.
Key points
Section titled “Key points”- The study used 10,000 pre-ChatGPT human articles and generated 90,000 AI articles across nine models.
- Researchers identified 12,877 words, phrases, and patterns that appeared at least twice as often in AI text.
- Less than half of the identified AI tells were shared between GPT-6 Astra and GPT-5.6 Soul.
- AI writing habits shift between model releases as models drop old patterns and acquire new ones.
- GPT and Gemini now use M dashes less often than people, while Claude Opus 5 uses them at about the human level.
- Only Claude’s overall word choices moved closer to human writing; GPT-6 Astra had 48% more AI tells than GPT-4.1.
- All nine models write more like one another than like people.
- The researchers advise against judging authorship based on individual AI tells.
- In the study’s data, humans used exclamation marks over 100 times as often as current models.
- The researchers released a data set that is free to download.
Chapters
Section titled “Chapters”| Start | Chapter | Summary |
|---|---|---|
| 00:00 | How the study compared human and AI writing | The video introduces a study that compared pre-ChatGPT human articles with AI articles generated from summaries of the same topics. |
| 00:30 | AI tells change between model versions | Researchers found thousands of patterns associated with AI text, but many differ between model releases. |
| 01:32 | M dashes and the blurred line between AI and human writing | The video describes how models have changed their use of M dashes and compares their writing patterns with human writing. |
| 02:04 | What still distinguishes human writing | The researchers caution against relying on individual AI tells and point to exclamation marks, asides, and personal anecdotes as human-writing features. |
Quotes
Section titled “Quotes”GPT-6 Astra now uses M dashes 88% less often than humans did before ChatGPT even existed. — 00:00
And across all nine models, they found 12,877 of such samples. — 00:30
GPT and Gemini cut back on it so much so that they now use it way less than normal people do. — 01:32
GPT-6 Astro now has 48% more AI tails than GPT-4.1 did. — 02:04
Humans actually use exclamation marks over 100 times as often as the current models. — 02:35
Concepts and entities
Section titled “Concepts and entities”Graphite, Common Crawl, GPT-4.1, GPT-6 Astra, GPT-5.6 Soul, Claude, Opus 5, Gemini, travessão longo, conjunto de dados
Full transcript (original language)
Section titled “Full transcript (original language)”00:00 If you’re still trying to spot AI writing by looking for M dashes, well, I’ve got bad news for you. GPT-6 Astra now uses M dashes 88% less often than humans did before ChatGPT even existed. And there are other indications that AI writing has gotten significantly less sloppy. Grafa I just published a massive study on the subject, so let’s go over what they found. So, they pulled 10,000 human-written articles from Common Crawl, all published before ChatGPT launched in November 2022. Then, they had GPT-4.1 summarize each one and asked nine different models to write a full
00:30 article from that summary. So, you end up with 90,000 AI articles on the exact same topics as the human ones. Then, they counted every word, phrase, and pattern that shows up at least as twice as often in AI text. And across all nine models, they found 12,877 of such samples. And it’s hard to map out all the AI tells exactly because when a model gets a new version, a lot of its AI tells change. For example, GPT-6 Astra and the version right before it, GPT-5.6 Soul, out of all the AI tells the researchers found in those models, less than half of them show up
01:01 in both versions. For example, Soul says in addition 14 times more often than Astra, while Astra says need not 17 times more often than Soul. And each new release drops some old habits and picks up new ones. For example, GPT has mostly stopped using salesy words like unlock and streamline, but it loves saying what something is not, like this is not simply a tool. GPT-6 Astra does that 12 times more than a human would. Claude has toned down the hype words like groundbreaking, but it now keeps writing comparisons like it’s less like a library and more like a toolkit. Opus 5
01:32 actually uses that pattern 105 times more than humans. And Gemini has almost stopped using contractions like it’s and don’t, while saying incredibly way more often than any other model. And the M dash debacle is super funny. GPT and Gemini cut back on it so much so that they now use it way less than normal people do. The researchers think this is because they overdid it. Claude also cut back on it in Opus 4.6, but on Opus 5, they started using M dashes again, and now it’s pretty much at the same level as humans use it. I was going to suggest putting them back in because now that would be a sign a human has written it,
02:04 but the lines are getting really blurred right now. So, you would think newer models would sound more human, right? But the researchers compared the overall word choices of each model to human writing, and only Claude actually got closer. GPT went the other way. GPT-6 Astro now has 48% more AI tails than GPT-4.1 did. Make that make sense. But the funniest thing is that all nine models now write more like each other rather than actual people. So, is AI writing actually getting harder to spot? The researchers themselves say you shouldn’t judge authorship on individual AI tails. What actually gives humans
02:35 away in their data are things like exclamation marks, parenthetical asides, and little personal anecdotes. Humans actually use exclamation marks over 100 times as often as the current models. So, humans still write in a genuinely unique way. And they also released a full data set, which is free to download if you want to dig deeper in the data. And if you want to stay up to date with the latest in AI research and engineering, be sure to subscribe to the Better Stack channel.