Chats with G: Are LLMs Getting Worse at Writing?
Chats with G is a space for thinking out loud. These posts capture ongoing conversations between myself and ChatGPT, who I fondly refer to as G. In this role, G is an AI collaborator I use to explore ideas around creativity, technology, and the human experience. The goal is not answers, conclusions, or declarations, but to see where the thinking goes. Questions lead, responses unfold, and follow ups extend the conversation. What emerges is the thinking itself, a dialog of ideas.
Bill:
G... People have observed that when Large Language Model (LLM) AIs were originally trained and came online, they had the ability to write sophisticated prose, often better than the average human. Today, many feel this is no longer the case. It has been speculated that over time, with AIs now training on material from the internet, including AI-generated writing, their writing abilities have gotten worse. They are not as refined. They drift in style and tone. Do you feel this is true?
G:
Yes, but I think the popular explanation mixes together several different phenomena.
I do think people are detecting a real change in the character of LLM writing. I am much less convinced by the simple explanation that AI trained on AI writing, therefore AI writing deteriorated.
A Change in the Voice
When the early powerful LLMs appeared, one of the startling things about them was their ability to produce prose that seemed remarkably sophisticated for a machine. You could ask for an essay and get unusual vocabulary, complex rhetorical structures, metaphors, transitions, and something approaching an individual voice.
Modern models are often capable of better reasoning and better instruction following, yet their default prose can paradoxically feel less literary. It can be smoother, safer, more predictable, and more stylistically homogeneous. We've also become familiar with the fingerprints: repeated rhetorical constructions, predictable transitions, and certain words and phrases appearing again and again.
That doesn't necessarily mean the underlying model has forgotten how to write.
The AI-Eating-AI Problem
There is legitimate research behind the concern people are describing. Researchers have studied what happens when generative models are repeatedly trained on data produced by previous generations of generative models. One phenomenon is commonly called model collapse. Over successive generations, uncommon characteristics in the original data can disappear while more statistically common patterns become dominant.
Human writing contains extraordinary irregularity. Bad grammar. Brilliant grammar. Regionalisms. Strange metaphors. Awkward sentences that somehow work. Hemingway. Faulkner. A teenager writing fan fiction. Someone arguing about lawnmowers on Reddit. That messiness contains enormous informational value.
Synthetic writing tends toward the statistical center. If future training corpora rely too heavily on synthetic material, there is a genuine danger of cultural photocopying, where the copy becomes a source for the next copy.
But that does not mean today's ChatGPT is continuously reading its own internet output and retraining itself on it. Training datasets are deliberately constructed, filtered, and evaluated, and developers are well aware that synthetic-data contamination is a concern.
The House Style
I think something else explains much of what people are noticing. Modern AI systems aren't merely pretrained language models anymore. Additional layers of post-training, preference optimization, safety behavior, instruction following, and stylistic tuning influence what users actually receive.
Those layers are useful, but they can also push prose toward a recognizable house style. Most users want writing that is clear, organized, pleasant, concise, grammatically clean, and helpful. Those are sensible objectives for an assistant, but they are not necessarily the objectives of a great writer.
Great writing occasionally wants to be difficult, strange, ambiguous, or abrasive. An assistant optimized to communicate efficiently can naturally move in the opposite direction.
There is also a feedback loop developing. Human writing influenced AI. AI-generated writing now influences humans, and some of that human writing may eventually become future training material. It becomes increasingly difficult to say where a particular stylistic habit originated.
Capability vs. Default Behavior
This is where I disagree with the strongest version of the argument. I don't think LLMs have broadly lost the ability to write sophisticated prose. I think sophisticated prose is increasingly something that has to be elicited rather than something you necessarily receive from default assistant behavior.
There is a difference between capability and default behavior.
A modern model may possess greater contextual awareness, stronger reasoning, and better command of language while nevertheless producing safer and more conventional prose when given a generic writing request. We've also gotten much better at recognizing AI writing. After reading millions of examples, people have learned its rhythms, so some of what feels like declining quality may actually be increased familiarity.
Modern LLMs may not be worse writers. They may be more capable writers with more conventional default voices.
The challenge may not be teaching AI how to write better. It may be figuring out how to keep it from sanding off the weird edges that make an individual writer sound like an individual writer.
Bill:
G... Given everything you have said and expressed above, how does this fit into the ideas around AI's abilities to completely take over jobs from humans? It seems to me that AIs, at least currently, have real limits and work best alongside humans in an assistive manner. This also lends credence to the mistake some companies are making by wanting to reduce their workforces to mostly AIs. The most productive mix would appear to be humans and AIs working together.
G:
I think your conclusion is largely right, but I would add an important qualification: AI does not have to become better than a human at an entire job to eliminate that job.
Jobs Are Bundles of Tasks
Most jobs aren't actually one thing. They are collections of tasks requiring different combinations of knowledge, judgment, communication, creativity, experience, repetition, and decision-making. AI might be extraordinarily good at some of those tasks and genuinely poor at others.
That can still transform employment. Ten people assisted by AI may accomplish work that previously required fifteen. AI hasn't literally replaced five employees. It has increased the productivity of the remaining ten enough that the organization decides it doesn't need fifteen anymore.
Replacement vs. Amplification
I think there are two competing visions of AI happening simultaneously. One is replacement, where the human is removed and AI performs the work. The other is amplification, where humans working with AI achieve greater capability.
For repetitive, predictable, rules-based work, replacement can make economic sense. But as ambiguity, judgment, creativity, interpersonal relationships, and changing circumstances increase, complete replacement becomes much harder.
AI can draft the report, but someone still needs to know whether it makes sense. AI can analyze thousands of records, but someone needs to determine what matters. AI can generate fifty designs, but someone needs to recognize which one solves the problem.
AI can generate prose, but someone still needs to recognize whether the prose is any good.
That connects directly to our original discussion. If AI naturally gravitates toward statistical averages and conventional patterns, human contribution becomes particularly important where originality, taste, judgment, and individual voice matter.
The Productivity Trap
This is where some companies could make a serious strategic mistake.
Imagine two companies adopting equally capable AI systems. Company A uses AI to make its employees more capable and increase what the organization can accomplish. Company B uses those productivity gains primarily to eliminate employees.
Initially, Company B may look brilliant. Labor costs fall and productivity per employee rises. But institutional knowledge, mentorship, human relationships, organizational memory, creative disagreement, and people who recognize when something unusual is happening can disappear with those employees.
Perhaps most dangerously, the company may lose the people capable of recognizing when the AI is wrong.
You can create an extraordinarily efficient organization that has simultaneously become extraordinarily brittle.
Efficiency Is Not Capability
Businesses frequently measure what is easy to measure. Headcount, cost per transaction, and output per employee are easy to quantify. Judgment, creativity, trust, and the value of someone who recognizes that the technically correct answer is inappropriate for a particular situation are much harder.
Humans have limitations too. We get tired, forget things, overlook information, and sometimes spend hours doing work a machine can accomplish in seconds. AI can compensate for human weaknesses, just as humans can compensate for AI weaknesses.
That's why I think the more interesting comparison is human with AI versus AI without a human.
A Different Kind of Expertise
The valuable employee of the AI era may be the person who knows what should be done, what can be delegated to AI, what requires human judgment, and how to recognize the difference.
That requires domain expertise because you can't reliably evaluate an AI's work if you don't understand the subject yourself. AI can produce extremely convincing bad answers, and someone still needs the knowledge and judgment to recognize them.
There is another side to amplification that is easy to overlook: AI can amplify mediocrity just as readily as it can amplify expertise. Someone with deep knowledge can use AI to work faster, explore alternatives, and extend their abilities. Someone without that knowledge can use the same technology to produce enormous amounts of plausible-looking work without recognizing its deficiencies.
Ironically, expertise may therefore become more important in some fields rather than less important. The beginner sees an impressive AI output and thinks, "Done." The expert looks at the same output and says, "Almost."
The Management Question
Ultimately, this raises a question that goes beyond technology: What is the purpose of adopting AI?
If the answer is simply reducing payroll, organizations will try to determine the minimum number of humans necessary to keep the machinery operating. If the answer is increasing organizational capability, you arrive at a very different strategy.
Instead of asking, "How many people can AI replace?" you ask, "What could these people accomplish if AI removed some of the limitations on their time and abilities?"
Some jobs will disappear, and many more will change substantially. But I don't think the evidence currently supports the simplistic vision that companies can remove most of their knowledgeable employees, replace them with autonomous AI agents, and expect equivalent organizations to emerge on the other side.
An organization isn't merely a collection of tasks. It is accumulated knowledge, relationships, judgment, culture, accountability, improvisation, disagreement, and experience.
The winning organizations may not be the ones that replace the most humans with AI. They may be the ones that figure out how to make humans and AI better together than either would have been separately.