What Breaks in AI-Written Blog Posts

I spent a weekend auditing eight articles a generator had written for this site, expecting to find clumsy prose. The prose was the part that held up. What had failed was everything a reader takes on trust: a factual claim repeated four times, a code sample that raises an exception on line two, and structural damage that never once appeared on the rendered page.

That last category is the interesting one. Every defect below survived because the review step was reading the published article, and a published article looks fine right up until someone tries to use it.

The factual error that survived four repetitions

One article opened with this:

Gemini, a project from OpenAI, is an advanced AI model…

Gemini is Google DeepMind's. The article said otherwise in the opening paragraph, in the meta description, and in two of its FAQ answers — so the claim was in the text, in the snippet Google displayed, and in the structured data feeding the rich result.

This is the failure mode worth internalising. A model does not get a fact wrong once and then recover. It establishes the wrong premise and then stays impressively consistent with it, which is exactly what makes the error hard to catch by skimming. Internal consistency reads as confidence.

The practical consequence is that spot-checking does not work. If you check one paragraph and it agrees with the next, you have learned nothing about whether either is true.

The code sample that cannot run

The same article offered this, in a post about calling AI models:

from openai import OpenAI
openai.api_key = 'my-api-key'
response = openai.Completion.create(...)

Line one imports the v1 client class. Line two uses the v0 module-level API on a module that was never imported, so it raises NameError before reaching the request. Then it calls the completions endpoint with a chat model.

It is not quite wrong in any single place. It is three API generations interleaved — the shape of code that has been pattern-matched rather than run. Models are trained on years of documentation and blog posts, and older API styles are heavily represented in that corpus, so the generated sample averages across versions that never coexisted.

A code sample nobody executed is worse than no code sample. A reader who copies it gets an error, cannot tell whether the fault is theirs, and leaves.

The markup damage nobody could see

Two faults hid inside the HTML while the page looked correct.

Five headings contained their own label as visible text:

<h2 id="h2-understanding-gemini">H2: Understanding Gemini</h2>

The generator had been asked to mark up headings and had helpfully written the instruction into the output. On screen it reads as a slightly odd heading. In search results it is the page announcing its own scaffolding.

The second was mine, not the model's, and that distinction matters. My renderer ran its paragraph pass across code blocks it had already emitted, so this:

x = a * b

became x = a <em> b in the stored HTML, along with <p> and <br /> tags nested inside <pre><code>. Indentation in every Python sample in two articles was destroyed.

One fault was the model. One was my own pipeline. Both survived for months for the same reason: nobody read the output as markup. Generated content tends to be reviewed like a finished page, and a finished page hides its own structure.

The metadata artifacts

The stored SEO title was:

Gemini vs ChatGPT: An In-Depth Comparison of AI Models | OpenAI

Nothing appended that | OpenAI. The model produced it because titles in its training data often end with a pipe and a brand, so it supplied one — inventing a publisher, on a page about a competitor's product.

Metadata gets the least review of anything on a page, because it is the part you never see while reading. It is also the part search engines read first.

Why ordinary review misses all of this

Every one of these defects is obvious once you look in the right place, and invisible everywhere else:

They share a shape: the article is coherent. Coherence is what these models are best at, and it is precisely the quality that makes a proofread feel complete when it is not.

A review pass that catches them

This is what I now do before anything generated goes live, and it takes about ten minutes:

1. Verify every proper noun. Companies, products, people, versions, dates. Not the claims around them — the nouns themselves. That single check would have caught the largest error here.
2. Run every code sample. Not read it. Run it, in a clean environment, with the current library version.
3. Read the markdown source, not the page. Headings, list nesting, fences. Paste it into a Markdown Previewer if you want both views side by side.
4. Read the metadata out loud. Title, description, and any FAQ answers, separately from the body — they are separate surfaces and they fail separately. The Headline Analyzer is useful for length, but reading it aloud is what catches an invented brand.
5. Check the article against itself. Does the conclusion claim anything the body did not establish? Generated closings restate, and restatement is where an unsupported claim gets smuggled in.

A Readability Analyzer will tell you whether the prose flows, which — as this audit showed — is the one thing you probably did not need to check.

What I actually changed

I stopped generating articles. Not because the writing was bad, but because the review cost was higher than the drafting saving. Verifying every noun and running every sample in something I did not write takes longer than writing it myself, and it is a duller job, which means it gets done worse.

That is a judgement about this site, not about the tools. If you are producing volume where a wrong publisher name is survivable, the arithmetic is different. For a site whose whole proposition is that the technical content is correct, one confidently wrong sentence costs more than a whole post saves.

The two articles above are fixed. I corrected the attribution, rewrote both code samples against current SDKs, and left the rest as it was — including the opening line about diving in, which is its own kind of tell.