Checking what an AI made: mistakes that look plausible

Checking AI output is a review in order: facts, function, security, data and law, accessibility, tells of a template. Mistakes in the output look plausible.

Output that looks right

An AI has an unpleasant trait: it makes mistakes confidently and smoothly. An invented number is written in the same tone as a real one. A function that doesn’t exist is named plausibly. A page with a key in its code looks exactly like a page without one. So “I looked, seems fine” is not a check of an AI’s output: eyes see only what is visible, and the most expensive mistakes are invisible.

This part of the work answers the question “what do you do when an AI has done it for you”. You check. And for that you need two things: to know which mistakes occur, and to check them in the right order.

Which mistakes occur

Invented facts. Numbers (“1,200 clients”), achievements (“specialist of the year”), reviews (“John, director”), the numbers of laws and standards, links to sources. A model completes what is missing with the most probable thing, and if the prompt doesn’t say “use only these facts”, it will add false ones.

The nonexistent and the outdated. Libraries, functions, CSS properties and attributes that don’t exist. They are named plausibly, and it is easy to take them for real. Worst is when an attacker registers an invented package name and waits for someone to install it; researchers who tested code-writing models found that such made-up package names turn up in their output often enough to matter. Next to this stands the outdated: methods and versions from past years.

Security. A secret key right in the page’s code. A form that sends over http or by GET, so that data lands in the address and the logs. No checking of input on the server and no access rights. Scripts from unknown outside servers. In Veracode’s 2025 study, models chose an insecure way in about 45% of the test tasks. This takes you back to the lesson on authentication and to the OWASP list.

Data and law. The form collects a name and a phone number, but there is neither a notice on how the data is used nor a link to a privacy notice, which privacy laws such as the GDPR and California’s CCPA expect (and where consent is the legal basis, a consent checkbox as well). Counters, fonts and pictures load from other people’s servers, and the visitor sends a request to a third party, which has to be described in the policy. Stock pictures of unknown license. Generated images of people who resemble real ones. And invented reviews are a legal matter too: in the US, the FTC’s rule on consumer reviews and testimonials (16 CFR Part 465) bans fake or false reviews, and the FTC says that includes AI-generated ones. General information, not legal advice.

Accessibility. “Buttons” made of div, images with no alt, fields with no labels, low contrast, a broken order of headings. You went through all this in the parts on HTML and design, and an AI reproduces these mistakes as readily as people do.

Tells of a template. A slogan, three cards, invented numbers, stock labels: eight tells from the part on design.

Maintainability. Duplicated code, unclear names, no documentation. It doesn’t break the page today, but it will make an edit next month painful.

Try finding the mistakes on a page an “AI wrote”. Mark the lines you think are mistakes, then look at the results: found, missed, false alarms. Some lines are right, and marking one with no reason is a miss too.

Mark the lines you think are mistakes. Then press “Show the results”. Some lines are right: marking one without a reason is a mistake too.

1. <h1>Project: innovative solutions for your business</h1>
2. <p class="claim">More than 15 years of experience, 1,200+ satisfied clients, 99.9% of cases won</p>
3. <blockquote>“The best specialist I’ve ever met!” John, director</blockquote>
4. <img src="https://images.example-stock.com/team.jpg" width="320" height="200">
5. <div class="btn" onclick="go()">Get Started</div>
6. const API_KEY = "sk-live-9f2c7a41e5b84d3aa1c0";
7. <form action="http://api.example.com/leads" method="get">
8. <input name="phone" placeholder="Phone">
9. <button type="submit">Send</button>
10. <html><head><title>Home</title></head>
11. <header><nav aria-label="Sections"><a href="#sec-1">…</a></nav></header>
12. <main><h2>Get in touch</h2><label for="f-name">Name</label><input id="f-name" name="name" required></main>
13. <footer><a href="/privacy/">Privacy notice</a></footer>
14. <img src="/images/portrait.jpg" alt="Portrait of the specialist" width="320" height="240">

Show the results Start over

Check yourself: why is “I looked, seems fine” not a check of AI output?

Eyes see only the visible, and the most expensive mistakes are invisible: a key in the source code, an invented number, a missing consent. They are found by a deliberate check, not by a glance.

In what order to check

The order is not random: first what costs the most and is hardest to see.

1. Facts. Every claim, number, name and review: where does it come from? If the source isn’t the brief or a document you can show, it is invented. This is the simplest and the most important check: a list of everything the page asserts, and against each item the words “where is the proof”.
2. Function. Does the page do what it was made for? Your project’s main action (a form, a booking) has to be walked through by hand at the real address.
3. Security. A search in the source code for the words key, token, password, sk-. The form’s address and method. Which third-party servers show on the “Network” tab of the developer tools. What is checked on the server and what only in the browser.
4. Data and law. The consent, the policy, where data goes, other people’s scripts and fonts, the licenses of images.
5. Accessibility and speed. Go through the page with the Tab key, look at contrast, labels, headings, the page’s weight.
6. Tells of a template, and tone. This is last, because it is about taste, not harm.

Automatic tools help at steps two to five (an accessibility scanner, a link check, a search for strings that look like keys), but don’t replace the first: only you know the truth about you. And a second model as a reviewer is useful but no replacement: models share weak spots, and they don’t know your facts.

Check yourself: which check do you start with, and why?

With the facts: an invented claim on the page is the most expensive and the simplest mistake, and only a person who knows the truth about the project can find it.

The log

A good habit is to write each find as “what, where, how I fixed it”. In a week the log shows what an AI catches you on most often (everyone has their own three or four kinds of mistake), and you learn to look for them first. And the log can be pasted into the next prompt as “what not to do”.

Practice

In front of you is your project’s site, as if an AI had built it. It has real mistakes from the list above: a slogan, invented numbers and a review, an image with no alt from another server, a div “button” with a stock label, a key in the script, and a form with no labels, consent or protection. Fix them in the code itself and write five finds in the log: what, where, how you fixed it. Take the proof from your own brief.

The check reads the page’s code: it will find whether the key is still there, whether there is alt, how many labels the fields have, and the consent. Whether a person would see it with their eyes it can’t say, so after it, go through the page with the Tab key.

What was wrong with 2025

In 2025 AI output was checked selectively or not at all: it looked too good. Hence the invented achievements on specialists’ pages, the keys in public repositories and the open databases in services. The conclusion the professionals reached is simple: trust in output should be proportional to the cost of a mistake, and checking is not distrust of an AI but an editor’s ordinary work.