ChatGPT sent visitors to a page that did not exist
In our September 2026 crawl of our own site, ChatGPT was sending clicks to a URL that returned a 404. The referral logged cleanly as utm_source=chatgpt.com. An AI assistant had cited the page, a real person had clicked through, and they landed on an error. Nothing in a normal analytics review had flagged it.
Why this is a category of problem, not a typo
Citation by an assistant is a traffic source. It behaves unlike every other one in a specific way: you did not submit anything to start it, and you are not told when it breaks.
With Google you have Search Console. A spike in 404s shows up in a report somebody eventually opens. With assistants there is no console and no coverage report. The model formed an impression of your URL structure at some point, and your site has moved on without telling it. The failure is silent by design.
Why your analytics review will not surface it
Volume. AI referrals are small in absolute terms today, often single or double digit sessions a month, and they scatter across Referral and Direct depending on how your tagging is set up. Nobody builds an alert for a source sending nine sessions.
But consider what those nine sessions are. A person asked a question. An assistant answered it and named you. The person chose to click a citation rather than accept the summary. That is about as pre-qualified as inbound traffic gets. Losing it to a 404 is not a rounding error.
How to find AI referrals in your own data
Two places. First, your analytics. Build an exploration on session source and filter for hosts containing chatgpt.com, perplexity.ai, copilot.microsoft.com, gemini.google.com and claude.ai. Set the window to the last twelve months, not the last month, because the pattern only shows with volume behind it.
Second, your server logs, which are better. Logs separate two things analytics blurs together. Crawler user agents (GPTBot, PerplexityBot, ClaudeBot) tell you what is being indexed. A user agent like ChatGPT-User tells you a live person’s question triggered a fetch of that URL a second ago. The second group is the one that converts, and it is invisible unless you read logs.
How to check whether any of them hit a 404
Filter your logs for status code 404 where the referrer or user agent matches any assistant host, across the same twelve months. That single query is the check. If it returns rows, an assistant is currently recommending a page you deleted or renamed.
Then look at the other direction: take every URL that has ever received an AI referral and confirm each one still returns a 200 and still contains the content that earned the citation. A page that redirects to a generic services hub technically resolves, but it does not answer the question the assistant promised it would answer. That is a soft failure and it is more common than the hard 404.
Why redirect hygiene now protects citations, not just links
The old argument for maintaining redirects was link equity. Keep the 301 so the authority of inbound links survives the migration. That argument still holds, and it is now the smaller half.
A model’s picture of your site is older than your last redesign, and it does not refresh on your schedule. Every URL you have ever published is a potential citation target for as long as it sits in training data or a retrieval index. Delete it without a redirect and you break an answer that may keep being given for months.
We have seen the compounding version of this. On an audit of a US retention-marketing agency, robots.txt was blocking the major AI crawlers through a default CDN setting nobody had touched, only 5 of 52 pages were indexed, there was no schema markup anywhere, and the title and meta description were identical across the entire site. Form submissions had fallen to zero over twelve months while paid traffic doubled. They were paying more to send traffic to a site assistants could not read.
Start here: https://growthym.com/audit/explains the crawl we run, but you do not need us for the first pass. Pull twelve months of referral data filtered to assistant hosts, cross-reference it against your 404 log, and see what comes back. When it returns rows worth acting on, Growthym can take the remediation from there.
