Straight Answers · Evidence
How do you catch an AI that made up a citation?
Open the source. Every time, before you use it. Fabricated references are fluent, correctly formatted, and often attached to a real journal and a real author, so nothing about their appearance will warn you. The only reliable check is resolving the identifier yourself.
There is no tell. That is the part people want to be false and it is the part that matters most. A fabricated citation has a plausible author, a real journal, a believable year, and correct formatting, because producing text that looks right is exactly what the model is optimized to do. Nothing on the surface separates it from a real one.
So stop looking for a tell and build a step instead.
The failure mode has a shape
A physician collecting real examples of ambient scribe errors opened with these:
A normal heart exam somehow became "ECG normal." A breast exam turned into "mammography normal." No ECG. No mammogram. Just vibes, apparently.
r/medicine
That is the whole pattern in two lines. The model did not invent a finding. It invented the instrument that produced the finding. An exam got upgraded to a test.
A fabricated citation is the identical move on different material. The claim is usually reasonable. What got invented is the authority underneath it. Once you see it that way you stop checking whether the sentence sounds right and start checking whether the thing it points to exists.
The failure I actually had to catch
I built a continuing education deck with AI assistance and inspected it before it went anywhere. Three separate failures were sitting in it, and only one was the kind people warn you about.
- A citation freeze. The deck kept growing. The reference list stopped. New slides carried claims that inherited authority from citations attached to entirely different content. Nothing was fabricated. The sourcing simply stopped tracking the material, which produces the same false impression and is much harder to see.
- A camouflaged authoring instruction. A line written to guide the build survived into content that read as substantive.
- Notes drift. Speaker notes and slides gradually stopped saying the same thing.
The citation freeze is the one worth internalizing, because it is invisible to the check most people run. If you verify that every listed reference is real, it passes. Every reference was real. The defect was in what those references were being used to support.
The check
- Resolve every identifier yourself. Paste the PMID or DOI into PubMed or doi.org. If it does not resolve, it does not exist. If it resolves to a different paper than the one named, treat everything else in that output as suspect.
- Open the abstract and read what it actually claims. Real paper, real finding, wrong strength is the most common failure once outright fabrication is ruled out. Watch for a hedged or conditional result being reported as a clean one.
- Check the population. A result in post-surgical knees is not a result in chronic low back pain. Models flatten this constantly and it is the error most likely to survive into something you say out loud.
- Count claims against sources. This is what catches a freeze. If the document grew and the reference list did not, the sourcing has stopped tracking the content.
- Hand over the document instead of describing it. Most fabrication happens in the gap where you asked the model to recall something rather than giving it the source. Attach the paper, the policy, the transcript. A model working from a document you provided has far less room to invent one.
None of this is a reason to avoid using AI for evidence work. It is genuinely good at finding threads you would have missed and at compressing a literature you do not have time to read. It is not a librarian and it does not know the difference between a source it retrieved and a source it produced. You do. That is the job that stays yours.