What an AI citation tells you, and what it doesn't

Are AI citations accurate?

Often, but not reliably enough to skip checking. When researchers audited four generative search engines in 2023, they found that on average a mere 51.5% of generated sentences were fully supported by citations, and only 74.5% of citations supported their associated sentence (Liu, Zhang & Liang, 2023). In 2025, the Tow Center for Digital Journalism gave eight AI search tools excerpts from news articles and asked them to identify the source. Collectively, they gave incorrect answers to more than 60 percent of queries (Jaźwińska & Chandrasekar, 2025).

Tools change quickly, so treat those figures as a snapshot rather than today's rate. The lesson holds anyway: a citation is a pointer to where the answer may have come from. It is not proof that the source says what the answer says.

What a citation does tell you

• Where to start checking. Without a source, you'd have to search from scratch.
• What kind of source the tool relied on: a peer-reviewed paper, a news article, a forum post, your own document.
• Often a date and an author or publisher, which tell you whether the source is current and who stands behind it.

That's real value. An answer with sources can be checked in a minute. An answer without them asks you to take it on trust.

What a citation doesn't tell you

• That the source exists. Some tools have produced references to papers nobody wrote.
• That the cited passage supports the sentence next to it, rather than just covering the same topic.
• That the source is correct, current, or the original rather than a copy.
• That the answer used every relevant source, or the one that disagrees.

Two different failures: invented sources and real ones that don't say it

The first failure is fabrication. In one study, researchers asked two versions of ChatGPT to write short literature reviews on 42 topics and checked the 636 references they produced. Within that set, 55% of the GPT-3.5 citations but just 18% of the GPT-4 citations were fabricated (Walters & Wilder, 2023). Of the real papers cited, 43% of the GPT-3.5 citations and 24% of the GPT-4 citations had substantive errors.

The newer model fabricated far fewer references in that study. The second failure is subtler: a real source that doesn't support the claim. That's what the 74.5% figure above measures. A benchmark built to test citation quality in language models found that on one question set, even the top models lacked complete citation support 50% of the time (Gao et al., 2023).

The second failure is harder to catch, because the link works and the page is on topic. You only see the gap when you read the sentence the answer depends on.

A link is not the same as the exact passage

Citations come in different levels of precision. Some point to a website's home page. Some point to a 40-page report. The precise ones point to the paragraph or line that supports the claim.

The Tow Center study found tools linking to home pages instead of articles, and to syndicated copies instead of the original publisher, even when they named the right article. A vague link isn't wrong, exactly, but it pushes the work back to you. If checking takes ten minutes, most people won't do it, and an unchecked citation protects nobody.

For your own documents, the same rule applies. "Source: Lecture 6" is a start. A source that opens Lecture 6 at the sentence about the mitral valve is something you can verify before you've finished reading the answer.

A one-minute check for any AI citation

1. 1Pick the claim that matters: the number, the date, the name or the instruction you're going to act on.
2. 2Open the source for that claim. If it doesn't load, or doesn't exist, stop treating the answer as sourced.
3. 3Find the actual sentence. Search the page for a key term from the claim. If you can't find it, the citation doesn't support it.
4. 4Compare wording. Watch for small shifts: "may reduce" becoming "reduces", "in mice" disappearing, a range turning into a single number.
5. 5Check the context: when it was published, who wrote it, and whether it's the original or a summary of someone else's work.

When an AI answer says it doesn't know

An answer that says "the sources don't cover this" can feel like a failure. It's often the most useful thing a tool can say. It tells you to look elsewhere, instead of handing you a confident sentence with a citation that doesn't hold up. Research on why models guess is covered in more depth in why AI answers make things up.

How citations work in Lumyo

We built Lumyo around this check. You choose which notes a chat reads, every answer cites the notes it used, and tapping a source opens the exact line, not just the note. When your notes don't cover a question, it says so instead of guessing. It still makes mistakes, which is why the source is one tap away: the citation is there so you can check, not so you don't have to.