Why can AI invent a source with complete confidence

A language model predicts plausible sequences from training and context. It may reproduce a known reference correctly, but it can also combine a real author, a plausible title and an existing journal into a publication that never existed.

NIST’s generative-AI profile describes confabulation as confidently presented false or erroneous content. The problem applies to facts and to references. A link, DOI or exact date should therefore be checked rather than trusted because it looks unusually specific.

A system with document retrieval or database access can cite material it actually fetched. It can still select the wrong passage, summarize poorly or attach a source that does not support the conclusion. Provenance improves auditability; it does not automate judgement.

In a paper, legal note, article or business decision, an invented reference can be copied by several people and gain a false appearance of legitimacy. The more precise it looks, the easier it is to reuse without checking.

Search the publisher or database, open the original and confirm that the cited passage actually supports the statement. For sensitive subjects, preserve the link, consultation date and, where possible, the exact page or section.