When Deloitte submitted a government report to Australian authorities in October 2025, it included peer-reviewed citations, court references, and expert sources that looked legitimate but didn't actually exist. The fabricated citations weren't caught until after the report was filed. This wasn't a one-time mistake; the same firm did it again the following month with a $1.6 million health policy report for the Government of Newfoundland, complete with four citations to research papers that don't exist in any journal database .

Why Do AI Models Confidently Make Up Information?

The problem isn't that AI systems are broken. It's that they're working exactly as designed, but in ways that create real liability. AI tools fail at two separate points when handling information they don't have: retrieval (finding the wrong document or outdated version) and generation (misrepresenting what a document actually says by filling gaps with statistically plausible invention). Both produce the same output: a confident, well-structured answer with no signal that anything went wrong .

MIT researchers confirmed in January 2025 that models were 34% more likely to use words like "definitely" and "certainly" when generating incorrect information than when stating facts they actually had. This isn't a model-specific bug; it's a structural feature of how these systems produce text .

OpenAI's own research team published a paper in 2025 with an unambiguous conclusion: hallucinations are structurally inevitable under current training architectures. The model is mathematically rewarded for guessing rather than admitting uncertainty. A model that guesses correctly scores higher than one that says it doesn't know. The training pipeline optimizes for the behavior that produced the Deloitte reports .

How Bad Is the Hallucination Problem Across Different AI Tools?

The numbers vary dramatically depending on the task. When AI systems are tested on controlled summarization tasks (give the model a document, ask it to summarize faithfully, check the output), leading models achieve hallucination rates below 5%. Vendors put these numbers in their marketing decks. They're not lying; they're measuring something that has almost nothing to do with what you'll actually use the tool for .

On real analytical tasks, the picture is much darker. Here's what researchers documented across different tools and scenarios :

• Perplexity AI: A 37% citation error rate, where the model returns real URLs but invents the attributions. The URL exists; the information is fabricated. That's the variety of hallucination hardest to catch because it passes the first verification check.
• OpenAI's o3 and o4-mini: Their most advanced reasoning models hallucinate at higher rates on factual recall than their predecessors. More computation spent reasoning means more opportunities to drift from what the source actually said. More capable doesn't mean more accurate; sometimes it means more confidently wrong.
• General-purpose AI on specialized markets: When asked about financial metrics for companies with limited analyst coverage or minimal international press presence, models return numbers with full confidence that came from nowhere you can point to. Statistical patterns from similar companies substitute for actual data.

The Charlotin database, a global tracker of AI hallucination incidents in legal filings, catalogued over 1,200 cases by early 2026, growing at five to six new entries per day. Every single case shares the same root cause: someone assumed the AI was more reliable than it actually was .

Who Gets Held Responsible When AI Hallucinations Cause Damage?

In every documented case, from Deloitte to Air Canada to Mata v. Avianca, the organization that deployed the AI was held responsible. No court, no tribunal, no government accepted "the AI generated it" as a defense. Liability belongs to whoever trusted the output without verifying it .

This matters especially in financial research, where hallucinations are far subtler than in legal documents. A revenue figure off by 8%, a management quote from Q2 attributed to Q4, a segment margin that matches industry averages but not this company's actual filing. Each passes a casual read. The feedback loop is slow; a flawed assumption doesn't surface until capital has already moved on it .

How to Reduce Hallucination Risk in Your Organization

Retrieval-Augmented Generation (RAG), a technique that grounds AI responses in actual documents rather than just statistical patterns, reduces hallucination by up to 71% versus base models and is the most effective mitigation available. However, it's not sufficient on its own. Stanford's research on Lexis+ AI and Westlaw, purpose-built RAG tools from companies with decades in legal search, found hallucination rates of 17-34% even with RAG active. The model still fills gaps when retrieval falls short .

If you're using AI for research that moves capital, consider these practical safeguards:

• Citation Enforcement at Output: Require every claim to be anchored to a specific document and page. Return "not disclosed" when the source is absent rather than allowing the system to generate something plausible in its place. A "not disclosed" is data; a generated estimate dressed as data is liability.
• Source Verification Before Use: Don't assume that real URLs or cited sources mean the information is accurate. Verify that the source actually contains the claim being attributed to it. Perplexity's 37% error rate shows that real citations can mask fabricated attributions.
• Domain-Specific Tools Over General Models: For specialized markets like Indian equities or niche financial segments, general-purpose AI trained on global data will hallucinate. Purpose-built platforms designed around specific source documents (BSE filings, NSE disclosures, etc.) eliminate the gap that statistical memory fills with invention.
• Human Review for High-Stakes Decisions: The Deloitte reports passed every internal review before being filed. The citations looked real. The numbers looked right. Nobody checked whether the sources actually existed until someone outside the firm did after the reports were submitted. Build verification into your workflow, not after it.

What's the Real Cost of Trusting AI Without Verification?

The Deloitte cases cost $440,000 and $1.6 million respectively, but the real damage extends beyond the immediate financial hit. Institutional credibility, regulatory relationships, and legal exposure compound the problem. When an organization submits a government report with fabricated sources, the liability doesn't disappear when the AI is blamed .

For financial teams specifically, the risk is amplified. Financial documents are numerically dense, with dozens of figures in close proximity in training data, producing metric substitution errors invisible to anyone not working from the source directly. The same metric appears differently across document types (standalone vs. consolidated, pre-restatement vs. post, Ind AS vs. GAAP), and a model without document provenance cannot tell you which version it cited .

The uncomfortable truth is that this problem doesn't improve with the next model version. It improves incrementally while the architecture that incentivizes confident guessing stays constant. Better reasoning doesn't solve hallucinations on factual recall; it just makes the model reason its way to a plausible answer instead of acknowledging it doesn't have the data .