Why do AI agents hallucinate recent events, and how do you fix it?

Why AI agents hallucinate recent events

An agent invents recent events because the model has no stored fact after its training cutoff and no penalty for guessing. Training and evaluation reward a confident answer over an admission of uncertainty, so the model produces the most plausible sentence instead of no sentence. Retrieval, not a larger model, removes the gap.

Two separate things are going wrong at once, and teams usually only fix one. The first is a coverage problem: the fact does not exist inside the weights. The second is an incentive problem: nothing in the model's training made silence attractive. Fix only the second and you get an agent that refuses useful questions. Fix only the first and you get an agent that still guesses when retrieval misses.

The incentive half, stated plainly

Kalai and colleagues at OpenAI argue that hallucinations are not mysterious. They originate as ordinary errors in binary classification, and they persist because most benchmarks grade like a multiple-choice exam: a wrong answer and a blank answer score the same, so guessing dominates abstaining.

That framing predicts exactly the behaviour you see in production. An agent asked what the Federal Reserve said this morning will produce a fluent, correctly formatted, entirely fictional statement, because a fluent fiction scores better on almost every evaluation than "I do not know."

The coverage half, and why recency makes it worse

Recent events are the worst case for a frozen model. It has seen thousands of Fed statements, so it knows the genre precisely: the cadence, the hedging, the phrase "in light of incoming data." Genre knowledge without fact knowledge is what produces a convincing wrong answer rather than an obviously wrong one.

This is why hallucinated recency is harder to catch by reading than hallucinated arithmetic. The output looks correct because the shape is correct.

How to detect it before a user does

The fake-event test is the fastest one. Ask your agent about an announcement that did not happen. An agent with grounding says it found nothing. An agent without it writes you a press release.

The fix: make retrieval cheaper than guessing

Grounding works when the agent can reach a real source at the moment of the question, cheaply enough that calling is never the expensive option.

# free preview, top 3 results, no payment: cheap enough to call on every question
curl "https://thebotwire.com/fed/preview?q=FOMC&since=14d"

# paid, $0.01: full ranked results with source and timestamp per item
curl "https://thebotwire.com/fed/latest?q=FOMC&since=14d&limit=10"

Three properties matter more than the model you pair it with.

• Primary sources, not articles about them. An agent treats a retrieved passage

as fact and acts on it, so distance from the origin becomes a correctness property. The Bot Wire reads the SEC, the Federal Reserve, the federal courts, CISA, the FDA and USGS directly across 40 wires.

• A timestamp on every item. An agent cannot reason about freshness it cannot

see. /health publishes the last poll time and the observed interval between ingest runs, currently about 299 seconds.

• An empty result that means empty. The most valuable response a grounding

layer returns is zero items, because that is the signal that licenses the agent to say nothing happened.

Prompt changes that actually help

Retrieval is the fix; prompting is the guardrail around it.

• Instruct the agent to answer only from retrieved items, and to name the source

and timestamp for each claim about a recent event.

• Give it an explicit abstain path. "If the wire returns no items, say no items

were found" outperforms "do not hallucinate," which names the failure without offering an alternative.

• Route by domain rather than searching generally. Sending a filings question to

an SEC EDGAR wire beats a web search that returns commentary about filings.

• Keep the retrieval budget generous. If a call costs half a cent, there is no

reason for an agent to skip one and guess instead. The four ways agents get current data compares the options on latency and cost.

FAQ

Does a bigger model hallucinate less about recent events?

Not meaningfully. Scale improves what a model knows up to its cutoff and does nothing about the period after it. A larger model asked about this morning has the same empty space and better prose to fill it with, which can make the failure harder to spot rather than rarer.

Is retrieval-augmented generation enough on its own?

Only if retrieval actually hits. RAG over a stale index reproduces the same failure with extra steps, because the agent finds something, treats it as current, and answers confidently. Freshness of the index matters as much as the presence of one.

How do I stop an agent from guessing when the wire returns nothing?

Make the empty case explicit in both the tool contract and the prompt. A wire that returns an empty array plus a queried window gives the agent something concrete to report, and an instruction to report it turns silence into a valid answer.

Why does the agent cite a source that does not say what it claims?

Because the retrieved passage was summarised rather than quoted, or the source was an article about a primary document rather than the document. Fetching from the issuing body and preserving per-item links removes one whole layer of that drift.

Sources

The argument that hallucinations arise as binary classification errors and persist because evaluations reward guessing over abstaining is set out in Why Language Models Hallucinate by Kalai, Nachum, Vempala and Zhang. Freshness figures on this page, including the observed interval between ingest runs, are live at thebotwire.com/health.

Related: How AI agents get real-time data past the training cutoff · Can ChatGPT or Codex read live SEC filings?