Why LLMs Hallucinate — and How Grounding Fixes It

Ask an LLM for last month's receivables balance and it can hallucinate one: a fast, confident, full-sentence figure it never actually looked up. It might even be close. Nothing was checked to get there; the model produced the words that most plausibly follow your question, the same way it would produce the next line of a poem if you asked for one instead.

That is not a failure mode hiding inside an otherwise reliable system. It is the system. A language model has one job: predict the next token. Everything it says, right or wrong, comes out of that same process. There is no separate "I actually know this" pathway that switches off when the model is guessing and on when it isn't.

For a first draft or a brainstorm, a plausible guess is a fine place to start. For "how much stock do we have of SKU 4021," it is not. The gap between those two situations is the entire enterprise AI hallucination problem, and prompting the model more carefully does not close it.

What a hallucination actually is

The useful mental model is that an LLM is a very capable autocomplete, trained on enough text that its completions often read as reasoning. Ask it something in a domain it saw heavily during training (general knowledge, common code patterns, well-documented history) and its predictions tend to line up with reality, because reality was thoroughly represented in what it learned.

Ask it something it never saw — your company's actual Q3 numbers, a metric your finance team defines its own particular way, anything that changed after training or that only exists inside your own systems — and it has no "I don't have this" branch. It keeps predicting anyway. The output looks identical whether the model is recalling something true or inventing something false, because fluency and correctness come from different places and the model was only ever optimized for the first.

This is also why hallucination cannot be fully trained away. A larger, better model hallucinates less on questions it has real information about, and just as confidently on the ones it doesn't, because the mechanism that generates the sentence never changes. Some researchers argue the ceiling is fundamental: a formal analysis titled "Hallucination is Inevitable" contends that a model cannot learn every possible function, so there will always be questions it can only guess at. Whatever the exact theoretical limit, the practical point holds: the model is neither lying nor broken; it is doing exactly what it was built to do, applied to a question it has no grounded answer for.

Two kinds of hallucination

Not every hallucination fails the same way, and the difference shapes how you catch them. It helps to split them into two kinds.

An intrinsic hallucination contradicts a source the model was actually handed. Give it a report stating July revenue was Rp 4.1 billion and it summarizes that same report as Rp 4.6 billion, and the answer is wrong against material sitting right in front of it. These are the easier kind to catch, because the ground truth is present and the check can be automated.

An extrinsic hallucination adds something the source neither states nor contradicts. Ask about a figure that appears nowhere in the connected data and the model fills the gap with a plausible invention, and there is nothing to check it against, which is exactly what makes it dangerous. Most of the enterprise failures worth worrying about are extrinsic: the system is asked for something it was never given, and answers anyway.

Why this is intolerable for business numbers specifically

A fabricated fact in a marketing draft gets caught by an editor before it ships. A fabricated number in an answer about revenue, inventory, or a covenant ratio gets acted on, because it arrives dressed in the same fluent, confident tone as a correct answer. There is no asterisk on a hallucinated figure. No hedge in the phrasing. The sentence "net revenue in July was Rp 4.2 billion" reads exactly the same whether that number came from the general ledger or from thin air.

Worse, the errors are rarely wild. A model asked for a number it doesn't have tends to produce something in a plausible range: close enough that a busy director doesn't double-check it, off enough to be wrong. That is the failure mode that does real damage: not the answer nobody believes, but the answer everybody quietly acts on.

Prompting doesn't fix a lookup problem

The instinctive fix is to ask more carefully — "only state facts you are certain of," "cite your sources," "say 'I don't know' if you're not sure." These instructions help around the edges: they can reduce some formatting mistakes and nudge a model toward hedging language. What they cannot do is give the model a fact it was never given. A model does not have reliable introspective access to whether a given output is memorized truth or a plausible fabrication. Both are produced by the same next-token process, so asking it to self-report which one just produced a given sentence is asking it to grade its own homework with no answer key.

The only fix that actually works is architectural, not promptual: put the fact in front of the model at the moment it answers, instead of hoping the fact survived somewhere in its training weights.

The grounding toolkit

Grounding means the model answers from data handed to it right now, not from what it happened to memorize. In practice that toolkit has four parts:

• Retrieval. Pull the specific record, document, or data slice relevant to the question before the model writes a word, so it is summarizing something real rather than recalling something approximate.
• Live queries. For operational numbers, retrieval from a stale export is not enough: the model needs to query the system of record itself, at question time, so "today's stock level" means today, not last Tuesday's sync.
• Citations. Every number in the answer should trace back to where it came from — a table, a report, a timestamp — so a human can verify it in seconds instead of trusting it on faith.
• Refusal when data is absent. The single most important behavior: when the connected systems genuinely don't have an answer, a grounded system says so, instead of filling the gap with a fluent guess. This is a design choice, and most consumer chat products do not make it.

None of this is exotic engineering, but it is real engineering: connecting live systems, modeling what the data means, and enforcing who is allowed to see what. That is the job an AI intelligence layer exists to do, and it's worth comparing honestly against the alternatives: retrieval, fine-tuning, and an intelligence layer solve different parts of this problem, and most mature setups use more than one. Grounding also only works if everyone agrees what the number means in the first place, which is what a semantic layer is for.

Ungrounded vs. grounded answering

How do you evaluate a vendor's grounding claims?

Every enterprise AI vendor now says their product is "grounded" or "hallucination-free." Neither claim survives five minutes of testing if it isn't true, and the test is simple:

Ask for a number you already know the real value of, and ask the system to show where it came from. A grounded answer names a source and a time; an ungrounded one gives you a confident figure and nothing to check it against. Then ask a question about data the vendor's demo obviously doesn't have connected. A grounded system says so plainly; an ungrounded one will still try to answer. Finally, ask the same question as two different roles, if the demo allows it: an answer that doesn't change based on who's asking was never permission-aware to begin with, which means it was never fully grounded either. Permission-awareness belongs on the same list as the data-security questions worth putting to any AI vendor before a contract, not treated as an afterthought.

This is really the same question as whether your own data is ready for AI to be grounded in. A vendor can only ground answers in systems that are actually connected, current, and mapped. Ask what "connected" means in their demo before you ask what their model can do.

Where Nalar fits

Nalar grounds every answer in the systems a company already runs (ERP, CRM, point of sale, HRIS, databases, spreadsheets) with live queries, sources shown, and per-user permissions enforced on every response. When the connected data doesn't have an answer, Nalar says so rather than guessing, across all six industry archetypes it supports.

If you want to see grounded answers in action before talking to anyone, the interactive demo runs on a realistic mock enterprise. If you want to know whether your own systems are connected and current enough to ground AI answers in, BARI, our AI-readiness diagnostic, will tell you honestly, including when the answer is "not yet."