How we stopped our assistant from making things up

Confidently wrong

We built a home AI assistant, and early on it had a habit we did not like: it would answer with total confidence using details it had simply made up. Not a hedge, not an "I'm not sure" — a clean, fluent answer that happened to be invented. Those are the worst kind of wrong answers, because they sound exactly like the right ones.

Our first instinct was the common one: write better instructions. Tell the model not to guess. Add rules, add warnings, add "only use real data." It helped a little. It did not fix the problem, because it could not.

Grounding beats prompting

The real fix was structural. We switched the assistant to native tool-calling — giving the model actual tools it can call to fetch real data, rather than asking it to recite facts from memory.

The difference is where the answer comes from. When a model has no way to look something up, its only option is to produce something plausible; guessing is not a bug in that situation, it is the only move available. Give it a real tool that returns real data, and the plausible-sounding guess stops being necessary. It asks, gets a fact, and answers from that.

A model with nothing to check against will always, eventually, make something up.

The lesson

Fabrication dropped sharply once the model no longer had to imagine answers. That reframed how we think about reliability: hallucination is often not a discipline problem to be scolded away with prompting, but a grounding problem to be engineered away with access to truth.

The assistant itself has since been retired — that is a separate story for another post. But the lesson outlived it. If you want a model to stop making things up, do not just tell it to stop. Give it something real to reach for.