de
When an AI agent produces poor results in an organisation, suspicion almost always falls on the model first. Too small, too old, not specialised enough. So the next model gets tested, then a fine-tuned one, then another vendor. The results become slightly better phrased and stay just as wrong in substance.
Why the assumption is so natural
It is understandable. Models are the most visible part of AI, they are compared in benchmarks, and each new version really is better than the last. In a demo on public knowledge a current model answers almost any question impressively. Anyone who has seen that looks for the fault in their own project in the technology first.
The numbers tell a different story, though. 95% of AI pilots show no measurable ROI (MIT NANDA, 2025). That is hardly because everyone picked the wrong model.
What an agent actually does not know
An agent that is supposed to handle a complaint, check an order or route a request needs more than language understanding. It has to know which rule applies here, which exception a particular customer has, above what amount someone has to approve and who that is. Part of this is in documents. A large part is not.
And even what is in documents is hard for an agent to judge. It sees three versions of a policy and does not know which applies. It finds a guide from 2019 and no hint that the process has long changed. A person with ten years of experience spots that immediately. An agent does not.
Meanwhile employees are going their own way: more than half use private GenAI accounts for work (Gartner, February 2026). If the organisation’s agent gets no better context than a freely available tool, it loses those users.
What holds instead
In practice, agent enablement means preparing context the way you would for a new specialist, only more explicitly. Four things make the difference, and none of them is a model switch.
Verified knowledge instead of all documents. Every piece of information an agent accesses has a source, an owner and a date until which it applies. Whatever lacks these three does not go into the context.
Experiential knowledge where documents stop. The exceptions, rules of thumb and warning signs experienced staff know are captured from them, for instance in short AI-guided interviews, and verified before the agent uses them.
Clear handover limits. The agent knows in which cases it does not decide but passes on to a human, and to whom.
A route for corrections. When a person corrects an agent’s answer, the correction flows back into the verified base instead of vanishing in a chat history.
Why retrieval architectures fail on their sources without this groundwork is covered in the piece on RAG architecture for enterprise. What happens when knowledge is derived only automatically from documents is shown by the look at Viva Topics.
To be fair, there are cases where the model really is the bottleneck, for example with very long documents or complex calculations. They are just rarer in enterprise projects than troubleshooting suggests.
So before testing the next model, ask the agent a question that only one particular person in your organisation can answer. If it gets it wrong, you know where the problem lies.
