The retrieval part is straightforward. Curation is not
Pointing a retrieval system at your documents is a weekend of work. Deciding which documents are true is the project. Most organisations have three versions of the same policy, one of them current, and nobody has flagged which.
So the first deliverable is not a chatbot. It is an approved source list with an owner. Until somebody is accountable for saying this document is authoritative and that one is superseded, the system will confidently repeat obsolete guidance, and it will do so in a tone that sounds more reliable than the person it replaced.
Citation and refusal are the two features that matter
A knowledge agent that answers without citing is unusable in any regulated or technical setting, because nobody can check it. Every answer should carry its source, and the person reading should be able to open that source in one click.
The second feature is refusal. The system must be willing to say the knowledge base does not cover this, and route to a person. This is the opposite of how these products are usually demonstrated, and it is the difference between a tool your senior staff trust and one they quietly stop using after being burned twice.
Tacit knowledge resists capture, and that is fine
The most valuable thing a thirty-year employee knows is often a judgment: this supplier's lead times slip in August, this customer needs the call before the email, this tolerance is nominal but nobody actually works to it. Some of that can be written down and captured. Some cannot, and pretending otherwise is how these projects lose credibility internally.
The realistic goal is to absorb the documented and semi-documented layer, which is usually the bulk of the daily question volume, and to route the rest to the expert with better context than they get today. That frees their time for the judgment calls rather than replacing them.
Use this before you buy or build.
- An approved source list exists with a named owner before the build starts.
- Every answer cites its source and the source is one click away.
- The system refuses and routes when the knowledge base does not cover a question.
- A review cadence exists to retire superseded documents from the index.
Fund the curation, not just the retrieval. An agent that cites its sources and admits what it does not know will be trusted; one that always has an answer will not.
