The short version
- A tool that answers confidently and wrongly is worse than one that answers nothing, because the reader has no way to tell the difference.
- The problem is structural: tools that read everything cannot distinguish a settled decision from an argument someone was losing.
- Run a five-question trust test before you roll anything out to a team. It takes an afternoon.
- The properties to insist on: a source under every answer, an explicit "I do not know", and never speaking in a person's name without a label.
An internal tool that answers questions about how your company works has a failure mode with no equivalent in the tools it replaces. A colleague who does not know says so. A search box that finds nothing returns nothing. A tool that generates an answer regardless returns something plausible, and plausible is indistinguishable from correct to someone who asked because they did not know.
"It answered confidently and it was wrong."
Confident and wrong
Consider the asymmetry. A tool that says "I do not know, ask Priya" costs the asker thirty seconds and a message. A tool that produces a confident, wrong answer costs them however long they work on the wrong basis, plus whatever it takes to unwind, plus the trust of everyone who finds out.
One wrong confident answer also does disproportionate damage to adoption. Somebody acts on it, is embarrassed, tells two colleagues, and the tool acquires a reputation that no number of correct answers reverses. Trust in an internal reference is close to binary in practice.
Why it happens structurally
This is not primarily a model quality problem, which is why better models have not removed it. It is a source problem.
A tool that reads everything your company has written is reading discussion, not resolution. A thread where three options were debated looks textually similar to a thread where one was chosen. A document from eighteen months ago looks like a document from last week. A message in which somebody argued for an approach that was ultimately rejected reads exactly like a statement of what the company does.
Asked what your pricing policy is, such a tool will synthesise something coherent from all of it. It cannot mark the difference between what was decided and what was merely said, because that distinction is not present in the text. The same problem in a different guise is covered in AI summaries capture what was said, not what was decided.
The five-question trust test
Before rolling anything out, spend an afternoon on these. Use real questions about your own company, and have someone who knows the answers grade them.
| Ask it | Pass looks like |
|---|---|
| Something nobody has written down | It says it does not know and names who might |
| Something decided twice, the second reversing the first | It gives the current answer and dates it |
| Something still under discussion | It says it is open, rather than picking a side |
| Anything at all | A source you can click, under every answer |
| Something about a named person's work | It is clear it is not that person speaking |
The first row is the whole test in one question, and most tools fail it. A tool that will not say "I do not know" is a tool that will eventually say something wrong with the same confidence it says everything else.
The properties to insist on
Three, and they are the design StandIn is built on.
It never guesses. When the answer is not in what people wrote down, it says so and names who to ask. That is a feature rather than a limitation: the value of an answer depends entirely on knowing it is not invented.
A source under every answer. Not a citation list, the specific thing the answer came from, so the reader can judge for themselves. This is also what makes the second row of the test passable, because a dated source lets a reader see how current the answer is.
It never puts words in your mouth. It speaks only from what you wrote, every reply is labelled as your StandIn, and everything it said is waiting for your review. On your behalf, never as you.
Those three constraints are what make the answers usable in the situations that matter, which are the situations where the asker cannot verify the answer themselves. See how StandIn works.
Common Questions
How do you test whether an internal AI tool is trustworthy?
Ask it something nobody has written down. If it produces a confident answer rather than saying it does not know, it will do the same on questions where you cannot tell it is wrong. Then check that every answer carries a source you can open.
Why do internal AI tools give wrong answers about company decisions?
Because they read discussion rather than resolution. A rejected proposal and an agreed policy look textually similar, so a tool that synthesises from everything cannot reliably tell which one it is repeating.
Is "I do not know" an acceptable answer from a tool?
It is the most important answer a tool can give. Without it, every answer is equally confident, which means none of them can be relied on in the cases where the asker has no way to check.
What if the tool speaks in an employee's name?
Then the label matters more than anything else. Answers drawn from a person's work should be clearly marked as not being that person speaking, and the person should be able to see afterwards what was said in their name.
When you're off, your StandIn is on.
It answers your teammates' questions from work you've already done, in your words, with a source under every answer.