The short version
- AI agents rarely crash. They fail without warning: answers drift, look right but are wrong, or credit the wrong person.
- Most failure modes come from the same limits: an agent has no stake in being right and no sense that something is off.
- Catch them with written boundaries, sampled reviews, sources on every answer, and a record of what the agent did.
- The most dangerous failure is the one that does not look like a failure.
AI agents rarely fail loudly. They fail by drifting slowly, by giving answers that look right and are not, by acting outside the scope someone assumed they had, and by never signaling that anything went wrong. The ten failure modes below are the ones teams most often meet only after launch.
They share a root. An agent has no stake in being right, no experience that tells it something is off, and no habit of stopping to check. The guide to what AI agents cannot do explains why those limits do not go away with better models. Knowing the patterns lets you design checks for them.
1. Quality drift over weeks
The agent works well at launch and slowly gets worse: the data it reads changes, the questions shift, a prompt gets edited. Nobody notices because no single answer is clearly broken. Review a fixed sample of outputs every week against the same standard, so drift shows up as a trend.
2. Looks right, is wrong
Fluent, confident, well-formatted, and wrong. This is the most common failure and the hardest to catch, because the format signals quality. Require a source for every factual answer, so a reviewer can check it in seconds, and prefer tools that say when they do not know.
3. Behavior change after a model update
The vendor ships a new model version, and the agent starts answering differently: shorter, longer, more willing, less willing. Treat model and prompt changes like software releases. Test a fixed set of questions before and after, and keep a way to roll back.
4. Bad responses to hostile input
A cleverly worded message, or a document with hidden instructions, gets the agent to ignore its rules. Instructions alone will not stop this. Enforce the important limits outside the agent, in what its account can access and which actions its tools allow; see the boundaries every AI agent needs.
5. Slow chains of calls
An agent that calls tools, which call other tools, can take minutes to answer, or loop. Users give up or retry, which makes it worse. Set time and cost limits per task, and stop the task when it hits them.
6. Hidden state building up
Agents that remember past conversations can carry forward a mistake, a wrong assumption, or one user's preferences into another user's answers. Decide what the agent may remember and for how long, and clear it on a schedule.
7. Permission creep
The agent was given broad access "to get started," and nobody narrowed it later. Now it can read or change far more than its job needs. Review each agent's access every quarter, and give it its own account so its access can be checked on its own.
8. Users talking the agent into things
People find that phrasing a request a certain way gets the agent to do what it should not: approve a refund, share a document, skip a step. Watch for patterns in what people ask, and close the gaps in the system, not just the prompt.
9. Information crossing between users
An agent serving many users or customers can let details from one conversation surface in another. Separate data by user or customer at the system level, and test for it before launch.
10. Failing without anyone seeing
The agent hits an error, cannot find an answer, or skips a step, and nothing tells a person. The user assumes it worked. Make failures visible: a clear message to the user, and a record someone reviews.
How agents fail in engineering work
Engineering teams see these failures in particular shapes. An AI coding tool writes a test that passes because it tests nothing. An agent summarizing incidents credits the fix to the wrong engineer. A tool answering "is this ready to merge?" reads an old thread and says yes. A bot that files tickets creates duplicates nobody triages.
The common thread is that each output looks plausible. The defense is the same as above: sources on answers, review on anything that changes production, and a named person who owns each tool.
Right facts, wrong owner
One failure deserves its own name. The agent gets the facts right but credits them to the wrong person: it says Priya decided something that Sam decided, or quotes a plan as current when its author replaced it last week. The answer passes a quick check, because the facts are real. The damage is to trust between people, and to anyone who acts on the wrong person's authority. Answers that name their source, and only speak from what that person wrote, prevent most of it.
What to do about it
- Write the agent's boundaries down before launch, and enforce the important ones outside the prompt.
- Require a source on every factual answer.
- Review a fixed sample of outputs every week.
- Treat model, prompt and tool changes as releases.
- Name one accountable person per agent; see keeping human accountability with AI.
- Keep a record of what the agent said and did. A governance layer gives that record a home.
Where StandIn fits
StandIn is not an agent, and it is built to avoid these failures. It answers your teammates' questions only from the work you have already done and what you wrote down, with your name and a source under every answer. It takes no actions and writes nothing into your tools. When the answer is not there, it never guesses: it says so and points to the person who knows. See how StandIn works.
Parker Miller
Founder, StandIn
Parker Miller founded StandIn. He writes about coverage, handoffs, and what it takes for software to answer for a team from what people actually wrote down.
More about StandInWhen you're off, your StandIn is on.
It answers your teammates' questions from work you've already done, in your words, with a source under every answer.