The Judgment That Left With the Employee

A financial services firm deployed an agent to qualify leads. It worked well for months. Then the manager who had configured it was promoted to a different team. Six months later, the sales team noticed the agent was discarding mid-market companies that clearly belonged in the funnel. No one could explain why. The prompt was still there, unchanged. The problem was what was never in the prompt to begin with.

Configuration is not the same as judgment

When people talk about configuring an AI agent, they tend to think about the visible layer: the prompt, the workflows, the CRM integrations, the automation triggers. All of that can be documented, versioned, and handed off.

But there is a layer beneath it. It is the layer of judgment: why this criterion and not that one? Why were mid-market companies with fewer than 200 employees deprioritized, except when they operated in certain industries? That reasoning was never written down anywhere. It lived in the manager's head, built over two years of sales calls, lost deals, and customer feedback.

This is what researchers call tacit knowledge: the kind of knowing that cannot be fully articulated because it is embedded in practical experience. An experienced doctor knows when a scan warrants a second look before she can explain why. A senior salesperson senses when a prospect is genuinely interested versus just collecting proposals. That kind of knowing does not transfer to a document. And it definitely does not transfer to a prompt.

The agent as a cast of whoever was available

Here is the core problem: when a company deploys an AI agent, it is encoding the judgment of whoever was available to do the configuration at that moment. Not necessarily the best judgment available in the organization. Not the collective wisdom of the team. The judgment of that one person, in that window of time, with the blind spots and priorities they happened to have.

If that person carried biases, the agent carries them too. If she had blind spots, the agent reproduces them. If she was optimizing for speed over precision because the business context at the time demanded it, the agent will keep optimizing for speed long after the context has shifted.

And when she leaves, no one knows what is inside the cast. The agent keeps running. Results keep coming in. The team assumes things are fine because the numbers have not dropped dramatically. The problem compounds quietly.

Why this is different from the same problem in a manual process

Manual processes have this problem too. When someone leaves, they take knowledge with them. That is not new.

The difference with AI agents is scale and invisibility. A manual process that depended on someone's judgment simply stops working well when that person is gone. The team notices. Something is clearly missing.

An AI agent keeps functioning. It responds, qualifies, categorizes, sends emails, updates records. It looks like it is operating normally. The signal that something is wrong is much harder to detect because the agent never stops working. It just works with the wrong logic, consistently, and at scale.

What to do before someone walks out

The solution is not to write longer prompts. It is to build a deliberate process for externalizing judgment before any implementation goes live.

In practice, that means asking the uncomfortable questions during configuration: which cases will this agent get wrong, and why? What kinds of exceptions do you usually open manually that the agent will not be able to detect? When you look at a lead and decide it is worth a call even though the data does not justify it, what is actually going through your head?

Those questions surface knowledge that is not in the standard prompt. They need to become living documentation: real examples of decisions made, notes on edge cases, records of when the agent was wrong and what the right call would have been.

AI agents do not replace human judgment. They amplify the judgment of whoever configured them. The gap between amplifying good judgment and amplifying mediocre judgment can be enormous in terms of outcomes.

The question worth asking today

If the person who configured your AI agents left tomorrow, could you explain why each criterion was chosen? Could you describe which exceptions were accounted for and which were left to human discretion?

If the answer is no, the risk is already there. The agent is already running on knowledge that belongs to a person, not to the company. While that person is still around, the problem stays invisible. When they leave, it becomes expensive.

The concrete next step is straightforward: before the next implementation, or as a review of the ones already running, schedule a one-hour session with whoever configured the agent. Record it. Ask the questions about exceptions, edge cases, and the judgment that never made it into the prompt. That will not solve everything, but it will reduce how much institutional knowledge walks out the door with the people who built your systems.