The AI era is a source-of-truth problem
6 min read
Most AI work starts with a normal request.
Deploy this. Create a ticket. Read this document. Find the answer. Fix the test.
The model can usually produce something useful in seconds. That part is getting less interesting by the month.
The hard part starts when the system has to decide what counts as true.
I have been building and testing agents around actual work, not just chat windows. The same problem keeps showing up. The agent is often willing to act before the surrounding system has made the important things clear.
Which issue does this change belong to? Which document pages matter? Is a failed evaluation a product problem, a stale environment, or a bad test? Who gets to decide?
People call this a prompting problem because prompts are visible. Most of the time it is an evidence problem.
A title is not the work
One workflow had to connect a pull request to the right work item before it could deploy anything. There were three clues: a branch name, a pull request title, and linked issue metadata.
They could disagree.
Reading the title first felt reasonable. It was also wrong. Titles get copied, edited, and reused. Linked metadata can be stale. The branch was the thing that actually pointed to the code being checked out and deployed.
So the rule became simple. The branch key is canonical. The title can help when the branch has nothing. Linked metadata can explain a disagreement. Neither gets to override the branch.
That is not an AI breakthrough. It is a boring rule about authority.
But without it, an agent can be very confident while doing the wrong thing to the wrong work item. This is the sort of failure that looks intelligent right up until somebody has to clean it up.
A lot of AI product work is deciding this order before the agent does anything:
- What is the primary evidence?
- What is useful but secondary evidence?
- What happens when they disagree?
If the answer is "the model will figure it out," then there is no answer yet.
Filters should not become invisible vetoes
Another system had to read visual documents. The first version used keyword rules to decide which pages were worth sending to a vision model.
It saved calls. It also missed pages that mattered.
Floor plans and product diagrams do not always contain the vocabulary a text filter expects. A page can look empty to a keyword matcher and still be the whole reason someone uploaded the document.
The fix was not to throw away heuristics. Heuristics are useful. They became a hint instead of a gatekeeper.
The vision model looked at every page. The classifier still offered a prior, but it no longer had the power to quietly exclude a page from inspection.
That distinction matters more than it sounds.
A lot of AI systems have hidden rules that decide what never reaches the model, the reviewer, or the user. Those rules are often more dangerous than a visible model mistake because nobody notices them until a real case falls through the floor.
If a filter can exclude something important, it needs to be visible, measurable, and easy to challenge.
A red evaluation is not a diagnosis
Agent evaluations are useful. They are also easy to misuse.
A test fails and everyone wants one answer: did the model get worse?
Sometimes it did. Sometimes the test hit an old server. Sometimes the check was brittle. Sometimes the response changed a little but still did the useful thing. Sometimes the response passed every format check while failing the user.
Those are different problems. Treating them as one number turns evaluation into theatre.
The work is to classify the failure before fixing it.
Was the answer unsupported by the data? Did the system promise an output it did not deliver? Did the environment run an old build? Did the assertion mistake a changing answer for a broken one?
Only one of those is a model-quality problem. The others still matter, but they need different owners and different fixes.
This is why I do not trust a dashboard that says an AI system is 94 percent reliable. Reliable at what? Against which cases? With which version running? What happened in the six percent?
The uncomfortable cases are the useful ones. They tell you where the system stops being a demo and starts becoming work.
A work agent needs boundaries
A chatbot can answer a question and disappear. A work agent leaves traces.
It creates tickets. It moves work between states. It schedules a reminder. It comments on something another person now has to deal with.
That changes the job.
The agent needs a clear scope. It needs to know what it can do without asking, what needs confirmation, and what it should refuse. It needs a way to expose its own state. It needs manual controls when automation is wrong, late, or simply annoying.
It also needs a human who owns the outcome.
This is not an argument for slowing everything down. I want agents to do more of the boring work. I want them to search, draft, classify, test, summarize, and move obvious things forward.
But speed makes bad assumptions travel faster too.
The useful version of an AI worker is not one that acts everywhere. It is one that acts inside a boundary people understand.
What changes in the AI era
The people who do well with AI will not be the ones with the most elaborate prompts.
They will be the ones who can turn a vague request into a system that has a source of truth, a visible decision path, a way to recover, and someone responsible when it gets things wrong.
That sounds less glamorous than an autonomous software factory. It is also closer to what survives contact with real work.
Models are getting better at producing an answer. The rest of the system still has to decide whether that answer should do anything.