Why AI Keeps Using the Wrong Business Definition
The problem is usually not that the AI lacks information. It is that your company has not given the definition a maintained structure, owner, and history.
If your AI keeps using the wrong definition of revenue, retention, activation, or an active customer, stop rewriting the prompt for a minute. The prompt may not be the real problem.
An AI can read your data dictionary, meeting notes, dashboards, and warehouse schema and still choose the wrong meaning. Those files contain evidence. They do not necessarily say which definition is approved now, where it applies, who owns it, what changed, or which reports and workflows must reuse it.
I have written before that file access is not the same as understanding your business. This is the narrower operational consequence: the AI reconstructs meaning each time it answers.
The symptomOne question produces several believable answers
Ask, “What was total sales last quarter?” One system counts signed contracts. Another counts orders. Finance uses payments received. All three answers can be technically coherent.
The AI does not fail because it cannot add numbers. It fails because the phrase total sales hides a business decision. If the approved decision is not explicit and connected to the analytical work, the model has to infer it from whatever context it finds.
The same problem appears with retention. Does a customer count as retained after another order, an active subscription, a product session, or any revenue event? Is the window 30 days, 90 days, six months, or one year? Which definition applies to the board report, the lifecycle campaign, and the product analysis?
- 01
Files and notesUseful evidence
Several explanations may exist, and some may be old or scoped to one team.
- 02
Query logicOne implementation
A dashboard can encode a definition without explaining why it is the company standard.
- 03
Maintained definitionApproved and reusable
Meaning, owner, scope, examples, dependencies, and history travel together.
The AI should retrieve the approved definition and its relationships, not infer authority from the loudest file.
Start with retrieval, but do not stop at retrieval
A better prompt can absolutely improve an answer. Saved instructions, skills, examples, memory, retrieval, a semantic layer, and MCP connections are all useful. If the question is bounded and the definition rarely changes, that may be enough.
For most teams, I would start by building retrieval over the materials that already carry business meaning: approved definitions, metric documentation, meeting decisions, report descriptions, transformation notes, and current owners. Good retrieval, often called RAG, is the quickest way to keep the AI from relying on whichever file happened to be attached to the conversation.
But retrieval answers, “What relevant evidence can I find?” It does not answer, “Which definition is approved, how is it assembled, and where must it be reused?” A RAG index can faithfully retrieve three contradictory definitions. The system still needs an authority model.
Anthropic's guidance on context engineering makes the broader point clearly: useful agent behavior depends on curating and maintaining the context available across instructions, tools, MCP, external data, and history. More tokens are not automatically better. The system has to retrieve the smallest high-signal context for the job.
Find approved definitions, decisions, examples, and owners.
Select the current meaning and state where it applies.
Show the Metric and simple building blocks in business language.
Arrange those blocks into a population, window, and formula.
Open SQL only when a technical reviewer needs the implementation.
The failure begins when the company treats the prompt as the permanent home of business meaning. A golden prompt can repeat an instruction. It cannot guarantee that the Metric behind the dashboard, the customer list, and the automation changed with it.
A semantic layer improves this substantially by governing Metrics and Dimensions. But even a strong Metric definition can sit apart from the customer behavior, Dataset population, report, workflow, decision, and change history around it. The remaining question is not whether the definition exists. It is whether every consequential use can find and reuse the same maintained meaning.
Use a coding agent to build and audit the system
If you are assembling this yourself, I would usually reach for Codex or Claude Code before trying to express the whole investigation as a sequence of narrow point tools. This is not because MCP is bad or because coding agents have a magical understanding of the business. It is because this job requires exploration and iteration.
The official Codex CLI documentation describes a terminal agent that can inspect a local repository, edit files, and run the tools already installed on the machine. Claude Code documents a similar agentic loop: gather context, take action, run tests, inspect the result, and adjust based on what it learns.
That loop matters here. A coding agent can search the definition catalog, modeling code, saved queries, tests, and report configuration; compare how the same term is implemented; create the retrieval layer; add checks; run them; and revise the system when the evidence disagrees. A tool that only says “get metric” or “run query” may be useful inside that loop, but its description does not expose every hidden assumption in the system behind it.
Use the coding agent where it has a legitimate, permissioned workspace to inspect. Use MCP and other connectors for focused access to systems outside that workspace. Then make the approved definition and the result durable so the next answer does not depend on reconstructing the same investigation from scratch.
The practical testTest one disputed definition before building more infrastructure
Choose one term your team already argues about. Total sales, activated account, retained customer, qualified lead, or healthy location are good candidates. Then run this test.
- Ask for the plain-language meaning. The answer should state what qualifies and what does not.
- Ask for the authority. Which approved Metric, Dataset, report, decision, or owner makes this the current definition?
- Ask for examples at the boundary. Show one included and one excluded customer or entity and explain why.
- Ask where the definition is reused. The dashboard, investigation, list, presentation, and workflow should not silently rebuild it.
- Change the definition deliberately. Record who approved the change, why, when it applies, and which downstream uses must be reviewed.
If your current stack passes that test, do not replace it. You may already have the semantic model, retrieval, interfaces, and ownership required for this problem.
If it fails, adding another folder or another chat instruction is a bandage over a structural wound. It may improve the next answer while leaving the company with no dependable way to carry the same meaning into the next report or workflow.
What a maintained definition actually contains
A maintained business definition is not one sentence in a glossary. It connects the business concept to the work that makes the concept true.
What the term means in plain language.
Who or what can qualify.
Which business events establish it.
Which focused transformations produce it.
Which reports and workflows depend on it.
Who reviews changes and when.
This is close to how we built Polyform's business context for AI. Retrieval brings the relevant context to the question, but the context lives as a maintained record rather than a larger document folder. The dedicated expert meets with the customer, turns domain truth into approved Metrics, Dimensions, Activities, definitions, decisions, and Tasks, and keeps those records connected to the product work that uses them. The customer remains the authority on the business. The expert owns making that authority usable and current.
The Activity Schema approach gives the analytical part a reusable grammar. A person should be able to open the Metric, understand its plain meaning, see the Activities and Transformations used to construct it, and understand how those blocks are arranged. SQL remains available when a technical reviewer needs to trace the exact implementation. It should not be the first or only explanation offered to everyone else.
The real fix is not a smarter chat
The model will keep getting better. Better retrieval and better tools will reduce many mistakes. But the company still has to decide what its terms mean, record that decision, connect it to the work, and maintain it when the business changes.
That is the part another layer in chat cannot own for you.
If this is the stop you keep hitting, start with one definition. Make it approved, inspectable, reusable, and owned. Then see whether the next AI answer, dashboard, and workflow all use the same thing. If they do not, you have found the system you need to fix.
Continue the five-part series
- Part 1: Why AI keeps using the wrong business definition.
- Part 2: How to verify an AI-generated data answer.
- Part 3: Why more MCP servers do not finish the workflow.
- Part 4: Why AI can diagnose a broken dashboard but not repair it.
- Part 5: How to turn AI analysis into a reusable dashboard or workflow.
Or use the five-situation guide to find the stop closest to your team.