Agent Context Management: How to Manage Context for AI Agents After Launch

Most advice on agent context management covers one agent at a time: sequence the context in the window, compact the history, retrieve on demand. That work matters, but it assumes the context the agent retrieves is already correct. Inside an enterprise, that assumption breaks early. A second agent needs the same revenue definition. A third team writes its own. Two agents give two answers to the same question, and nobody owns the fix.

The teams getting agents into production treat context as something they run continuously. They scope agents to a domain, define what a correct answer looks like before adding anything, keep context current from the systems that already hold it, and route every gap an agent finds back to someone who can fix it. A context platform is the infrastructure that makes that work repeatable across every agent.

“Agentic AI fails without business context. Agents can turn an inaccurate answer into a bad decision or action.”

Kevin PetrieVP of Research, BARC US

Agent context management happens at two levels

Agent context management is the ongoing work of making sure every AI agent has accurate, current, and governed context to act on. It happens at two levels, run by different people on different timescales:

1. Inside each agent, the harness manages the context window: The harness is the code and framework that run the agent loop around a large language model (LLM). Agent builders use it to decide what enters the window on each turn, from the system prompt and tools to retrieved documents and conversation history, and how to keep it focused as a task runs. Techniques include compaction, just-in-time retrieval, sub-agents for context isolation, and notes stored outside the window. Anthropic‘s engineering team describes this work as the natural progression of prompt engineering and treats context as a finite resource with diminishing returns. This level operates per request, inside one agent.
   Learn more about context window optimization.
2. Across agents, the organization manages the context itself: Data and domain teams decide what the business’s definitions are, which sources are trustworthy, and who answers for them. Then they serve that context to every agent the same way. This level operates per domain, over weeks and months.

Both levels are necessary, but enterprise agents tend to fail on the second. Runtime techniques decide how much context reaches the model and in what order, but they work on whatever context retrieval hands them. If the definition of “active customer” is stale or contested, compaction preserves the error faithfully.

When each team handles the second level inside its own agent, it rebuilds the same context its own way. Definitions drift apart, and two agents give different answers to the same question. That’s the core difference between context engineering and context management. Context engineering still matters inside each agent. It just shouldn’t be where your context lives.

“In every enterprise I talk to, the harder problem isn’t engineering the context. It’s having context worth engineering in the first place.”

Shirshanka DasCo-Founder and CTO, DataHub

How to manage context for AI agents in six steps

Managing context across agents works best as a loop you repeat for each domain. Evals run through every step, so each change to context gets measured rather than assumed. The Build a Data Agent guide in the DataHub docs follows the same loop, and it’s a useful worked example if you want to see each step in a real product.

1. Scope agents to one domain, with an owner

Start with one business domain, such as finance, sales, or product, rather than pointing an agent at your whole data estate. A good first domain has questions people ask every week and a team that already knows what the right answers look like.

Next, name an owner for that domain. This is the person or small group who will write or review the evals in step two, approve changes to context, and work through feedback once the agent is live. If nobody will take that on, pick a different domain.

Then decide which tables and documents the agent can see. Limit it to the assets you trust for that domain, so the agent retrieves relevant context instead of the closest-sounding table, and leave everything else out for now. You can widen the scope later. Miro had more than 20,000 datasets across Snowflake, dbt, and Looker. It documented its critical data products and kept the remaining assets out of scope.

Expect to grow in small increments. Pinterest rolled out its governance tiers to five tables first, then 10, then the whole company.

In DataHub, you set this up with Domains, which group data and context by business area. Build a domain-scoped agent first, and add a global agent for your data team once the context underneath it is in good shape. If you’re tempted to model everything before you start, read how to implement an enterprise context layer first.

2. Define evals before you add context

Before you add any context, collect 20 to 50 real business questions from the people who will use the agent. Ask your domain owner or a data expert to write or review the correct answer to each one. These are your evals, and every change you make from here gets measured against them.

Run the evals once as a baseline, and expect a low score. Then open the failures and compare the agent’s SQL with the expert’s, along with the tables and documents the agent found. Most of the time you’ll find missing context, which the next two steps fix.

Before you roll the agent out more widely, agree with your stakeholders on the pass rate it has to reach. The DataHub docs use 85% as an example. Then schedule the evals to run daily.

Miro works this way. It runs 900 questions written by data experts across finance, marketing, and product every day, and updates them with each model change. Its first benchmark, with the agent connected directly to Snowflake, came back under 40%.

“Analytics accuracy is not a code generation problem. LLMs are good already at writing SQL. The problem is context and routing.”

Ronald AngelLead Product Manager, Miro

In DataHub Cloud, Context Evals run on a schedule, with an AI judge scoring each answer. If your agent runs outside DataHub, it can report its answers for grading, so every agent is held to the same standard.

3. Ingest what’s already governed, and generate the long tail

Start with the context you already have. Connect the systems that hold governed definitions:

• Semantic models
• dbt projects
• BI tools
• Glossaries
• Documentation in places like Confluence and Notion

DataHub connects to more than 150 sources and brings technical, operational, business, and organizational context into one graph. (For what each of those four layers does, see AI agent context.)

Next, capture what doesn’t fit in a schema. Runbooks, FAQs, and notes on how to use a metric often hold the most important context an agent needs. They belong in Context Documents, attached to the assets and domains they explain.

Then fill in the long tail that nobody will document by hand. Your query history already records the joins, filters, and metric logic analysts use to answer real questions. In DataHub Cloud, Context Generation turns query history, BI definitions, and existing metadata into context documents for a domain. It works from metadata and never reads the data in your tables. Set it to run on a schedule so your context keeps pace as your data and queries change.

Pinterest did this at scale. Governance tiers reduced 400,000 tables to 100,000 curated, AI-ready assets. AI-generated documentation, lineage-based propagation, and vector similarity search then eliminated roughly 70% of the manual documentation work. Pinterest Engineering has documented how the team indexed query history by analytical intent.

“We treat this history of queries as a library of real solutions written by real analysts.”

Keqiang LiSoftware Engineer, Pinterest

4. Decide where people review context, and where evals do

Your experts’ time is limited, so decide up front where to spend it and let evals check the rest. Use expert time in two places: agreeing on what good answers look like (step two) and reviewing critical context in sensitive domains.

For generated context, choose one of two modes for each domain:

• Publish and verify: Let generated documents go live right away, and let your evals do the checking. If the pass rate holds or rises after a document publishes, keep it. If it drops, revise or remove the document. Use this mode for most domains.
• Review first: Name reviewers who approve new or changed context before agents see it. Use this mode for sensitive domains. Reviewers can run evals against a proposed change before it publishes.

Pinterest drew the line by tier. It used AI to draft documentation across the warehouse and kept human review for Tier 1 tables, the cross-team production assets that matter most. Whichever mode you choose, ask experts to confirm context rather than write it from scratch.

Learn more about human-validated semantic context.

In DataHub Cloud, you set review for each Context Generation job. If you leave the reviewer list empty, requests go to the owners of the tables and domains involved, plus your admins.

5. Serve the same context to every agent through open interfaces

Once your evals show the context works, connect your agents to it. Pick the pattern that fits how your teams already work:

1. Use a built-in agent: If people just need answers, point them to Ask DataHub in DataHub Cloud, Slack, or Teams. There’s nothing to build.
2. Hand data questions to a DataHub agent: If you already have an agent in Claude, ChatGPT, or a LangChain app, have it pass data questions to a DataHub agent that applies your domain’s instructions and scope.
3. Give your agent DataHub’s tools: If your agent builders want more control, connect the agent to the DataHub MCP server or the Agent Context Kit and let it do its own reasoning.

Choose one of the first two if you want your data team to improve answers in one place for everyone. Choose the third if your agent builders need more say in how data is found. The Activate Context docs compare the three in detail.

Whichever pattern you choose, serving pre-validated context instead of raw schema also lowers token usage per question.

Don’t confuse context access with reliable context. Model Context Protocol (MCP) gives agents reach, not trust, and a consistent interface to inconsistent context still returns wrong answers. Reliability comes from what sits behind the server: owned definitions, trust signals, and evals.

Learn more about MCP context management.

6. Close the loop with feedback, then expand

Your context starts to drift the day the agent goes live. Metrics get redefined, sources get retired, and people ask questions your evals never covered. Feedback is how you keep up.

Turn on feedback so agents record a note whenever they hit a gap. Each note falls into one of three types:

• Missing context: The agent couldn’t find what it needed
• Incorrect context: What it found was wrong or out of date
• Conflicting context: Two sources disagree

Agents also record answers a user corrected, which is often how a gap first surfaces.

Ask your domain owners to review those notes on a regular schedule. For each one, confirm the gap is real, then fix the context behind it: write the missing definition, clarify a description, or deprecate an outdated table. Let people outside the data team help too. Anyone can propose an edit, including by asking an agent in Slack or Teams, and the proposal goes to the domain’s owners.

Because each fix lands in shared context, every agent that uses the domain benefits. When your organization changes, such as a new source, a retired system, or a redefined metric, update the context and the evals together. Once the domain is stable and in regular use, start the loop again with the next one.

In DataHub Cloud, Context Feedback collects these notes, which never include data values or query results. For why agents should inherit each other’s corrections, see AI agent onboarding.

Who does what, and how often

Poor context management rarely fails loudly. It usually starts when everyone assumes someone else owns the context. Here’s a responsibility map for a single domain:

Activity	Owner	Cadence or trigger
Write and review evals	Domain expert	Before adding context, then as new questions surface
Run evals	Automated	Daily
Update evals	Domain expert with the data team	Each model change, and each organizational change (new source, retired system, redefined metric)
Ingest governed sources	Data and platform team	Scheduled, or as sources change
Generate context for the long tail	Data and platform team	Scheduled
Review generated context in sensitive domains	Named reviewers or domain owners	When context is proposed
Review feedback and proposed edits	Domain owners	A regular cadence the domain sets
Manage the context window inside each agent	Agent builders	Per agent, as the agent changes
Expand to a new domain	Data team with the new domain’s owner	When the current domain is stable and in regular use

The split that matters most is between the last two kinds of work:

• Agent builders still own each agent’s runtime: prompts, tools, memory, and the agent loop
• Domain owners and the data team own the context those agents draw on

For more on dividing responsibility, see who owns context management.

How do you know agent context management is working?

Agent observability and context evals answer different questions:

• Observability traces a run: which tools the agent called, where it stalled, and whether it reached its goal
• Context evals test the context: given what’s in your shared context, does an agent answer real business questions correctly?

An agent can run cleanly from start to finish and still return the wrong number, because the definition it retrieved was wrong. Production agents need both.

These signals show that context management is working:

• The pass rate holds at or above target: A falling pass rate is the earliest warning that context is drifting.
• Failures point somewhere specific: Breaking failures down by cause (routing, sparse documentation, or the model itself) tells you what to fix next.
• Flaky evals get fixed: An eval that alternates between passing and failing usually signals conflicting documents or an ambiguous question.
• Every domain has owners reviewing feedback: Feedback that sits unreviewed is context that’s drifting.
• Different agents give the same answer: Consistency across agents is the clearest sign that context lives in one place.

The discipline tracks with AI outcomes more broadly: In BARC‘s 2026 study of 285 data, AI, IT, and business stakeholders, context leaders were four times as likely as other organizations to also qualify as AI leaders (49% versus 12%). BARC notes that this shows association, not causation. And in DataHub’s State of Context Management Report 2026, 83% of IT and data leaders agreed that agentic AI can’t reach production value without a context platform.

You don’t need to model your whole data estate to start managing context for AI agents. Pick one domain with an owner who cares about the answers, write 20 questions with known-correct answers, and run a baseline. The number you get back tells you where to start.

To see the full loop in practice, follow the Build a Data Agent guide.

FAQs

How do you manage context for AI agents?

Managing AI agent context happens at two levels. Context engineering assembles dynamic context for each agent turn, deciding what goes into the window. Context management makes sure what goes in is accurate, current, governed, and consistent across every agent. Effective context engineering depends on the second, because the best assembly logic still fails if the underlying context is wrong. At enterprise scale, that second job needs a context platform that unifies technical, operational, business, and organizational context in one graph and delivers it to agents through standard interfaces. Learn more about context management.

How many eval questions do you need to test an AI agent’s context?

Start with 20 to 50 real business questions, each with an answer written or reviewed by a data expert. That’s enough to set a baseline and agree on a target pass rate for one domain. Grow the set as real usage surfaces new questions. Miro, for example, runs 900 expert-written questions across finance, marketing, and product every day.

What’s the difference between agent observability and context evals?

Agent observability measures how a run behaved: tool calls, latency, and whether the agent completed its task. Context evals measure whether the context an agent relies on produces correct answers to real business questions. An agent can pass every observability check and still return a wrong number built on a stale definition. Production agents need both.

What’s the difference between context rot and stale context?

Context rot happens inside a single run. As the context window fills, the model recalls what’s in it less reliably, which is why agent builders trim context bloat, compact history, or restart in a new context window. Stale context builds up across agents and over time, when a definition, table, or owner changes and nobody updates the shared context. Runtime techniques address the first. Ongoing context management addresses the second.

Should AI agents be scoped to one domain or have access to all your data?

Start with domain-scoped agents. A domain such as finance or sales is small enough for its experts to define correct answers and review context, and narrow enough to keep agents on relevant data and away from untrusted tables. A global agent for the data team can sit alongside domain agents once the context underneath it is in good shape.

Who reviews AI-generated context before agents use it?

It depends on the domain. In most domains, generated context can publish right away, with evals confirming that it improves answers. In sensitive domains, named reviewers or domain owners approve changes first. In DataHub Cloud, review is configured per Context Generation job, and if no reviewers are set, requests go to the owners of the tables and domains involved, plus admins. Either way, the goal is for experts to confirm context rather than write it from scratch.

How do you keep AI agent context from going stale?

Maintain context on a schedule. Re-ingest governed sources and regenerate context from query history regularly, run evals daily so a falling pass rate flags drift early, and have agents record missing, incorrect, or conflicting context as they hit it. When a metric is redefined or a system is retired, update the context and the evals together.

How do you manage context across multiple AI agents?

Manage it once, in shared infrastructure, rather than separately inside each agent. That holds whether agents work independently or together in multi-agent systems. When every team builds its own context strategies for its own agent, definitions drift and agents give different answers to the same question. A context platform holds one governed version of each definition, serves it to every agent through standard interfaces such as MCP, and applies each fix everywhere at once.