Context Engineering for AI Agents: Why the Hard Part Isn’t the Context Window

By: Lakshay Nasa

09.16.26

Quick definition: What is context engineering for AI agents?

Context engineering for AI agents is the practice of assembling the instructions, tools, and validated information an agent needs at each step of a task. It covers both what enters the model’s context window at runtime and where that information comes from: how it is ingested, defined, governed, and approved before an agent can retrieve it.

Most writing on context engineering for AI agents is about the context window: What to put in it, what order to put it in, and how to compress it when it fills up. That’s real work, but it isn’t where enterprise agents fail. They fail because the context they retrieved was never correct to begin with.

The agent found a table, read its schema, wrote fluent SQL, and returned an answer built on a staging table nobody has trusted since last quarter. No amount of window curation catches that. The fix sits upstream, in the context platform that governs what context an agent retrieves in the first place.

What does context engineering for AI agents actually cover?

Context engineering is the successor to prompt engineering, and the runtime half of the discipline is in good shape: Anthropic‘s engineering team frames the core constraint clearly in its guide to effective context engineering: context is a finite resource with diminishing returns, and models lose precision as the window fills. The goal is the smallest set of high-signal tokens that produces the outcome you want, which means treating the system prompt, tool calls, external data, and conversation history as a single budget rather than as separate concerns.

These four techniques do most of the work:

Compaction, which summarizes a conversation approaching the window limit and restarts from the summary
Structured note-taking, where the agent writes state to a file and pulls it back in later
Sub-agent architectures, where specialists explore deeply and return condensed findings
Just-in-time retrieval, where the agent holds lightweight references and loads data on demand

If you want the mechanics, we cover them in what a context window is and context window optimization.

All four techniques share an assumption: Each one manages information that is already correct, already defined, and already reachable. In a coding agent working over a repository, that assumption usually holds (the files are the truth). But in an enterprise, it almost never does.

Where the runtime techniques stall

A coding agent reads a repository, where the files are the record and there is one place to look. An enterprise data estate offers no equivalent. It holds thousands of sources, most of them thinly documented, several of them contradicting each other, and nothing on the surface marking which one to believe. That can change what failure looks like.

An agent can retrieve exactly the right row from exactly the right table and still be wrong, because the definition attached to that table is wrong, stale, or contested. Retrieval accuracy and answer accuracy are not the same measurement.

These are the failures curation cannot reach:

Two teams define “active customer” differently, and both definitions are live in the warehouse
A raw staging table and a production semantic view look nearly identical to an agent reading schema alone
Deprecated columns sit alongside current ones with nothing marking the difference
The institutional knowledge that resolves any of the above lives in people, not systems

Every one of these produces a confident, fluent, wrong answer. And a wrong answer that looks right is worse than an obvious failure, because it can slip by undetected.

What 20,000 datasets look like to an agent

Miro ran into this at full scale. It connected Claude Code directly to Snowflake to let business users query in natural language across more than 20,000 datasets spanning Snowflake, dbt, and Looker. Simple questions worked. Then the wrong tables started appearing.

Measured against an evaluation bank of 900 questions written by hand by data experts across finance, marketing, and product, accuracy came in below 40%. A question as basic as asking for active users would return something plausible built on the wrong join.

Analytics accuracy is not a code generation problem. LLMs are good already at writing SQL. The problem is context and routing.
Ronald AngelLead Product Manager, Miro

When the team analyzed why evaluations were failing, the most common cause was the agent pulling from raw tables that were not ready for use: The model wasn’t the bottleneck; it was the supply of trustworthy context.

Miro closed the gap in four steps:

A governed metadata layer between the orchestrator and the warehouse: Entity descriptions, glossary terms, column definitions, lineage, and example queries, exposed to the agent so it could understand what each model represented rather than only what it contained. Accuracy rose to roughly 65%.
Question decomposition and parallel search: Business questions rarely resolve to a single entity, so each one was split into its constituent facts and dimensions and resolved in parallel before SQL generation.
A ranking framework built on five trust signals: Proximity to semantic views, downstream lineage into Looker, a usage quality gate, data product membership, and decomposed joins, together surfacing the most trusted table rather than the most popular one. Accuracy reached about 85%.
Documentation curated where it mattered: Critical data products documented with business definitions, dimensions, and measures, deprecated columns tagged explicitly, and the remaining assets left deliberately out of scope. Accuracy crossed 90%.

What’s notable here is that none of those four steps changed the model, the prompt, or the context window strategy.

Bar chart titled "From under 40% to over 90% — in four moves," showing the accuracy of Miro's analytics agent rising across four configurations: direct to Snowflake, under 40%; plus DataHub metadata, 65%; plus semantic signals, 85%; plus docs and semantic views, over 90%. — Trustworthy context took the accuracy of Miro’s analytics agent from under 40% to over 90%.

Seven characteristics of a context engineering program that scales

Getting one agent working is a project. Getting context to serve every agent in the organization is a program, and the difference is something we can put measurements behind:

In Context Engineering for Agentic AI, a 2026 study of 285 data, AI, IT, and business stakeholders, BARC found that 42% of organizations qualify as context leaders. Those leaders were four times as likely as everyone else to also qualify as AI leaders, with strategic, executive-led, governed AI programs in place: 49% against 12%. BARC is careful to note that this does not prove causation, and that context engineering appears to advance alongside a disciplined overall approach to AI rather than causing it.

The same study lays out seven characteristics that a context engineering program needs.

1. Controlled

Governance has to travel with the context itself, and the reason becomes obvious the first time an agent asks for something a user should not see. If access rules live inside each agent instead, every new agent becomes a fresh opportunity to get permissions wrong. Privacy, accuracy, and protection of intellectual property need to be properties of the context, not of whatever happens to be reading it.

2. Flexible

Definitions move. Teams reorganize, metrics get redefined, and new sources land faster than anyone documents them. Enterprise context is dynamic whether or not anyone planned for it. What matters is the cost of changing a program after the business has already changed underneath it, and when every redefinition needs an engineering ticket, that cost compounds until the program is describing a company that no longer exists.

3. Modular

Add a source, retire a model, or swap an embedding approach, and the rest of the program should not need to know. Without that, every agent ends up with its own context pipeline, which multiplies the maintenance surface and all but guarantees that two agents will eventually answer the same question differently because someone fed them different definitions of the same metric.

4. Open

Portability across data platforms, AI/ML tools, and large language models is a hedge, and what it hedges against is the pace of the ecosystem. Context wired into one vendor’s agent framework has to be rebuilt when that framework changes, and frameworks are changing constantly. The work of establishing what a metric means should outlive whatever is consuming it this year.

5. AI-guided

The people holding the definitions that matter most are the ones with the least time to write them down. That is the central difficulty in getting context out of domain experts, and it is why assistance has to reach every persona rather than only the data team. If contributing a definition means learning a metadata tool first, the definitions do not get contributed.

6. Integrated

Trust is rarely inferred from a single signal. An analyst deciding whether to use a table weighs who owns it, whether it is fresh, what feeds it, and who else depends on it, more or less at once. Agents need the same combination, which is why technical metadata, semantic definitions, lineage, and usage signals are worth considerably more joined up than they are sitting in separate systems.

7. Multimodal

Enterprise context is not only relational. Definitions live in Confluence and Notion, decisions live in tickets and threads, and relationships live in graphs, so a program that only reaches the warehouse reaches roughly half of what an agent needs to answer a real question. Tables, vector databases, and knowledge graphs have to feed one program rather than three parallel ones.

BARC also found that fewer than half of organizations manage context beyond a single agent or team. That number is the scaling problem stated plainly, and the difference between context engineering as a project and context management as an organizational capability.

What supplying validated context actually requires

If you’re the one who has to make an agent give trustworthy answers over thousands of tables you did not build, the work starts well before anything reaches the model. Each piece has to be in place before the first request, and each one depends on the piece before it.

Ingestion across the whole estate: Integrations spanning more than 150 sources, covering warehouses, lakes, BI platforms, and unstructured sources including Confluence and Notion, with everything chunked, embedded, and retrievable in real time. The unstructured half matters as much as the structured half, because that is where definitions and decisions are usually written down.
Semantics derived from what the warehouse already produces: Query history, dbt projects, and BI definitions are technical exhaust that most organizations never read. In DataHub Cloud, they are converted into a structured semantic index that gives agents validated query patterns rather than bare schema: the joins, filters, and aggregation logic that have already answered similar questions. This is what compresses semantic modelling from a multi-hour documentation workshop into a short review.
Expert validation as a workflow: In DataHub Cloud, domain experts review, correct, and approve machine-proposed context before any agent sees it, and can simulate the effect of a change on text-to-SQL results before publishing. The design principle is that experts confirm rather than create, which is the only version of this that survives contact with a busy calendar.
Activation everywhere: Validated context reaches Snowflake Intelligence, Databricks Genie, Claude, Cursor, and custom agents through an MCP server, pre-built skills, and APIs. Build once, activate everywhere. (Learn more about how to build a context layer and MCP context management.)

Pinterest shows what this looks like at scale. It faced more than 400,000 ungoverned tables with 500 more created daily and no lifecycle policy. Tiered governance combined with AI-generated documentation cut manual effort by roughly 70% and produced 100,000 curated, AI-ready assets. The analytics agent built on that foundation reached 10 times the usage of the next-most-used internal agent, with 40% analyst adoption in two months.

Pinterest’s engineering team has documented the text-to-SQL architecture independently, and it required exactly this combination of technical metadata and business semantics.

Where automated context generation stops

Deriving definitions from schema, query history, and existing documentation covers more ground than most teams expect. Where it stops is always the same place: meaning that was never written down anywhere.

FIS builds software for more than 20,000 bank clients and processes 75 billion transactions a year on mainframe systems whose column names carry no semantic meaning at all. Ontology derivation in Databricks Genie handled part of the problem.

In any large enterprise, the real knowledge is gonna be tied up in people’s heads, and I don’t know anybody that’s writing an adapter yet that can go into someone’s brain and pull that out.
Frank ShowalterVP of Data Engineering and Governance, FIS

That’s the ceiling on generation, and no model release moves it. Genie writes the SQL. Something else has to ground that SQL in definitions, ownership, and validated query patterns that a mainframe column name cannot supply on its own. For FIS, that meant capturing the business and technical context together, with domains and ontologies attached, in a form the organization could share.

Which is the whole argument, compressed. Curating the context window is necessary and it isn’t sufficient. Every technique in the published literature operates on information that someone, somewhere, has to have made correct first. So getting reliable results from agents requires really treating the supply of validated context as engineering work, and not just assuming it away.

FAQs

Context engineering for AI agents is the practice of assembling the instructions, tools, and validated information an agent needs at each step of a task. It has two halves. The runtime half governs what enters the context window and in what order. The supply half governs where that information comes from, how it is defined, and who has approved it before an agent can retrieve it.

Retrieval-augmented generation (RAG) solves reach, not correctness. It connects an agent to information outside its training data, but it does not establish which of several conflicting definitions is authoritative, whether a table is production-grade, or whether a column has been deprecated. In an enterprise, those questions cause more agent failures than retrieval quality does. RAG works well when it retrieves from a governed source and poorly when it retrieves from an ungoverned one.

Through a context layer that sits between the agent and the data platform, usually exposed over an MCP server. Miro uses this pattern: Claude Code orchestrates question decomposition, DataHub supplies governed metadata including entity descriptions, glossary terms, lineage, and example queries, and Snowflake Cortex executes the SQL. Without that layer, a coding agent connected directly to a warehouse sees schema and nothing else, which is why Miro’s accuracy sat below 40% before the change.

Both, working on different halves. AI engineers own the runtime: prompts, tools, memory, and the agent loop. Data engineers own the supply: ingestion, semantic definitions, lineage, and governance. Most published guidance addresses the first group, which is why the second half is often unowned. DataHub covers the data engineering side in the data engineer’s guide to context engineering.

Build an evaluation bank of real business questions with known-correct answers, and score against it continuously. Miro used 900 questions written by domain experts across finance, marketing, and product, run daily and refreshed with each model change. Aggregate accuracy is the headline number, but the more useful diagnostic is the failure breakdown: how many errors trace to routing, to sparse documentation, or to the model itself.

No. Automated derivation from query history, dbt projects, and BI definitions covers a large share of the work and should be used for all of it. But semantic meaning that was never written down anywhere cannot be inferred from a schema, which is why expert validation remains part of every program that works at scale. The realistic target is generation proposing and experts confirming, not experts authoring from scratch.

A data catalog is built for humans searching for data. A context platform is built to serve validated context to agents at machine speed. The practical differences are activation, validation, and freshness: an MCP server, skills, and APIs that agents consume directly, a DataHub Cloud workflow where domain experts approve machine-proposed definitions before agents see them, and event-driven updates that keep context current as the estate changes.

The runtime literature names four. Context poisoning, where an error enters the context and compounds as the agent builds on it. Context distraction, where accumulated history crowds out fresh reasoning. Context confusion, where irrelevant tools or documents pull the model toward the wrong action. And context clash, where contradictory material leaves the agent stuck between assumptions. All four describe what happens inside the context window during multi-step tasks. A fifth category sits upstream and is more common in enterprises: the retrieved context was accurate, well-formed, and wrong, because the definition behind it was never validated.

Not reliably. A context window acts as the model’s short-term memory during inference, holding instructions, retrieved documents, tool calls, and conversation history for the current task. It is finite, and expanding it does not necessarily improve performance. Research on long-context behaviour consistently finds that accuracy degrades as input grows, well before the advertised limit, and that models attend less reliably to material buried in the middle of a long context than to material at the start or the end. Efficient management matters more than raw size, because unnecessary token usage crowds out what the agent actually needs.