Data Lineage for AI: What Changes When Agents Read Your Lineage Graph

For most of its history, data lineage had one kind of reader: An engineer opened the graph during an incident, or an analyst traced a metric before a board meeting. Those readers brought judgment with them, so when something in the graph looked off, they usually noticed.

AI agents in production now read lineage too, often many times a day, and they read it differently than those human readers. That changes what lineage needs to do and how good it must be.

Why data lineage is important for AI systems

Data lineage is important for AI systems because agents act on it without stopping to question it. Data lineage tells an agent how data flows between tables, which source is authoritative, and what sits downstream. An agent uses that to decide where to get its answer, whereas teams use it afterward to work out where the answer came from.

That gives lineage two jobs in an AI system:

1. At query time, it helps an agent pick a source and check that the source is healthy
2. After the fact, it lets anyone trace an answer back through every system it touched

Neither job replaces what lineage already does well: It still underpins trust, compliance, root cause analysis, and debugging, and automated column-level capture still matters. What changes is the cost of getting lineage wrong.

Wrong lineage is worse than no lineage when an agent reads it

A person who spots a missing edge in a lineage graph goes and asks someone. That gap prompts a Slack message, a look at the query, or a conversation with the table’s owner. Judgment fills the gap.

An agent has no such instinct. It routes on the graph as given. If an edge is missing, the agent treats the dependency as absent. If an edge points to the wrong upstream table, the agent trusts it. Either way, it returns an answer that reads just as confidently as a correct one.

An agent with no lineage is at least visibly working blind, and the team running it knows that. But an agent with wrong lineage looks well informed. That makes the way lineage is derived an AI risk in its own right, and it’s why accuracy matters much more now than it did when people were the only readers.

Where lineage fits: one input to the context layer

Lineage on its own tells an agent where data came from and what depends on it. But here’s what it doesn’t say:

• Whether that data is fresh
• Whether a quality check failed this morning
• Who owns it, and whether it holds sensitive data
• What the organization means by “active user” (or any other term)

Those signals live in data quality assertions, ownership records, business glossaries, and access policies.

Lineage becomes useful to an agent when it sits in the same graph as those signals. That combined graph is the context layer: the architecture that gives agents the technical, operational, business, and organizational knowledge they need to answer correctly. A context platform is the infrastructure that builds and maintains it. A lineage path that also carries a failing assertion and a named owner tells an agent something it can act on. The same path alone is a road map with no traffic information.

Data leaders increasingly see that layer as a prerequisite. In DataHub’s State of Context Management Report 2026, 83% of IT and data leaders agreed that agentic AI can’t reach production value without a context platform. Lineage is a critical input to that layer, and it’s one input among several.

The two jobs lineage does for AI agents

Lineage works on both sides of an agent’s answer. In most content on AI and data lineage, the second job (the audit trail) gets the most focus. But for agents in production, the first job is where accuracy is really won or lost.

1. Before the answer: routing and health checks

At query time, lineage helps an agent choose where to look. A warehouse with thousands of tables usually holds several candidates for any question: a raw ingestion table, a staging model, a derived table built for a one-off analysis, and a certified production view. Names don’t reliably separate them. Lineage does. A table close to the semantic layer, feeding dashboards business users actually rely on, is a stronger candidate than one that feeds nothing.

Lineage also lets an agent check the path before it answers. If an upstream table has an active incident, or a freshness check failed overnight, lineage connects that problem to the table the agent is about to query. The agent can flag the issue, choose another source, or decline to answer. Without that check, the problem surfaces only after someone has acted on the number.

Miro saw what happens without this. When it connected Claude Code directly to Snowflake across more than 20,000 datasets, its analytics agent couldn’t tell staging tables from production views and routed to the wrong table in over 30% of cases. Accuracy on Miro’s 900-question benchmark sat below 40%.

“When we started analyzing why zombie evals were failing, one of the most common reasons was because it was getting the information from the raw table that was not ready.”

Ronald AngelLead Product Manager, Miro

Miro put DataHub between the agent and the warehouse and built a ranking framework on metadata signals. Looker lineage was one of them: data models with downstream lineage into Looker, where most of Miro’s metrics surface, ranked higher. Wrong-table routing fell below 10%, and accuracy passed 90% in a mode scoped to governed data products.

Lineage was one signal in that framework, alongside schema position, usage, data product membership, and curated documentation. The gain came from those signals working together, which is exactly what the context layer is for.

2. After the answer: tracing what the answer depended on

When an agent’s answer is challenged, lineage lets a team trace it back through every table, transformation, and system the data passed through. That’s how teams debug a wrong number, size the blast radius of a bad upstream change, and show auditors where data came from.

Provenance gaps already cause real problems. In the State of Context Management Report 2026, 53% of IT and data leaders said they frequently or very frequently face AI-related compliance issues caused by a lack of data provenance.

Block shows what fast tracing looks like across a sprawling estate. It manages more than 50 data platforms under strict financial compliance requirements. Its engineers paired Block’s open source agent, Goose, with the DataHub MCP Server so they could verify tables, assess downstream impact, and find data owners during incidents through conversation.

“Something that might have taken hours, or days, or even weeks turns into just a few simple, short conversation messages.”

Sam OsbornSenior Software Engineer, Block

Data lineage covers the data side of that trace. Tracing the prompts an agent received, the tools it called, and the actions it took is related but separate work, closer to LLM observability. A full audit of an AI system usually needs both.

Related:

• For point-in-time questions, such as how a dataset was classified when a model used it, see metadata lineage
• For lineage through feature engineering and model training, see data lineage for machine learning

What lineage has to get right before an agent can act on it

Both of the above jobs depend on the graph being right. Three things decide that:

1. Whether lineage follows data across tools
2. How accurately each edge was derived
3. Whether it keeps up as systems change

Lineage has to cross tool boundaries

Native lineage in Snowflake, Databricks, or BigQuery is accurate within its platform and stops at that platform’s edge. An agent answering a business question rarely stays inside one platform. It pulls from a warehouse table built by a dbt model, fed by an ingestion tool, and surfaced to data consumers in a BI dashboard. Lineage that stops at each boundary leaves gaps in exactly the places an agent crosses.

Cross-system lineage fixes that, and independent data catalogs now offer it widely, usually with automated column-level capture. That makes it the baseline, which leaves a harder question: how well lineage handles the queries that are difficult to parse.

Lineage has to be derived accurately

Most automated lineage comes from SQL parsing, which works out which source columns feed which output columns. Simple queries parse cleanly. The hard cases show up in every real warehouse:

• Unqualified columns: A query joins three tables and selects customer_id without saying which table it comes from.
• SELECT *: A view selects every column from another view, which selects every column from a table. The column list exists only in the schemas.
• Temp tables: A pipeline writes to a temporary table, transforms it, loads the result into the final table, and drops the temp table.
• Table swaps and renames: A blue/green deployment builds a new table and swaps it into place, so the name downstream queries use now points at different data.
• Multi-dialect estates: One organization runs Snowflake, BigQuery, and Postgres, each with its own SQL syntax.

A parser that reads each query in isolation, without the underlying table schemas, has to guess which table an unqualified column belongs to, and it can’t expand SELECT * at all. Temp tables and swaps break the chain unless something tracks statements across queries. Whatever the parser decides, the agent inherits, and a guessed edge looks identical to a resolved one. Breadth gets you a graph; accuracy decides whether an agent should act on it.

Lineage has to stay current

Lineage also goes wrong when it falls behind the systems it describes. A table gets deprecated, a new upstream source is added, or a team refactors its data pipelines. An agent reading last week’s lineage routes to the deprecated table or misses the new dependency, and answers with the same confidence. Lineage for AI has to update as systems change.

How to test lineage before you trust an agent with it

You can check lineage accuracy yourself during a proof of concept. Set up the four cases below in a test environment and run each of the data lineage tools you’re evaluating against them. Then compare the edges each tool returns with the edges you know exist.

Test case	What it tests	What to count
An unqualified column in a multi-table join	Whether the parser uses table schemas to resolve which table a column belongs to	Correct column-level edges vs. guessed or missing edges
SELECT * through three stacked views	Whether the parser expands wildcards from known schemas at every layer	Column-level edges from the top view back to the base table
A table rename or swap mid-pipeline	Whether lineage survives a blue/green deployment pattern	Downstream assets still linked to the correct upstream data after the swap
Nested CTEs and subqueries across two or more warehouse dialects in one estate	Whether parsing holds across SQL dialects and complex query structure	Edges returned per dialect vs. edges that exist

A missing edge is a failure. A wrong edge is a worse one, because an agent will act on it. To check that lineage reaches every system in your stack, use the scope test in our end-to-end data lineage guide.

How DataHub builds lineage agents can act on

DataHub starts with getting the edges right, because every other use of lineage depends on them. From there, lineage lives in the same context graph as quality, ownership, and business meaning, and agents can query all of it together.

Lineage you can audit

DataHub’s SQL parser is schema-aware. Because it runs inside the metadata graph, it can resolve any table’s schema at parse time. That’s how it qualifies unqualified columns in joins and expands SELECT * from the schemas registered in DataHub instead of dropping the edge. DataHub’s SQL parsing aggregator also resolves lineage across temp tables and table renames, and across table swaps on Snowflake.

In DataHub’s published benchmarks, the parser generates lineage with 97 to 99% of queries. That’s DataHub’s own benchmark, and the parser is open source, so any team can run its own query corpus against it before committing. It’s built on the open source sqlglot library, and DataHub’s technical write-up explains how schema-aware parsing works. Under audit, you can show how your lineage was derived, not only what it says.

The same documentation lists the parser’s limits, including SQL constructs it handles on a best-effort basis and some it doesn’t support. Because the parser relies on schemas in the graph, it also depends on those schemas being current.^[a]

Lineage in the context graph

DataHub captures lineage automatically through connectors to 150+ data sources, generates column-level lineage across the major platforms through SQL parsing, and keeps the graph current through event-driven and batch ingestion.

In that graph, lineage sits alongside the signals an agent needs to judge a path: quality assertions, Freshness Assertions (DataHub Cloud), incidents, ownership, glossary terms, and policies attached to the nodes along it. In DataHub Cloud, glossary terms and documentation also propagate downstream through lineage, so business meaning follows the data an agent can reach.

“They can just look at the lineage, and they can find if there is any node that has an active incident there, find their owners, and reach out to them.”

Sherin ThomasSoftware Engineer, Chime

Lineage agents can query

The DataHub MCP Server exposes lineage, along with the rest of the context graph, to Claude, Cursor, Copilot, and custom agents. The Agent Context Kit gives agent builders tools such as get_lineage to retrieve upstream or downstream lineage mid-task. In DataHub Cloud, Ask DataHub uses lineage relationships to answer questions and run impact analysis.

Trustpilot uses lineage this way across more than 500,000 cataloged data assets. Engineers planning a migration off self-hosted MongoDB trace downstream impact by clicking through the DataHub UI, asking in chat, or pulling lineage straight into Claude Code or Copilot through MCP.

“Lineage is one of the really big selling points. What people really want to do with that information is usually lineage-based — who do I need to go and ask, who’s querying my table, is it being used?”

Hugo HobsonSenior Data Engineer, Trustpilot

In IDC’s 2026 Business Value study of DataHub Cloud customers, organizations moved 119% more AI and ML models into production, saw 24% lower AI and ML project failure rates, and resolved data-related outages 58% faster.

Run DataHub’s open source SQL parser against your own queries, or book a demo to see lineage working inside the DataHub context platform.

FAQs

How do AI agents use data lineage at query time?

AI agents use lineage at query time to choose and check a source before answering. Lineage shows which tables are certified and close to the semantic layer, and which are raw or abandoned, so the agent can rank candidates. It also connects upstream incidents or failed freshness checks to the table the agent is about to query, so it can flag the issue or route elsewhere.

Does data lineage track prompts, tool calls, or agent actions?

No. Data lineage tracks how data moves and transforms across systems. Tracing the prompts an agent received, the tools it called, and the actions it took is a related discipline, closer to LLM observability. A complete audit of an AI system usually combines both: data lineage shows where the data came from, and observability shows what the agent did with it.

Is native warehouse lineage enough for AI agents?

Usually not. Native lineage in a warehouse such as Snowflake, Databricks, or BigQuery is accurate inside that platform but stops at its edge. Agents answer questions that span ingestion tools, transformation layers, warehouses, and BI platforms, so they need lineage that follows data across those boundaries, along with quality, ownership, and business context attached to it.

How accurate is automated column-level lineage?

It depends on how the lineage is derived. Parsers that know the underlying table schemas can resolve unqualified columns and expand SELECT *, while parsers that read queries in isolation guess or drop those edges. DataHub’s open source SQL parser generates lineage with 97 to 99% coverage in DataHub’s published benchmarks. The most reliable check is to run each tool against your own query corpus and count correct edges.

Can data lineage help with AI compliance requirements like the EU AI Act?

Yes, as one part of the evidence. Lineage documents where data originated and how it was processed, which supports requirements such as the EU AI Act’s Article 10 data governance obligations for high-risk AI systems. Compliance also depends on broader data management practices, including policies, ownership, and documented processes, and specific obligations vary by system and risk category, so confirm requirements with your legal team.

What’s the difference between data lineage and data provenance for AI?

Data provenance records where data originated, who created it, and what authority it carries. Data lineage tracks how data moves and transforms across systems over the data lifecycle. For AI, the two work together: provenance tells an agent whether a source is authoritative, and lineage shows the path from that source to the answer. Learn more about data lineage vs. data provenance.