Real Stories in Production: July 2026 Town Hall Highlights

A demo has to work once. Production has to keep working: accurate as the data shifts, stable under continuous ingestion at scale, and observable enough to catch a pod that was evicted mid-run while the UI still shows the job as running.

That gap, between something that works in a demo and something that holds up in production, was the through-line of the July town hall. We brought three teams that had crossed it and were candid about what it took. Ronald Angel from Miro walked through making a text-to-SQL agent accurate enough to trust. Maksym Oliinyk from Just Eat Takeaway walked through running self-hosted DataHub at millions of metadata entities. And Tim Bossenmaier, a DataHub Champion and Data Architect at Cloudflight, traced the multi-year path from a rough early catalog to a context platform, told from inside the community that carried it.

The theme was scale, context, and community. These were field reports from teams running DataHub in production, including the parts that broke.

What’s happening across the community: a hackathon, a summit, and events

A quick recap around the community.

• The Agent Hackathon has wrapped. Build with DataHub: The Agent Hackathon ran fully online for five weeks and closed for submissions on August 10. It drew more than 3,000 participants and over 600 submissions, with 100+ pull requests and open-source contributions opened along the way. Judging is underway, and winners will be announced on September 8. The $20,500 prize pool, including a $6,000 grand prize. Browse the entries on the hackathon page and follow the results in the agent-hackathon channel in the DataHub Slack.
• CONTEXT 2026 is set for November 4. The half-day virtual summit is built for practitioners putting context into production, not for vendor pitches. Register now at datahub.com/context.
• A new co-founder podcast is out. Swaroop Jagadish and Shirshanka Das recorded an episode on data infrastructure and the context layer, including the founding story of the past five years. Check it out on YouTube.
• The DataHub Slack community has passed 16,000 members. It remains the fastest place to get a question answered, most of the time by another practitioner rather than the DataHub team.

A first look at the DataHub Metrics Catalog

Shirshanka Das, Co-founder and CTO of DataHub, gave a short preview of something the team has been building: the DataHub Metrics Catalog, shown first as a teaser.

The idea is to make business metrics first-class entities in DataHub rather than notes buried in a glossary. Metrics group into semantic models, and each one carries its own owners, lineage, and governance. Drill into a metric and you get its certified SQL definition. Because a metric can also carry AI context, Ask DataHub and the DataHub Model Context Protocol (MCP) server can answer from that definition directly, so an agent reads the same certified logic a human would.

The catalog answers a recurring community request. The business glossary is useful, but teams wanted metrics as their own entity type. The timing also lines up with the Open Semantic Interchange (OSI) standard, so the work stays interoperable with the rest of the warehouse and metrics stack.

We are finally making metrics first class citizens in DataHub.

Shirshanka DasCo-founder and CTO, DataHub

The Metrics Catalog ships in DataHub Cloud 2.1, with open source support to follow shortly after. This was a preview rather than a full launch, with more registries still in the works. The DataHub team will take it apart in full at the August town hall, so join us there for a deep dive into the Metrics Catalog, the semantic models it lives in, and how agents answer from its certified definitions.

Miro: enhancing text-to-SQL accuracy with metadata signals

Ronald Angel, Lead Product Manager at Miro, opened the sessions with the problem every team building a talk-to-data agent runs into. Point an LLM at the warehouse and it is magical for about a week. Then it picks the wrong table, computes “active users” three different ways in an afternoon, and people quietly stop trusting it.

Miro measured exactly how bad that gets. With Claude wired directly to Snowflake across more than 20,000 datasets, an end-to-end accuracy benchmark came back under 40%. Ronald’s diagnosis reframes the whole problem.

Analytics accuracy is a context and routing problem, not a code-generation problem.

Ronald AngelLead Product Manager, Miro

How does DataHub context lift accuracy?

The climb happened in stages, each one adding context rather than changing the model.

The first move was to put DataHub in the middle. Miro runs no dbt Cloud, so dbt documentation lives only in DataHub and is reached through the DataHub MCP. Exposing descriptions, tags, glossary terms, columns, and lineage to the agent’s search let it finally read what each model means. That alone took accuracy from under 40% to 65%. Better, but as Ronald put it, still fluent SQL against the wrong table.

The second move was a diagnostic framework. Miro decomposes each question into fact concepts, what to measure, and dimension concepts, how to slice and group, then runs a search per concept in parallel before validating and assembling the SQL. On top of that, it re-ranks every candidate with a runtime confirmation boost built from DataHub metadata signals, applied before the quality gates run. Signals include whether a table sits behind a curated semantic view, whether it is owned by a recognized analytics team, how relevant its columns are to the question, whether it belongs to a curated data product, and whether real analysts actually query it. Quality and usage gates then drop anything stale: a table needs recent usage and passing assertions to survive. This stage moved accuracy from 65% to 85%.

The gap	The fix
Answers came from raw schemas, not metric tables	Schema ranking that prefers shared marts and boosts data products
Curated data was hard to tell from raw	Ranking by Snowflake query history and Looker lineage, so only analyst-curated SQL surfaces
No single front door to a domain	Routing to the semantic layer first, probing and boosting semantic views
Stale, unmaintained tables still got picked	Quality and usage gates: usage above a threshold, assertions passing
Metrics got confused with attributes	Decomposing every question into facts versus dimensions

The last mile was about meaning, not mechanics. Some key models had no documentation, so Miro documented them with their grain and valid ranges. Deprecated columns were still live and unflagged, so the team added deprecation checks and wrote down the gotchas. Terms like “MAU,” “adoption,” and “active user” mean specific things at Miro, so those went into the business-glossary catalog as terms and tags. And the most important data products got curated semantic views as a front door. Together these pushed accuracy above 90%.

How does DataHub context lift accuracy?

Ronald closed on a refinement that came from watching how people actually ask. Some users want to explore broadly. Others want a high level of determinism for a question that has to be right. So Miro’s data-discovery skill asks up front which mode you want. General search spans every dbt model and table and lands in the 85% to 90% range. Scoping to data products only, the curated and analyst-validated semantic views, pushes past 90%. Ask “how many active users do we have?” in the second mode and the query routes straight to the product engagement data product and its semantic view, where a correct answer is far more likely.

Just Eat Takeaway: running DataHub self-hosted at scale

Maksym Oliinyk, Data Engineer on the Data Platform Governance Team at Just Eat Takeaway (JET), gave the deepest session of the day: a candid operations story about running self-hosted DataHub as a production metadata service. 

For JET, DataHub serves as the metadata discovery and governance capability inside a wider governance-first platform, the layer that brings distributed technical and business metadata together with the context needed to use data responsibly. His team operates that capability, enables glossary, domains, ownership, and data products, and automates metadata from contracts and pipelines. Business and data owners stay accountable for the content and the decisions.

Governance as code

The split that organizes the whole operation is between two kinds of metadata. Technical metadata, the datasets, schemas, pipelines, lineage, and usage, is discovered continuously from the platform and explains what exists. Business and governance metadata, the glossary definitions, domains, ownership, tags, and data products, explains what things mean and who is accountable, and it needs a controlled, reviewable way to change.

So JET manages that second kind as code. A contributor defines or updates a glossary term, owner, domain, subdomain, tag, or data product as versioned YAML in GitHub, then opens a pull request. Automated checks validate structure against JET’s standards before anything merges, and a business owner approves the change. On merge, the definition syncs to DataHub and maps tagged assets to their domains and data products. The result is a clear audit trail of what changed, why, and who signed off. JET runs this across two DataHub instances, one for development and one for production.

What does it take to keep ingestion healthy at scale?

Ingestion volume is the pressure behind everything else. Since mid-2024 it has grown steadily, peaking at roughly 1 million metadata entities in a single month and reaching about 12 million cumulative entities by November 2025. Maksym was careful to note this is ingestion volume, the continuous change generated as pipelines and data products evolve, not the current count of catalog assets. Two GMS memory incidents, caught through monitoring, prompted a full review of capacity, scaling, and resilience.

The architecture is intentionally hybrid. JET self-operates the application and metadata-processing path on Google Kubernetes Engine, deployed through Terraform and Helm, and relies on managed services for the parts where durability matters most.

• Sources. BigQuery and Tableau are pull-based, run through DataHub’s UI ingestion recipes. Airflow is push-based through the DataHub Airflow plugin v2, using OpenLineage and SQL parsing so a DAG becomes a data flow and a task becomes a data job, building table-level lineage through the orchestration layer. dbt is also push-based: it builds artifacts and test results, pushes them to Google Cloud Storage, and a separate Airflow DAG loads them into DataHub.
• In-cluster on GKE. Kafka and the Avro schema registry move metadata change events, MCE consumers materialize them into GMS, GMS serves the core metadata service and API behind OIDC, the frontend serves the UI and search, and DataHub Actions executes ingestion.
• Managed off-cluster. Cloud SQL for PostgreSQL is the highly available system of record, and Elastic Cloud provides search and discovery over Private Service Connect.

Three production gotchas, and the fixes

Maksym picked the three most useful failures for anyone about to self-host.

An available UI can hide a failed ingestion 

DataHub Actions is a separate critical path for metadata freshness. In development, GKE evicted an Actions pod during an active ingestion. The UI kept showing the run as “Running,” but execution went silent: no new logs, no new metadata. The lesson is that a healthy-looking UI or a replaced pod does not prove a specific ingestion is still making progress. Cloud monitoring covers executor health, CPU, memory, uptime, pod restarts, and replica mismatch, but not whether a given run is advancing. JET’s response is a run-activity alert that fires when a run shows active but produces no execution logs for a set period, plus a dedicated GKE node pool for Actions with requests sized equal to limits so the pod gets Guaranteed QoS and stops being evicted.

JVM memory pressure becomes UI latency 

GMS is the serving path behind the UI and the API. During the incident, maximum memory utilization repeatedly reached 83%, which left the JVM too little headroom and triggered frequent blocking garbage collection. Those pauses showed up to users as a flickering UI and failing API requests. JET treated it as a resilience problem rather than waiting for an out-of-memory crash: it raised the GMS memory limit from 10 to 12 GiB, lifted the JVM MaxRAMPercentage from 75% to 85%, ran two replicas with pod anti-affinity, and added an alert at sustained utilization above 75% for 15 minutes. Garbage-collection time dropped from about 11 minutes to 27 seconds, with a lower memory baseline and steadier UI.

A visible lineage graph can still be incomplete

This was the subtlest one. JET’s dbt models create temporary projects and tables in BigQuery as a feature of the platform. Because those temporary tables never persist, they fell outside standard lineage inference, and some upstream and downstream dependencies were missing from the graph. As Maksym put it, that partial graph was the real risk: it created false confidence, looking complete while hiding dependencies that teams and tools were relying on. The fix is a scheduled weekly BigQuery audit-log ingestion over a rolling seven-day window. Audit logs preserve the executed query history, which gives DataHub the evidence to infer lineage through those temporary tables even though the intermediate table never persists as a catalog entity.

A lineage graph is useful only when it represents the actual flow.

Maksym OliinykData Engineer, Just Eat Takeaway

Where JET is heading: a declarative data-product marketplace

Maksym closed on what the team is building next, an automated data-product marketplace driven by a data-product operator. The shift is that a data product stops being a page someone fills in after the data already exists and becomes a declarative platform resource defined in YAML. The definition starts with the product itself, its name, domain, subdomain, owners, and governance policies. A Kubernetes operator validates it against the Data Product Standard, checks that lifecycle changes are valid, and applies the required approval gates.

The core of the model is the contract interface. Each data product declares input and output ports, and each port carries a contract: the agreement that a producer will deliver a given schema under given quality rules. A streaming or batch implementation connects the declared inputs to the declared outputs, which produces a connected dependency graph by design. DataHub then becomes the marketplace projection of that resource, receiving the data product, its contracts, datasets, pipeline jobs, and the lineage that follows from the declared input-to-output relationship.

The through-line of the whole session was what it actually takes to run DataHub in production at this scale: ingestion resilience, resource headroom, lineage you can trust, and governance you can review.

From catalog to context: a Champion’s long view

The last session was a community spotlight conversation, with Lakshay Nasa, Developer Advocate at DataHub, and Tim Bossenmaier, a DataHub Champion and Data Architect at Cloudflight, joining from Innsbruck in the Austrian mountains.

Tim has been around DataHub for about four years. He first reached for it in his inovex GmbH days, building a data platform for a client that had plenty of tables and data products but no real way to discover them. The prevailing “solution” at the time was Confluence pages documenting table schemas by hand. Looking for something better, the team landed on DataHub, then a much less mature catalog focused on the fundamentals: discoverability, lineage, and documentation. What stood out even then was the community.

His path from user to contributor started with a gap. That platform ran on AWS with Athena and EMR rather than the more common Databricks or open-source stack, and DataHub’s Athena connector, while it existed, did not cover everything the team needed. So they extended it, then contributed the work back. Tim was candid about the motivation, and it was not only altruism. Maintaining a fork creates a debt you pay forever, because you have to keep it compatible with a project that keeps moving. Upstreaming the changes handed that maintenance to the community and, as a bonus, put his company’s name on visible open-source work. That Athena connector is still part of what people build on today.

Asked what has kept him in the community and in the Champions program, Tim pointed to proximity to the team, early visibility into the roadmap, and the chance to give feedback on features before they ship. Mostly, though, he pointed to the people, and the fact that in the early days any question in Slack got an answer, often from a stranger who later became a Champion themselves.

On the shift from catalog to context, his read was refreshingly unhyped: it feels like the natural next step. Teams that invested early in good-quality metadata are the ones best positioned for agentic and context use cases now, because that metadata is the foundation the context layer sits on.

Stop waiting until you feel ready or expert enough. Just go out there.

Tim BossenmaierDataHub Champion

And it does not have to be code. Raise a question, answer someone else’s in Slack, write a blog post about what you are doing, or come speak at a town hall. The community grows when people show up before they feel ready.

What it all adds up to

Three sessions, three angles on one question: what it takes to run DataHub in production once real users depend on it. Miro showed that grounding an agent in governed context, not swapping in a better model, is what makes text-to-SQL trustworthy. Just Eat Takeaway showed what it takes to keep that service healthy at millions of entities: resilient ingestion, memory headroom, and lineage validated against the source. Tim showed the longer arc, and the community that carries a project from a rough catalog to a context platform over four years.

A few ways to go deeper:

• Watch the full town hall recording for the live demos.
• Read the Miro customer story for the full text-to-SQL accuracy build.
• Explore the DataHub Context Platform and where the Metrics Catalog fits.
• Register for CONTEXT 2026 on November 4.
• Browse the DataHub integrations index for the connectors behind these stories.
• Come find the next contributor’s story in the DataHub Slack.
• Register for the next DataHub town hall to catch the next set of stories live.

Getting something to work in a demo was never the hard part. What these teams showed is the work that keeps it running after.