Chime’s Data Now Works in Harmony With Their Teams

Note: This story was originally published October 2023.

Challenge

As Chime scaled, so did its data and the complexity of managing it. Multiple teams were managing data independently, with an ever-increasing number of tools adding to the complexity. 

Business-critical metrics were scattered across systems. Source-of-truth datasets were hidden or duplicated. And when dashboards broke, no one knew whether it was a real business issue or just bad data.

The root cause? A disconnect between the teams producing data and the teams using it.

“In a lot of organizations, the producers (product engineering) and consumers (analytics teams) are in separate orgs… Because these two groups are not talking to each other, there are a lot of problems related to consumer expectations, producers not knowing how their data is being used, and so on and so forth.”

 — Sherin Thomas, Software Engineer, Chime

Chime needed a unified approach to data management; one that would streamline workflows, improve data discovery, and foster better collaboration across teams.

Solution

To break down silos and restore trust in their data, Chime implemented DataHub Cloud as its centralized discovery and governance platform. 

Instead of making data engineers the middlemen between data consumers and producers, Chime brought everyone (engineers, PMs, analysts, BI teams) into DataHub Cloud. In doing so, they established a “water cooler” for all data stakeholders: a shared space where everyone could access, contribute to, and collaborate around metadata.

A key reason for this approach’s effectiveness lies in DataHub’s X-platform lineage functionality. It bridges the gap between data producers and consumers. With lineage, producers can see exactly who is using their data and how. Consumers, on the other hand, can trace where the data comes from and how it’s been transformed. Besides ensuring transparency, this also simplifies accountability. Everyone knows what’s happening and why.

“My favorite part about DataHub is the lineage because this is one really easy way of connecting the producers to the consumers. Now the producers know who is using their data. Consumers know where the data is coming from. And it is easier to have accountability mechanisms.”

 — Sherin Thomas, Software Engineer, Chime

Chime also embraced a shift-left approach to collecting metadata at the source. Using DataHub SDKs, vital context like schema definitions, documentation, and tags are transformed into searchable glossary terms, tags, and descriptions, making it accessible across teams. Effectively making metadata a first-class citizen at Chime.

Impact

With DataHub Cloud, Chime manages data across the organization, improving collaboration, efficiency, and data quality.

Key outcomes included:

• Centralized platform for all teams, ensuring alignment and better collaboration
• Enhanced visibility with lineage to track data flows and quickly spot issues
• Streamlined metadata management using crowdsourced metadata ingestion and schema integration
• Clear ownership and accountability by designating data stewards for each dataset and enforcing ownership policies
• Proactive data quality monitoring with assertions to set and monitor data quality standards, automatically detecting issues early

“Now our engineers, PMs, analysts, and BI folks, everybody is using the same tool … They can just look at the lineage, and they can find if there is any node that has an active incident there, find their owners, and reach out to them.”

 — Sherin Thomas, Software Engineer, Chime