Quick answer:
Data observability is automated monitoring for the data itself: is it fresh, complete, in the expected shape, and behaving like it usually does? Pipelines can run green while delivering garbage, and observability tools exist to catch that quietly-wrong state before an executive finds it in a board deck.
Application monitoring asks “is the service up?”. Data observability asks “is the data right?”, which turns out to be a much sneakier question.
What is data observability?
The nasty property of data failures is silence. An API change upstream makes a column arrive empty. A timezone bug halves Tuesday’s revenue. A sync stops and nobody notices for 9 days because the dashboard still renders, just with old numbers. No error was thrown anywhere. Every job “succeeded”.
Observability tools attack this by learning what normal looks like and alerting on deviation. They watch a handful of signals across every table, the ones the category settled on as its five pillars:
- Freshness: this table updates every 6 hours; it’s been 26.
- Volume: yesterday had 2 million rows; today has 40,000.
- Schema: a column vanished, appeared, or changed type.
- Distribution: a field that’s normally 2% null is suddenly 60% null.
- Lineage: when something breaks, which upstream change caused it and which dashboards downstream are now lying. (See data lineage.)
The point of doing this with machine-learned baselines instead of hand-written rules is coverage. Nobody writes tests for all 3,000 tables; the monitor watches all of them by default.
Where did the category come from?
It’s young. Monte Carlo, founded in 2019, effectively created it, borrowing the observability framing from software tools like Datadog and coining “data downtime” for the periods when your data is wrong or missing. The pitch to data leaders was blunt: you’d never run production services without monitoring, yet you run the data feeding your decisions on hope.
Competition arrived fast (Bigeye, Metaplane, Sifflet, Elementary on the dbt-native side, Anomalo for ML-heavy checks), and then the platforms started absorbing the idea: warehouse-native quality checks, dbt tests and freshness built into the transform layer, catalogs adding monitors. It’s a familiar arc, and the same one reverse ETL is on.
How is this different from data testing?
Complementary, and easily confused. Tests (like dbt tests) are assertions you write on things you know matter: this key is unique, this column is never null, order totals are non-negative. They run during pipeline execution and fail loudly. They’re precise, and they only cover what you thought to write.
Observability is the wide net: unwritten expectations, learned from history, over everything. Tests catch the failures you predicted; monitors catch the ones you didn’t. Mature teams run both and route the alerts to the same place.
What does failure look like without it?
A composite that every data person will recognize: marketing spend data stops syncing on the 3rd. Dashboards keep showing the last loaded numbers. On the 14th, someone notices ROAS has been “flat” for 11 days, which it never is. Three analysts spend 2 days tracing it to an expired API credential. Meanwhile budget got reallocated based on frozen numbers, and the data team’s credibility takes the real hit. Total error messages produced by the entire stack during those 11 days: zero.
That story is why this category exists. The dollar cost varies; the trust cost is remarkably consistent.
What are the benefits and drawbacks of data observability?
Benefits of data observability
Coverage you didn’t have to write
Learned baselines watch every table by default, including the 2,900 nobody wrote tests for. The failures you didn’t predict are exactly the ones this catches.
You find out before the CFO does
Detection time drops from “someone noticed in a meeting” to minutes. The credibility saved is worth more than the analyst hours, and it’s the analysts who’ll tell you that.
Incidents come with a map
Lineage-aware alerts say what broke, what caused it, and which dashboards downstream are affected. Triage starts at the cause instead of at a Slack thread of guesses.
Trust compounds
When stakeholders stop catching errors themselves, they stop double-checking the data team, and start acting on the numbers. That behavioral shift is the actual product.
Drawbacks of data observability
Alert fatigue is the default failure
Anomaly detection on thousands of tables produces noise until tuned, and an ignored alert channel is worse than none. Budget real weeks for tuning, not the demo’s minutes.
It observes; it doesn’t fix
The tool tells you the table is stale. Someone still owns the pipeline, the fix, and the postmortem. Buying observability without incident ownership just documents your failures faster.
Warehouse-bill side effects
Monitors run queries against your warehouse, and metadata scanning isn’t free at scale. Ask vendors precisely what their checks will cost you in compute.
Enterprise pricing, hazy ROI math
The category sells insurance, and insurance is hard to price. Anchor negotiations to your actual incident history, since that’s the claim rate you’re insuring.
When should you buy it?
The honest gate is dependency, and dbt tests plus freshness checks are the right starting point (they’re free and you should max them out first). Move to a dedicated tool when the surface outgrows hand-written checks: hundreds of tables, multiple pipelines, downstream consumers who act on the data without asking, especially anything feeding operational syncs or customer-facing numbers.
And price it against incidents, since that’s the actual unit. If a silent failure costs you days of analyst archaeology plus a dent in trust, and you have a few of those a quarter, the math gets easy. If your whole estate is 30 tables one person understands end to end, that person is your observability tool, and cheaper.