Data Quality That Lives in a Contract

Most data quality lives in the wrong place. It’s a pile of dbt tests, a Great Expectations suite, a few cron’d SQL checks — scattered across repos, owned by whoever wrote them, invisible to the people consuming the data. The check knows the data is bad. The consumer finds out in a dashboard three days later.

A data contract moves the quality rules to where they belong: into a document attached to the table, written in a standard, owned by a named team, and enforced on a schedule. The contract says what the data should look like, how good it will be, how fresh, and who to call when it breaks. We build on the Open Data Contract Standard, run the checks on the live engine, and surface the results next to the table in the catalog.

This post covers the background — what ODCS is and why contracts matter — then the implementation, then the honest accounting of which pieces are live and which are built but not yet plugged in. That last part is longer than I’d like.

Background: what a data contract is

The data contract idea came out of the data-mesh and data-product conversations around 2022–2024, pushed by people like Andrew Jones and Chad Sanderson, as a reaction to a specific failure mode. Producer changes a column. Consumer’s pipeline breaks. Nobody agreed to anything, so nobody’s at fault, and the fix is a Slack thread. The interface between producer and consumer was implicit, and implicit interfaces rot.

A contract makes the interface explicit. It’s the API contract idea, applied to a dataset. Schema: these columns, these types. Quality: this column is never null, this one is unique, row count stays above a floor. SLA: refreshed every two hours, retained for a year. Ownership: this team owns it, here’s the Slack channel. Write it down, version it, enforce it.

The Open Data Contract Standard — ODCS, now stewarded by the Bitol project under the Linux Foundation — is one of two main specs for writing that down (the other being datacontract.com’s Data Contract Specification). ODCS is a YAML structure: a schema of tables and their properties, quality rules at the column and table level, slaProperties, a team, support channels. It’s engine-agnostic on purpose — the rules live in the contract, not in a testing framework’s config. That last property is the one we lean on hardest: if the rules are just data, you can compile them to whatever your engine speaks.

We use ODCS v3.1.0, via the Bitol Python library, which gives us the parsing and validation for free. A contract looks like this, trimmed:

apiVersion: v3.1.0
kind: DataContract
name: analytics_orders
version: 1.0.0
status: active
schema:
  - name: orders
    physicalName: main_warehouse.analytics.orders
    properties:
      - name: order_id
        primaryKey: true
        required: true
        quality:
          - type: library
            metric: duplicateValues
            mustBe: 0
team:
  members:
    - username: jverhoeks
      role: Owner
slaProperties:
  - property: frequency
    scheduler: cron
    schedule: "0 */2 * * *"

The physicalName — catalog.namespace.table — is the join key. It’s how a contract finds its table, and how a table finds its contract.

The check compiler

The heart of the implementation is the part that turns a quality rule into SQL.

ODCS quality rules come in three flavors: library (a named metric like nullValues, duplicateValues, rowCount), sql (a raw query you wrote), and custom. The compiler translates each into a CompiledCheck — a SQL string, an operator, and an expected value. Library metrics compile to portable patterns: nullValues mustBe 0 becomes a SUM(CASE WHEN col IS NULL THEN 1 ELSE 0 END), deliberately not COUNT(*) FILTER (WHERE ...), because the former runs everywhere the latter doesn’t. Custom SQL passes through a write-keyword block and identifier validation before it’s allowed near the engine.

There’s a quietly important behavior on top: implicit checks from schema flags. Mark a property required and you get a not-null check whether or not you wrote one. Mark it primaryKey or unique and you get a uniqueness check. The schema isn’t just documentation — it generates enforcement. That’s the contract earning its keep.

One metric is a no-op. invalidValues compiles to SELECT 0 ... LIMIT 1 — it always returns zero, so it always passes. It shows up in the editor’s metric list, but it enforces nothing. If you build a contract around invalidValues, you’ve built a contract that’s green by construction.

Running the checks

A quality run is synchronous and runs as you. The endpoint pulls your JWT and forwards it to the engine over Arrow Flight SQL, so the quality SQL executes under your identity, not a service principal — the checks see exactly the data you’re allowed to see.

It collects every check: table-level rules from the contract, column-level rules from each property, plus the implicit ones from required/primaryKey/unique. It compiles them. It runs each one, pulls the scalar result, applies the operator, records pass or fail — and stores the compiled SQL alongside each result, so when a check fails you can see exactly the query that failed it. The run gets a score: passed over total, on a 0-to-1 scale. One-point-zero is passed, 0.8 and up is warning, below is failed.

A scheduler closes the loop. A background loop wakes every 60 seconds, takes a Postgres advisory lock so only one backend instance runs the cycle, finds contracts whose cron is due, runs their checks, and computes the next fire time with croniter. The cron comes straight out of the contract’s slaProperties.frequency. Write the schedule in the contract; the platform honors it.

The catalog tie-in

Open any table in the explorer and there’s a Contract tab. It calls a by-table lookup — the physicalName join key in action — and shows the contract: name, version, domain, owner, property count, and how many columns carry a classification (the PII/PHI signal). No contract? A “create one” link that drops you into the editor with an Iceberg import pre-triggered, so you start from the table’s actual schema instead of a blank page.

The editor itself is a seven-tab form — General, Server, Schema, Quality, Team & Support, SLA, Description — with a live ODCS YAML preview that re-serializes on every keystroke. Import from a live table, get suggestions per column, see the YAML you’re about to save.

The honest accounting

This is where I have to be straight, because the gap between what’s built and what’s wired is unusually wide in this subsystem. A lot of correct, tested code is sitting one connection away from being live.

Working end to end: ODCS parsing and validation, full contract CRUD, the check compiler, synchronous quality runs on the engine with per-check SQL stored, implicit checks from schema flags, the cron scheduler with advisory locking, the contract tab in the catalog, the seven-tab editor with live YAML, import from Iceberg, export to dbt/SQL/JSON-Schema, and the AI assistant’s contract tools over MCP.

Built, tested, not wired:

• The editor can’t update. The save button always calls the create endpoint, even in edit mode. The server has a correct PUT route; the frontend never calls it. So editing an existing contract through the UI fails with a 409 conflict on the contract ID. Today the only path that actually updates a contract is the AI assistant’s MCP tool, which does call PUT. A real, user-visible gap.
• Profiling is dark. There’s a whole profiler module — table stats, column stats, histograms — with tests. It is never instantiated in the app. The profiles endpoints return stubs. And because the smart quality suggestions need profile data, the suggest endpoint 404s for every live table. The suggestion logic is correct and tested. It just has no data to stand on.
• Polaris sync isn’t called. There’s a module that writes the quality score back to Iceberg table properties under a dq.* namespace, so the score is discoverable through the catalog. It’s implemented and tested. The quality run doesn’t call it.
• Schema drift isn’t exposed. There’s a detect_drift function — added, removed, retyped columns — implemented and tested. No endpoint calls it. The schema_drift field on the response is hardcoded False.
• Git repo sync is a UI and a table. There’s a contract-repositories page and a contract_repositories table for syncing contracts from git. The service that does the actual pull-scan-upsert loop isn’t there.
• There’s an orphaned second implementation. A QualityService class exists alongside the ContractService that does the real work. It’s instantiated only in its own unit tests, scores 0–100 instead of 0–1, and reads the data in a different shape. It’s a fork that lost.

The pattern across these is the same: the hard part — the compiler, the profiler, the drift detector, the sync — got built and tested in isolation, and the last mile of wiring it into the live request path didn’t happen. That’s a specific kind of technical debt, and it’s recoverable, but it means the demo and the reality diverge in exactly the places a buyer would poke. Better to say so.

Where this goes

The core is genuinely good. Quality rules as standardized, versioned, owned documents, compiled to portable SQL and run on your own engine under your own identity, on a schedule the contract itself declares — that’s the right architecture, and it’s working. The roadmap isn’t “build more.” It’s “wire up what’s built”: fix the editor’s update path, turn on profiling so suggestions work, call the Polaris sync after a run, expose drift. Each of those is a connection, not a project.

The contract idea is sound, the standard is the right one, and the engine-agnostic enforcement is exactly what ODCS was designed to enable. The work left is plumbing — and plumbing is a much better problem to have than architecture.