Topic
Data Quality
2 entries filed under this topic
Data Contracts — the schema is a shared interface, not a private detail
Most data incidents that reach a dashboard begin as an ordinary application release. Contracts enforced in the producing application's build, with the consumer list generated from what pipelines already declare rather than maintained by hand — and running code for both halves.
The pipeline half, part 1 — a pipeline is a descriptor, not a program
Metadata-first is usually read as an emission story — lineage and quality events coming out of a pipeline. That is half of it. This post walks the other half: the schemas, descriptors, contracts, expectation suites, and policy metadata a pipeline reads on the way in, and the one generic executor that interprets them instead of a DAG per pipeline. Real descriptor YAML, a complete Python executor, the six rules for defining a descriptor schema, and honest limits on where the abstraction stops working.