Topic
DataZone
2 entries filed under this topic
The pipeline half, part 2 — OpenLineage, Great Expectations, and the evidence a pipeline emits
Part 1 covered what a pipeline reads. This part covers what it emits, and why that is the whole governance story: OpenLineage as the vendor-neutral lineage backbone, the AWS-native adoption playbook via DataZone's PostLineageEvent API, assertion results as a data-quality facet a grant can be suspended on, and the provenance facet that answers the auditor's 'which code produced this number'. Plus why the scheduler stays central while compute decentralises, and the one hop this architecture cannot see. Real payloads and real emitter config throughout.
Fabric + Mesh on AWS — LakeFormation, DataZone, and one governance plane
The DataEng pillar named the compositional choice but left the code unwalked. Here it is in Terraform: central governance account, per-domain producer accounts, LF-tag-based grants, DataZone publishing in two layers — plus the trap of running two catalog planes over the same data, and three DACH compliance checkpoints hit on the way.