Data Engineering — the Fabric + Mesh Hybrid

Working draft — pre-publication review. This pillar carries the Fabric half of Cordata's argument about data platforms. The Mesh half — organizational adoption, domain ownership, cultural change — sits in the Governance pillar (also drafting).

Data platforms can fail for many reasons — two stand out

A DACH mid-market insurance company decides in early 2023 to consolidate its analytics stack. Snowflake wins the evaluation. An eight-person team takes fourteen months to migrate 200 pipelines, rebuild 40 dashboards, and consolidate the master data model. The go-live lands on time and under budget. There is champagne.

Eighteen months later, half those pipelines still run their nightly loads. The other half were deprecated by the domain teams they were built for, replaced by ad-hoc SQL and Excel exports that bypass the platform entirely. Two dashboards have been touched in the last quarter. The master data model was frozen by year one because nobody's job included owning it.

We at Cordata have watched this pattern repeat — in variations — across banking, insurance, and telco over years of engagements as engineers on these projects. The vendor logo changes. The consultancy logo changes. The two failure modes don't:

(a) The organization would not adapt its structure to a data-platform-first way of working. Ownership stayed central; the domains that produced the data never accepted product responsibility for it; shadow-IT stayed cheaper than platform use.
(b) The platform was not metadata-driven enough to stay manageable as it grew. Every new pipeline was a custom build; every access request went through a ticket queue; the catalog drifted from reality within a quarter of launch.

The usual excuses — wrong vendor, wrong cloud, insufficient training, bad change management — are downstream of these two. The actual problem is that the industry keeps selling one-half answers to a two-halves problem.

Why DACH mid-market — and why now

Two terms need pinning down before this makes sense.

Data Fabric is a technical architecture that unifies distributed data through active metadata, automated lineage, unified catalog, and policy-as-code access — call it the machinery.

Data Mesh is an operating model where each business domain owns its data as a product with SLAs and consumers, while the central team provides self-serve tooling instead of gates — call it the accountability.

Fabric handles failure mode (b); Mesh handles (a). Everything below assumes both.

Two forces make the twin failure modes bite DACH mid-market harder than they bite anyone else — and one of those forces is new.

DORA is new. The Digital Operational Resilience Act came into force in January 2025 and enforcement went live end-2025. For anything touching financial services in the EU it mandates, in plain terms:

a critical-ICT-asset inventory that must be current at all times
incident reporting with downstream-blast-radius analysis inside tight regulatory windows (you must know within hours what breaks when component X goes down)
threat-led resilience testing against that inventory
a third-party processor and data-flow catalog with contractually enforceable exit strategies

Each of those is a Data Fabric feature mature teams built anyway for engineering reasons. DORA does not ask you to build new things; it asks you to prove that the things you built are complete, current, and answerable to a supervisor at short notice. DORA moves Data Fabric from "nice to have" to "supervisory finding if missing".

The older pressures have not moved:

BaFin and FINMA outsourcing rules make central-team gatekeeping a de facto compliance model that scales badly past a hundred pipelines.
Works-council (Betriebsrat / Personalrat) dynamics turn central-team RBAC changes into calendar-months rather than sprints; decentralized ownership shrinks scope and accelerates approval.
GDPR's data-processing accountability rewards federated ownership: each domain keeping its own processing records (RoPA) is the only way that stays current at scale. A central registry drifts.
On-prem legacy that will not leave for another decade makes hybrid architecture a reality to design for, not an aspiration to grow out of.

Every pressure makes both failure modes worse. Together with DORA, they make the Fabric + Mesh Hybrid — treated as a stack, not a shopping list — the only stable answer for a DACH mid-market shop today.

The correlated failure

The industry treats the two failure modes as independent variables. Vendors sell you Data Mesh workshops or Data Fabric platforms; consultancies pitch strategy engagements or implementation engagements. Each half is a real product with a defensible thesis. Neither half, by itself, has ever prevented the story above from playing out.

The reason both halves are needed is worth stating in one sentence you can quote back at the next vendor slide:

A perfectly automated platform that no domain owns is an abandoned museum. A perfect ownership model on top of a manual platform is a gated bottleneck.

Automation without ownership degrades into stagnation: every pipeline is technically fine and nobody feels responsible for whether it still serves a business decision. Ownership without automation degrades into gatekeeping: every schema change is technically owned and moves at the speed of the ticket queue. Neither shape is stable, and neither shape is what a serious data platform in 2026 looks like.

The rest of this pillar makes the case for how Data Mesh and Data Fabric compose into one answer, why the composition is more than the sum of its parts, and what Cordata refuses to sell as a substitute.

Data Mesh — in brief

Data Mesh, coined by Zhamak Dehghani in 2019 and mainstream by 2023, rests on four principles: domain-oriented ownership (fraud team owns fraud events; policy team owns policy state; not "the data team owns everything"); data-as-product (SLAs, docs, versioning, discoverability — the same rigor you apply to a public API); self-serve platform (the central team provides tools; domains consume them without tickets); and federated computational governance (policies apply everywhere but are agreed on together, not decreed from a central authority).

Adopting Data Mesh without changing team structures and incentives is the most expensive way to keep a monolith. The Governance pillar (drafting) unpacks the organizational side — the RACI shifts, the incentive changes, the works-council conversations, the cultural work that a rename doesn't achieve. This pillar takes ownership as given and turns to the machinery.

Data Fabric — in brief

Data Fabric is a technical architecture, not a product. It's the layer that makes a distributed data landscape — multi-cloud, on-prem, hybrid — manageable at all. Five capabilities carry it:

Active metadata — metadata that isn't a wiki. Schemas, freshness signals, data-quality checks, drift alerts all live as first-class citizens, updated by pipelines rather than typed in by hand.
Unified catalog — one place to find every data product, spanning clouds and on-prem. Catalog entries are generated from the source (dbt models, Delta table properties, Snowflake information_schema), not maintained separately.
Automated lineage — every transformation registers its inputs and outputs; the resulting DAG answers "what upstream feeds this KPI?" and "what breaks if I change this column?" without a human in the loop.
Knowledge-graph overlay — entities and relationships across domains (customer ↔ policy ↔ claim) surfaced as a graph, not just a list of tables. Enables semantic search, impact analysis, and (increasingly) LLM-grounded retrieval.
Policy-as-code access — permissions expressed in version-controlled config (Open Policy Agent, LakeFormation permissions, Unity Catalog grants), reviewed like code, applied automatically.

Tools that materialise these today, non-exhaustively: OpenLineage + Marquez or DataHub for lineage + catalog; Atlan, Collibra, Alation for enterprise catalog + governance UX; AWS LakeFormation + DataZone, Databricks Unity Catalog, Microsoft Purview for cloud-native governance; Open Policy Agent for cross-cutting policy. This pillar refuses to pick a winner — the companion post on AWS walks through the LakeFormation + DataZone vs. Unity Catalog trade-off in production shape.

The Hybrid — how they compose

Fabric and Mesh solve different halves of the same problem. Composed, they close both. This is what the composition looks like:

Diagram read-out. Three layers, bottom to top. The Data Fabric substrate (bottom) contains the unified catalog, the active-metadata + lineage graph, the policy-as-code engine, and the self-serve provisioning surface — this is what the central platform team builds and operates, and it exposes three interfaces to the layer above: a catalog SDK for registering data products, a provisioning API for spinning up new storage / compute / access grants, and a policy engine the domains author against. The Data Mesh (middle) is one node per business domain — fraud, policy, claims, and so on — each publishing one or more data products through those interfaces. A data product is a versioned schema plus a SQL or API endpoint plus documentation plus a declared SLA, registered in the catalog so consumers can find it. The Consumers layer (top) is everyone reading those products: analytics teams, ML models, downstream domains, external partners.

Fabric is the substrate. The central platform team builds and operates the catalog, lineage graph, policy engine, and self-serve provisioning surface. They own no data. They own the plumbing that lets domains own data safely at scale.

Mesh is the operating model. Domain teams — fraud, policy, claims, marketing, finance, and so on — each own one or more data products. They publish through the catalog, declare SLAs and consumers, register lineage, express access policies as code. They own the content of their data — its correctness, its currency, its meaning — because they own the systems that produce it.

The critical property is that Fabric is what the platform team builds, and Mesh is how the domain teams consume it. Fabric without Mesh is a self-serve platform with no tenants. Mesh without Fabric is an aspiration on a slide. The composition is not a compromise between two ideologies; it's the shape the answer was always going to take once both failure modes had to be solved.

The composition also concentrates DORA compliance in the right place: the Fabric holds the technical inventory (assets, lineage, third-party data flows) that a supervisor asks for; the Mesh holds the accountability (who owns which product, who accepts risk, who signs off on changes) that the supervisor also asks for. Neither side alone answers DORA.

What each half owes the other

Composition is stable only under a two-way contract. Both halves have obligations to the other that are non-negotiable.

Fabric owes Mesh:

Self-serve provisioning of a new data product in under a day, not a sprint — otherwise domains route around the platform.
Per-domain quota and cost visibility, so a domain can answer "am I spending budget on the right things?" without a central-team query.
Drift alerts scoped to the affected domain, not sprayed at a central inbox nobody reads.
A policy engine that domains can author against, not just consume from — a policy language domain engineers can write directly beats one that requires a central-team translation.

Mesh owes Fabric:

Product responsibility for each data product: SLA declared, documentation current, consumers listed, versioning honoured, breaking changes announced.
Catalog-first: schemas and lineage flow through the catalog, not through DMs to downstream teams.
Federated-governance decisions enforced, not lobbied around — if the community agrees on a rule, exceptions require the same governance path as the rule itself.

Where either side stops meeting its side of the contract, the composition degrades into failure mode (a) or (b) from § 1. Cordata engagements start by testing which obligations are already in place, which are aspirational, and which have quietly collapsed — because that's what determines whether the platform is fixable or needs re-founding.

Guardrails, not gates

Cordata's position on governance: automation makes domain teams faster and safer, and never forces them through a central bottleneck.

The distinction is concrete. A gate is a central-team ticket queue for schema changes — every change waits for review, and review capacity is fixed. A guardrail is a CI check on the schema change that fails when it would break a downstream consumer, and passes when it wouldn't. Gates protect through veto and become the bottleneck they were meant to prevent. Guardrails protect through information — the domain team learns immediately what their change would do, and either amends it or coordinates the downstream update.

Every governance-through-veto system Cordata has audited eventually devolves into shadow-IT workarounds, because the veto queue outgrows its human capacity within a year. Governance-through-information scales with the platform, because the information is generated automatically from the same active metadata the Fabric already maintains. This is one of the reasons the active metadata capability is load-bearing.

What Cordata refuses to sell

Cordata refuses to sell (a) strategy engagements without implementation accountability, or (b) platform builds without an organizational-adoption plan. Both refusals come from experience.

Strategy without implementation is a deck. It survives the workshop, gets circulated, and is quietly ignored six months later when the recommended structural changes turn out to conflict with someone's Q3 objectives.

Implementation without adoption is a museum. It ships on time, wins an internal award, and stagnates. The domains it was built for continue to route around it, because nobody asked them what they needed and nobody put ownership incentives in place after go-live.

Cordata brings both because both failure modes are correlated. We are willing to lose the deal when a client wants only one half — because we know how that story ends, and it isn't a client the engagement helps.

DACH-specific hooks

DACH mid-market has regulatory and organizational realities that make Fabric + Mesh especially apt. Concisely:

DORA (Digital Operational Resilience Act) — since January 2025, EU financial services must maintain a current critical-ICT-asset inventory, produce incident reports with downstream blast-radius analysis inside tight windows, submit to threat-led resilience testing (TIBER-EU), and hold a third-party processor and data-flow catalog with exit strategies. Each maps directly to a Fabric capability.
BaFin and FINMA outsourcing rules — central-team gatekeeping is a de facto compliance model that scales badly past a hundred pipelines. Federated ownership scales cleanly.
Works-council (Betriebsrat / Personalrat) dynamics — approval time for a data-access change scales with the scope of the change. Decentralized ownership shrinks scope, and approval time, per change.
GDPR data-processing accountability — per-domain processing records stay current at scale. A central registry drifts.
Multinational org structures — HQ Munich, ops Zurich, back-office Bratislava is a common shape. One central team cannot hold this.
On-prem legacy that will not leave for another decade — hybrid architecture is a reality to design for, not an aspiration to grow out of.

The Governance pillar (drafting) unpacks the organizational side of each: works-council paths, processing-records (RoPA) maintenance, DORA's board-level oversight chain.

Where AI belongs in a Fabric + Mesh Hybrid

AI belongs in the Fabric substrate, as active-metadata infrastructure. Concretely:

Agentic data catalogs that infer types, classify PII candidates at ingest, detect drift when a source's distribution shifts, and propose lineage links between similarly-named columns across domains.
Natural-language lineage queries — "what upstream feeds the quarterly regulator report?" answerable by an LLM grounded in the lineage graph, without a human tracing through a DAG viewer.
LLM-assisted schema mapping when a new source lands — the model proposes column mappings against the domain's canonical schema, a human reviews, the mapping becomes catalog metadata.
Automated documentation — from lineage plus query patterns, the catalog can auto-generate first-draft docs for a data product, which the owning domain refines.

There is one more piece the Hybrid needs before AI consumption is safe: a semantic layer — a governed, versioned definitions tier (dbt Semantic Layer, Cube, Malloy) that sits above the data products and exposes typed metrics with owned business meaning. Agents that query raw SQL hallucinate joins and misuse columns; agents that query a semantic layer ask for monthly_recurring_revenue by dimension and get the definition — the single one, versioned in Git, owned by a domain. Cordata's position: the semantic layer is not optional if the platform is going to serve AI consumers. A follow-up post treats it in depth.

Each of these must earn its keep. The pattern we refuse is "we added AI because everyone does" — the LLM tacked onto a workflow it doesn't improve, whose only measurable effect is a token bill. If the AI feature does not reduce a specific human hour per week, or does not reduce the mean time to detect a specific class of failure, it does not belong in the platform. The Governance pillar (drafting) covers the flip side — where AI belongs inside governance workflows themselves.

Where AI does not belong

AI belongs on top of the platform. It does not belong inside decision paths that produce compliance artifacts, and it never belongs in a shape that turns the platform into a covert data-exfiltration channel.

Do not use LLMs to auto-approve data-access requests. The audit trail cannot survive an LLM in the loop.
Do not let agentic systems make cross-domain data-sharing decisions. Sharing decisions are governance decisions and belong to humans with named accountability.
Do not embed vector stores of production data in third-party APIs — the data leaves your compliance boundary the moment it enters the embedding call. If retrieval-augmented use cases are legitimate, run the embedding infrastructure inside the boundary.

Every DACH prospect we've spoken to about AI features asks these questions inside the first fifteen minutes. Answering them convincingly is table stakes.

Reference implementation — preview

The Fabric + Mesh Hybrid can be built cleanly on any major cloud. Cordata's canonical reference implementation runs on AWS: LakeFormation + DataZone for governance and the domain-publishing surface, with Databricks Unity Catalog covering the cases where multi-cloud lineage matters.

The decision this pillar deliberately does not make: whether LakeFormation + DataZone or Unity Catalog is the right primary governance layer for a given shop. They compete; they don't complement. The trap is running both simultaneously and letting them fight for the same governance role.

The companion post — Fabric + Mesh on AWS walks through the trade-off with real Terraform, a two-layer DataZone segmentation (platform-owned root, domain-owned sub-domains), the catalog-of-catalogs trap, and the three DACH-specific compliance checkpoints hit along the way. It is a shared reference — the same walkthrough is linked from the forthcoming Governance pillar as its governance-plane companion. Read it for the "we've built this, not just diagrammed it" evidence.

When Fabric + Mesh is the wrong answer

Fabric + Mesh Hybrid is wrong for at least three shop shapes, and pretending otherwise wastes the client's money:

Single-domain shops where "the org" is one team of ten. There is no Mesh to build — everyone owns everything, and imposing Data Mesh vocabulary on a group that fits in a Slack channel is theatre. A well-run centralized platform is the right answer.
Greenfield teams under six engineers. The Fabric overhead — catalog, lineage, policy engine — costs more to set up than the manual coordination it replaces at that scale. Grow into it. Six months in, revisit.
Narrow single-source-of-truth requirements, particularly around regulatory reporting where one canonical figure must exist. Central ownership of that slice is often the right answer even inside a Fabric + Mesh shop. The pillar doesn't forbid centralization; it forbids centralizing everything.

A pillar that never says "no" isn't defensible. This one does.

Cordata's engagement model

Cordata engagements start with a shared diagnosis of which failure mode currently dominates. That's a two-week discovery: architecture review of the current platform, structured interviews with two domains and one central team, an honest map of where (a) and (b) each sit today.

From there, engagements are scoped 6–12 weeks, not open-ended. Deliverables are always both:

Organizational — a RACI for data-product ownership, decision rights across the federated governance, a change plan the works-council can approve.
Technical — catalog + lineage + policy engine landed in production, one or two domains onboarded end-to-end as the reference shape.

Handover-first. Every engagement builds our replaceability into the deliverable. The measure of a successful engagement is whether the client's team can extend the platform without us six months later.

If this sounds like a fit for what you're building, get in touch — the first conversation is a 30-minute discovery call, no charge.