Enterprise Context Strategy Data Flows: One Entity, End-to-End Walkthrough

Updated October 2026

An enterprise context data flow moves one business entity from source systems to AI agents through seven governed steps. Each step produces a durable artifact, such as a semantic contract, a stable ID, a validation report, or a provenance record. When a step is skipped, a predictable failure appears downstream, and the fix becomes a search through every pipeline instead of a single query.

What does an end to end context flow cover?

It covers everything that happens to one record between a source system and the person or agent that uses it. Most architecture diagrams stop at boxes and arrows. This walkthrough stays at ground level and follows a single Customer through capture, mapping, resolution, validation, provenance, publication, and serving.

It is the operational companion to the enterprise context strategy reference blueprint, which sets scope, owners, and stages. The blueprint says what must exist. This article shows how one entity actually moves through it. The walkthrough is implementation neutral, so it names open standards where they help but assumes no particular database, orchestrator, or vendor.

Why follow one entity instead of drawing the whole architecture?

Following one entity exposes the gaps that diagrams hide. A box labeled "identity resolution" looks complete until someone asks how a bad merge gets reversed. Tracing one record forces an answer for every handoff, and each answer becomes an artifact the team can review.

The stakes are real. Gartner predicts that through 2026, organizations will abandon 60 percent of AI projects unsupported by AI-ready data (Gartner). The same research found 63 percent of organizations lack, or are unsure they have, the right data management practices for AI. A traced flow is one of the cheapest ways to find out which group you are in.

Structure also changes what models can do with the data. In a benchmark by Juan Sequeda, Dean Allemang, and Bryon Jacob, GPT-4 answered 16 percent of enterprise questions correctly over raw SQL and 54 percent over a knowledge graph of the same data (arXiv). Governed entities with clear relationships are what make that kind of gain possible.

Which Customer record does the walkthrough follow?

The walkthrough follows a Customer because every company has one and it touches several systems. The entity carries four fields and two relationships. That is small enough to follow and rich enough to exercise every step.

The fields are legal_name (required string), email (format constrained), mailing_address (structured object), and account_id (reference to Account). Each Customer belongs to exactly one Account and can have many Orders. The same person appears in three source systems, each with its own key and naming habits.

These three records will become one governed entity with the stable ID ctx-00382 by the end of Step 2.

What artifacts should the flow produce?

The flow should produce six durable artifacts. Think of them as the paperwork that makes context trustworthy rather than merely available. Every step creates or consumes at least one of them, and a step that produces nothing durable deserves a second look.

Several of these formats matured recently. Apache Ossie, formerly Open Semantic Interchange, entered the Apache Incubator in July 2026 as a vendor neutral format for business metrics and definitions (Apache Ossie). The Open Data Contract Standard defines a YAML contract with sections for schema, data quality, service levels, roles, and authoritative definitions.

What happens at each step of the flow?

Each step has one control point, one artifact, and one failure mode. The steps run in order for a new record, but changes to definitions or rules can send an entity back through the flow at any point.

Step 0, source capture

The Customer starts in a CRM, an ERP, and a support desk. The discipline at this step is capturing metadata at the edge instead of reconstructing it later. Each record should carry its source system, extraction time, source schema version, and a batch or event ID for replay.

Schema registries show how cheap this can be. Confluent Schema Registry stores versions of each schema, and messages carry a small schema ID rather than the full schema (Confluent). If those tags are missing and the CRM team renames cust_email to contact_email, the mapping breaks silently because nobody can tell which schema version a record came from.

Step 1, mapping to shared definitions

Raw fields arrive as cust_nm, LEGAL_NAME, and customerFullName. Step 1 maps each one to Customer.legal_name through a semantic contract. The contract is versioned and machine readable, and it says which field from which source schema version feeds which governed property.

Versioning is not optional. When a source changes, the team publishes a new contract version and keeps the old one so historical data can be reprocessed. The contract also maps relationships, so the CRM acct_id becomes the governed link from Customer to Account. The ontology management operating model covers who approves changes to the definitions themselves.

The failure here is mapping drift. Two teams map the same source field to different properties, or one pipeline picks up a new mapping while another keeps the old one. The architecture then recreates the conflicting definitions it was built to remove.

Step 2, identity resolution and entity mastering

Step 2 decides that the three records describe one Customer, assigns the stable ID ctx-00382, and applies survivorship rules to choose each value. A typical rule prefers the ERP for legal_name and the CRM for email. The stable ID must stay the same through later merges and source migrations.

Most teams run deterministic rules first, such as an exact match on tax ID, and then score the remaining records probabilistically. Splink, an open source Python library, implements probabilistic linkage based on the Fellegi Sunter model (Splink). Its documentation notes that 1 million records produce about 500 billion candidate pairs, which is why blocking rules narrow the comparisons first (Splink blocking).

Pairs that score between the auto merge and auto reject thresholds go to a review queue. Every merge should write a decision record listing the source records, the rule that fired, and the confidence score. Without that record, two different customers who share a name and address can be collapsed with no practical way to split them.

The identity resolution and entity mastering guide covers survivorship and stable IDs in depth.

Step 3, constraint validation

The merged entity is now a candidate for publication. Before it reaches the store, it passes a quality gate that checks it against governed constraints. The W3C Shapes Constraint Language, a Recommendation since 2017, frames this as validating a data graph against a shapes graph (W3C SHACL).

The pattern does not require RDF. JSON Schema describes "the shape and limits" a JSON document should follow, and the same idea applies to tables. The key is separation, with rules kept in a versioned artifact instead of buried in pipeline code. SHACL 1.2 is now a W3C Working Draft, dated 18 September 2026 (W3C SHACL 1.2).

For this Customer, the rules are simple. The legal name is present, the email matches a format pattern, the Customer links to at most one Account, and that Account already exists in the store. A failing entity goes to a remediation queue with a report naming the rule, the property, and the value. The context QA gate guide covers gate design in detail.

Step 4, provenance and lineage capture

Provenance answers where a fact came from and why it should be trusted. The W3C PROV family describes provenance as information about the entities, activities, and people involved in producing data (W3C PROV). In this flow, it needs to exist at two levels.

Pipeline lineage records which jobs ran, what they read, and what they wrote. OpenLineage models this with jobs, runs, and datasets, plus facets for schema and version (OpenLineage). Fact level provenance goes further and records, for each property, which source record supplied the value and which rule selected it.

With these events in place, a wrong answer from an agent becomes a targeted query instead of a review of every pipeline. The provenance and lineage guide explains how to store this evidence alongside the entity.

Step 5, publishing to the context store

Publishing is the moment the entity becomes trusted, servable context. It means three things. The entity passed validation, its provenance is attached, and it is written with a version identifier and a publication time.

Teams handle versions in two common ways. Some keep immutable snapshots, so every change creates a queryable version. Others overwrite the current state and keep a full audit log. Table formats make the first approach practical, and Delta Lake, for example, supports queries with TIMESTAMP AS OF and VERSION AS OF (Delta Lake).

Either pattern works if a consumer can retrieve the entity as it stood at any past time. The store can be a knowledge graph, a property graph, or warehouse tables with semantic metadata. Without a version identifier, nobody can diff the entity before and after a report changes.

Step 6, serving to analytics and AI agents

The published Customer now serves dashboards, applications, and agents. A serving contract states which fields each endpoint exposes, the expected schema, freshness targets, access rules, and deprecation timelines. Consumers that break the contract should fail loudly rather than receive stale or partial data.

For agents, the Model Context Protocol makes the contract concrete. Each MCP tool declares an input schema, and may declare an output schema that servers must conform to (MCP specification). A tool such as get_customer can then return ctx-00382 with its governed fields, and the agent knows a legal name will always be present.

Most deployments combine two or more of these behind one contract. The failure to avoid is quiet drift. An agent built months ago asks for a renamed property, receives nulls, and reasons as if the data were missing.

Which failure modes break the flow most often?

Four failures cause most context incidents. Each one maps to a skipped or weakened artifact, so each has a structural fix rather than a heroic one.

Mapping drift

Two pipelines read the same source with different contract versions. One maps cust_email to Customer.email and the other creates a second property for the same fact. The fix is one versioned semantic contract per source, enforced at ingestion.

Bad merges

Loose match rules collapse two real customers into one, while strict rules leave duplicates. The fix is a merge decision record with confidence scores, a review queue for borderline pairs, and an unmerge operation that uses those records.

Constraint bypass

A team disables the quality gate "temporarily" during a migration. Months later the store holds thousands of entities that break its own rules, and agents treat them as facts. The fix is making the gate mandatory in every publication path, with a logged exception process instead of a bypass flag.

Missing provenance

An executive asks why a revenue figure moved after a Customer update. Without fact level provenance, the team compares snapshots for days. With it, they find the survivorship change that swapped the source for account_id and resolve it within hours.

Who should own the artifacts in this flow?

The company that depends on the flow should own every artifact in it. Definitions, contracts, match rules, constraints, and serving schemas are institutional knowledge that takes years to get right. Kept in open formats, in your own repository, they survive any change of database, catalog, or model provider.

Ownership inside the company follows the artifacts. Domain stewards approve definitions and survivorship rules. A platform team runs the pipeline and the quality gate, and security owns access rules in the serving contract.

Some teams bring in a partner to stand this up on the stack they already run. Galaxy is one such partner, using internal tooling and building in the open with its Apache 2.0 data movement project Filament, with plans to open source more.

How do you check that the flow is working?

Ask one question per step and expect a confident yes. If any answer is "we would have to check," that step is the highest priority gap. Run the checklist for one entity before you widen to the next domain.

Frequently asked questions

What is an enterprise context data flow?

An enterprise context data flow is the path a business entity takes from source systems to the consumers that rely on it. It covers capture, mapping to shared definitions, identity resolution, validation, provenance, publication, and serving. Each step produces a durable artifact, so any fact an analyst or AI agent sees can be traced, checked, and corrected.

What is a semantic contract in a data pipeline?

A semantic contract is a versioned, machine-readable mapping from source fields to governed properties in your shared definitions. It names the source system and schema version, the target property, and any transformation. Old versions stay available so historical data can be reprocessed. Standards such as the Open Data Contract Standard give teams a YAML format to start from.

Why not use the source system key as the entity ID?

A source key belongs to one system and changes when that system is migrated, merged, or replaced. A stable ID is assigned during identity resolution and stays the same across all of those events. Every downstream artifact, from provenance records to agent tool outputs, points at the stable ID, so history survives when the sources underneath it change.

Does constraint validation require SHACL or RDF?

No. SHACL is a useful model because it separates a shapes graph of rules from the data graph being checked. The same pattern works with JSON Schema, dbt tests, or a custom validator. What matters is that constraints live in a versioned artifact outside pipeline code, and that a failed check blocks publication and produces a structured report.

What is the difference between pipeline lineage and fact level provenance?

Pipeline lineage records which jobs ran, what they read, and what they wrote. OpenLineage models this with jobs, runs, and datasets. Fact level provenance goes further and records, for one property on one entity, which source record supplied the value, which survivorship rule chose it, and when it was last validated. Agents and auditors usually need the second kind.

What is a serving contract?

A serving contract is the agreement between the context store and its consumers. It states which entities and fields each endpoint exposes, the expected schema, freshness targets, access rules, and how long deprecated fields keep working. For AI agents, the input and output schemas of each tool are part of that contract and should be versioned like any other interface.

How do entity constraints help AI agents?

Constraints give an agent a reliable schema to reason against. If the agent knows a Customer always has a legal name and belongs to exactly one Account, it can build queries and check answers against those rules. A 2023 benchmark found GPT-4 accuracy rose from 16 to 54 percent when questions ran over a knowledge graph instead of raw SQL.

What happens when an entity fails validation?

The entity is not published. It goes to a remediation queue with a structured report naming the failed constraint, the property, and the value that caused it. A steward or the owning source team fixes the issue, and the entity runs through validation again. The previous published version stays live in the meantime, so consumers never see the failing state.

Which step should a team build first?

Start with Step 0 and Step 1. Capturing source system, schema version, and batch ID at ingestion is cheap, and a single versioned semantic contract per source prevents mapping drift. Identity resolution and validation depend on both. Once those exist for one entity, add the stable ID, the quality gate, and provenance before you serve the entity to an agent.

Who should own the artifacts in this flow?

The company that relies on them should own them. Definitions, mappings, match rules, constraints, and serving contracts are institutional knowledge. Keep them in open formats, in your own repository, on infrastructure you control. Domain stewards approve definition changes, a platform team runs the pipeline, and security owns access rules in the serving contract.