Skip to content
datastudyguides

DP-750 · 3.4.3 · Prepare and process data

Schema enforcement and drift

Implement schema enforcement and manage schema drift

Enforce stable target contracts while configuring each ingestion, parser, streaming, and table layer for expected schema changes.

Connect schema rules to pipeline outcomes

Microsoft Learn flow diagram showing pipeline expectation outcomes: warn, drop, or fail the flow.

Schema drift becomes an operational issue when incoming structure no longer matches the contract. The expectation flow shows how a governed pipeline responds; the diagrams below distinguish enforcement from controlled evolution.

Separate protection from change

  • Schema enforcement

    Focus
    Protect target contract
    Risk
    Incompatible write
    Response
    Reject or control

  • Schema drift

    Focus
    Detect source change
    Risk
    New or changed fields
    Response
    Evolve, quarantine, or fail
Enforcement and drift

Schema enforcement prevents incompatible writes from silently changing the target table.

Evolution spans the whole path

  1. 1SourceAdds or changes a field
  2. 2Connector or parserDetects the new shape
  3. 3Streaming engineApplies its evolution policy
  4. 4Delta targetAccepts or rejects the change
Schema change path

Each layer can have different schema behavior.

Be explicit in production

Allow only changes you intend to support. Auto Loader discovering a new source column does not automatically mean the downstream Delta table should evolve.

Key points

  • Enforcement: Rejects incompatible writes to a defined schema
  • Evolution: Allows selected schema changes such as new columns
  • Drift: Source structure changes over time
  • Layers: Connector, parser, engine, and dataset evolve independently

Exam trap

Auto Loader schema evolution does not automatically evolve the downstream Delta table unless that layer is configured too.

Sources · 2 Microsoft Learn pages Show ▾

Checked on October 4, 2026. Report a problem

Go deeper on Microsoft Learn

Checked against Microsoft Learn on October 4, 2026. Report a problem with this card

How well do you know this?