Skip to content
datastudyguides

DP-750 · 3.2.6 · Prepare and process data

Event Hubs ingestion

Ingest streaming data from Azure Event Hubs

Consume Azure Event Hubs through its Kafka-compatible endpoint with Structured Streaming and keep credentials outside code.

See the Event Hubs partition model

Azure Event Hubs architecture showing producers, protocols, event hubs with partitions, consumer groups, and consumers.

Event Hubs scales ingestion through partitions and lets independent consumer groups read the same stream. The diagrams below then connect that broker model to Spark Structured Streaming.

Use Event Hubs through Kafka

  1. 1Azure Event HubsReceives events
  2. 2Kafka-compatible endpointExposes the stream
  3. 3Spark Kafka sourceReads incrementally
  4. 4CheckpointPersists offsets and recovery state
  5. 5Delta targetStores governed output
Event Hubs to Databricks

The supported Databricks path is the Kafka-compatible endpoint with the built-in Structured Streaming Kafka source.

Keep secrets outside code

  1. 1Event Hubs policy keySensitive credential
  2. 2Databricks secretStores it securely
  3. 3Notebook or pipelineRetrieves at runtime
  4. 4Kafka optionsBuild the connection configuration
Secure connection

Do not hard-code policy keys in notebook or pipeline source.

Avoid the wrong connector

Lakeflow pipelines should use the Kafka-compatible integration rather than an unsupported third-party Event Hubs JVM connector.

Key points

  • Kafka endpoint: Event Hubs exposes Kafka-compatible connectivity
  • Kafka connector: Native Structured Streaming integration
  • Secrets: Store connection credentials securely
  • Checkpoint: Persist streaming progress for recovery

Exam trap

Lakeflow pipelines should use the Kafka-compatible endpoint, not the unavailable Event Hubs JVM connector.

Sources · 2 Microsoft Learn pages Show ▾

Checked on October 4, 2026. Report a problem

Go deeper on Microsoft Learn

Checked against Microsoft Learn on October 4, 2026. Report a problem with this card

How well do you know this?