Skip to content
datastudyguides

DP-750 · 4.4.5 · Deploy and maintain data pipelines and workloads

Optimize Delta tables

Optimize Delta tables for performance and cost, including OPTIMIZE and VACUUM commands

Use OPTIMIZE to improve file layout and VACUUM to remove expired unreferenced files, with predictive optimization preferred for managed tables.

Diagnose before you optimize

Microsoft Learn Azure Databricks Spark UI screen for a running compute resource.

Table optimization should respond to observable scan, shuffle, skew, and task behavior rather than habit. The Spark UI gives the execution evidence; the diagrams below explain compaction and data layout choices.

Separate layout from cleanup

  • OPTIMIZE

    Purpose
    Improve active file layout
    Logical rows
    Unchanged
    Primary benefit
    Better reads and fewer small files
    Main risk
    Unnecessary rewrite work

  • VACUUM

    Purpose
    Remove expired unreferenced files
    Logical rows
    Unchanged
    Primary benefit
    Reclaim storage
    Main risk
    Too-short history retention
OPTIMIZE and VACUUM

OPTIMIZE rewrites active data files. VACUUM removes old unreferenced files after the retention threshold.

Protect recoverability

  1. 1Query patternDecide whether layout needs work
  2. 2OPTIMIZECompact or apply clustering layout
  3. 3Retention requirementProtect history and long readers
  4. 4VACUUMRemove only eligible files
Safe maintenance

Do not shorten retention merely to reduce storage if time travel or long-running operations still need those files.

Prefer managed automation

For supported Unity Catalog managed tables, predictive optimization can schedule OPTIMIZE, VACUUM, and ANALYZE when Databricks determines maintenance is worthwhile.

Key points

  • OPTIMIZE: Rewrites files for better layout and compaction
  • Liquid clustering: OPTIMIZE applies clustering layout
  • VACUUM: Deletes unreferenced files older than retention
  • Predictive optimization: Automates OPTIMIZE, VACUUM, and ANALYZE for managed tables

Exam trap

VACUUM with overly short retention can destroy time-travel history and endanger long-running operations.

Sources · 2 Microsoft Learn pages Show ▾

Checked on October 4, 2026. Report a problem

Go deeper on Microsoft Learn

Checked against Microsoft Learn on October 4, 2026. Report a problem with this card

How well do you know this?