DP-600 · 2.2.7 · Prepare data
Duplicates and NULLs
Identify and resolve duplicate data, missing data, or null values
intentionally, replace missing values only when a meaningful default exists, and test with IS NULL or IS NOT NULL.
The core idea

Profile data before fixing it. Exact duplicate rows can be removed directly, while duplicate business keys often need a rule such as keeping the latest row with ROW_NUMBER.
means unknown, not zero or an empty string. can replace it when a business-valid default exists.
Do not compare a value with = NULL. Use IS NULL or IS NOT NULL.
Key points
- Exact duplicates: DISTINCT or Remove duplicates
- Duplicate keys: rank rows and keep the intended record
- Missing values: only when the default is meaningful
- : use IS NULL or IS NOT NULL
Click an underlined term for a one-line explanation, or see them all in the glossary .
Exam trap
WHERE Phone = never returns a row, because a comparison with NULL is never true. Use IS NULL.
Sources · 4 Microsoft Learn pages Show ▾
- NULL and UNKNOWN (Transact-SQL)
- COALESCE (Transact-SQL)
- Simplify the data structure (Clean, transform, and load data in Power BI)
- Explore and transform Spark data with Data Wrangler (Microsoft Fabric)
Checked on October 4, 2026. Report a problem
Go deeper on Microsoft Learn
- NULL and UNKNOWN (Transact-SQL)
- COALESCE (Transact-SQL)
- Simplify the data structure (Clean, transform, and load data in Power BI)
- Explore and transform Spark data with Data Wrangler (Microsoft Fabric)
Checked against Microsoft Learn on October 4, 2026. Report a problem with this card