← Data Engineering & ETL learning pathData Engineering & ETL · Module 7 of 10
Data quality and testing
Catch bad data before it reaches users.
📖 Key ideas
- Test for nulls, uniqueness, valid values and relationships.
- dbt tests: unique, not_null, accepted_values, relationships.
- Great Expectations and Soda run data checks.
- Monitor freshness and volume anomalies.
- Fail fast and alert instead of publishing wrong numbers.
💡 Real-life example: Data quality checks are quality inspectors on an assembly line, rejecting faulty products.
not_null testEnsures no missing values
FreshnessHow recent data is
Anomaly detectionFinding unusual changes
Data observabilityMonitoring health of data