Data Quality Fundamentals: A Practitioner’s Guide to Building Trustworthy Data Pipelines

Reliable data pipelines are only part of the job: the data flowing through them needs to be dependable, too. Data Quality Fundamentals: A Practitioner’s Guide to Building Trustworthy Data Pipelines looks at how teams can detect, investigate, and prevent data quality problems across modern data systems.

Start with the systems behind the data

Barr Moses, Lior Gavish, and Molly Vorwerck begin with the foundations: what data quality means, why it matters, and how operational and analytical data systems differ. The book explores warehouses, lakes, and lakehouses, along with quality metrics and data catalogs—helpful context for understanding where problems arise and how teams can make their data easier to assess.

Put checks into the pipeline

Practical coverage follows data through collection, cleaning, transformation, and testing. Topics include batch and stream processing, schemas and type handling, ETL, and tests using tools such as dbt, Great Expectations, and Deequ. Examples involving Apache Airflow, Kafka, and AWS Kinesis connect quality practices to familiar pipeline technologies.

Monitor, investigate, and improve reliability

The chapters on monitoring and anomaly detection consider freshness, data distributions, schema changes, lineage, and the challenges of false positives and false negatives. Later material turns to data reliability architecture, service-level agreements and objectives, incident response, root-cause analysis, and end-to-end lineage. Case studies and discussion of governance, ownership, and treating data as a product broaden the perspective beyond code and tooling.

For working data teams

This guide is aimed at practitioners concerned with the quality and reliability of data pipelines, including data engineers and other members of data platform teams. Its central idea is practical: trustworthy data depends on a combination of technical checks, visibility into how data moves, and clear organizational responsibility.