Abstract: This talk is for data engineers and scientists who are interested in the art of data quality. Too often, data reliability is centered on finding what was already broken. Meanwhile, many data quality issues result from bugs introduced in the code that processes data and can be prevented with better testing pre-production.
We will discuss why the cultural shift from “How fast can I find and fix pipeline issues?” to “How can I prevent 90% of my pipeline issues from occurring in the first place?” is critical to achieving consistent data quality. This talk also explains why data engineers and scientists need to undertake this cultural shift in thinking before seeking out tooling (and not the other way around).
By attending this session, you’ll understand:
1. Why achieving data quality is so difficult
2. Why proactive data QA processes are the foundation of true data quality
3. How data catalogs and lineage can be used to enhance stakeholder relationships and build more trust
Bio: Bio Coming Soon!