Overview
Discover the basics of big data with a data science expert. Learn about how to perform core data engineering tasks including staging, profiling, cleansing, and migrating data.
Syllabus
Introduction
- Welcome
- What you should know before watching this course
- Using the exercise files
1. Ecosystem Overview
- Data science system overview
- Star schema design overview
- Where does data engineering fit?
- Components of a good data pipeline
- Environment setup
2. Staging Data
- Loading and profiling data
- Data quality testing
3. Cleansing Data
- Adding data types
- Handling missing values
- Verifying addresses
4. Conforming Data
- Performing master data lookups
- Handling inferred members
5. Delivering Analytical Data Sets
- Loading the star schema
- Loading dimension tables
- Loading fact tables
- Creating views
- Next steps
Conclusion