YouTubePrice
This course is free to watch on its platform.
Watch free on YouTube →Mark your own learning status. Opening a course link does not establish enrollment.
Loading saved status…
The course starts with Spark's history, architecture and APIs, then moves through reading data in several formats and into transformations and performance. Delta Lake, the Databricks platform and production topics follow, and the sequence ends with full projects and interview question sets.
Complete beginners who want a single path rather than scattered tutorials will find the structure useful. Working engineers who need to fill specific gaps can dip into the relevant section if they already know the basics.
Work through the Spark foundations before the platform sections, since later topics assume the core API is familiar.
A complete PySpark and Databricks course, free and in order. 58 videos, around 30 hours, from what Spark is through to production pipelines and interview preparation.
Watch from the top. Each section builds on the one before it.
SPARK FOUNDATIONS
History and architecture, deployment modes, the Spark APIs, and setting up a free Databricks account to follow along.
READING DATA
CSV, JSON, Parquet, unstructured text and binary files, JDBC sources, and handling corrupted records with read modes.
TRANSFORMATIONS AND PERFORMANCE
Narrow and wide transformations, joins and join strategies, aggregations and window functions, arrays and structs, the Catalyst optimizer and execution plans, reading the Spark UI, cache vs persist, write modes and partitioning, and UDFs. Every transformation is shown in both PySpark and SQL.
DELTA LAKE
OPTIMIZE, ZORDER and VACUUM, shallow vs deep cloning, liquid clustering vs partitioning, and change data feed.
DATABRICKS PLATFORM
Notebooks and magic commands, secret scopes, DBUtils, jobs with loops and failure handling, Auto Loader, cluster types and costs, Unity Catalog, debugging, and job run cost calculation.
PRODUCTION AND ADVANCED
LakeFlow Connect, declarative pipelines, CI/CD with Databricks Asset Bundles, Metric Views and the semantic layer, and Genie.
PROJECTS AND INTERVIEW PREP
A finance domain end-to-end build, a retail project, a full data modeling course, and interview question sets for Spark, Delta Lake and Databricks.
Every hands-on demo runs on the Databricks Free Edition, so following along costs nothing.
Who it is for: complete beginners who want a path rather than scattered tutorials, and working engineers filling specific gaps. New videos are added to this playlist as they are published.
#PySpark #Databricks #ApacheSpark #DataEngineering #DeltaLake #SparkTutorial
Advertisement
No Korshub reviews yet.
Data Engineering, Big Data, and Machine Learning on GCP
