Skip to main content
MindLeap AI MindLeap AI
Intermediate Level

PySpark Certification

★ 4.8 (300 learners) 60 hrs total 20 modules
Databricks Apache Spark PySpark
Only 5 seats left · Next batch starts 22 Aug 2026
Download Curriculum
60 hrs
of content
20 modules
in curriculum
16 projects
hands-on builds
Intermediate
difficulty

Curriculum covers

Frameworks you'll master

3 tools & frameworks
Databricks Apache Spark PySpark

Course Curriculum

20 modules · 60 hours · 24 live sessions

Understanding Big Data and Distributed Data Processing 1 sessions
  • Understanding Big Data and Distributed Data Processing
  • What is Big Data and How it Started
  • Hadoop Architecture, History and Evolution
  • Data Lake and Lakehouse Architecture
  • Introduction to Apache Spark (Spark Installation, Components & Architecture)
  • What is Apache Spark
  • Apache Spark System Architecture
  • Apache Spark Components & Architecture
Spark Platform and Development Environment 1 sessions
  • Spark Platform and Development Environment
  • What is Databricks Cloud
  • Create your Databricks Free Account
  • Setup your handson environment
  • Spark Installation
  • Spark Main Components
  • Spark & its features
  • Deployment Modes of Spark
Introduction to PySpark Shell 1 sessions
  • Introduction to PySpark Shell
  • Submitting PySpark Jobs
  • Utilizing Spark Web UI
  • Getting started with Spark Programming
  • Starting Point Spark Session
  • Introduction to Spark Session
  • Spark Object and Spark Submit Part 1
  • Spark Object and Spark Submit Part 2
Spark Object and Spark Submit Part 3 1 sessions
  • Spark Object and Spark Submit Part 3
  • DataFrame – A view to Structured data
  • DataFrame Transformations and Actions
  • DataFrame Concepts
  • Exploring DataFrame Transformations
  • Creating Spark DataFrame
  • Spark Cluster Execution Architecture
  • Full Architecture
YARN As Spark Cluster Manager 1 sessions
  • YARN As Spark Cluster Manager
  • JVMs across Clusters
  • Commonly Used Terms in Execution Frame
  • Narrow and Wide Transformations
  • DAG Scheduler Part 1
  • DAG Scheduler Part 2
  • DAG Scheduler Part 3
  • Task Scheduler
Spark Shared Variables 1 sessions
  • Spark Shared Variables
  • Broadcast Variable
  • Accumulator Variable
  • Spark SQL Introduction & Architecture
  • Spark SQL Architecture Part 1
  • Spark SQL Architecture Part 2
  • Spark SQL Architecture Part 3
  • Spark SQL Architecture Part 4
Spark SQL Architecture Part 5 1 sessions
  • Spark SQL Architecture Part 5
  • Spark SQL Architecture Part 6
  • Spark SQL Architecture Part 7
  • Spark SQL Architecture Part 8
  • Working with SQL Context and Schema RDDs
  • User Defined Functions (UDFs) in Spark SQL
  • Data Frames, Datasets, and Interoperability with RDDs
  • Loading Data from Different Sources
Integration of Spark with Hive for Data Warehousing 1 sessions
  • Integration of Spark with Hive for Data Warehousing
  • Spark Shared Variables
  • Broadcast Variable
  • Accumulator Variable
  • Spark Session Features
  • Introduction to Spark Session
  • Spark Object and Spark Submit Part 1
  • Spark Object and Spark Submit Part 2
Spark Object and Spark Submit Part 3 1 sessions
  • Spark Object and Spark Submit Part 3
  • Version and Range
  • createDataFrame
  • sql
  • Table
  • sparkContext
  • udf
  • readcsv
readtext 1 sessions
  • readtext
  • readorc and parquet
  • readjson
  • readavro
  • readhive
  • readjdbc
  • 23min
  • Catalog
23min 1 sessions
  • 23min
  • newSession and stop
  • DataFrame Fundamentals
  • Introduction to DataFrame
  • DataFrame Features Distributed
  • DataFrame Features Lazy Evaluation.
  • DataFrame Features Immutability
  • DataFrame Features Other Features
Organization 1 sessions
  • Organization
  • DataFrame ETL
  • Introduction to Transformations and Extraction
  • DataFrame APIs Introduction Extraction
  • DataFrame APIs Selection
  • DataFrame APIs Filter or Where
  • DataFrame APIs Sorting
  • DataFrame APIs Set
DataFrame APIs Join 1 sessions
  • DataFrame APIs Join
  • DataFrame APIs Aggregation
  • DataFrame APIs GroupBy
  • DataFrame APIs Window Part 1
  • DataFrame APIs Window Part 2
  • DataFrame APIs Sampling Functions
  • DataFrame APIs Other Aggregate Functions
  • DataFrame Builtin Functions Introduction
DataFrame Builtin Functions New Column Functions 1 sessions
  • DataFrame Builtin Functions New Column Functions
  • DataFrame Builtin Functions Column Encryption
  • DataFrame Builtin Functions String Functions
  • DataFrame Builtin Functions RegExp Functions
  • DataFrame Builtin Functions Date Functions
  • DataFrame Builtin Functions Null Functions
  • DataFrame Builtin Functions Collection Functions
  • DataFrame Builtin Functions na Functions
DataFrame Builtin Functions Math and Statistics Functions 1 sessions
  • DataFrame Builtin Functions Math and Statistics Functions
  • DataFrame Builtin Functions Explode and Flatten Functions
  • DataFrame Builtin Functions Formatting Functions
  • DataFrame Builtin Functions Json Functions
  • Need of Repartition and Coalesce
  • How to Repartition a DataFrame
  • How to Coalesce a DataFrame
  • Repartition Vs Coalesce Method of a DataFrame
DataFrame Extraction Introduction 1 sessions
  • DataFrame Extraction Introduction
  • DataFrame Extraction csv
  • DataFrame Extraction text
  • DataFrame Extraction Parquet
  • DataFrame Extraction orc json
  • DataFrame Extraction avro
  • DataFrame Extraction hive
  • DataFrame Extraction jdbc
DataFrame Transformations 5 sessions
  • Adding, Removing, and Renaming Columns
  • DataFrame Column Expressions
  • Filtering and removing duplicates
  • Sorting, Limiting and Collecting
  • Transforming Unstructured data
  • Transforming data with LLM
  • Working with Nulls
  • Working with Numbers
  • Manipulating Strings
  • Working with Date
  • Working with Timestamps
  • Handling Time Zone Information
  • Working with Complex Data Types
  • Working with JSON data
  • Working with Variant Type
  • Introduction to Joins in Spark
  • Inner Joins
  • Outer Joins
  • Lateral Join
  • Other Types of Joins
  • Simple Aggregation
  • Grouping Aggregation
  • Multilevel Aggregation
  • Windowing Aggregation
  • UserDefined Functions
  • Vectorized UDF
  • User Defined Table Functions
  • Unit Testing Spark Code
  • Execution Methods How to Run Spark Programs?
  • Spark Distributed Processing Model How your program runs?
  • Spark Execution Modes and Cluster Managers
  • Summarizing Spark Execution Models When to use What?
  • Working with PySpark Shell Demo
  • Installing MultiNode Spark Cluster Demo
  • Working with Notebooks in Cluster Demo
  • Working with Spark Submit Demo
  • Creating Spark Project Build Configuration
  • Configuring Spark Project Application Logs
  • Creating Spark Session
  • Configuring Spark Session
  • Data Frame Introduction
  • Data Frame Partitions and Executors
  • Spark Transformations and Actions
  • Spark Jobs Stages and Task
  • Understanding your Execution Plan
  • Introduction to Spark APIs
  • Introduction to Spark RDD API
  • Working with Spark SQL
  • Spark SQL Engine and Catalyst Optimizer
Spark Data Sources and Sinks 3 sessions
  • Spark Data Sources and Sinks
  • Spark DataFrameReader API
  • Reading CSV, JSON and Parquet files
  • Creating Spark DataFrame Schema
  • Spark DataFrameWriter API
  • Writing Your Data and Managing Layout
  • Spark Databases and Tables
  • Working with Spark SQL Tables
  • Introduction to Data Transformation
  • Working with DataFrame Rows
  • DataFrame Rows and Unit Testing
  • DataFrame Rows and Unstructured data
  • Working with DataFrame Columns
  • Misc Transformations
  • DataFrame Joins and column name ambiguity
  • Outer Joins in DataFrame
  • Internals of Spark Join and shuffle
  • Optimizing your joins
  • Implementing Bucket Joins
  • Optimizing Data Frame Transformations
  • General Join Mechanisms
  • Shuffled Join
  • Optimized Join
  • Broadcast Join
Caching & Checkpointing 1 sessions
  • Caching & Checkpointing
  • Skew Joins
  • Partitioning
  • Repartition & Coalesce
  • Shuffle Partitions
  • Performance Tuning Problems
  • Optimizing Cluster Resource Allocation
  • Serialization Problem
Fixing Data Skew (Data Skewness) & Straggling Task 1 sessions
  • Fixing Data Skew (Data Skewness) & Straggling Task
  • Partitioning
  • Repartition & Coalesce
  • Shuffle Partitions
  • Custom Partitioners
  • Column Pruning
  • Optimizing Apache Spark on Databricks (Performance & Optimization)
  • Five Most Common Problems with Spark Applications
Download Full Curriculum

Batch Schedule

⚡ Seats filling fast — only 5 remaining across all batches

Available Batches
Batch Type Schedule Duration Next Start Date Availability
Weekend Sat & Sun, 10am–1pm 10 weeks 22 Aug 2026 Available
Weekday Mon–Fri, 7–9pm 8 weeks 24 Aug 2026 Available
Fast-track Mon–Sun, 6 hours/day 2 weeks 31 Aug 2026 Only 5 seats left

Ready to Get Started?

Download the curriculum to see every module, batch dates and fees. Book a free 30-minute session with our advisors to get your questions answered.

Book Free Demo

Download the Full Curriculum

See every module, capstone project, and tool covered — free, instant access.

Similar Courses You Might Like

View Course →
Data Engineering
data engineering

AWS GLUE Lambda Certification

★ 4.8 · 300 learners · Intermediate
Starts 8 Aug 2026 · ~1–2 months
View Course →
Data Engineering
data engineering

Databricks - Master Databricks for Data Engineers

★ 4.8 · 300 learners · Intermediate
Starts 15 Aug 2026 · ~1–2 months
View Course →
Data Engineering
data engineering

Databricks Certified Data Engineer Associate

★ 4.8 · 300 learners · Intermediate
Starts 22 Aug 2026 · ~1–2 months
View Course →
Data Engineering
data engineering

Databricks Certified Data Engineer Professional

★ 4.8 · 300 learners · Advanced
Starts 29 Aug 2026 · ~1–2 months
Book Free Demo