Table of Content Spark Architecture Spark Clusters RDD (Resilient Distributed Dataset) Spark Job, Task and Stage Spark Operations Spark Memory Management Spark Compression Spark Joins Spark SQL Optimization Spark Queries RDD Commands Dataframe Commands User Defined Schema Join Query Loading Table from Database UDF (User Defined Function) Spark SQL Syntax Partitioning & Bucketing Spark Streaming Spark Streaming Spark Structure Streaming Stream Processing Spark Streaming Joins Output Sinks Read from Folder Spark Streaming with Kafka Window Transformations Watermarks Miscellaneous Points Delta Lake (Spark Queries) Spark Config & Settings