Performance Optimization Guide

speed

Standard Optimizations

align_horizontal_left

Skewed Joins

Common

Handle data imbalance across nodes using salting or map-side joins.

memory

Memory Errors

Critical

Resolving OutOfMemory (OOM) exceptions in driver and executor.

Advanced Techniques

shuffle

Shuffle Tuning

High Impact

Configuring shuffle partitions and spill buffer sizes dynamically.

cell_tower

Broadcast Hints

Impactful

Force broadcast joins for small tables to prevent network shuffles.