In the Spark UI a data engineer sees one join stage where the max shuffle-read task is 5 GB while the median is 60 MB, and a single straggler task keeps running long after all others finish. AQE is enabled but the skew persists on a few extremely hot keys. Which action best mitigates the skew?
Answer & explanation
Correct answer: B
WHY B: Salting adds a random prefix to hot keys so their rows spread across many partitions; you join on the salted key then strip and re-aggregate, which is the standard fix when AQE alone cannot tame a few extreme keys. WHY NOT A: More executors do not help because a single oversized partition still runs on one core; the imbalance, not total capacity, is the bottleneck. WHY NOT C: Setting shuffle.partitions to 1 forces everything into one partition, making the straggler problem worse, not better. WHY NOT D: autoBroadcastJoinThreshold = -1 disables auto-broadcast rather than forcing it, and broadcasting a large table would OOM the driver. WHY NOT E: cache() avoids recomputation but does nothing about the uneven partition sizes causing the skew and straggler.