What is the difference between repartition and coalesce?

Answer»

Repartition	Coalesce
Usage repartition can increase/decrease the NUMBER of data partitions.	Spark coalesce can only reduce the number of data partitions.
Repartition creates NEW data partitions and performs a full SHUFFLE of evenly distributed data.	Coalesce makes use of already existing partitions to reduce the amount of shuffled data unevenly.
Repartition internally calls coalesce with shuffle parameter thereby making it SLOWER than coalesce.	Coalesce is faster than repartition. HOWEVER, if there are unequal-sized data partitions, the speed might be slightly slower.

Discussion