Partitioning & Bucketing

Dynamic Partition

salesDF.write.partitionBy("Country").save("path")

Bucketing

Number of files created = No. of partitions * No. of buckets

salesDF.write.bucketBy(4,"City").sortBy("City").saveAsTable("lti871.salebuck")

Partitioning & Bucketing

salesdf.write.partitionBy("Country").bucketBy(10,"city").sortBy("City").saveAsTable("lti871.salebuck")

NOTE

BucketBy() does not support .save(), .insertInto() and .jdbc() so use saveAsTable()