Amir A. Shabani
Amir A. Shabani
Programming language:Whatever
2021-05-31 06:43:54
0
Q:

spark aggregatebykey

Radz
Radz
Code: Whatever
2021-08-05 02:58:09 
val resReduce = pairs.reduceByKey(_ + _) //the same operation for everything
resReduce.collect
res3: Array[(String, Int)] = Array((b,7), (a,9))

//0 is initial value, _+_ inside partition, _+_ between partitions
val resAgg = pairs.aggregateByKey(0)(_+_,_+_)
resAgg.collect
res4: Array[(String, Int)] = Array((b,7), (a,9))
0

Related

New to Communities?

Join the community