What is the syntax to reverse the ordering for the takeOrdered() method of an RDD in Spark?
For bonus points, what is the syntax for custom-ordering for an RDD in Spark?
Reverse Order
val seq = Seq(3,9,2,3,5,4)
val rdd = sc.parallelize(seq,2)
rdd.takeOrdered(2)(Ordering[Int].reverse)
Result will be Array(9,5)
Custom Order
We will sort people by age.
case class Person(name:String, age:Int)
val people = Array(Person("bob", 30), Person("ann", 32), Person("carl", 19))
val rdd = sc.parallelize(people,2)
rdd.takeOrdered(1)(Ordering[Int].reverse.on(x=>x.age))
Result will be Array(Person(ann,32))