How to randomly sample from a Scala list or array?

arrays list scala apache-spark sample
Carter picture Carter · Oct 4, 2015 · Viewed 15.8k times · Source

I want to randomly sample from a Scala list or array (not an RDD), the sample size can be much longer than the length of the list or array, how can I do this efficiently? Because the sample size can be very big and the sampling (on different lists/arrays) needs to be done a large number of times.

I know for a Spark RDD we can use takeSample() to do it, is there an equivalent for Scala list/array?

Thank you very much.

Answer

Marius Soutier picture Marius Soutier · Oct 4, 2015

An easy-to-understand version would look like this:

import scala.util.Random

Random.shuffle(list).take(n)
Random.shuffle(array.toList).take(n)

// Seeded version
val r = new Random(seed)
r.shuffle(...)

Related questions

Difference between Array and List in scala

In what cases I should use Array(Buffer) and List(Buffer). Only one difference that I know is that arrays are nonvariant and lists are covariant. But what about performance and some other characteristics?

Read More
Why are Arrays invariant, but Lists covariant?

E.g. why does val list:List[Any] = List[Int](1,2,3) work, but val arr:Array[Any] = Array[Int](1,2,3) fails (because arrays are invariant). What is the desired effect behind this design decision?

Read More
Scala ListBuffer (or equivalent) shuffle

Is there a simple shuffle function for Scala lists? If not, whats the simplest way to implement? I have a lot of these things to do all over the code, so the simpler the call, the best it is An …

Read More