Unable to use for comprehension to map over List within Future

Question

I have this issue that I have to work around every time. I can't map over something that is contained within a Future using a for comprehension.

Example:

import scala.concurrent.ExecutionContext.Implicits.global
import scala.concurrent.Future

val f = Future( List("A", "B", "C") )
for {
  list <- f
  e <- list
} yield (e -> 1)

This gives me the error:

 error: type mismatch;
 found   : List[(String, Int)]
 required: scala.concurrent.Future[?]
              e <- list
                ^

But if I do this it works fine:

f.map( _.map( (_ -> 1) ) )

Should i not be able to do this by using a for comprehension, is the reason it works in my other example that I do not flatmap? I'm using Scala 2.10.0.

Answer 1

Well, when you have multiple generators in a single for comprehension, you are flattening the resulting type. That is, instead of getting a List[List[T]], you get a List[T]:

scala> val list = List(1, 2, 3)
list: List[Int] = List(1, 2, 3)

scala> for (a <- list) yield for (b <- list) yield (a, b)
res0: List[List[(Int, Int)]] = List(List((1,1), (1,2), (1,3)), List((2,1
), (2,2), (2,3)), List((3,1), (3,2), (3,3)))

scala> for (a <- list; b <- list) yield (a, b)
res1: List[(Int, Int)] = List((1,1), (1,2), (1,3), (2,1), (2,2), (2,3),
(3,1), (3,2), (3,3))

Now, how would you flatten a Future[List[T]]? It can't be a Future[T], because you'll be getting multiple T, and a Future (as opposed to a List) can only store one of them. The same problem happens with Option, by the way:

scala> for (a <- Some(3); b <- list) yield (a, b)
<console>:9: error: type mismatch;
 found   : List[(Int, Int)]
 required: Option[?]
              for (a <- Some(3); b <- list) yield (a, b)
                                   ^

The easiest way around it is to simply nest multiple for comprehensions:

scala> for {
     |   list <- f
     | } yield for {
     |   e <- list
     | } yield (e -> 1)
res3: scala.concurrent.Future[List[(String, Int)]] = scala.concurrent.im
pl.Promise$DefaultPromise@4f498585

In retrospect, this limitation should have been pretty obvious. The problem is that pretty much all examples use collections, and all collections are just GenTraversableOnce, so they can be mixed freely. Add to that, the CanBuildFrom mechanism for which Scala has been much criticized makes it possible to mix in arbitrary collections and get specific types back, instead of GenTraversableOnce.

And, to make things even more blurry, Option can be converted into an Iterable, which makes it possible to combine options with collections as long as the option doesn't come first.

But the main source of confusion, in my opinion, is that no one ever mentions this limitation when teaching for comprehensions.