2

I'm having syntax issues with a for-comprehension in Scala 2.10.

for(a <- List(Some(1,2)); b <- a) yield b evaluates to List(1,2)

so why doesn't for(a <- Some(List(1,2)); b <- a) yield b evaluate to the same thing?

The type-checker complains about the second expression (b <-a) saying it found a List[Int] when expecting an Option[?]

Daniel Schobel
  • 498
  • 4
  • 12
  • possible duplicate of [Type Mismatch on Scala For Comprehension](http://stackoverflow.com/questions/4719592/type-mismatch-on-scala-for-comprehension) – om-nom-nom Feb 19 '13 at 07:06
  • it's also duplicate of [Confused with the for-comprehension to flatMap/Map transformation](http://stackoverflow.com/questions/14598990/confused-with-the-for-comprehension-to-flatmap-map-transformation) – pagoda_5b Feb 19 '13 at 08:55

2 Answers2

7

I've explained this very recently -- hopefully someone can find the link. It's a recurring question, so it may well be closed.

At any rate, the outer generator controls the representation. That happens on each level, so if I have this:

for { 
  a <- A
  b <- B
  c <- C
} yield f(a, b, c)

Then the representation of f(a, b, c) is controlled by C, the representation of that is controlled by B, and the representation of the final result is controlled by A. So, for most practical purposes, the representation of the for comprehension is controlled by the first generator.

So what do I mean by "representation"? Well, a for comprehension is a monadic comprehension, usually (actually, it's just a set of calls to methods like flatMap and map, so it can be anything that typechecks). That means that given a monad M[A] and a function A => M[B], then you can transform M[A] in M[B], where M, the monad, is the "representation".

This means that, for the most part, it would be impossible to combine Option and List in a for comprehension. All the collections have a common parent in GenTraversableOnce, so there's no problem in combining them (though things are much more complicated than that under the hood).

However, there is an implicit conversion from Option to Iterable. That being the case, when Scala finds b <- a in the first example, and knowing it can't pass an Option because the comprehension is being "controlled" by a List, it converts the Option into an Iterable, and everything works.

However, that doesn't happen in the second case. Doing a for comprehension with an Option is ok, so there's no need to convert it into an Iterable. Unfortunately, one cannot convert a List into an Option (what would be the result of such a conversion?), which results in the error.

Scala will not "backtrack" to a <- Some(List(1, 2)) and apply an implicit conversion to it, as type inference in Scala only goes forward -- what it decided before will remain as is.

I heartily recommend you look at the related questions and understand how a for comprehension is translated.

Daniel C. Sobral
  • 295,120
  • 86
  • 501
  • 681
  • Thanks Daniel. I wouldn't be surprised if the question is a duplicate because it is hard to search for and the "Tour of Scala" doc on comprehensions doesn't explain this behaviour. The thing which really tripped me up was that for(a <- Some(List(1,2)); b <- a){println(b)} behaves as I expect, namely 'b' evaluates to the underlying Ints. – Daniel Schobel Feb 19 '13 at 15:53
  • @DanielSchobel Well, in the absence of `yield`, you are not returning anything, so there's no problem. Since you can't have a `Some(1, 2)`, you can't have the `yield` version. – Daniel C. Sobral Feb 19 '13 at 23:22
2

Daniel explained the intricacies of this. But I would like to add some detail, because as you I find this behaviour incredibly unintuitive, and I have come across this problem of mixing Option and List a few times myself. It is especially annoying, because as you see, it works in one direction but not the other.

So according to the for-comprehension rules you'll have

def test(a: Option[List[Int]]) = a.flatMap(_.map(identity))

failing with

<console>:7: error: type mismatch;
 found   : List[Int]
 required: Option[?]
           def test(a: Option[List[Int]]) = a.flatMap(_.map(identity))
                                                           ^

But you can make it work:

def test(a: Option[List[Int]]) = (a: Iterable[List[Int]]).flatMap(_.map(identity))

test(Some(List(1,2)))  // List(1,2)

Or back with for:

for(a <- Some(List(1,2)).toIterable; b <- a) yield b

Should for impose this conversion by itself? I don't know honestly, but I share your surprise that it doesn't work.

0__
  • 66,707
  • 21
  • 171
  • 266
  • Upvoted for the work-around, and I agree, it does seem like it _should_ work in its original form, especially as I point out in my comment to Daniel that for(a <- Some(List(1,2)); b <- a){println(b)} works exactly as you would expect, namely, b evals to the underlying Ints. – Daniel Schobel Feb 19 '13 at 16:01