Futures - map vs flatmap

Question

I've read the docs about map and flatMap and I understand that flatMap is used for an operation that accepts a Future parameter and returns another Future. What I don't fully understand is why I would want to do this. Take this example:

1. User hits my webservice asking to "do stuff"
2. I download a file (which is slow)
3. I process the file (which is CPU intensive)
4. Render the result

I understand that I would want to use a future to download the file but I have have two options re processing it:

val downloadFuture = Future {/* downloadFile */}
val processFuture = downloadFuture map {/* processFile */}
processFuture onSuccess { case r => renderResult(r) }

or

val downloadFuture = Future {/* download the file */}
val processFuture = downloadFuture flatMap { Future {/* processFile */} }
processFuture onSuccess { case r => renderResult(r) }

By adding debug statements (Thread.currentThread().getId) I see that in both cases download, process and render occur in the same thread (using ExecutionContext.Implicits.global).

Would I use flatMap simply to decouple downloadFile and processFile and ensure that processFile always runs in a Future even if it was not mapped from downloadFile?

Answer 1

If you have a future, let's say, Future[HttpResponse], and you want to specify what to do with that result when it is ready, such as write the body to a file, you may do something like responseF.map(response => write(response.body). However if write is also an asynchronous method which returns a future, this map call will return a type like Future[Future[Result]].

In the following code:

import scala.concurrent.Future
import scala.concurrent.ExecutionContext.Implicits.global

val numF = Future{ 3 }

val stringF = numF.map(n => Future(n.toString))

val flatStringF = numF.flatMap(n => Future(n.toString))

stringF is of type Future[Future[String]] while flatStringF is of type Future[String]. Most would agree, the second is more useful. Flat Map is therefore useful for composing multiple futures together.

When you use for comprehensions with Futures, under the hood flatMap is being used together with map.

import scala.concurrent.{Await, Future}
import scala.concurrent.ExecutionContext.Implicits.global
import scala.concurrent.duration._

val threeF = Future(3)
val fourF = Future(4)
val fiveF = Future(5)

val resultF = for{
  three <- threeF
  four <- fourF
  five <- fiveF
}yield{
  three * four * five
}

Await.result(resultF, 3 seconds)

This code will yield 60.

Under the hood, scala translates this to

val resultF = threeF.flatMap(three => fourF.flatMap(four => fiveF.map(five => three * four * five)))

Answer 2

ensure that processFile always runs in a Future even if it was not mapped from downloadFile?

Yes that is correct.

However most of the time you wouldn't use Future { ... } directly, you would use functions (from other libraries or your own) which return a Future.

Imagine the following functions :

def getFileNameFromDB{id: Int) : Future[String] = ???
def downloadFile(fileName: String) : Future[java.io.File] = ???
def processFile(file: java.io.File) : Future[ProcessResult] = ???

You could use flatMap to combine them :

val futResult: Future[ProcessResult] =
  getFileNameFromDB(1).flatMap( name =>
    downloadFile(name).flatMap( file =>
       processFile(file)
    )
  )

Or using a for comprehension :

val futResult: Future[ProcessResult] =
  for {
    name <- getFileNameFromDB(1)
    file <- downloadFile(name)
    result <- processFile(file)
  } yield result

Most of the time you would not call onSuccess (or onComplete). By using one of these functions you register a callback function which will be executed when the Future finishes.

If in our example you would like to render the result of the file processing, you would return something like Future[Result] instead of calling futResult.onSuccess(renderResult). In the last case your return type would be Unit, so you can not really return something.

In Play Framework this could look like :

def giveMeAFile(id: Int) = Action.async {
  for {
    name <- getFileNameFromDB(1)
    file <- downloadFile(name)
    processed <- processFile(file)
  } yield Ok(processed.byteArray).as(processed.mimeType))
}

Answer 3

def flatMap[B](f: A => Option[B]): Option[B] = 
  this match {
    case None => None
    case Some(a) => f(a)
  }

This is a simple example where how the flatMap works for Option, this can help to understand better, It is actually composing it is not adding a wrapper again.That's what we need.

Answer 4

Failure possibility from transformation function another reason why there exists flatMap on Future.

say you have a f: Future[T]. and a transformation func: T => B. however this func could fail due to some reason. so we want to indicate the caller it has failed.

with just Future.map it's not obvious how to achieve this. but with flatMap you can. because with flatMap it takes Future as return, and you can then easily do Future.failed(e) to bubble up the error to the caller. or if it succeeded, you can then use Future.success(r) to return the result.

aka. turn func into func: T => Future[B]

this is very useful when you are chaining operations with Future and operations could fail in the middle.