Warnings suppressed with mclapply in R

Question

With mclapply() all issued warnings seems get suppressed:

library(multicore) 
mclapply(1:3, function(x) warning(x))
[[1]]
[1] "1"

[[2]]
[1] "2"

[[3]]
[1] "3"

while lapply would give:

lapply(1:3, function(x) warning(x))
[[1]]
[1] "1"

[[2]]
[1] "2"

[[3]]
[1] "3"

Warning messages:
1: In FUN(1:3[[3L]], ...) : 1
2: In FUN(1:3[[3L]], ...) : 2
3: In FUN(1:3[[3L]], ...) : 3

Any tips on how to avoid loosing the warnings?

score 5 · Answer 1 · answered Feb 01 '14 at 11:22

According to mclapply's help page, in my opinion the argument mc.silent should allow you to chose if warnings are to be printed or not. Strangely, it does not do that. Setting it explictly to TRUE or FALSE does not have any effect in your situation.

So that leaves us only with a somewhat dirty hack: forcing R to print warnings as they occur.

options(warn=1)
mclapply(1:3, function(x) warning(x))

# Warning in FUN(1L[[1L]], ...) : 1
# Warning in FUN(2L[[1L]], ...) : 2
# Warning in FUN(3L[[1L]], ...) : 3
# [[1]]
# [1] "1"
#
# [[2]]
# [1] "2"
#
# [[3]]
# [1] "3"

Note that this doesn't work in GUI or RStudio. But the docs warn explicitly against using `mclapply` with these. — Roland, Feb 01 '14 at 16:32

score 2 · Answer 2 · answered Oct 30 '15 at 01:30

I have this problem too. If I'm reading the code correctly, parallel::mclapply() passes the mc.silent option to parallel:mcparallel(). But mcparallel() has this line:

sendMaster(try(eval(expr, env), silent = TRUE))

I think that's the place where the warnings would be sent back to the master process, but the mc.silent is not respected. That's my best guess about what is going on.

Anatoly Vasilyev · Answer 3 · 2020-08-12T09:38:49.593

For anyone who will come around the same issue, here is a workaround:

safe_mclapply <- function(X, FUN, mc.cores, stop.on.error=T, ...){
  fun <- function(x){
    res_inner <- tryCatch({
      withCallingHandlers(
        expr = {
          FUN(x, ...)
        }, 
        warning = function(e) {
          message_parallel(trimws(paste0("WARNING [element ", x,"]: ", e)))
          # this line is required to continue FUN execution after the warning
          invokeRestart("muffleWarning")
        },
        error = function(e) {
          message_parallel(trimws(paste0("ERROR [element ", x,"]: ", e)))
        }
      )},
      error = function(e){
        # error is returned gracefully; other results of this core won't be affected
        return(e)
      }
    )
    return(res_inner)
  }
  
  res <- mclapply(X, fun, mc.cores=mc.cores)
  failed <- sapply(res, inherits, what = "error")
  if (any(failed == T)){
    error_indices <- paste0(which(failed == T), collapse=", ")
    error_traces <- paste0(lapply(res[which(failed == T)], function(x) x$message), collapse="\n\n")
    error_message <- sprintf("Elements with following indices failed with an error: %s. Error messages: \n\n%s", 
                             error_indices,
                             error_traces)
    if (stop.on.error)
      stop(error_message)
    else
      warning(error_message, "\n\n### Errors will be ignored ###")
  }
  return(res[!failed])
}

#' Function which prints a message using shell echo; useful for printing messages from inside mclapply when running in Rstudio
message_parallel <- function(...){
  system(sprintf('echo "\n%s\n"', paste0(..., collapse="")))
}

safe_mclapply above is a wrapper around mclapply. For each iteration it uses withCallingHandlers to catch and print warnings and errors; note that invokeRestart("muffleWarning") is required in order to continue FUN exection and return the result. Printing is done via message_parallel function which uses shell echo to print messages to R console (tested to work in Rstudio).

safe_mclapply provides few more features which you might find optional:

along with the warning and error it prints character representation of x which I find useful because it gives an idea where the message comes from
tryCatch around withCallingHandlers helps to return an error gracefully so that other results of the core are not affected
after mclapply is executed, the indices of error results are printed
stop.on.error provides an option to ignore any results which contain error and continue despite the errors

Side note: I personally prefer pbmclapply function from pbmcapply over mclapply which adds a progress bar. You can change mclapply to pbmclapply in the code above.

Small snippet to test the code:

X <- list(1, 2, 3, 4, 5, 6)
f <- function(x){
  if (x == 3){
    warning("a warning")
    warning("second warning")
  }
  if (x == 6){
    stop("an error")
  }
  return(x + 1)
}

res <- safe_mclapply(X = X, FUN = f, mc.cores=16)
res_no_stop <- safe_mclapply(X = X, FUN = f, mc.cores=16, stop.on.error = F)

Warnings suppressed with mclapply in R

3 Answers3