How can I print when using %dopar%

Question

I have a foreach loop that uses %dopar% with doSNOW as the back-end. How can I have the loop print something out each iteration?

My code below is what I'm currently using, but its not printing anything.

foreach(ntree=rep(25,2),.combine=combine,.packages='randomForest',
    .inorder=FALSE) %dopar% {
        print("RANDOM FOREST")
        randomForest(classForm,data=data,na.action=na.action,do.trace=do.trace,ntree=ntree,mtry=mtry)
    }   

Answer 1

Output produced by the snow workers gets thrown away by default, but you can use the makeCluster "outfile" option to change that. Setting outfile to the empty string ("") will prevent snow from redirecting the output, often resulting in the output from your print messages showing up on the terminal of the master process.

Just create and register your cluster with something like:

library(doSNOW)
cl <- makeCluster(4, outfile="")
registerDoSNOW(cl)

Your foreach loop doesn't need to change at all.

This works for me with both SOCK clusters and MPI clusters using Rmpi built with Open MPI. On Windows, you won't see any output if you're using Rgui. If you use Rterm.exe instead, you will.

Note that in addition to your own output, you'll see messages produced by snow which can also be useful.

---

To use a progress bar, doSNOW version 1.0.14 has a progress option. Here is a complete example:

library(doSNOW)
library(tcltk)
library(randomForest)
cl <- makeSOCKcluster(3)
registerDoSNOW(cl)

ntasks <- 100
pb <- tkProgressBar(max=ntasks)
progress <- function(n) setTkProgressBar(pb, n)
opts <- list(progress=progress)

x <- matrix(runif(500), 100)
y <- gl(2, 50)

rf <- foreach(ntree=rep(25, ntasks), .combine=combine,
        .multicombine=TRUE, .packages='randomForest',
        .options.snow=opts) %dopar% {
  randomForest(x, y, ntree=ntree)
}

The progress option is fairly general, so you could simply print a message using a function such as:

progress <- function(n) cat(sprintf("task %d is complete\n", n))

The function can take 0, 1, or 2 arguments. The first supplied argument is the total number of completed tasks, and the second is the task number of the task that just finished.

The simplest example simply prints a . when a task completes:

progress <- function() cat('.')

This example displays both arguments and can be used to demonstrate that tasks aren't always completed in order:

progress <- function(nfin, tag) {
  cat(sprintf('tasks completed: %d; tag: %d\n', nfin, tag))
}