How to avoid 'sink stack is full' error when sink() is used to capture messages in foreach loop

Roger picture Roger · Oct 10, 2014 · Viewed 8.3k times · Source

In order to see the console messages output by a function running in a foreach() loop I followed the advice of this guy and added a sink() call like so:

   library(foreach)    
   library(doMC)
   cores <- detectCores()
   registerDoMC(cores)

   X <- foreach(i=1:100) %dopar%{
   sink("./out/log.branchpies.txt", append=TRUE)
   cat(paste("\n","Starting iteration",i,"\n"), append=TRUE)
   myFunction(data, argument1="foo", argument2="bar")
   }

However, at iteration 77 I got the error 'sink stack is full'. There are well-answered questions about avoiding this error when using for-loops, but not foreach. What's the best way to write the otherwise-hidden foreach output to a file?

Answer

Roland picture Roland · Oct 10, 2014

This runs without errors on my Mac:

library(foreach)    
library(doMC)
cores <- detectCores()
registerDoMC(cores)

X <- foreach(i=1:100) %dopar%{
  sink("log.branchpies.txt", append=TRUE)
  cat(paste("\n","Starting iteration",i,"\n"))
  sink() #end diversion of output
  rnorm(i*1e4)
}

This is better:

library(foreach)    
library(doMC)
cores <- detectCores()
registerDoMC(cores)
sink("log.branchpies.txt", append=TRUE)
X <- foreach(i=1:100) %dopar%{
  cat(paste("\n","Starting iteration",i,"\n"))
    rnorm(i*1e4)
}
sink() #end diversion of output

This works too:

library(foreach)    
library(doMC)
cores <- detectCores()
registerDoMC(cores)

X <- foreach(i=1:100) %dopar%{
  cat(paste("\n","Starting iteration",i,"\n"), 
       file="log.branchpies.txt", append=TRUE)
  rnorm(i*1e4)
}