How to let R use all the cores of the computer?

r parallel-processing multicore
Dail picture Dail · Nov 11, 2011 · Viewed 10.4k times · Source

I have read that R uses only a single CPU. How can I let R use all the available cores to run statistical algorithms?

Answer

Gavin Simpson picture Gavin Simpson · Nov 11, 2011

Yes, for starters, see the High Performance Computing Task View on CRAN. This lists details of packages that can be used in support of parallel computing on a single machine.

From R version 2.14.0, there is inbuilt support for parallel computing via the parallel package, which includes slightly modified versions of the existing snow and multicore packages. The parallel package has a vignette that you should read. You can view it using:

vignette(package="parallel", topic = "parallel")

There are other ways to exploit your multiple cores, for example via use of a multi-threaded BLAS for linear algebra computations.

Whether any of this will speed up the "statistics calculations" you want to do will depend on what those "statistics calculations" are. Spawning off multiple threads or workers entails an overhead cost to set them up, manage them and collect the results. Some operations see a benefit (some large, some small) of using multiple cores/threads, others are slowed down because of this extra overhead.

In short, do not expect to get an n times decrease in your compute time by using n cores instead of just 1.

Related questions

could not find function inside foreach loop

I'm trying to use foreach to do multicore computing in R. A <-function(....) { foreach(i=1:10) %dopar% { B() } } then I call function A in the console. The problem is I'm calling a function Posdef inside B that is defined in …

Read More
Parallelizing SQL queries in R

I have six SQL queries that I script though R that each take a very long time (~30 minutes each). Once each query returns I then manipulate the data for some standard reports. What I'd like to do is use my …

Read More
R: converting each row of a data frame into a list item

I have a number of operations on data frames which I would like to speed up using mclapply() or other lapply() like functions. One of the easiest ways for me to wrestle with this is to make each row of …

Read More