Automatically setting jobs (-j) flag for a multicore machine?

makefile parallel-processing build-script
KarateSnowMachine picture KarateSnowMachine · Jan 24, 2011 · Viewed 42.1k times · Source

I have a Makefile on a machine that has a ton of cores in it, but I always seem to forget to write -jX when compiling my project and it takes way longer than it should.

Is there some way I can set the -j flag through an environment variable or some other persistent config file so that make will automatically execute multiple jobs in parallel on this machine?

Answer

Sam Miller picture Sam Miller · Jan 24, 2011

I'm assuming you're using Linux. This is from my ~/.bashrc

# parallel make
export NUMCPUS=`grep -c '^processor' /proc/cpuinfo`
alias pmake='time nice make -j$NUMCPUS --load-average=$NUMCPUS'

sample usage

samm@host src> echo $NUMCPUS
8
samm@host src> pmake

becomes time nice make -j8 --load-average=8.

To answer your specific question about putting this into a Makefile, I don't find it practical to sprinkle this logic all over my Makefiles. Putting this into a top level Makefile also isn't a great solution since I often build from sub-directories and wish to build them in parallel as well. However, if you have a fairly flat source hierarchy, it may work for you.

Related questions

gcc makefile error: "No rule to make target ..."

I'm trying to use GCC (linux) with a makefile to compile my project. I get the following error which is can't seem to decipher in this context: "No rule to make target 'vertex.cpp', needed by 'vertex.o'. Stop." This …

Read More
How to run a makefile in Windows?

I have some demos that I downloaded and they come with a Makefile.win and a Makefile.sgi. How can I run these in Windows to compile the demos?

Read More
Passing additional variables from command line to make

Can I pass variables to a GNU Makefile as command line arguments? In other words, I want to pass some arguments which will eventually become variables in the Makefile.

Read More