How to optimize the startup time of an SCons script?

Question

I have an SCons script that takes around 10 seconds just to find out that nothing needs to be rebuild, which feels awfully long for what is essentially a rather small project. Reading the SConscript itself takes just a second or two, most of the time is spend at the:

scons: Building targets ...

step.

How can I find out what exactly scons is doing at this point? And what other general advise can be given on writing fast SCons scripts?

Answer 1

(Stolen directly from http://www.scons.org/wiki/GoFastButton)

The command 'scons --max-drift=1 --implicit-deps-unchanged' will execute your build as fast as possible.

OR:

• env.Decider('MD5-timestamp'): as of SCons 0.98, you can set the Decider function on an environment. MD5-timestamp says if the timestamp matches, don't bother re-MD5ing the file. This can give huge speedups. See the man page for info.
• --max-drift: By default SCons will calculate the MD5 checksum of every source file in your build each time it is run, and will only cache the checksum after the file is 2 days old. This default of 2 days is to protect from clock skew from NFS or revision control systems. You can tweak this delay using --max-drift=SECONDS where SECONDS is some number of seconds. Decreasing SECONDS can improve build speed by eliminating superfluous MD5 checksum calculations. Instead of specifying this on the command line each run, you can set this option inside your SConstruct or SConscript file using "SetOption('max_drift', SECONDS)".
• --implicit-deps-unchanged: By default SCons will rescan all source files for implicit dependencies (e.g. C/C++ header #includes), which can be an expensive process. You can tell SCons to cache the implicit dependencies between invocations using the --implicit-deps-unchanged. By using this options you are making a promise to SCons that you didn't change any of the implicit dependencies since it was last run. If you do change the implicit dependencies, then using --implicit-deps-changed will cause them to be rescaned and cached. (You cannot set this option from within the SConstruct or SConscript files.)
• --implicit-cache: This option tells SCons to intelligently cache implicit dependencies. It attempts to determine if the implicit dependencies have changed since the last build, and if so it will recalculate them. This is usually slower than using --implicit-deps-unchanged, but is also more accurate. Instead of specifying this on the command line each run, you can set this option inside your SConstruct or SConscript file using "SetOption('implicit_cache', 1)".
• CPPPATH: normally you tell Scons about include directories by setting the CPPPATH construction variable, which causes SCons to search those directories when doing implicit dependency scans and also includes those directories in the compile command line. If you have header files that never or rarely change (e.g. system headers, or C run-time headers), then you can exclude them from CPPPATH and include them in the CCFLAGS construction variable instead, which causes SCons to ignore those include directories when scanning for implicit dependencies. Carefully tuning the include directories in this way can usually result in a dramatic speed increase with very little loss of accuracy.
• Avoid RCS and SCCS scans by using env.SourceCode(".", None) - this is especially interesting if you are using lots of c or c++ headers in your program and that your file system is remote (nfs, samba).
• When using "BuildDir", use it with "duplicate" set to 0: "BuildDir( dir1, dir2, duplicate=0". This will cause scons to invoke Builders using the path names of source files in src_dir and the path names of derived files within build_dir. However, this may cause build problems if source files are generated during the build, if any invoked tools are hard-coded to put derived files in the same directory as the source files.
• On a multi-processor machine it may be beneficial to run multiple jobs at once - use the --jobs N (where N is the number of processors on your machine), or "SetOption('num_jobs', N)" inside your SConstruct or SConscript. On Windows machines, the number of processors is available in the environment variable 'NUMBER_OF_PROCESSORS'.
• If you have more than a few dozen preprocessor defines ("-DFOO1 -DFOO2") you may find from using --profile that SCons is spending a lot of time in subst() function, usually just adding the -D string to the defines over and over again. This can really slow down builds where nothing has changed. With 100+ defines I saw a do-nothing build time drop from 35s to 20s using the idea described in "Caching the CPPDEFINES" elsewhere on this page.

Another trick to making things faster is to avoid relinking programs when a shared library has been modified but not rebuilt. See SharedLibrarySignatureOverride