Improving CI build time (.NET)

Question

We are developing an application framework + "plugins" using TeamCity as a CI server.

Project Details

1. 4 Visual Studio solutions
2. ~70 projects (and increasing)
3. Currently running 2 builds using TeamCity: CI and FULL build.

CI - triggered on every commit.

FULL - runs nightly.

I would like to improve the performance of both builds (especially the CI build, as it needs to give its output as quickly as possible).

Are there any guidelines in general on what can be effectively and easily improved?

The build process simply builds a .sln file and runs some unit tests.

Directions Considered:

• MSBuild parallelization
• Overriding CopyFilesToLocal

Not sure these are applicable/will result in a performance gain.

I am looking for more ways to improve the build time (which takes around 3-4 minutes).

Answer 1

Minimise the work your ci builds do.

• set up the workspace with any unneeded folders cloaked, to minimise the number of files you need to get from source control. If necessary reorganise your source structure so that it's easy to knock whole folders of data out of the build with cloaking.

• make sure the ci build uses incremental get and incremental build.

• only build solutions/projects that you need to. If your libraries only get changed infrequently, can you precompile them and check the binaries into source control? If a project is not currently being actively developed, don't bother building it in ci builds.

• do you need to run unit tests for every check in? We run a simple ci build of code only, with a separate test build running as ci, but no more frequently than once per hour. This slashes our ci build time but still lets us know within one hour if we break any unit tests.

• Similarly, don't build documentation, obfuscate, build installers, sign assemblies with certificates, etc, and disable any build processes that copy the outputs to the drop folder. CI builds are there to tell you if you've broken the build asap, you don't care about generating useful binary outputs.

• optimise the build in general - merge projects together, use multi-threaded builds, use several build agents so ci builds don't have to wait for other build types to complete. Only do full builds overnight so your build server is dedicated to ci while you are working. Keep source files tidy (delete unused code rather than just commenting it out, remove unused usings/includes etc.)

• invest in better build server hardware. If you don't have a top spec machine, drop more RAM and an SSD into it for a cheap speed boost. Make sure your build server is dedicated to ci builds, and isn't being used for anything else that might slow it down. Make sure the network between the build server and tfs sever is gigabit. Ensure you don't have any anti-virus software running on the server, or at least that its scheduled-scans are run overnight and your build folders are in the real-time-scan exclusion lists.

• use tfs check in policies to stop devs checking in if the ci build has failed, so that you stop and fix breakages immediately.