I want to build projects from the command line. Is it possible to deploy a C# compiler without installing Visual Studio?
Sure, the framework includes a compiler, csc.exe. Look at this article for a quick how-to. The important parts:
You can get the command-line compiler (csc.exe) from Microsoft site http://msdn2.microsoft.com/en-us/netframework/aa731542.aspx.
Download the redistributable package of the .NET Framework, which includes the compiler and the .NET Framework with C# 2005 syntax support.
The compiler is located in the following directory: %windir%\Microsoft.NET\Framework\
Also look at this MSDN article for a full guide and explanation.
Note that for more recent versions, you will be looking for the MSBuild standalone package rather than the framework -- see @Vadzim's answer.