What is the best practice for "Copy Local" and with project references?

Question

I have a large c# solution file (~100 projects), and I am trying to improve build times. I think that "Copy Local" is wasteful in many cases for us, but I am wondering about best practices.

In our .sln, we have application A depending on assembly B which depends on assembly C. In our case, there are dozens of "B" and a handful of "C". Since these are all included in the .sln, we're using project references. All assemblies currently build into $(SolutionDir)/Debug (or Release).

By default, Visual Studio marks these project references as "Copy Local", which results in every "C" being copied into $(SolutionDir)/Debug once for every "B" that builds. This seems wasteful. What can go wrong if I just turn "Copy Local" off? What do other people with large systems do?

FOLLOWUP:

Lots of responses suggest breaking up the build into smaller .sln files... In the example above, I would build the foundation classes "C" first, followed by the bulk of the modules "B", and then a few applications, "A". In this model, I need to have non-project references to C from B. The problem I run into there is that "Debug" or "Release" gets baked into the hint path and I wind up building my Release builds of "B" against debug builds of "C".

For those of you that split the build up into multiple .sln files, how do you manage this problem?

Answer 1

In a previous project I worked with one big solution with project references and bumped into a performance problem as well. The solution was three fold:

1. Always set the Copy Local property to false and enforce this via a custom msbuild step

2. Set the output directory for each project to the same directory (preferably relative to $(SolutionDir)

3. The default cs targets that get shipped with the framework calculate the set of references to be copied to the output directory of the project currently being built. Since this requires calculating a transitive closure under the 'References' relation this can become VERY costly. My workaround for this was to redefine the GetCopyToOutputDirectoryItems target in a common targets file (eg. Common.targets ) that's imported in every project after the import of the Microsoft.CSharp.targets. Resulting in every project file to look like the following:

   <Project DefaultTargets="Build" xmlns="http://schemas.microsoft.com/developer/msbuild/2003">
     <PropertyGroup>
       ... snip ...
     </ItemGroup>
     <Import Project="$(MSBuildBinPath)\Microsoft.CSharp.targets" />
     <Import Project="[relative path to Common.targets]" />
     <!-- To modify your build process, add your task inside one of the targets below and uncomment it. 
          Other similar extension points exist, see Microsoft.Common.targets.
     <Target Name="BeforeBuild">
     </Target>
     <Target Name="AfterBuild">
     </Target>
     -->
   </Project>
   

This reduced our build time at a given time from a couple of hours (mostly due to memory constraints), to a couple of minutes.

The redefined GetCopyToOutputDirectoryItems can be created by copying the lines 2,438–2,450 and 2,474–2,524 from C:\WINDOWS\Microsoft.NET\Framework\v2.0.50727\Microsoft.Common.targets into Common.targets.

For completeness the resulting target definition then becomes:

<!-- This is a modified version of the Microsoft.Common.targets
     version of this target it does not include transitively
     referenced projects. Since this leads to enormous memory
     consumption and is not needed since we use the single
     output directory strategy.
============================================================
                    GetCopyToOutputDirectoryItems

Get all project items that may need to be transferred to the
output directory.
============================================================ -->
<Target
    Name="GetCopyToOutputDirectoryItems"
    Outputs="@(AllItemsFullPathWithTargetPath)"
    DependsOnTargets="AssignTargetPaths;_SplitProjectReferencesByFileExistence">

    <!-- Get items from this project last so that they will be copied last. -->
    <CreateItem
        Include="@(ContentWithTargetPath->'%(FullPath)')"
        Condition="'%(ContentWithTargetPath.CopyToOutputDirectory)'=='Always' or '%(ContentWithTargetPath.CopyToOutputDirectory)'=='PreserveNewest'"
            >
        <Output TaskParameter="Include" ItemName="AllItemsFullPathWithTargetPath"/>
        <Output TaskParameter="Include" ItemName="_SourceItemsToCopyToOutputDirectoryAlways"
                Condition="'%(ContentWithTargetPath.CopyToOutputDirectory)'=='Always'"/>
        <Output TaskParameter="Include" ItemName="_SourceItemsToCopyToOutputDirectory"
                Condition="'%(ContentWithTargetPath.CopyToOutputDirectory)'=='PreserveNewest'"/>
    </CreateItem>

    <CreateItem
        Include="@(_EmbeddedResourceWithTargetPath->'%(FullPath)')"
        Condition="'%(_EmbeddedResourceWithTargetPath.CopyToOutputDirectory)'=='Always' or '%(_EmbeddedResourceWithTargetPath.CopyToOutputDirectory)'=='PreserveNewest'"
            >
        <Output TaskParameter="Include" ItemName="AllItemsFullPathWithTargetPath"/>
        <Output TaskParameter="Include" ItemName="_SourceItemsToCopyToOutputDirectoryAlways"
                Condition="'%(_EmbeddedResourceWithTargetPath.CopyToOutputDirectory)'=='Always'"/>
        <Output TaskParameter="Include" ItemName="_SourceItemsToCopyToOutputDirectory"
                Condition="'%(_EmbeddedResourceWithTargetPath.CopyToOutputDirectory)'=='PreserveNewest'"/>
    </CreateItem>

    <CreateItem
        Include="@(Compile->'%(FullPath)')"
        Condition="'%(Compile.CopyToOutputDirectory)'=='Always' or '%(Compile.CopyToOutputDirectory)'=='PreserveNewest'">
        <Output TaskParameter="Include" ItemName="_CompileItemsToCopy"/>
    </CreateItem>
    <AssignTargetPath Files="@(_CompileItemsToCopy)" RootFolder="$(MSBuildProjectDirectory)">
        <Output TaskParameter="AssignedFiles" ItemName="_CompileItemsToCopyWithTargetPath" />
    </AssignTargetPath>
    <CreateItem Include="@(_CompileItemsToCopyWithTargetPath)">
        <Output TaskParameter="Include" ItemName="AllItemsFullPathWithTargetPath"/>
        <Output TaskParameter="Include" ItemName="_SourceItemsToCopyToOutputDirectoryAlways"
                Condition="'%(_CompileItemsToCopyWithTargetPath.CopyToOutputDirectory)'=='Always'"/>
        <Output TaskParameter="Include" ItemName="_SourceItemsToCopyToOutputDirectory"
                Condition="'%(_CompileItemsToCopyWithTargetPath.CopyToOutputDirectory)'=='PreserveNewest'"/>
    </CreateItem>

    <CreateItem
        Include="@(_NoneWithTargetPath->'%(FullPath)')"
        Condition="'%(_NoneWithTargetPath.CopyToOutputDirectory)'=='Always' or '%(_NoneWithTargetPath.CopyToOutputDirectory)'=='PreserveNewest'"
            >
        <Output TaskParameter="Include" ItemName="AllItemsFullPathWithTargetPath"/>
        <Output TaskParameter="Include" ItemName="_SourceItemsToCopyToOutputDirectoryAlways"
                Condition="'%(_NoneWithTargetPath.CopyToOutputDirectory)'=='Always'"/>
        <Output TaskParameter="Include" ItemName="_SourceItemsToCopyToOutputDirectory"
                Condition="'%(_NoneWithTargetPath.CopyToOutputDirectory)'=='PreserveNewest'"/>
    </CreateItem>
</Target>

With this workaround in place I found it workable to have as much as > 120 projects in one solution, this has the main benefit that the build order of the projects can still be determined by VS instead of doing that by hand by splitting up your solution.