MSBUILD: Build Package including extra files

Question

I've been having some serious challenges over the past week trying to implement the solution located at: http://blog.samstephens.co.nz/2010-10-18/msbuild-including-extra-files-multiple-builds/.

It's based off of Sayed's implementation: http://sedodream.com/CommentView,guid,803d77d7-a220-4cee-a803-f6291cd4ba71.aspx (which works perfectly), however I need files from multiple locations and Sayed's solution doesn't account for that scenario.

While in theory Sam's solution provides exactly what I need, I can't get it to build (despite cut and pasting his solution and only modifying the paths to reflect my environment). I've been banging my head against the following error for several days:

[15:31:30]: [CopyPipelineFiles] C:\Program Files (x86)\MSBuild\Microsoft\VisualStudio\v10.0\Web\Microsoft.Web.Publishing.targets(1852, 5): error MSB4018: The "CopyPipelineFiles" task failed unexpectedly.  
System.ArgumentException: Illegal characters in path.  
at System.IO.Path.CheckInvalidPathChars(String path) at System.IO.Path.Combine(String path1, String path2)  
at Microsoft.Web.Publishing.Tasks.CopyPipelineFiles.CopyPipelineFilesToFolder(TaskLoggingHelper log, ITaskItem[] allpipeLineItems, String sourceFolderName, String targetFolderName, ItemMetadataFilter itemMetadataSkipFilter, Boolean fUpdateItemSpec, Boolean deleteItemsMarkAsExcludeTrue, List`1 updatedPipeLineItems, List`1 failedPipeLineItems)  
at Microsoft.Web.Publishing.Tasks.CopyPipelineFiles.Execute()  
at Microsoft.Build.BackEnd.TaskExecutionHost.Microsoft.Build.BackEnd.ITaskExecutionHost.Execute() 
at Microsoft.Build.BackEnd.TaskBuilder.ExecuteInstantiatedTask(ITaskExecutionHost taskExecutionHost, TaskLoggingContext taskLoggingContext, TaskHost taskHost, ItemBucket bucket, TaskExecutionMode howToExecuteTask, Boolean& taskResult)    
[15:31:31]: Process exited with code 1  
[15:31:31]: MSBuild output:  
[15:31:31]: C:\Program Files (x86)\MSBuild\Microsoft\VisualStudio\v10.0\Web\Microsoft.Web.Publishing.targets(1852,5): error MSB4018: The "CopyPipelineFiles" task failed unexpectedly. [<PATH_TO>\MYPROJ.csproj]  
[15:31:31]: C:\Program Files (x86)\MSBuild\Microsoft\VisualStudio\v10.0\Web\Microsoft.Web.Publishing.targets(1852,5): error MSB4018: System.ArgumentException: Illegal characters in path. [<PATH_TO>\MYPROJ.csproj]  
[15:31:31]: C:\Program Files (x86)\MSBuild\Microsoft\VisualStudio\v10.0\Web\Microsoft.Web.Publishing.targets(1852,5): error MSB4018: at System.IO.Path.CheckInvalidPathChars(String path) [<PATH_TO>\MYPROJ.csproj]  
[15:31:31]: C:\Program Files (x86)\MSBuild\Microsoft\VisualStudio\v10.0\Web\Microsoft.Web.Publishing.targets(1852,5): error MSB4018: at System.IO.Path.Combine(String path1, String path2) [<PATH_TO>\MYPROJ.csproj]  
[15:31:31]: C:\Program Files (x86)\MSBuild\Microsoft\VisualStudio\v10.0\Web\Microsoft.Web.Publishing.targets(1852,5): error MSB4018: at Microsoft.Web.Publishing.Tasks.CopyPipelineFiles.CopyPipelineFilesToFolder(TaskLoggingHelper log, ITaskItem[] allpipeLineItems, String sourceFolderName, String targetFolderName, ItemMetadataFilter itemMetadataSkipFilter, Boolean fUpdateItemSpec, Boolean deleteItemsMarkAsExcludeTrue, List`1 updatedPipeLineItems, List`1 failedPipeLineItems) [<PATH_TO>\MYPROJ.csproj]  
[15:31:31]: C:\Program Files (x86)\MSBuild\Microsoft\VisualStudio\v10.0\Web\Microsoft.Web.Publishing.targets(1852,5): error MSB4018: at Microsoft.Web.Publishing.Tasks.CopyPipelineFiles.Execute() [<PATH_TO>\MYPROJ.csproj]  
[15:31:31]: C:\Program Files (x86)\MSBuild\Microsoft\VisualStudio\v10.0\Web\Microsoft.Web.Publishing.targets(1852,5): error MSB4018: at Microsoft.Build.BackEnd.TaskExecutionHost.Microsoft.Build.BackEnd.ITaskExecutionHost.Execute() [<PATH_TO>\MYPROJ.csproj]  
[15:31:31]: C:\Program Files (x86)\MSBuild\Microsoft\VisualStudio\v10.0\Web\Microsoft.Web.Publishing.targets(1852,5): error MSB4018: at Microsoft.Build.BackEnd.TaskBuilder.ExecuteInstantiatedTask(ITaskExecutionHost taskExecutionHost, TaskLoggingContext taskLoggingContext, TaskHost taskHost, ItemBucket bucket, TaskExecutionMode howToExecuteTask, Boolean& taskResult) [<PATH_TO>\MYPROJ.csproj]  
[15:31:31]: Done Building Project "<PATH_TO>\MYPROJ.csproj" (Package target(s)) -- FAILED.  
[15:31:31]: Done Building Project "C:\buildAgent\work\8aa6ae640d0f858b\main\scripts\MYPROJ.csproj" (BatchCopyPackage target(s)) -- FAILED.  
[15:31:31]: Build FAILED.  
[15:31:31]: "C:\buildAgent\work\8aa6ae640d0f858b\main\scripts\MYPROJ.csproj" (BatchCopyPackage target) (1) ->  
[15:31:31]: "<PATH_TO>\MYPROJ.csproj" (Package target) (2) ->  
[15:31:31]: (CopyAllFilesToSingleFolderForPackage target) ->  
[15:31:31]: C:\Program Files (x86)\MSBuild\Microsoft\VisualStudio\v10.0\Web\Microsoft.Web.Publishing.targets(1852,5): error MSB4018: The "CopyPipelineFiles" task failed unexpectedly. [<PATH_TO>\MYPROJ.csproj]  
[15:31:31]: C:\Program Files (x86)\MSBuild\Microsoft\VisualStudio\v10.0\Web\Microsoft.Web.Publishing.targets(1852,5): error MSB4018: System.ArgumentException: Illegal characters in path. [<PATH_TO>\MYPROJ.csproj]  
[15:31:31]: C:\Program Files (x86)\MSBuild\Microsoft\VisualStudio\v10.0\Web\Microsoft.Web.Publishing.targets(1852,5): error MSB4018: at System.IO.Path.CheckInvalidPathChars(String path) [<PATH_TO>\MYPROJ.csproj]  
[15:31:31]: C:\Program Files (x86)\MSBuild\Microsoft\VisualStudio\v10.0\Web\Microsoft.Web.Publishing.targets(1852,5): error MSB4018: at System.IO.Path.Combine(String path1, String path2) [<PATH_TO>\MYPROJ.csproj]  
[15:31:31]: C:\Program Files (x86)\MSBuild\Microsoft\VisualStudio\v10.0\Web\Microsoft.Web.Publishing.targets(1852,5): error MSB4018: at Microsoft.Web.Publishing.Tasks.CopyPipelineFiles.CopyPipelineFilesToFolder(TaskLoggingHelper log, ITaskItem[] allpipeLineItems, String sourceFolderName, String targetFolderName, ItemMetadataFilter itemMetadataSkipFilter, Boolean fUpdateItemSpec, Boolean deleteItemsMarkAsExcludeTrue, List`1 updatedPipeLineItems, List`1 failedPipeLineItems) [<PATH_TO>\MYPROJ.csproj]  
[15:31:31]: C:\Program Files (x86)\MSBuild\Microsoft\VisualStudio\v10.0\Web\Microsoft.Web.Publishing.targets(1852,5): error MSB4018: at Microsoft.Web.Publishing.Tasks.CopyPipelineFiles.Execute() [<PATH_TO>\MYPROJ.csproj]  
[15:31:31]: C:\Program Files (x86)\MSBuild\Microsoft\VisualStudio\v10.0\Web\Microsoft.Web.Publishing.targets(1852,5): error MSB4018: at Microsoft.Build.BackEnd.TaskExecutionHost.Microsoft.Build.BackEnd.ITaskExecutionHost.Execute() [<PATH_TO>\MYPROJ.csproj]  
[15:31:31]: C:\Program Files (x86)\MSBuild\Microsoft\VisualStudio\v10.0\Web\Microsoft.Web.Publishing.targets(1852,5): error MSB4018: at Microsoft.Build.BackEnd.TaskBuilder.ExecuteInstantiatedTask(ITaskExecutionHost taskExecutionHost, TaskLoggingContext taskLoggingContext, TaskHost taskHost, ItemBucket bucket, TaskExecutionMode howToExecuteTask, Boolean& taskResult) [<PATH_TO>\MYPROJ.csproj]  
[15:31:31]: 0 Warning(s)  
[15:31:31]: 1 Error(s)  
[15:31:31]: Time Elapsed 00:00:23.00  

Here's my implementation of Sam's targets:

<Target Name="DefineCustomFiles">
    <ItemGroup>
      <CustomFilesToInclude Include="$(workingDir)\main\img\**\*">
        <Dir>img</Dir>
      </CustomFilesToInclude>
      <CustomFilesToInclude Include="$(workingDir)\main\Service References\**\*">
        <Dir>ServiceReferences</Dir>
      </CustomFilesToInclude>
    </ItemGroup>
</Target>

<Target Name="CustomCollectFiles">
  <Message Text="Here is a file list: %(CustomFilesToInclude.Identity)" />
    <ItemGroup>
      <FilesForPackagingFromProject Include="@(CustomFilesToInclude)">
        <DestinationRelativePath>
          %(CustomFilesToInclude.Dir)\%(RecursiveDir)%(Filename)%(Extension)
        </DestinationRelativePath>
      </FilesForPackagingFromProject>
    </ItemGroup>
</Target>

I thought it may have something to do with spaces (or the wildcard symbols) so here's what I've tried:

• Examined the log with both "detailed" and "diagnostic" logging levels

• Various incarnations of mixing double quotes and single quotes

• Commenting out the "Service References" node. (The paths containing imgs have no spaces)

• Pointing to a single file with an absolute path. Therefore eliminating both wildcards and errors due to the path containing spaces. I thought FOR SURE this would resolve the issue, but it failed with the same error above.

After issue #3 I'm at a complete loss. It doesn't like any files passed to it.

Any insight would be GREATLY appreciated. Thanks in advance.

Here is how my target currently looks and the respective error message:

<Target Name="DefineCustomFiles">
    <ItemGroup>
      <CustomFilesToInclude Include="$(workingDir)\main\img\file.gif">
        <Dir>img</Dir>
      </CustomFilesToInclude>
    </ItemGroup>
</Target>

C:\Program Files (x86)\MSBuild\Microsoft\VisualStudio\v10.0\Web\Microsoft.Web.Publishing.targets(1852, 5): error MSB4018: The "CopyPipelineFiles" task failed unexpectedly.  
System.ArgumentException: Illegal characters in path.

Answer 1

I've found your issue, and I'm afraid it's my fault. The issue is that whitespace is significant within the DestinationRelativePath element of FilesForPackagingFromProject. I added whitespace in when I did the blog post to make the XML easier to read, without realising that it would cause the code to fail.

So if you try

<Target Name="CustomCollectFiles">
  <Message Text="Here is a file list: %(CustomFilesToInclude.Identity)" />
    <ItemGroup>
      <FilesForPackagingFromProject Include="@(CustomFilesToInclude)">
        <DestinationRelativePath>%(CustomFilesToInclude.Dir)\%(RecursiveDir)%(Filename)%(Extension)</DestinationRelativePath>
      </FilesForPackagingFromProject>
    </ItemGroup>
</Target>

Your problems should hopefully be fixed. I've updated the blog post to remove the offending whitespace.

Note that, as Scott Stafford points out in his comment below, CopyAllFilesToSingleFolderForPackageDependsOn has been renamed to CopyAllFilesToSingleFolderForMsdeployDependsOn in Visual Studio 2012.