I am curious whether I need two manifest files that are created when I publish my application. It works when I delete them. In the case they are needed, I have tried to embed (Project>Application>Embed manifest with default setting) them but they are still external.
Those are: (appname).exe.manifest and (appname).application.
Answer
The manifest file describes how your application should run. From MSDN:
Every assembly, whether static or dynamic, contains a collection of data that describes how the elements in the assembly relate to each other. The assembly manifest contains this assembly metadata. An assembly manifest contains all the metadata needed to specify the assembly's version requirements and security identity, and all metadata needed to define the scope of the assembly and resolve references to resources and classes. The assembly manifest can be stored in either a PE file (an .exe or .dll) with Microsoft intermediate language (MSIL) code or in a standalone PE file that contains only assembly manifest information.
So deleting them is probably the wrong thing to do, especially if you want your application to run elevated by default on Vista and beyond.
Here are details from MSDN on use of the mt tool, which is used to embed the manifest in your application.
Also note a really interesting issue concerning the caching of the manifest in Vista and beyond that looks like a real gotcha.