I copied the source code from one application to another, both running on the same machine. I am also using the same string for containerName below in both applications.
What is preventing my new application from reading the key that was saved in the other application? All other things are equal, logged in user account etc.
CspParameters cspParams = new CspParameters();
cspParams.KeyContainerName = containerName;
cspParams.Flags = CspProviderFlags.UseMachineKeyStore;
// Get error "object already exists" below.
RSACryptoServiceProvider rsaKey = new RSACryptoServiceProvider(cspParams);
Did you try to grant permissions to Everyone, for example, for files in "Documents and Settings\All Users\Application Data\Microsoft\Crypto\RSA\Machine Keys", as it described there:
http://social.msdn.microsoft.com/Forums/en/netfxbcl/thread/f7b9f928-a794-47f2-a5bd-9f64ca375040