Somehow during the upgrade to VS2012 and .NET 4.5, I've managed to get NuGet confused. There are packages that appear in the package manager (and the packages folder) that I cannot delete (I believe they are legacy ASP.NET NuGet packages that have been replaced with new package names with the new version. They each have only a Manage button -- but no uninstall button. And when you look to see where they are used, you can see that they are referenced by none of the solution's projects? They are NOT displaying an Uninstall?
I've found a workaround for this.
NuGet will restore only the packages used in your solution. You end up with a nice, streamlined set of packages.