I tried:
The GitExtensions toolbar shows up again on the second toolbar row after a computer restart.
I use:
The Git Extension files are stored in your 'My Documents' folder in a "Visual Studio 2012" subfolder. Removing these files will make the toolbar disappear from within MSVC.