I have just installed Git For Windows, and created my SSH key using SSH through Cygwin. It seems to be working from command-line and the Git For Windows GUI, but now I want to use TortoiseGit. TortoiseGit uses a Putty-like SSH client by default but apparently GitHub only accepts (open)SSH, and I don't really know how to proceed. I will be doing a clean TortoiseGit install, I already have Cygwin and Git for Windows installed.
For anyone looking to change an existing TortoiseGit installation to use PuTTY rather than OpenSSH, simply right-click, -> TortoiseGit -> Settings -> Network, then change 'ssh.exe' to 'tortoiseplink.exe'. From then on, doing a push/pull should be using Pageant (and any keys you have set up in there) if you also have that running.