I am looking at utilising the GPU for crunching some equations but cannot figure out how I can access it from C#. I know that the XNA and DirectX frameworks allow you to use shaders in order to access the GPU, but how would I go about accessing it without these frameworks?
I haven't done it from C#, but basically you use the CUDA (assuming you're using an nVidia card here, of course) SDK and CUDA toolkit to pull it off.
nVidia has ported (or written?) a BLAS implementation for use on CUDA-capable devices. They've provided plenty of examples for how to do number crunching, although you'll have to figure out how you're going to pull it off from C#. My bet is, you're going to have to write some stuff in un-managed C or C++ and link with it.
If you're not hung-up on using C#, take a look at Theano. It might be a bit overkill for your needs, since they're building a framework for doing machine learning on GPUs from Python, but ... it works, and works very well.