I have no knowledge of GPU programming concepts and APIs. I have a few questions:
I would also be glad for links to any kind of documentation with appropriate learning resources.
Best, Jozef
1) No - not for the general case of C# - obviously anything can be created for some subset of the language
2) Yes - HLSL using Direct X or Open GL
3) Not generally possible - CPU and GPU coding are fundamentally different
Basically you can't think of CPU and GPU coding as being comparable. A GPU is a highly specialised parallel processing tool - for lots of parallel simple calculations.
Trying to write a general progam in a GPU with lots of branches etc just won't be efficient - maybe not even possible.
Their memory access architectures are totally different.
You should write for the CPU but farm out appropriate parallel computations to the GPU.