Since graphics cards provide large amounts of RAM (0.5GiB to 2GiB) and API access to the GPU is not that difficult with CUDA, Stream and more portable OpenCL I wondered if it is possible to use graphics memory as RAM. Grahics RAM might have a larger latency (from CPU) than real RAM but its definitively faster than HDD so it could be optimal for caching.
Is it possible to access graphics memory directly or at least with a thin memory management layer within own applications (rather than free usable for the OS)? If so, what the the preferred way to do this?
Yes, you can use it as swap memory on Linux. Refer to the link here for more details.
With Linux, it's possible to use it as swap space, or even as RAM disk.
Be warned
It's nice to have fast swap or RAM disk on your home computer but be warned, if a binary driver is loaded for X, it may freeze the whole system or create graphical glitches. Usually there is no way to tell the driver how much memory could be used, so it won't know the upper limit. However, the VESA driver can be used because it provides the possibility to set the video RAM size.
So, Direct Rendering or fast swap. Your choice.
Unlike motherboard RAM and hard drives, there aren't any known video cards that have ECC memory. This may not be a big deal for graphics rendering, but you definitely don't want to put critical data in it or use this feature on servers.