High level GPU programming in C++
I've been looking into libraries/extensions for C++ that will allow GPU-based processing on a high level. I'm not an expert in GPU programming and I don't want to dig too deep. I have a neural network consisting of classes with virtual functions. I need a library that basically does the GPU allocation for me - on a high level. There is a guy who wrote a thesis on a system called GPU++ which does most of the GPU stuff for you. I can't find the code anywhere, just his thesis.
Does anyone know of a similar library, or does anyone have the code for GPU++? Libraries like CUDA are too low level and can't handle most of my operations (at least not without rewriting all my processes and algorithms - which I don't want to do).
Answer
There are many high-level libraries dedicated to GPGPU programming. Since they rely on CUDA and/or OpenCL, they have to be chosen wisely (a CUDA-based program will not run on AMD's GPUs, unless it goes through a pre-processing step with projects such as gpuocelot).
CUDA
You can find some examples of CUDA libraries on the NVIDIA website.
- Thrust: the official description speaks for itself
Thrust is a parallel algorithms library which resembles the C++ Standard Template Library (STL). Thrust's high-level interface greatly enhances programmer productivity while enabling performance portability between GPUs and multicore CPUs. Interoperability with established technologies (such as CUDA, TBB, and OpenMP) facilitates integration with existing software.
As @Ashwin pointed out, the STL-like syntax of Thrust makes it a widely chosen library when developing CUDA programs. A quick look at the examples shows the kind of code you will be writing if you decide to use this library. NVIDIA's website presents the key features of this library. A video presentation (from GTC 2012) is also available.
- CUB: the official description tells us:
CUB provides state-of-the-art, reusable software components for every layer of the CUDA programming mode. It is a flexible library of cooperative threadblock primitives and other utilities for CUDA kernel programming.
It provides device-wide, block-wide and warp-wide parallel primitives such as parallel sort, prefix scan, reduction, histogram etc.
It is open-source and available on GitHub. It is not high-level from an implementation point of view (you develop in CUDA kernels), but provides high-level algorithms and routines.
- mshadow: lightweight CPU/GPU matrix/tensor template library in C++/CUDA.
This library is mostly used for machine learning, and relies on expression templates.
- Eigen: support for CUDA with a new Tensor class have been added in version 3.3. It is used by Google in TensorFlow, and is still experimental.
Starting from Eigen 3.3, it is now possible to use Eigen's objects and algorithms within CUDA kernels. However, only a subset of features are supported to make sure that no dynamic allocation is triggered within a CUDA kernel.
OpenCL
Note that OpenCL does more than GPGPU computing, since it supports heterogeneous platforms (multi-core CPUs, GPUs etc.).
- OpenACC: this project provides OpenMP-like support for GPGPU. A large part of the programming is done implicitly by the compiler and the run-time API. You can find a sample code on their website.
The OpenACC Application Program Interface describes a collection of compiler directives to specify loops and regions of code in standard C, C++ and Fortran to be offloaded from a host CPU to an attached accelerator, providing portability across operating systems, host CPUs and accelerators.
- Bolt: open-source library with STL-like interface.
Bolt is a C++ template library optimized for heterogeneous computing. Bolt is designed to provide high-performance library implementations for common algorithms such as scan, reduce, transform, and sort. The Bolt interface was modeled on the C++ Standard Template Library (STL). Developers familiar with the STL will recognize many of the Bolt APIs and customization techniques.
Boost.Compute: as @Kyle Lutz said, Boost.Compute provides a STL-like interface for OpenCL. Note that this is not an official Boost library (yet).
SkelCL "is a library providing high-level abstractions for alleviated programming of modern parallel heterogeneous systems". This library relies on skeleton programming, and you can find more information in their research papers.
CUDA + OpenCL
- ArrayFire is an open-source (used to be proprietary) GPGPU programming library. They first targeted CUDA, but now support OpenCL as well. You can check the examples available online. NVIDIA's website provides a good summary of its key features.
Complementary information
Although this is not really in the scope of this question, there is also the same kind of support for other programming languages:
- Python: PyCUDA for CUDA, Clyther and PyOpenCL for OpenCL. There is a dedicated StackOverflow question for this.
- Java: JCuda for CUDA, and for OpenCL, you can check this other question.
- JavaScript: GPU.JS for WebGl.
If you need to do linear algebra (for instance) or other specific operations, dedicated math libraries are also available for CUDA and OpenCL (e.g. ViennaCL, CUBLAS, MAGMA etc.).
Also note that using these libraries does not prevent you from doing some low-level operations if you need to do some very specific computation.
Finally, we can mention the future of the C++ standard library. There has been extensive work to add parallelism support. This is still a technical specification, and GPUs are not explicitely mentioned AFAIK (although NVIDIA's Jared Hoberock, developer of Thrust, is directly involved), but the will to make this a reality is definitely there.