How can I check if keras/tensorflow is using cuDNN?

Question

I have installed CUDA and cuDNN, but the last was not working, giving a lot of error messages in theano. Now I am training moderate sized deep conv nets in Keras/Tensorflow, without getting any cuDNN error messages. How can I check if cuDNN is now being used?

Answer 1

tl;dr: If tensorflow-gpu works, then CuDNN is used.

The prebuilt binaries of TensorFlow (at least since version 1.3) link to the CuDNN library. If CuDNN is missing, an error message will tell you ImportError: Could not find 'cudnn64_7.dll'. TensorFlow requires that this DLL be installed....

According to the TensorFlow install documentation for version 1.5, CuDNN must be installed for GPU support even if you build it from source. There are still a lot of fallbacks in the TensorFlow code for the case of CuDNN not being available -- as far as I can tell it used to be optional in prior versions.

Here are two lines from the TensorFlow source that explicitly tell and force that CuDNN is required for gpu acceleration.

There is a special GPU version of TensorFlow that needs to be installed in order to use the GPU (and CuDNN). Make sure the installed python package is tensorflow-gpu and not just tensorflow.

You can list the packages containing "tensorflow" with conda list tensorflow (or just pip list, if you do not use anaconda), but make sure you have the right environment activated.

When you run your scripts with GPU support, they will start like this:

Using TensorFlow backend.

2018- ... C:\tf_jenkins\...\gpu\gpu_device.cc:1105] Found device 0 with properties:
name: GeForce GTX 1080 major: 6 minor: 1 memoryClockRate(GHz): 1.7845

To test it, just type into the console:

import tensorflow as tf
tf.Session()

To check if you "see" the CuDNN from your python environment and therewith validate a correct PATH variable, you can try this:

import ctypes
ctypes.WinDLL("cudnn64_7.dll") # use the file name of your cudnn version here.