tf.data.Dataset: how to get the dataset size (number of elements in a epoch)?

python python-3.x tensorflow tensorflow-datasets
nessuno picture nessuno · Jun 7, 2018 · Viewed 41k times · Source

Let's say I have defined a dataset in this way:

filename_dataset = tf.data.Dataset.list_files("{}/*.png".format(dataset))

how can I get the number of elements that are inside the dataset (hence, the number of single elements that compose an epoch)?

I know that tf.data.Dataset already knows the dimension of the dataset, because the repeat() method allows repeating the input pipeline for a specified number of epochs. So it must be a way to get this information.

Answer

markemus picture markemus · May 29, 2019

len(list(dataset)) works in eager mode, although that's obviously not a good general solution.

Related questions

Is there a way to stack two tensorflow datasets?

I want to stack two datasets objects in Tensorflow (rbind function in R). I have created one dataset A from tfRecord files and one dataset B from numpy arrays. Both have same variables. Do you know if there is a …

Read More
Could not find a version that satisfies the requirement tensorflow

I installed the latest version of Python (3.6.4 64-bit) and the latest version of PyCharm (2017.3.3 64-bit). Then I installed some modules in PyCharm (Numpy, Pandas, etc), but when I tried installing Tensorflow it didn't install, and I got the error message: …

Read More
Could not load dynamic library 'cudart64_101.dll' on tensorflow CPU-only installation

I just installed the latest version of Tensorflow via pip install tensorflow and whenever I run a program, I get the log message: W tensorflow/stream_executor/platform/default/dso_loader.cc:55] Could not load dynamic library 'cudart64_101.dll'; dlerror: …

Read More