Tensorflow fail with "Unable to get element from the feed as bytes." when attempting to restore checkpoint
I am using Tensorflow r0.12.
I use google-cloud-ml locally to run 2 different training jobs. In the first job, I find good initial values for my variables. I store them in a V2-checkpoint.
When I try to restore my variables for using them in the second job :
import tensorflow as tf
sess = tf.Session()
new_saver = tf.train.import_meta_graph('../variables_pred/model.ckpt-10151.meta', clear_devices=True)
new_saver.restore(sess, tf.train.latest_checkpoint('../variables_pred/'))
all_vars = tf.trainable_variables()
for v in all_vars:
print(v.name)
I got the following error message :
tensorflow.python.framework.errors_impl.InternalError: Unable to get element from the feed as bytes.
The checkpoint is created with these lines in the first job :
saver = tf.train.Saver()
saver.export_meta_graph(filename=os.path.join(output_dir, 'export.meta'))
saver.save(sess, os.path.join(output_dir, 'export'), write_meta_graph=False)
According to this answer, it could come from the absence of metadata file but I am loading the metadata file.
PS : I use the argument clear_devices=True because the device specifications generated by a launch on google-cloud-ml are quite intricated and I don't need to necessarily get the same dispatch.
Answer
The error message was due to the absence of the file named "checkpoint" by inadvertency.
After the reintroduction of this file in the appropriate folder, it appears that the loading of the checkpoint is working.
Sorry for having omitted this key point.