Zero initialiser for biases using get_variable in tensorflow

machine-learning initialization tensorflow bias-neuron zero-initialization
Keir Simmons picture Keir Simmons · Jan 24, 2017 · Viewed 10.9k times · Source

A code I'm modifying is using tf.get_variable for weight variables, and tf.Variable for bias initialisation. After some searching, it seems that get_variable should always be favoured due to its portability in regards to sharing. So I tried to change the bias variable to get_variable but can't seem to get it to work.

Original: tf.Variable(tf.zeros([128]), trainable=True, name="b1")

My attempt: tf.get_variable(name="b1", shape=[128], initializer=tf.zeros_initializer(shape=[128]))

I get an error saying that the shape should not be specified for constants. But removing the shape then throws an error for no arguments.

I'm very new to tf so I'm probably misunderstanding something fundamental here. Thanks for the help in advance :)

Answer

user1454804 picture user1454804 · Jan 26, 2017

Following should work: tf.get_variable(name="b1", shape=[128], initializer=tf.zeros_initializer())

Related questions

Tensorflow: how to save/restore a model?

After you train a model in Tensorflow: How do you save the trained model? How do you later restore this saved model?

Read More
A simple explanation of Naive Bayes Classification

I am finding it hard to understand the process of Naive Bayes, and I was wondering if someone could explain it with a simple step by step process in English. I understand it takes comparisons by times occurred as a …

Read More
What is the role of the bias in neural networks?

I'm aware of the gradient descent and the back-propagation algorithm. What I don't get is: when is using a bias important and how do you use it? For example, when mapping the AND function, when I use 2 inputs and 1 output, …

Read More