What's the difference between tensorflow dynamic_rnn and rnn?

tensorflow recurrent-neural-network
Mad Wombat picture Mad Wombat · Sep 27, 2016 · Viewed 15.9k times · Source

There are several classes in tf.nn that relate to RNNs. In the examples I find on the web, tf.nn.dynamic_rnn and tf.nn.rnn seem to be used interchangeably or at least I cannot seem to figure out why one is used in place of the other. What is the difference?

Answer

Abhishek Mishra picture Abhishek Mishra · Dec 6, 2016

From RNNs in Tensorflow, a Practical Guide and Undocumented Features by Denny Britz, published in August 21, 2016.

tf.nn.rnn creates an unrolled graph for a fixed RNN length. That means, if you call tf.nn.rnn with inputs having 200 time steps you are creating a static graph with 200 RNN steps. First, graph creation is slow. Second, you’re unable to pass in longer sequences (> 200) than you’ve originally specified.

tf.nn.dynamic_rnn solves this. It uses a tf.While loop to dynamically construct the graph when it is executed. That means graph creation is faster and you can feed batches of variable size.

Related questions

How to deal with batches with variable-length sequences in TensorFlow?

I was trying to use an RNN (specifically, LSTM) for sequence prediction. However, I ran into an issue with variable sequence lengths. For example, sent_1 = "I am flying to Dubain" sent_2 = "I was traveling from US to Dubai" I am …

Read More
ValueError: Tensor must be from the same graph as Tensor with Bidirectinal RNN in Tensorflow

I'm doing text tagger using Bidirectional dynamic RNN in tensorflow. After maching input's dimension, I tried to run a Session. this is blstm setting parts: fw_lstm_cell = BasicLSTMCell(LSTM_DIMS) bw_lstm_cell = BasicLSTMCell(LSTM_DIMS) (fw_outputs, bw_…

Read More
What is num_units in tensorflow BasicLSTMCell?

In MNIST LSTM examples, I don't understand what "hidden layer" means. Is it the imaginary-layer formed when you represent an unrolled RNN over time? Why is the num_units = 128 in most cases ?

Read More