Default activation function in Keras

neural-network deep-learning keras recurrent-neural-network keras-layer
Kiran Baktha picture Kiran Baktha · Mar 18, 2017 · Viewed 7.2k times · Source

Does anyone know the default activation function used in the recurrent layers in Keras? https://keras.io/layers/recurrent/

It says the default activation function is linear. But what about the default recurrent activation function. Nothing is mentioned about that. Any help would be highly appreciated. Thanks in advance

Answer

Marcin Możejko picture Marcin Możejko · Mar 18, 2017

Keras Recurrent is an abstact class for recurrent layers. In Keras 2.0 all default activations are linear for all implemented RNNs (LSTM, GRU and SimpleRNN). In previous versions you had:

  • linear for SimpleRNN,
  • tanh for LSTM and GRU.

Related questions

Many to one and many to many LSTM examples in Keras

I try to understand LSTMs and how to build them with Keras. I found out, that there are principally the 4 modes to run a RNN (the 4 right ones in the picture) Image source: Andrej Karpathy Now I wonder how a …

Read More
LSTM Followed by Mean Pooling

I'm using Keras 1.0. My problem is identical to this one (How to implement a Mean Pooling layer in Keras), but the answer there does not seem to be sufficient for me. I want to implement this network: The following code …

Read More
Why binary_crossentropy and categorical_crossentropy give different performances for the same problem?

I'm trying to train a CNN to categorize text by topic. When I use binary cross-entropy I get ~80% accuracy, with categorical cross-entropy I get ~50% accuracy. I don't understand why this is. It's a multiclass problem, doesn't that mean that I …

Read More