Activation function for output layer for regression models in Neural Networks

neural-network regression activation-function
user7400738 picture user7400738 · Oct 11, 2017 · Viewed 22k times · Source

I have been experimenting with neural networks these days. I have come across a general question regarding the activation function to use. This might be a well known fact to but I couldn't understand properly. A lot of the examples and papers I have seen are working on classification problems and they either use sigmoid (in binary case) or softmax (in multi-class case) as the activation function in the out put layer and it makes sense. But I haven't seen any activation function used in the output layer of a regression model.

So my question is that is it by choice we don't use any activation function in the output layer of a regression model as we don't want the activation function to limit or put restrictions on the value. The output value can be any number and as big as thousands so the activation function like sigmoid to tanh won't make sense. Or is there any other reason? Or we actually can use some activation function which are made for these kind of problems?

Answer

Vaibhav K picture Vaibhav K · Apr 10, 2018

for linear regression type of problem, you can simply create the Output layer without any activation function as we are interested in numerical values without any transformation.

more info :

https://machinelearningmastery.com/regression-tutorial-keras-deep-learning-library-python/

for classification : You can use sigmoid, tanh, Softmax etc.

Related questions

Keras - How to perform a prediction using KerasRegressor?

I am new to machine learning, and I am trying to handle Keras to perform regression tasks. I have implemented this code, based on this example. X = df[['full_sq','floor','build_year','num_room','sub_area_2','sub_area_3…

Read More
tensorflow deep neural network for regression always predict same results in one batch

I use a tensorflow to implement a simple multi-layer perceptron for regression. The code is modified from standard mnist classifier, that I only changed the output cost to MSE (use tf.reduce_mean(tf.square(pred-y))), and some input, output …

Read More
How to implement the Softmax derivative independently from any loss function?

For a neural networks library I implemented some activation functions and loss functions and their derivatives. They can be combined arbitrarily and the derivative at the output layers just becomes the product of the loss derivative and the activation derivative. …

Read More