softmax and sigmoid function for the output layer

tensorflow computer-vision deep-learning theano keras
user288609 picture user288609 · Dec 31, 2016 · Viewed 15.8k times · Source

In the deep learning implementations related to object detection and semantic segmentation, I have seen the output layers using either sigmoid or softmax. I am not very clear when to use which? It seems to me both of them can support these tasks. Are there any guidelines for this choice?

Answer

martianwars picture martianwars · Dec 31, 2016

softmax() helps when you want a probability distribution, which sums up to 1. sigmoid is used when you want the output to be ranging from 0 to 1, but need not sum to 1.

In your case, you wish to classify and choose between two alternatives. I would recommend using softmax() as you will get a probability distribution which you can apply cross entropy loss function on.

Related questions

Custom loss function in Keras

I'm working on a image class-incremental classifier approach using a CNN as a feature extractor and a fully-connected block for classifying. First, I did a fine-tuning of a VGG per-trained network to do a new task. Once the net is …

Read More
Tensorflow Slim: TypeError: Expected int32, got list containing Tensors of type '_Message' instead

I am following this tutorial for learning TensorFlow Slim but upon running the following code for Inception: import numpy as np import os import tensorflow as tf import urllib2 from datasets import imagenet from nets import inception from preprocessing import …

Read More
TensorFlow: does tf.train.batch automatically load the next batch when the batch has finished training?

For instance, after I have created my operations, fed the batch data through the operation and run the operation, does tf.train.batch automatically feed in another batch of data to the session? I ask this because tf.train.batch …

Read More