Sigmoid layer in Keras

Question

I have a list of values, ranging from 15000 to 25000. I have to separate them into two categories, such that (approx) 20000 will end up in category 1 and the rest in category 2. I figured out that the sigmoid activation should work for this. I am using the following layers in keras for that:

model = Sequential()

model.add(Dense(1 , input_dim =1 ))
model.add(Activation('sigmoid'))
model.add(Dense(2 , init='normal' , activation = 'softmax'))
model.compile(loss='mean_absolute_error', optimizer='rmsprop')
model.fit(X_train, y_train, validation_data=(X_test, y_test),epochs=10,batch_size=200,verbose=2)

However, when I run the model for my sample cases, all values end up in category 2. How can I improve this?

Answer 1

If you have a list of values that range between 15000 and 25000 then sigmoid is going to give you near 1.0 for all of those. sigmoid squashes everything to fit between 0 and 1 so very large values like that are going to asymptotically approach 1. Quick and dirty test shows this:

>>> import math
>>> def s(x):
...     return 1.0/(1.0+math.exp(-x))
... 
>>> s(15000)
1.0
>>> s(25000)
1.0

Either normalize the values, which shouldn't be hard since you know the range, or use a different activation function.

Question is sort of ambiguous when you mention that 20000 of them will end up in one category and the rest in the other, so if you mean that you have between 15000 and 25000 values (not referring to the values of the values) that need to be classified, then nevermind