How to avoid an overflow in numpy.exp()

Question

I read a lot on stack Overflow but i still can't understand how avoiding the overflow error. I'am building a Neural network which use the sigmoid function. But I cant go on without converting or finding a workaround for these errors.

def activation(x):
    return  1/(1+np.exp(-x))

  def dactivation(x):
    return  activation(x)*(1-activation(x))


  def propagateb(self, target, lrate=8.1, momentum=0.1):
        deltas = []
        error = target - self.layers[-1]
        delta = error*dactivation(self.layers[-1])
        deltas.append(delta)
        for i in range(len(self.shape)-2,0,-1):
            delta =np.dot(deltas[0],self.weights[i].T)*dactivation(self.layers[i])
            deltas.insert(0,delta)
        for i in range(len(self.weights)):
            layer = np.atleast_2d(self.layers[i])
            delta = np.atleast_2d(deltas[i])
            dw = np.dot(layer.T,delta)
            self.weights[i] += lrate*dw + momentum*self.dw[i]
            self.dw[i] = dw

        # Return error
        return (error**2).sum()

raise

ann.py:5: RuntimeWarning: overflow encountered in exp
  return  1/(1+np.exp(-x))

Answer 1

SciPy comes with a function to do that, which won't give you that warning:

scipy.special.expit(x)