Speeding up Math calculations in Java

Question

I have a neural network written in Java which uses a sigmoid transfer function defined as follows:

private static double sigmoid(double x)
{
    return 1 / (1 + Math.exp(-x));
}

and this is called many times during training and computation using the network. Is there any way of speeding this up? It's not that it's slow, it's just that it is used a lot, so a small optimisation here would be a big overall gain.

Answer 1

For neural networks, you don't need the exact value of the sigmoid function. So you can precalculate 100 values and reuse the value that is closest to your input, or even better (as a comment stated) do an interpolation from the neighbour values.

How you can do this is described in this article (link stolen from the answer of s-lott).

This is the sigmoid function:

As you can see, only values of -10 < x < 10 are interesting at all. And, as another comment stated, the function is symmetric. You only have to store half of the values at all.

---

Edit: I'm sorry that I showed the wrong graph here. I've corrected it.