Top "Numerical-stability" questions

In the mathematical subfield of numerical analysis, numerical stability is a generally desirable property of numerical algorithms.

Calculating an inverse matrix in Matlab

I'm running an optimization algorithm that requires calculation of the inverse of a matrix. The goal of the algorithm is …

Read More
matlab linear-algebra matrix-inverse numerical-stability
In Python small floats tending to zero

I have a Bayesian Classifier programmed in Python, the problem is that when I multiply the features probabilities I get …

Read More
python floating-point numerical-stability
Robust atan(y,x) on GLSL for converting XY coordinate to angle

In GLSL (specifically 3.00 that I'm using), there are two versions of atan(): atan(y_over_x) can only return angles …

Read More
c++ glsl coordinates atan2 numerical-stability