For questions related to numerical precision in programming.
Is there a printf width specifier which can be applied to a floating point specifier that would automatically format the …
c floating-point printf c99 floating-point-precisionI have an XSD that requires me to use a BigDecimal for a lat/lon. I currently have the lat/…
java bigdecimal floating-point-precisionHow can I find the index of the minimum item in a Python list of floats? If they were integers, …
python list floating-point minimum floating-point-precisionIn JavaScript, I have a number which is 21 digits, and I want to parse it. Does a parseDouble method exist …
javascript floating-point-precisionIn C++ (and C), a floating point literal without suffix defaults to double, while the suffix f implies a float. …
c++ c literals floating-point-precisionI often need to floor or ceil a CGFloat to an int, for calculation of an array index. The problem …
ios math floating-point floating-accuracy floating-point-precisionI was trying to write a function to approximate square roots (I know there's the math module...I want to …
python python-3.x floating-point floating-point-precisionI have a list of floats in Python and when I convert it into a string, I get the follwoing [1883.95, 1878.3299999999999, 1869.4300000000001, 1863.4000000000001] …
python list formatting floating-point-precision floating-point-conversionI am having a hard time understanding the precision parameter p for float(p) in PostgreSQL. For example, float(1) and …
postgresql floating-point precision sqldatatypes floating-point-precisionFor what values of x does the test (x == 0) return true? Is there some kind of margin or does the …
java zero floating-point-precision