java integer rounding (division related)

Keybounce picture Keybounce · Jul 22, 2016 · Viewed 28.9k times · Source

I have run into a surprise with integer division not rounding down as expected.

Simple code:

public class HelloMath {

    public static void main(String[] args) {
        for (int s=1; s< 20; s++)
        {
            int div = 1<<s;
            int res = (int) ((float)-8/ s);
            System.out.printf("Bit %d, result %d\n", s, res);
        }
    }

}

Even with the explicit (float) casts, the output is:

Bit 1, result -8
Bit 2, result -4
Bit 3, result -2
Bit 4, result -2
Bit 5, result -1
Bit 6, result -1
Bit 7, result -1
Bit 8, result -1
Bit 9, result 0
Bit 10, result 0
Bit 11, result 0
Bit 12, result 0
Bit 13, result 0
Bit 14, result 0
Bit 15, result 0
Bit 16, result 0
Bit 17, result 0
Bit 18, result 0
Bit 19, result 0

I was expecting -1 all the way down.

The real code where this is happening does this:

public static int fluidTo8th(int fluid)
{
    if (0 == fluid)
        return 0;   // Technically, this isn't needed :-).
    int wholePart = (fluid-1) * 8 / RealisticFluids.MAX_FLUID; // -1 gets rounding correct;
    // consider fluid of exactly 1/8th.
    return 1+wholePart;
}

RealisticFluids.MAX_FLUID has the value (1<<20). The code is supposed to take an input that is 1 to MAX_FLUID (0 should only happen if the input block is air), and return a number from 0 to 7 -- 1 to 1/8th max is 0, 1/8th +1 to 2/8th is 2, up to 7/8th + 1 to 8/8th is 7.

I'm expecting integer math to round all fractions down. But that's not happening -- I wind up with off-by-one errors all over the place as 5.999 winds up becoming 6 instead of 5.

  1. Where is this behavior documented?
  2. What's the easiest work-around to get the rounding down that I expected?

(full code context: https://github.com/keybounce/Finite-Fluids/blob/master/src/main/java/com/mcfht/realisticfluids/Util.java)

Answer

Solomon Slow picture Solomon Slow · Jul 22, 2016

I'm expecting integer math to round all fractions down.

First of all, you're not doing integer division: You're doing floating point division. (float)-8 is a floating point expression. Therefore, the s in (float)-8/s gets promoted to (float) before the division happens.

Then the floating point result is converted to an int. Reuseman said that integer division rounds toward zero. Well, float->int conversion also rounds toward zero.