1

You know how sometimes a code error feels like you must be losing your mind? I thought I was and then I determined that there's some sort of rounding error here.

Why is the first cell in result not equal to the same cell in expected? This is obviously rounding (see second to last test), but it feels like bug-level rounding to me. Second, if 1 - .9 * .9 is not .19, why isn't is .19000000000004 or whatever I usually see when there's a rounding thing happening.

def calc_negation(negations):
    takens = 1. - negations
    return 1. - np.prod(takens, axis=1, keepdims=True)

result = calc_negation(np.array([[.1, .1], [.5, .5]]))
expected =  np.array([[.19], [.75]])

print(result)
print(result == expected)  # show the failed equality.

assert result.shape == (2, 1)
assert result.shape == expected.shape
assert result.dtype == expected.dtype

assert expected[0, 0] == .19  # why is this okay!?!
assert expected[1, 0] == .75
assert np.allclose(result, expected)  # why is numpy rounding 1 - (.9 * .9)? 
assert np.array_equal(result, expected)  # FAILS!
philosofool
  • 773
  • 4
  • 12
  • 3
    https://stackoverflow.com/questions/588004/is-floating-point-math-broken – ddejohn Dec 06 '22 at 00:17
  • 3
    Right. You can't ever reliably compare floating point numbers for exactness. The answer probably IS .1900000000004, but numpy tries to present human-friendly numbers by rounding them for display. – Tim Roberts Dec 06 '22 at 00:18
  • You don't like 53 bits of FP mantissa? Ok, try `import decimal`. https://docs.python.org/3/library/decimal.html – J_H Dec 06 '22 at 00:21

1 Answers1

3

You should read the linked answers about floating point in general. But here is what specifically happened in this case.

Since expected = np.array([[.19], [.75]]), assert expected[0, 0] == .19 had better be OK: you wrote 0.19 both times. Whatever 0.19 parses to, you should at least get the same value both times. Namely 0.190000000000000002220446049250313080847263336181640625 (this is the closest double precision IEEE float to 0.19, converted to decimal exactly as it is, without rounding as we usually do)

Second, if 1 - .9 * .9 is not .19, why isn't is .19000000000004 or whatever I usually see when there's a rounding thing happening.

It was 0.18999999999999994671 actually, but by default floats are not printed to such a precision.

harold
  • 61,398
  • 6
  • 86
  • 164