100

Let:

double d = 0.1;
float f = 0.1;

should the expression

(f > d)

return true or false?

Empirically, the answer is true. However, I expected it to be false.

As 0.1 cannot be perfectly represented in binary, while double has 15 to 16 decimal digits of precision, and float has only 7. So, they both are less than 0.1, while the double is more close to 0.1.

I need an exact explanation for the true.

Mark Lakata
  • 19,989
  • 5
  • 106
  • 123
Hesham Eraqi
  • 2,444
  • 4
  • 23
  • 45
  • 63
    Less precision doesn't mean lesser number. – zakinster Oct 10 '13 at 09:43
  • 8
    Why did you expect the reverse? – user207421 Oct 10 '13 at 09:47
  • The rounding off value of the float, simply happens to be more than the precise value of double... – Barath Ravikumar Oct 10 '13 at 09:50
  • 7
    @BeagleBone In IEEE 754 floating point, every 32-bit binary value is also exactly representable as a 64-bit binary value. In any system using them for float and double, a decimal-to-float conversion will never be more precise than the corresponding decimal-to-double conversion. As shown in the answers, the double is more precise, and smaller because both numbers are larger than decimal 0.1. – Patricia Shanahan Oct 10 '13 at 10:02
  • 2
    IEEE [`double(0.1)`](http://www.binaryconvert.com/result_double.html?decimal=048046049) and [`float(0.1)`](http://www.binaryconvert.com/result_float.html?decimal=048046049) – Grijesh Chauhan Oct 10 '13 at 11:19
  • 11
    The only given thing is, double precision will be CLOSER to the wanted value. It could be smaller, it could be bigger. – SinisterMJ Oct 10 '13 at 11:27
  • 1
    @SinisterMJ: Or it could be the same. – Sven Marnach Oct 10 '13 at 12:01
  • 3
    Just for the record, since no one mentioned yet: there is a great article called "What Every Computer Scientist Should Know About Floating-Point Arithmetic", that give some details about all those troubles of using floating point. available at http://docs.oracle.com/cd/E19957-01/806-3568/ncg_goldberg.html – woliveirajr Oct 10 '13 at 12:27
  • 9
    @heshameraqi: The answers (especially Kenny's) are great in showing what happens. I believe you could benefit from reading: **[What Every Computer Scientist Should Know About Floating-Point Arithmetic](http://docs.oracle.com/cd/E19957-01/806-3568/ncg_goldberg.html)** – Olivier Dulac Oct 10 '13 at 12:58
  • 5
    There should be a close reason on StackOverflow called "Refer to stock computer science paper Everyone should have already read", including WECSSKAFPA, and others. I have posted the above comments 100+ times in the last 3 years. – Warren P Oct 10 '13 at 13:10
  • 8
    Suppose you are computing 1/9 in 3-digit decimal and 6-digit decimal. `0.111 < 0.111111`, right? Now suppose you are computing 6/9. `0.667 > 0.666667`, right? You can't have it that 6/9 in three digit decimal is `0.666` because that is not the closest 3-digit decimal to 6/9! – Eric Lippert Oct 10 '13 at 13:26
  • 1
    @PatriciaShanahan: In the .NET framework, conversion of other types to `float` is performed by conversion to `double` followed by conversion to `float`, so conversions "directly" to `float` cannot be more precise than conversions via `double` [since all conversions to `float` are done via `double`], but float rounding could be better. The `long` numbers 9007199254740992 (0x20000000000000) and 9007200328482816 (0x20000040000000) are both representable as `float`; although the number 9007199791611905 (0x20000020000001) is obviously closer to the latter, (float)9007199791611905 yields the former. – supercat Oct 10 '13 at 16:20
  • @supercat Interesting. Is there freely available documentation where this kind of information about .NET's floating-point can be found, instead of having to discover this one comment at a time? (Also I admit that some kind of morbid curiosity makes me want to see this for myself in some sort of official documentation. Not that I disbelieve you, but it is the difference between having a road crash described to you by a third party and taking a peek yourself) – Pascal Cuoq Oct 10 '13 at 16:58
  • @PascalCuoq: I don't know of any guarantee that no legitimate future implementation of .NET will ever make (float)X perform rounding differently from (float)(double)X. You should, however, easily be able to test what a present implementation of .NET does when converting various renditions of the numeric value 9007199791611905 to float, without having to take my or anyone else's word for it. – supercat Oct 10 '13 at 17:21
  • Please don't close question it is not very much similar – Grijesh Chauhan Oct 11 '13 at 09:35
  • @EricLippert Clearest explanation :) – Hesham Eraqi Oct 17 '13 at 08:53

7 Answers7

173

I'd say the answer depends on the rounding mode when converting the double to float. float has 24 binary bits of precision, and double has 53. In binary, 0.1 is:

0.1₁₀ = 0.0001100110011001100110011001100110011001100110011…₂
             ^        ^         ^   ^ 
             1       10        20  24

So if we round up at the 24th digit, we'll get

0.1₁₀ ~ 0.000110011001100110011001101

which is greater than the exact value and the more precise approximation at 53 digits.

kennytm
  • 510,854
  • 105
  • 1,084
  • 1,005
  • 15
    precise answer...! – Barath Ravikumar Oct 10 '13 at 09:48
  • Logical Explanation. What do you think about Sven's answer though? – Hesham Eraqi Oct 10 '13 at 10:21
  • in other words `0.2d > 0.2f` – ratchet freak Oct 10 '13 at 12:27
  • 1
    @ratchetfreak ... or not... depends if the language implements IEEE 754, in which platform it is run... – woliveirajr Oct 10 '13 at 12:29
  • 2
    @HeshamERAQI Both answers are correct - I guess this one makes it easier to see *why* this happens. For both explanations though, it is critical to take into account how the result is rounded at the cut-off point. Like other comments have said, IEEE754, the de-facto standard on FP representations, does it this way, but in theory there could be FP implementations that simply truncated instead of rounding. – us2012 Oct 10 '13 at 14:18
55

The number 0.1 will be rounded to the closest floating-point representation with the given precision. This approximation might be either greater than or less than 0.1, so without looking at the actual values, you can't predict whether the single precision or double precision approximation is greater.

Here's what the double precision value gets rounded to (using a Python interpreter):

>>> "%.55f" % 0.1
'0.1000000000000000055511151231257827021181583404541015625'

And here's the single precision value:

>>> "%.55f" % numpy.float32("0.1")
'0.1000000014901161193847656250000000000000000000000000000'

So you can see that the single precision approximation is greater.

Sven Marnach
  • 574,206
  • 118
  • 941
  • 841
  • 4
    But this will depend on the implementation. I suspect that your results are for IEEE `double` and `float`, since that is by far the most commonly used today, but you might get different results on an IBM or a Unisys mainframe. – James Kanze Oct 10 '13 at 09:56
  • 3
    @JamesKanze: This is on a x86 machine. Python uses native floating-point operations, and the x86 implementation (more or less) conforms to IEEE-754. – Sven Marnach Oct 10 '13 at 10:03
  • That's more or less what I assumed (IEEE-745, at least): I think almost all machines for which Python is implemented use IEEE-754. My point is just that the corresponding values may be different on another architecture (like some of the mainframes, which don't use IEEE). – James Kanze Oct 10 '13 at 10:59
19

If you convert .1 to binary you get:

0.000110011001100110011001100110011001100110011001100...

repeating forever

Mapping to data types, you get:

float(.1)  = %.00011001100110011001101
                                     ^--- note rounding
double(.1) = %.0001100110011001100110011001100110011001100110011010

Convert that to base 10:

float(.1)  = .10000002384185791015625
double(.1) = .100000000000000088817841970012523233890533447265625

This was taken from an article written by Bruce Dawson. it can be found here:
Doubles are not floats, so don’t compare them

Grijesh Chauhan
  • 57,103
  • 20
  • 141
  • 208
static_cast
  • 1,174
  • 1
  • 15
  • 21
5

Since it can't be exactly represented, comparing 1/10 in base 2 is like comparing 1/7 in base 10.

1/7 = 0.142857142857... but comparing at different base 10 precisions (3 versus 6 decimal places) we have 0.143 > 0.142857.

xan
  • 7,511
  • 2
  • 32
  • 45
5

I think Eric Lippert's comment on the question is actually the clearest explanation, so I'll repost it as an answer:

Suppose you are computing 1/9 in 3-digit decimal and 6-digit decimal. 0.111 < 0.111111, right?

Now suppose you are computing 6/9. 0.667 > 0.666667, right?

You can't have it that 6/9 in three digit decimal is 0.666 because that is not the closest 3-digit decimal to 6/9!

Community
  • 1
  • 1
Kip
  • 107,154
  • 87
  • 232
  • 265
1

Just to add to the other answers talking about IEEE-754 and x86: the issue is even more complicated than they make it seem. There is not "one" representation of 0.1 in IEEE-754 - there are two. Either rounding the last digit down or up would be valid. This difference can and does actually occur, because x86 does not use 64-bits for its internal floating-point computations; it actually uses 80-bits! This is called double extended-precision.

So, even among just x86 compilers, it sometimes happen that the same number is represented two different ways, because some computes its binary representation with 64-bits, while others use 80.

In fact, it can happen even with the same compiler, even on the same machine!

#include <iostream>
#include <cmath>

void foo(double x, double y)
{
  if (std::cos(x) != std::cos(y)) {
    std::cout << "Huh?!?\n";  //← you might end up here when x == y!!
  }
}

int main()
{
  foo(1.0, 1.0);
  return 0;
}

See Why is cos(x) != cos(y) even though x == y? for more info.

BlueRaja - Danny Pflughoeft
  • 84,206
  • 33
  • 197
  • 283
  • A much better description of the issues you are trying to refer to can be found in the article http://arxiv.org/abs/cs/0701192 by David Monniaux. The only drawback of that article is that it describes the situation before Joseph S. Myers decided to fix the situation for GCC: http://gcc.gnu.org/ml/gcc-patches/2008-11/msg00105.html . His post is a good explanation of how a compiler with FLT_EVAL_METHOD>0 should work like. With such a compiler, the same computation produces the same results. `cos(1.0)` is `cos(1.0)`. It may not be what you'd get with FLT_EVAL_METHOD=0, but it's always the same. – Pascal Cuoq Oct 10 '13 at 17:08
  • Some x86 compilers in some modes may use the SSE hardware even for scalar math, so intermediate precision will be 32 or 64 bits rather than the 80 bits of the x87 ALU. – Russell Borogove Oct 11 '13 at 16:22
  • 1
    I once got mad with a piece of code deemed to take the largest value computed by a function. Because of these unexpected rounding artifacts, the code was returning the first largest value instead of the last largest value (in case of ties). Took me a while to realize that x > x can hold ! –  Oct 16 '13 at 06:38
1

The rank of double is greater than that of float in conversions. By doing a logical comparison, f is cast to double and maybe the implementation you are using is giving inconsistent results. If you suffix f so the compiler registers it as a float, then you get 0.00 which is false in double type. Unsuffixed floating types are double. 

#include <stdio.h>
#include <float.h>

int main()
{
     double d = 0.1;
     float f = 0.1f;
     printf("%f\n", (f > d));

     return 0;
}
Odimegwu David
  • 43
  • 7