Strange compiler behavior with float literals vs float variables

Question

I have noticed an interesting behavior with float rounding / truncation by the C# compiler. Namely, when a float literal is beyond the guaranteed representable range (7 decimal digits), then a) explicitly casting a float result to float (a semantically unnecessary operation) and b) storing intermediate calculation results in a local variable both change the output. An example:

using System;

class Program
{
    static void Main()
    {
        float f = 2.0499999f;
        var a = f * 100f;
        var b = (int) (f * 100f);
        var c = (int) (float) (f * 100f);
        var d = (int) a;
        var e = (int) (float) a;
        Console.WriteLine(a);
        Console.WriteLine(b);
        Console.WriteLine(c);
        Console.WriteLine(d);
        Console.WriteLine(e);
    }
}

The output is:

In the JITted debug build on my computer, b is calculated as follows:

          var b = (int) (f * 100f);
0000005a  fld         dword ptr [ebp-3Ch] 
0000005d  fmul        dword ptr ds:[035E1648h] 
00000063  fstp        qword ptr [ebp-5Ch] 
00000066  movsd       xmm0,mmword ptr [ebp-5Ch] 
0000006b  cvttsd2si   eax,xmm0 
0000006f  mov         dword ptr [ebp-44h],eax

whereas d is calculated as

          var d = (int) a;
00000096  fld         dword ptr [ebp-40h] 
00000099  fstp        qword ptr [ebp-5Ch] 
0000009c  movsd       xmm0,mmword ptr [ebp-5Ch] 
000000a1  cvttsd2si   eax,xmm0 
000000a5  mov         dword ptr [ebp-4Ch],eax

Finally, my question: why is the second line of the output different from the fourth? Does that extra fmul make such a difference? Also note that if the last (already unrepresentable) digit from the float f is removed or even reduced, everything "falls in place".

i saw answer to this question here but can't find it – Andrey Jun 21 '10 at 20:54 — Andrey, Jun 21 '10 at 20:54

Mark Byers · Accepted Answer · 2010-06-21T21:15:29.067

5

Your question can be simplified to asking why these two results are different:

float f = 2.0499999f;
var a = f * 100f;
var b = (int)(f * 100f);
var d = (int)a;
Console.WriteLine(b);
Console.WriteLine(d);

If you look at the code in .NET Reflector you can see that the above code is actually compiled as if it were the following code:

float f = 2.05f;
float a = f * 100f;
int b = (int) (f * 100f);
int d = (int) a;
Console.WriteLine(b);
Console.WriteLine(d);

Floating point calculations cannot always be made exactly. The result of 2.05 * 100f is not exactly equal to 205, but just a little less due to rounding errors. When this intermediate result is converted to an integer is truncated. When stored as a float it is rounded to the nearest representable form. These two methods of rounding give different results.

Regarding your comment to my answer when you write this:

Console.WriteLine((int) (2.0499999f * 100f));
Console.WriteLine((int)(float)(2.0499999f * 100f));

The calculations are done entirely in the compiler. The above code is equivalent to this:

Console.WriteLine(204);
Console.WriteLine(205);

edited Jun 21 '10 at 21:15

answered Jun 21 '10 at 20:56

Mark Byers

811,555
193
1,581
1,452

So you say the reason is that (int) is done by truncation and (float) means rounding. If that's the case, then why is the output different for Console.WriteLine((int) (2.0499999f * 100f)) and Console.WriteLine((int) (float) (2.0499999f * 100f)) ? – Alan Jun 21 '10 at 21:05
@Alan, check my answer. reason is that float can hold only 7 digits. log (2^23) = 6.9 – Andrey Jun 21 '10 at 21:07
@Alan: When you use hard-coded constants the calculations are done entirely in the compiler and using the compiler's rules, not in the .NET runtime. – Mark Byers Jun 21 '10 at 21:12
@Andrey, thanks. I know that the float exceeds the representable range (see question, I did not edit that part), but then, with your confirmation, this looks a little scary to me - casting a float to float should not make a difference, but in this case, it does. – Alan Jun 21 '10 at 21:13
@Mark: Are these rules different? And if yes, am I supposed to know this, either from the C# language reference doc or MSDN, or is this just an occasional discrepancy between the compiler and the runtime? – Alan Jun 21 '10 at 21:17
@Alan: Apparently they are different as your example shows. I think in general though relying on the least significant digit of a floating point calculation is an incredibly bad idea. – Mark Byers Jun 21 '10 at 21:19

score 4 · Answer 2 · edited May 23 '17 at 12:06

In a comment you asked

Are these rules different?

Yes. Or, rather, the rules allow for different behaviour.

And if yes, am I supposed to know this, either from the C# language reference doc or MSDN, or is this just an occasional discrepancy between the compiler and the runtime

It's implied by the specification. Floating point operations have a certain minimum level of precision that must be met, but the compiler or runtime is permitted to use more precision if it sees fit. That can cause large, observable changes when you do operations that magnify small changes. Rounding, for example, can turn an extremely small change into an extremely large one.

This fact leads to fairly frequently asked questions here. For some background on this situation and other situations that can produce similar discrepancies, see the following:

Why does this floating-point calculation give different results on different machines?

C# XNA Visual Studio: Difference between "release" and "debug" modes?

CLR JIT optimizations violates causality?

https://stackoverflow.com/questions/2494724

Eric, thank you very much. Your last link was especially enlightening. I actually searched for similar scenarios before I posted the question, but apparently my scope was too narrow. — Alan, Jun 23 '10 at 16:46

score 2 · Answer 3 · answered Jun 21 '10 at 21:02

Mark is right about compiler. Now let's fool the compiler:

    float f = (Math.Sin(0.5) < 5) ? 2.0499999f : -1;
    var a = f * 100f;
    var b = (int) (f * 100f);
    var c = (int) (float) (f * 100f);
    var d = (int) a;
    var e = (int) (float) a;
    Console.WriteLine(a);
    Console.WriteLine(b);
    Console.WriteLine(c);
    Console.WriteLine(d);
    Console.WriteLine(e);

first expression is meaningless but prevents compiler from optimization. Result is:

ok, i found the explanation.

2.0499999f can't be stored as float, because it can hold only 7 10-based digits. and this literal is 8 digits, so compiler rounded it because could not store. (should give a warning IMO)

if you change to 2.049999f result will be expected.

Thanks Andrey, I chose Mark's reply on the basis of the compiler vs runtime info, but yours is also relevant. — Alan, Jun 21 '10 at 21:26

Strange compiler behavior with float literals vs float variables

3 Answers3

Linked