2

When I do the following double multiplication in C# 100.0 * 1.005 I get 100,49999999999999 as a result. I believe this is because the exact number (or some intermedia result when evaluting the expression) cannot be represented. When I do the same computation in calc.exe I get 100.5 as expected.

Another example is the ninefold incrementation of 0.001 (that is the first time a deviation occurs) so basically 9d * 0.001d = 0,0090000000000000011. When I do the same computation in calc.exe I get 0.009 as expected.

Now one can argue, that I should choose decimal instead. But with decimal I get the problem with other computations for example with ((1M / 3M) * 3M) = 0,9999999999999999999999999999 while calc.exe says 1.

With calc.exe I can divide 1 by 3 several times until some real small number and then multiply with 3 again as many several times and then I reach exacty 1. I therefore suspect, that calc.exe computes internally with fractions, but obviously with real big ones, because it computes 

(677605234775492641 / 116759166847407000) + (932737194383944703 / 2451942503795547000)

where the common denominator is -3422539506717149376 (an overflow occured) when doing a long computation, so it must be at least ulong. Does anybody know how computation in calc.exe is implemented? Is this implementation made somewhere public for reuse?

David
  • 2,426
  • 3
  • 24
  • 36
  • What type are you casting to for your numbers? – Mark Schultheiss Sep 08 '16 at 12:07
  • 2
    Microsoft was getting complaints about their old version producing 9,999... so they switched to another computation engine that doesn't lose any digits. So 1,234 x 4,567 has a result with 6 digits in the fraction. And uses base 10 instead of base 2, like System.Decimal. But it is only partial, transcendental functions don't use the expanded range. Such an engine is both slow and requires a lot of storage. Doesn't matter to calc.exe because it runs at human-time, a C# program runs at processor-time. You'll have to go shopping. – Hans Passant Sep 08 '16 at 12:21
  • 1
    @MarkSchultheiss: Sorry, I don't understand. Please clarify. – David Sep 08 '16 at 12:24
  • Are you using decimal, double,float etc. For example, double has a limit to the storage bits that impacts the calculation capability. https://msdn.microsoft.com/en-us/library/bfft1t3c.aspx With a double you WILL experience the issues you describe at some point and this is one reason you should not use it for some calculations. THIS might be a duplicate of http://stackoverflow.com/questions/2741903/c-sharp-4-double-minus-double-giving-precision-problems – Mark Schultheiss Sep 08 '16 at 13:29
  • I have added `d` and `M` in my code above to emphasize to which data type the examples refer to. Our code should be as fast as possible, so double is a better choice than decimal. – David Sep 08 '16 at 13:32
  • You should read the specification (download from here https://www.microsoft.com/en-us/download/details.aspx?id=7029) and understand the impact of the precision and number of bits used for each. IF you want to truly emulate calc you should consider rounding, possibly at each step of the formula. – Mark Schultheiss Sep 08 '16 at 14:09
  • @MarkSchultheiss: calc uses an arbitrary-precision engine for algorithmic operations. Rounding at every step actually does the *opposite* of giving you more precision. – Joey Sep 09 '16 at 07:13

1 Answers1

10

As described here, calc uses an arbitrary-precision engine for its calculations, while double is standard IEEE-754 arithmetic, and decimal is also floating-point arithmetic, just in decimal, which, as you point out, has the same problems, just in another base.

You can try finding such an arbitrary-precision arithmetic library for C# and use it, e.g. this one (no idea whether it's good; was the first result). The one inside calc is not available as an API, so you cannot use it.

Another point is that when you round the result to a certain number of places (less than 15), you'd also get the intuitively "correct" result in a lot of cases. C# already does some rounding to hide the exact value of a double from you (where 0.3 is definitely not exactly 0.3, but probably something like 0.30000000000000004). By reducing the number of digits you display you lessen the occurrence of such very small differences from the correct value.

Joey
  • 344,408
  • 85
  • 689
  • 683
  • "The one inside calc is not available as an API, so you cannot use it." - The Windows Calculator is open source now: https://github.com/microsoft/calculator Is its API available now? I don't use windows 10 so I can't build the code to test it. – 123iamking Nov 09 '21 at 10:13