-1

Here is how my calculator should work:

There is a JSON value where I can write the first multiplier - something like this:

{
    "value1": 1.4
}

On the calculator I can write the second multiplier - only 10^n numbers (10, 100, ..., 10000000). And my calc should return me an integer, as I know that always people who use my calc with write less numbers after the decimal point for the first multiplier than we have 0s on the calc for the second multiplier. Yes, my calc is a very-very strange one.

Here are valid inputs:

v1=1.4; v2=100;
v1=1.414; v2=100000;
v1=1.1; v2=100;

What happens when I do this, for example for value1=1.4 and value2=10000 I get 13900. As far as float cannot hold any number sometimes it stores different numbers. For 1.4 internally it stores 1.399999 on my machine. I know why, but you know the QA engineer who tests my app tells me that I need to get 14000. Your calc does not work. How to make my calc so that I will print correct number?

P.S. Of course I have cut out my real problem from the context but the thing is that I have a float in a file and a 10^n number in my program as a user input. How to get correct result?

EDIT1: I don't ask why float works that way. I know why. I ask how to solve the problem even when float works that way.

EDIT2: I use RapidJson to read the JSON file which already returns me wrong number as a double precision number. I can't use libraries that provide with higher precision floating points.

Narek
  • 38,779
  • 79
  • 233
  • 389
  • If you know the valid ranges of user inputs, you can, internally, store your float as integer + expoents. – Amadeus Mar 14 '18 at 15:22
  • Oh, I cannot do that any more. This solution is valid, but it breaks backward compatibility :( – Narek Mar 14 '18 at 15:23
  • Still, if you know the user inputs, you know the lowest valid number, just round to the lowest – Amadeus Mar 14 '18 at 15:24
  • Look into a fixed point or a floating point library that gives you more precision. – NathanOliver Mar 14 '18 at 15:25
  • @NathanOliver cannot do that either. I use RapidJSON to read the JSON file. It returns me a double. That double number is already the wrong number (1.399... instead of 1.4) – Narek Mar 14 '18 at 15:26
  • @Narek Can you instead have it return to you a string? – NathanOliver Mar 14 '18 at 15:27
  • "breaks backward compatibility" every suggestion will do that; because you've not given your specifications well enough – UKMonkey Mar 14 '18 at 15:27
  • @Ron this is not a duplicate. I am not surprised why float works that way. I ask how to solve the issue, even when float works that way. – Narek Mar 14 '18 at 15:28
  • If the second multiplier is 10^n can't you read the integer before the dot `.` and multiply it by 10^n and then read the integer after and multiply it by 10^(n-1) and then add them? – imreal Mar 14 '18 at 15:29
  • 2
    Anyway, if your QA engineer is working with integer and you with float, soon or later you will need to convert it to integer. Just look into your specification to see what is the common denominator. – Amadeus Mar 14 '18 at 15:38
  • Is there any way you can have RapidJason give you the value in a non floating point form? – NathanOliver Mar 14 '18 at 15:51

1 Answers1

0

Round the result when you format it for display. A double precision value is correct to about 15 significant digits, so if you round the result to 12 significant digits you're not going to surprise the user.

Joni
  • 108,737
  • 14
  • 143
  • 193
  • It's about 15-17 digits: *The 53-bit significand precision gives from 15 to 17 significant decimal digits precision (2−53 ≈ 1.11 × 10−16). If a decimal string with at most 15 significant digits is converted to IEEE 754 double-precision representation, and then converted back to a decimal string with the same number of digits, the final result should match the original string. If an IEEE 754 double-precision number is converted to a decimal string with at least 17 significant digits, and then converted back to double-precision representation, the final result must match the original number.* – phuclv Mar 14 '18 at 16:40
  • https://en.wikipedia.org/wiki/Double-precision_floating-point_format Excel always round to 15 digits before displaying, so even if you specify more than 15 decimal digits you'll always get 0s from the 16th digit – phuclv Mar 14 '18 at 16:41