0

This question is similar to In Excel, when I enter 22222.09482 then I see 22222.0948199999 number in the formula bar but it isn't the same. It's about how to verify with precision.

A simple calculation seems to be incorrect, or more accurately, it seems to be imprecise. I verified this using an IF formula. Also tested this in Excel. Basically, I noticed that when increasing the number of significant figures or decimal points the result doesn't appear to be the same.

//Why is this? Have I done something wrong? Can anyone replicate this on their end?//

It seems this appears to be related to the Floating-point arithmetic (see link as pointed out by Cybernetic.Nomad in comments below).

However, the goal here is to verify calculations when Excel and Google Sheet give imprecise results. I thought a simple IF formula can do this but it doesn't due to the issue above and illustrated below. The only option as per the comments is combining IF with ROUND but that only gives precision to a specified decimal point.

19157.39-19052 should and must equal 105.39. However, Google Sheets and Excel seems to disagree giving 105.3899999999999 as per screen shots below: enter image description here enter image description here

Please see example Google Sheet below (you may comment on this sheet as well). https://docs.google.com/spreadsheets/d/1UZQ0FukVw0smbtUJYhZQxwRhmqdEycQI93SrYV_mbQk/edit?usp=sharing

questioner56
  • 187
  • 1
  • 1
  • 8

0 Answers0