1

Yesterday I asked why a adding 10 times 0.10 to a double is not equal to int 1;

in VB Why (1 = 1) is False

I got an excellent answer. The overview is because:

  • Floating point types and integer types cannot be compared directly, as their binary representations are different.
  • The result of adding 0.1 ten times as a floating point type may well be a value that is close to 1, but not exactly

I can see the reason why now. However, if I do something like:

  Dim d As Double

  For i = 1 To 4
        d = d + 0.25
  Next

  MsgBox(d)  'output 1 
  MsgBox(d = 1) 'True
  MsgBox(1 - d) ' 0
  Console.WriteLine(d) '1

In this case I really obtain equality between double 1.0 and int 1. I thought then that double were only approximations so I would expect d to be somehow a value close to 1 as in the original first question. Why is not a good idea to compare directly different data types (in this case double - integer) and why I this time I obtain equality ??

Community
  • 1
  • 1
Bartzilla
  • 2,768
  • 4
  • 28
  • 37
  • Try comparison between dim d as double and dim i as Int , this will be false i guess. To make cleaner code type casting is needed so programmer knows what is being assigned to what? :) – TeaCupApp Sep 17 '11 at 08:32

2 Answers2

4

Yes, doubles are usually aproximations. But some numbers work better than others.

Just like decimal numbers: you can write 1/10 exactly (0.1), but not 1/3 (0.33333...).

So it happens that 1/4 can be converted exactly to a binary floating point number, where 1/10 can't be.

EDIT
A decimal (floating point) numbers works with powers of 10, so if you can write some number as a combination of 1/10, 1/100, 1/1000 etc (multiples allowed) then you can write that number exactly as a decimal number.

For binary floating point numbers it works the same, only the sequence is 1/2, 1/4, 1/8, 1/16 etc. Plus in computers there is a limit in the precision: some details are just too small to represent exactly.

Hans Kesting
  • 38,117
  • 9
  • 79
  • 111
  • You might want to add that any number of the form j/(2^k), where j and k are integers, can be represented exactly in floating point (up to some limit of j and k). – RBarryYoung Sep 17 '11 at 16:46
1

A double can represent integers precisely. However, you can introduce errors a number of ways, especially when using values less than one. For example, this will usually result with x not precisely equal to 1:

x = 1
x = x / 3
x = x * 3

But in this example, x will be precisely 3

x = 3
x = x / 3
x = x * 3

If you expect a double to have an integer value, you can use Round(x) to round off precisely to the nearest integer.

If Round(x) = 1 Then...

What Every Computer Scientist Should Know About Floating-Point Arithmetic has a really good explanation of floating point numbers (which is what Double is) and their errors.

xpda
  • 15,585
  • 8
  • 51
  • 82