Comparing decimals of given numbers VB.NET

Question

I'm trying to see if two given numbers, for example 1.134 and 1.135 have the same decimals. So far everything works correctly, but when working with number with a large number of decimals I have a problem: I get the following error for the following input 0.841666666666667, 0.841468253968254:

    Unhandled Exception:
System.OverflowException: Arithmetic operation resulted in an overflow.

This is my code

Function TrueOrFalse(ByVal x As Double, ByVal y As Double, ByVal n as Integer) As Boolean
   
    For i As Integer = 1 To n+1
        
        If  CInt(x) <> CInt(y) Then
            Return False
        End If
        x *= 10
        y *= 10
    Next
    Return True
End Function

I understand that it's because the Int type cannot contains that much digits, and when changing CInt to CLng everything works perfectly. Problem: I cannot use CLng. Is there any alternatives ? Thanks!

You could "cut off" the part on the left side of the `.` after you compared it for equality, since you already know that this part is equal (i.e. `x = x mod 1`, and the same for `y`). — Heinzi, Jan 04 '21 at 16:42
EDIT: It worked, thank you very much for your help, if you want you can submit your answer so I can accept it! — MM1, Jan 04 '21 at 16:49
I have also added a simpler solution to solve your underlying problem as a separate answer, just in case you are interested. — Heinzi, Jan 04 '21 at 22:18

Heinzi · Accepted Answer · 2021-01-04T22:17:29.203

You could "cut off" the integer part after you compared it for equality, since you already know that this part is equal, i.e., instead of

CInt(1.134) = CInt(1.135)
CInt(11.34) = CInt(11.35)
CInt(113.4) = CInt(113.5)
etc.

you just compare

CInt(1.134) = CInt(1.135)
CInt(1.34) = CInt(1.35)
CInt(3.4) = CInt(3.5)

That way, your values won't get too large.

How to do that? Before multiplying x and y by ten, you remove the integer part, either straightforward with

x = x - CInt(x)
y = y - CInt(y)

or by abusing the Mod operator:

x = x Mod 1
y = y Mod 1

A few side notes:

TrueOrFalse is not a great name for your method, since it does not describe what the method does. AreEqualUpToNDecimalPlaces would be a better name.
Beware of floating point imprecisions when using the Double data type.
In case this is not a toy problem for educational purposes, note that there are simpler solutions to your concrete problem (I have added one example as a separate answer).

Heinzi · Answer 2 · 2021-01-05T08:03:23.233

If you exceed the range of an Integer, don't use an Integer. You can truncate Doubles just fine using Math.Truncate.

Using that built-in functionality, your method could be reduced to the following one-liner:

Function AreEqualUpToNDecimalPlaces(ByVal x As Double, ByVal y As Double, ByVal n as Integer) As Boolean
    Return Math.Truncate(x * (10.0^n)) = Math.Truncate(y * (10.0^n))
End Function

Test cases:

Console.WriteLine(AreEqualUpToNDecimalPlaces(1.134, 1.135, 2))  ' True
Console.WriteLine(AreEqualUpToNDecimalPlaces(1.134, 1.135, 3))  ' False

Console.WriteLine(AreEqualUpToNDecimalPlaces(0.841666666666667, 0.841468253968254, 3))  ' True
Console.WriteLine(AreEqualUpToNDecimalPlaces(0.841666666666667, 0.841468253968254, 4))  ' False

Comparing decimals of given numbers VB.NET

2 Answers2