Why is result NSOrderedDescending?
NSDecimalNumber *decimal = [[NSDecimalNumber alloc] initWithString:@"2.6"];
NSComparisonResult result = [decimal compare:[NSNumber numberWithFloat:2.6]];
Asked
Active
Viewed 228 times
2
Rüdiger Hanke
- 6,215
- 2
- 38
- 45
Gonzalo
- 187
- 1
- 10
-
1I bet it's an internal rounding issue. Did you try @"2.60" and 2.6 ? – Mark Granoff Jul 31 '12 at 21:12
-
Thanks Mark. Thought I was being smart using NSDecimalNumber but this rounding err thing runs deeper. Ended up using NSDecimal. – Gonzalo Jul 31 '12 at 22:00
1 Answers
3
2.6 cannot be represented exactly as a floating-point number, but NSDecimal can represent this value exactly (that's why we need NSDecimals in the first place).
According to IEEE 754 Calculator, 2.6 gets converted to 2.5999999046325684, which is less than 2.6, and therefore compares as NSOrderedDescending.
Sergey Kalinichenko
- 714,442
- 84
- 1,110
- 1,523
-
Using NSDecimal works. I also used handy CPDecimalFromString from http://stackoverflow.com/questions/2035421/creating-nsdecimal – Gonzalo Jul 31 '12 at 22:01