Round off error in C (Forward and Backward sum)

Question

#include <stdio.h>
#include <string.h>
#include <math.h>
#include <stdlib.h>

int main() {

    /* Enter your code here. Read input from STDIN. Print output to STDOUT */

    float sum=0.0;

    for(int i=1;i<=1000000;i++)
        sum+=(1.0)/i;

    printf("Forward sum is %f ",sum);

    sum=0.0;
    for(int i=1000000;i>=1;i--)
        sum+=(1.0)/i;

    printf("Backward sum is %f ",sum);

    return 0;
}

Output:- Forward sum is :- 14.357358 Backward sum is :- 14.392652.

Why is there a difference in both sums ? I think that there is some precision error which is causing the difference in both the sums but I am not able to get a clear picture of why is this happening.

OT: regarding: `float sum=0.0;` and `sum=0.0;` The variable 'sum' is declared as `float`, but the literals are `doubles`!. Suggest using: `float sum=0.0f;` and `sum=0.0f;` Note the trailing `f` that makes the literals `float` — user3629249, Aug 12 '18 at 14:36
Possible duplicate of [Is floating point math broken?](https://stackoverflow.com/questions/588004/is-floating-point-math-broken) — 0___________, Aug 12 '18 at 14:38
use search to find duplicatres https://stackoverflow.com/questions/588004/is-floating-point-math-broken — 0___________, Aug 12 '18 at 14:38
OT: regarding the statements: `sum+=(1.0)/i;` the literal `1.0` is a `double`, which the compiler should have told you about converting that `double` to `float` — user3629249, Aug 12 '18 at 14:45
OT: it is poor programming practice to include header files those contents are not used. Suggest removing: `#include #include #include ` — user3629249, Aug 12 '18 at 14:51
@P__J__: That is not a duplicate. It covers general information about floating-point, not specific issues that arise, and not this particular issue. We do not mark various C questions as duplicates of one “Is C broken?” question. We answer the individual issues. The same should be done with floating-point. — Eric Postpischil, Aug 12 '18 at 14:57

Paul Ogilvie · Answer 1 · 2018-08-12T17:22:55.337

The first loop starts with adding relatively large parts to the sum, and decreasingly smaller parts when the sum gets larger. So while more bits are needed to represent the sum, less bits are available for the small parts.

In the second loop, small parts are added to the sum, and increasingly larger parts are added when the sum gets larger. So less bits are required to store the the newly added part relative to the current value of sum.

(Not a very scientific explanation, but i hope this verbal attempt makes the principle clear)

N.b.: it also means the second result is more accurate.

In an attempt to be more precise: in order to add two floating point numbers they need to be scaled to have the same number of bits for mantissa and exponent. When the sum gets larger, the item added to will be scaled so as not to loose significance of this sum. As a result, the least significant bits of the part to be added will be scaled out of the register before the addition. For example (hypothetical) adding 0.00000001 to 1,000,000,000 will result in adding zero to this large number.

@rici, important semantics: updated. However, I added some precision to my answer :-) — Paul Ogilvie, Aug 12 '18 at 17:23

Steve Summit · Accepted Answer · 2018-08-12T14:57:26.427

This is one of the surprising aspects of floating-point arithmetic: it actually matters what order you do things like addition in. (Formally, we say that floating-point addition is not commutative.)

It's pretty easy to see why this is the case, with a simpler, slightly artificial example. Let's say you have this addition problem:

1000000. + 0.1 + 0.1 + 0.1 + 0.1 + 0.1 + 0.1 + 0.1 + 0.1 + 0.1 + 0.1

But let's say that you're using a single-precision floating-point format that has only 7 digits of precision. So even though you might think that 1000000.0 + 0.1 would be 1000000.1, actually it would be rounded off to 1000000.. So 1000000.0 + 0.1 + 0.1 would also be 1000000., and adding in all 10 copies of 0.1 would still result in just 1000000., also.

But if instead you tried this:

0.1 + 0.1 + 0.1 + 0.1 + 0.1 + 0.1 + 0.1 + 0.1 + 0.1 + 0.1 + 1000000.

Now, when you add 0.1 + 0.1, there's no problem with precision, so you get 0.2. So it you add 0.1 ten times, you get 1.0. So if you do the whole problem in that order, you'll get 1000001..

You can see this yourself. Try this program:

#include <stdio.h>

int main()
{
    float f1 = 100000.0, f2 = 0.0;
    int i;
    for(i = 0; i < 10; i++) {
        f1 += 0.1;
        f2 += 0.1;
    }
    f2 += 100000.0;
    printf("%.1f %.1f\n", f1, f2);
}

On my computer, this prints 100001.0 100001.0, as expected. But if I change the two big numbers to 10000000.0, then it prints 10000000.0 10000001.0. The two numbers are clearly unequal.

Round off error in C (Forward and Backward sum)

2 Answers2