how to handle floating point imprecision?

Question

I am converting an array of bytes to a 32 bit floating point. Sometimes the numbers are slightly off.

Example:

10.1 becomes 10.100000381469727 when I serialize the value in RapidJSON. How can I normalize this?

I can't share that code. What I can share is this to prove it:

std::string BytesToHexString(
    unsigned char* data,
    size_t len
)
{
    std::stringstream ss;
    ss << std::hex << std::setfill('0');
    for (size_t i = len - 1; i >= 0 && i < len; --i)
        ss << std::setw(2) << static_cast<int>(data[i]);
    return ss.str();
}

std::string FLOATvalueToHexString(
    float value
)
{
    union FloatToUChar {
        float f;
        unsigned char  c[sizeof(float)];
    };

    FloatToUChar floatUnion;
    floatUnion.f = value;

    return BytesToHexString(
        floatUnion.c,
        sizeof(float)
    );
}

int main()
{
    std::string sFloatValue = "10.100000";

    float fltValue = atof(sFloatValue.c_str());

    std::string strHexFloatValue = FLOATvalueToHexString(fltValue);

    std::cout << sFloatValue << " " << fltValue << " " << strHexFloatValue << std::endl;

    return 0;
}

It prints: 10.100000 10.1 4121999a The debugger says fltValue is 10.1000004.

If I convert 4121999a then this confirms that the internal storage is indeed off:

https://babbage.cs.qc.cuny.edu/IEEE-754.old/32bit.html

10.100000381469727

How can I normalize the floating point so I can at least get the correct hexadecimal value?

What is "the correct hexadecimal value" then? 10.1 simply _does not exist_ as a possible value for the `float` type. https://stackoverflow.com/questions/588004/is-floating-point-math-broken has some good answers that explain this. — Thomas, Mar 17 '20 at 14:45
You could try an arbitrary precision library like [GMP](https://gmplib.org/). — Galik, Mar 17 '20 at 14:50
Does this answer your question? [Is floating point math broken?](https://stackoverflow.com/questions/588004/is-floating-point-math-broken) Canonical duplicate with lots of help / explanation — Richard Critten, Mar 17 '20 at 14:57
Why are you using `float` in preference to `double`? If you can use `double`, you will get the same effect but in my lower significance digits. — Patricia Shanahan, Mar 17 '20 at 16:33

score 1 · Answer 1 · answered Mar 17 '20 at 14:50

1

Just like an int type can't be used to non-whole numbers, a double can only store a subset of the real numbers too.

If you want to be able to store 0.1 exactly, then use a decimal type. See C++ decimal data types for a starting point.

Job done!

answered Mar 17 '20 at 14:50

Bathsheba

231,907
34
361
483

how to handle floating point imprecision?

1 Answers1