2

I need to read values from a binary file. The data format is IBM single Precision Floating Point (4-byte Hexadecimal Exponent Data). I have C++ code that reads from the file and takes out each byte and stores it like so

 unsigned char buf[BUF_LEN];

        for (long position = 0; position < fileLength; position += BUF_LEN) {
            file.read((char* )(&buf[0]), BUF_LEN);

           // printf("\n%8ld:  ", pos);

            for (int byte = 0; byte < BUF_LEN; byte++) {
               // printf(" 0x%-2x", buf[byte]);
            }
        }

This prints out the hexadecimal values of each byte.

this picture specifies IBM single precision floating point IBM single precision floating point

How do I convert the buffer into floating point values?

CrippledTable
  • 784
  • 5
  • 20
  • 1
    Examples are available to covert hex to decimal in below link https://stackoverflow.com/questions/11031159/converting-hexadecimal-to-decimal – Hariom Singh Jul 21 '17 at 02:16

1 Answers1

6

The format is actually quite simple, and not particularly different than IEEE 754 binary32 format (it's actually simpler, not supporting any of the "magic" NaN/Inf values, and having no subnormal numbers, because the mantissa here has an implicit 0 on the left instead of an implicit 1).

As Wikipedia puts it,

The number is represented as the following formula: (−1)sign × 0.significand × 16exponent−64.

If we imagine that the bytes you read are in a uint8_t b[4], then the resulting value should be something like:

uint32_t mantissa = (b[1]<<16) | (b[2]<<8) | b[3];
int exponent = (b[0] & 127) - 64;
double ret = mantissa * exp2(-24 + 4*exponent);
if(b[0] & 128) ret *= -1.;

Notice that here I calculated the result in a double, as the range of a IEEE 754 float is not enough to represent the same-sized IBM single precision value (also the opposite holds). Also, keep in mind that, due to endian issues, you may have to revert the indexes in my code above.

Edit: @Eric Postpischil correctly points out that, if you have C99 or POSIX 2001 available, instead of mantissa * exp2(-24 + 4*exponent) you should use ldexp(mantissa, -24 + 4*exponent), which should be more precise (and possibly faster) across implementations.

Matteo Italia
  • 123,740
  • 17
  • 206
  • 299
  • sorry could you explain what the mantissa is as well as what you would mean by reverting the indexes – CrippledTable Jul 21 '17 at 03:06
  • For clarity that picture that I added is essentially exactly how the data will be stored in the file. So given that in the table it seems as though the most significant bit is in the right most column. So I believe this means it is little endian? and therefore I would have to revert the indexes. I'm not sure how that would work in your code though. Would it require another function that swaps around the bits. – CrippledTable Jul 21 '17 at 03:35
  • With reverting the indexes I mean change `b[0]` <=> `b[3]` and `b[1]` <=> `b[2]`; that should account for byte endian differences. That 2^-1 it's taking about is the most significant *bit* of the *mantissa*, i.e. it's just saying that the MSb of the mantissa corresponds to the value 2^-1 (IOW, you have to multiply the 24-bit integer of the mantissa by 2^-24). – Matteo Italia Jul 21 '17 at 03:36
  • The code that I wrote follows the byte ordering shown by your picture, which is big endian. If it's actually like that, my code should work as is. Also, don't confuse bit ordering with byte ordering; when you deal with software, bits inside a byte are already ordered correctly, you generally have to worry only about byte ordering. – Matteo Italia Jul 21 '17 at 03:40
  • Would you happen to know why when I declare my buf as uint8_t it says there's an invalid conversion from uint8_t* (aka unsigned char*) to std::basic_istream,char.::char_type* {aka char*} [-fpermissive] using int8_t gives the same error but with (aka signed char*) – CrippledTable Jul 21 '17 at 04:51
  • You can use an `unsigned char` buffer and add a cast to `char *` when reading. – Matteo Italia Jul 21 '17 at 06:25
  • 2
    `ldexp(mantissa, -24 + 4*exponent)` is preferable to `mantissa * exp2(-24 + 4*exponent)`. `ldexp` is designed for just these sorts of situations, assembling a floating-point value from a significand and exponent (or adjusting an existing value by integer values of the exponent), and it avoids the numerical problems of `exp2`. Since `exp2` has to handle non-integer arguments, it is implemented with various approximations and may return imperfect results in some implementations. – Eric Postpischil Feb 18 '19 at 20:29