Dumping a C++ array into a file with a certain endianness

Question

I have something like the following :

boost::uint32_t data[ 256 ];
fillMyArray( data );

std::ofstream output( "this.raw", std::ios_base::out | std::ios_base::binary | std::ios_base::trunc );
output.write( reinterpret_cast< char * >( & data ), 256 * 4 );
output.close( );

However, the array will be saved with the local endianness. How can I ensure that it will be stored with little endian ? I can use Boost library if simpler.

Why not simply loop over the data in the array, and write each value with the correct endianess? — Some programmer dude, Aug 15 '13 at 12:12
@JoachimPileborg: Part of the problem with that is to identify what is "correct", and the writes will have to be done "backwards" if the machine is bigendian... — Mats Petersson, Aug 15 '13 at 12:14
Are you trying to write the whole array? Because the actual number of bytes is 256 * sizeof(uint32_t). Or are you thinking that you're casting each element of the array? Because that is not what happens here. — davmac, Aug 15 '13 at 12:16

score 6 · Accepted Answer · answered Aug 18 '13 at 15:01

Using karma::big_word or karma::little_word could be an option:

See it live on Coliru

Output of /a.out && xxd this.raw:

0000000: f000 f001 f002 f003 f004 f005 f006 f007  ................
0000010: f008 f009 f00a f00b f00c f00d f00e f00f  ................
0000020: f010 f011 f012 f013 f014 f015 f016 f017  ................
0000030: f018 f019 f01a f01b f01c f01d f01e f01f  ................
0000040: f020 f021 f022 f023 f024 f025 f026 f027  . .!.".#.$.%.&.'
// ...
0000200: 00f0 01f0 02f0 03f0 04f0 05f0 06f0 07f0  ................
0000210: 08f0 09f0 0af0 0bf0 0cf0 0df0 0ef0 0ff0  ................
0000220: 10f0 11f0 12f0 13f0 14f0 15f0 16f0 17f0  ................
0000230: 18f0 19f0 1af0 1bf0 1cf0 1df0 1ef0 1ff0  ................
0000240: 20f0 21f0 22f0 23f0 24f0 25f0 26f0 27f0   .!.".#.$.%.&.'.

Full code:

#include <boost/spirit/include/karma.hpp>
#include <fstream>

namespace karma = boost::spirit::karma;

template <typename C>
void fillMyArray(C& data) 
{
    std::iota(begin(data), end(data), 0xf000);
}


int main()
{
    std::vector<boost::uint32_t> data(256);
    fillMyArray(data);

    std::ofstream output( "this.raw", std::ios_base::out | std::ios_base::binary | std::ios_base::trunc );
    boost::spirit::karma::ostream_iterator<char> outit(output);

    karma::generate(outit, +karma::big_word,    data);
    karma::generate(outit, +karma::little_word, data);
}

score 1 · Answer 2 · answered Aug 15 '13 at 12:27

I think this can be split into three problem phases:

Identify if the system is little- or big-endian. There are numerous ways to do that, runtime or compile time (generally, compile time is fine).
Make the transformation (if needed).
Write the data.

For 1. boost has a endian.hpp that provides this feature for most setups - it defines BOOST_BIG_ENDIAN or BOOST_LITTLE_ENDIAN.

In case of 2, iterating over the buffer and either copying or converting in place. Most compilers have a built-in function to swap bytes for the data, MSVC provides _byteswap_ulong and GCC has __builtin_bswap32 - for other compilers, check their respective documentation.

For the part 3 shouldn't need to change if the byteswap is done "in place". If it's a copy, obviously the byteswapped data should be provided to write.

Or you can have Boost do all that for you out of the box :) Karma does, and Boost Serialization does too. See my answer. — sehe, Aug 18 '13 at 15:10

Some programmer dude · Answer 3 · 2013-08-18T15:11:16.693

0

While it doesn't write all data in one call, you could use e.g. std::transform to write the values:

std::transform(std::begin(data), std::end(data)
               std::ostream_iterator<uint32_t>(output),
               [](const uint32_t& value) -> uint32_t {
                   return convert_endianess(value);
               });

Where the convert_endianess function is the function to do the endianess conversion.

edited Aug 18 '13 at 15:11

answered Aug 15 '13 at 12:16

Some programmer dude

400,186
35
402
621

what instantiation of `std::ostream_iterator` would that ... be? – sehe Aug 18 '13 at 14:53
`ostream_iterator is a class template last time I checked. Even if you made it compile, the stream would contain stringified numbers, right? – sehe Aug 18 '13 at 15:11
@sehe Ah, it took a little thinking to understand what you meant. Updated answer :) – Some programmer dude Aug 18 '13 at 15:11
Now, I still think this will only with with non standard `basic_ofstream` if you used `ostreambuf_iterator<>`. But that would likely break down or be non-portable due to locale restrictions on implementations – sehe Aug 18 '13 at 15:13

score 0 · Answer 4 · edited May 23 '17 at 12:14

0

Joachim Pileborg has given a good answer. Instead of writing your own function, you can also use std::reverse() as described here How to manage endianess of double from network

edited May 23 '17 at 12:14

Community

1
1

answered Aug 18 '13 at 12:23

O.C.

6,711
1
25
26

Dumping a C++ array into a file with a certain endianness

4 Answers4