1

I have created a multi-platform application using FLTK which using the boost text serialisation functionality to achieve portable cross-platform document saving like functionality. (If this is a bad idea: too late I'm afraid!).

This has worked fine on Mac and on the Windows machine I compiled it on. I however used the compiled binary on a lower specced Windows machine, and, whilst it ran fine with all the other functionality, when opening the save files (ie when deserializing) the program would crash.

Looking at the memory usage in task manager on the lower specced Windows machine and in the Activity monitor I noticed this different behaviour

Windows: memory usage is the problem, it keeps climbing until the program uses around 800Mb of RAM and then the program crashes

Mac: memory use for the same compiled code when deserializing the exact same file tops out at 40Mb.

The windows binary is Win32 (32bit) and I have verified the same normal working behaviour in both 64bit and 32bit Mac versions. All the code was linked using static libraries (except where it isn't possible, for instance for the CRT in Mac OS).

Why would using the exact same code using boost functionality that is self contained in a header only file lead to this wayward memory usage on the Windows machine (it worked fine on a higher specced machine although I did not track the memory use) whilst all the other programs functionality work fine?

And what can I do about it?

Thanks.

user3353819
  • 911
  • 2
  • 8
  • 21
  • To narrow down the problem, have you tried just a simple console program to test the boost serialization similar to what you have written it in the GUI version. Does it also leak memory? – cup May 25 '15 at 21:00

1 Answers1

0

I suspect that the types used are simply not the same.

E.g. on a 32 bit compiler (or target architecture, really) long is usually 32 bits (4 byte).

On a 64 bit compiler, long is usually twice that size. This can trip you up even though you used text representation.

Consider what happens when a number is read that exceeds int32_t capacity. It might result in a large negative number. Now, interpret that as size_t and you will get breakage: it will result in an even greater size, and if the code makes allocations to match ...

So, first thing:

  1. Check all your datatypes to see whether they are architecture-independent. E.g. using

    #include <cstdint>

    Spell or your types like int32_t or uint64_t. Never depend on defaults.

  2. Once that's done, consider using the EPA portable archive implementation found here: https://epa.codeplex.com/

    This is binary archive specifically designed to be portable. I hope this might give you compile-time validation of all the types/sizes involved.

sehe
  • 374,641
  • 47
  • 450
  • 633
  • Both good points, thanks, but haven't been able to implement/test as of yet. Just checking though, the windows version *that did work* on the higher specced machine was compiled for 32bit target architecture, so I'm a little unclear as to why the 32bit version would only fail on the 32bit machine? That aside, given I that I want portability, should I change everything to int32_t (as opposed to int64_t)? – user3353819 May 26 '15 at 17:26
  • int64_t is a perfectly valid type on 32 bit systems. It might just not be as efficient because that exceeds the native register size. However, of course, the application domain dictates what range of integral values is required. The machine never comes into that at all. That would be optimization. Remember [what they say about premature optimization](http://c2.com/cgi/wiki?PrematureOptimization). – sehe May 26 '15 at 20:49
  • I'm replacing everything with int32_t, but I'm doubtful as the text serialisation is identical to before the change. I think I didn't make something clear: the crash on the lower specced 32bit machine used the same binary/executable as on the higher specced 64 bit machine. It was compiled on the higher specced machine but for 32bit architecture. I fail to see how wrong type would lead to the same executable crashing on one but not the other with the same text file. Am I missing something? – user3353819 May 27 '15 at 23:02
  • Mmm mm. It could just be heap overflow perhaps... I guess you would have noticed this though, in terms of memory use – sehe May 27 '15 at 23:05
  • I haven't seen any wayward memory issues before this if that's what you mean. – user3353819 May 27 '15 at 23:22
  • If it is of any relevance, all the objects are, basically, just POD classes (with a couple of exceptions and of course the serialise functions). None contain any pointers (so this, http://www.boost.org/doc/libs/1_54_0/libs/serialization/doc/exception_safety.html, isn't relevant) , just vectors, ints (int32_t now) doubles, and nested classes that contain things like vectors ints and doubles. – user3353819 May 28 '15 at 15:45