Convert an array of doubles to an array of structs with only double members without copying data

Question

I am using a third-party C++ library to do some heavy lifting in Julia. On the Julia side, data is stored in an object of type Array{Float64, 2} (this is roughly similar to a 2D array of doubles). I can pass this to C++ using a pointer to a double. On the C++ side, however, data is stored in a struct called vector3:

typedef struct _vector3
{
    double x, y, z;
} vector3;

My quick and dirty approach is a five-step process:

Dynamically allocate an array of structs on the C++ side
Copy input data from double* to vector3*
Do heavy lifting
Copy output data from vector3* to double*
Delete dynamically allocated arrays

Copying large amounts of data is very inefficient. Is there some arcane trickery I can use to avoid copying data from double to struct and back? I want to somehow interpret a 1D array of double (with a size that is a multiple of 3) as a 1D array of a struct with 3 double members.

Beware of [strict aliasing rule](https://stackoverflow.com/q/98650/2486888). — , Jul 16 '18 at 07:31
I bet that the copying of the 3 64bits values will have marginal impact on the calculations ( assume that the calculations are no complex than adding only for example) — 0___________, Jul 16 '18 at 08:45
@PeterJ_01 I am always copying arrays of structs, hence I copy N * 3 * 64 bits, where N can be bigger than a million. — Numaerius, Jul 16 '18 at 10:29
and them how many operations you do with the single one? If not too much - do it in julia. — 0___________, Jul 16 '18 at 10:43
The reason for using the third-party library is because in Julia I have an algorithm which is of O(N^2) complexity and the third-party library has a O(N) complexity algorithm. The O(N) algorithm is very complex and I do not intend to implement it myself. — Numaerius, Jul 16 '18 at 11:28

bolov · Answer 1 · 2018-07-16T07:39:57.573

Unfortunately no you cannot. And this is because of the aliasing rule that C++ has. In short if you have an object T you cannot legally access it from a pointer of incompatible type U. In this sense you cannot access an object of type double or double* via a pointer of type struct _vector3 or vice-versa.

If you dig deep enough you will find reinterpret_cast and maybe think "Oh this is exactly what I need" but it is not. No matter what trickery you do (reinterpret_cast or otherwise) to bypass the language restrictions (also known as just make it compile) the fact remains that you can legally access the object of type double only via pointers of type double.

One trick that is often used to type-pune is to use union. Legal in C it is however illegal in C++ but some compilers allow it. In your case however I don't think there is a way to use union.

The ideal situation would be to do the heavy lifting on the double* data directly. If that is feasible on your workflow.

score 3 · Answer 2 · answered Jul 16 '18 at 08:05

Strictly speaking, you cannot. I asked a similar question some times ago (Aliasing struct and array the C++ way), and answers explained why direct aliasing would invoke Undefined Behaviour, and gave some hints on possible workarounds.

That being said you are already in a corner case, because the original data come from a different language. That means that the processing of that data is not covered by the C++ standard and is only defined by the implementation that you are using (gcc/version or clang/version or...)

For your implementation, it might be legal to alias an external array to a C++ struct or an external struct to a C++ array. You shuld carefully the documentation of mixed language programming for your precise implementation.

also take care of possible struct padding – bolov Jul 16 '18 at 10:06 — bolov, Jul 16 '18 at 10:06

score 0 · Answer 3 · answered Jul 21 '18 at 10:04

The other answers mention real issues (the conversion may not work properly). I will add a small runtime check which verifies that aliasing works, and provide a placeholder where you would call/use your copy-heavy code.

int aliasing_supported_internal() {
    double testvec[6];
    _vector3* testptr = (_vector3*)(void*)testvec;
    // check that the pointer wasn't changed
    if (testvec != (void*)testptr) return 0;
    // check for structure padding
    if (testvec+3 != (void*)(testptr+1)) return 0;
    // TODO other checks?
    return 1;
}
int aliasing_supported() {
    static int cached_result = aliasing_supported_internal();
    return cached_result;
}

This code converts a small array of doubles to an array of structures aliasing (not copying), then checks if it is valid. If the conversion works (the function returns 1), you are likely to be able to use the same kind of aliasing (converting via void pointer) yourself.

BEWARE THAT THE CODE CAN STILL BE BROKEN IN UNEXPECTED WAYS. The strict aliasing rules state that even the above check is undefined behavior. This may work or may fail horribly. Only conversions that are allowed to work properly are to void* and back to the original pointer type. Also, the above check may be completely wrong in multiple inheritance hierarchies or with virtual base classes (both are in a sense unsafe to convert to void* because the actual pointer value may be shifted, that is, may change due to constraints other than alignment and by quite a few bytes)

Convert an array of doubles to an array of structs with only double members without copying data

3 Answers3