Missing data while copying buffer to struct

Question

I have a TCP server socket which receives a 16-byte request message. The request message will have several field and based on the field values i need to proceed with different actions.

When I tried to copy the buffer to struct, I could see missing data. I tried all possible methods but couldn't able to figure out whether I need to do structure padding or not.

My structure looks like,

struct stRequestMsg {
    uint16_t startTag;
    uint32_t messageSize;
    uint16_t messageID;
    uint16_t sequenceNumber;
    uint16_t messageType;
    uint32_t reserved;
};

all I do is,

char buff[1024]
result = static_cast<int>(recv(sockDesc, buff, sizeof(stRequestMsg), 0));
    if (0 < result) {

        printf("\n Actual value on buffer");
        for (int i = 0; i < result; i++)
        {
            printf("\n buff[%d] = 0x%x", i,buff[i]);
        }

        reqMessage = *(stRequestMsg *)buff;
        printf("\n RESULT of reqMessage = *(stRequestMsg *)buff;");
        printf("\nstartTag : 0x%x", reqMessage.startTag);
    printf("\nmessageSize : 0x%x", reqMessage.messageSize);
    printf("\nmessageID : 0x%x", reqMessage.messageID);
    printf("\nsequenceNumber : 0x%x", reqMessage.sequenceNumber);
    printf("\nmessageType : 0x%x", reqMessage.messageType);
printf("\nreserved : 0x%x", reqMessage.reserved);

        stRequestMsg hdr;
        std::copy(&hdr, &hdr + 1, reinterpret_cast<stRequestMsg*>(buff));
        printf("\n RESULT of std::copy(&hdr, &hdr + 1, reinterpret_cast<stRequestMsg*>(buff));");
        printf("\nstartTag : 0x%x", reqMessage.startTag);
    printf("\nmessageSize : 0x%x", reqMessage.messageSize);
    printf("\nmessageID : 0x%x", reqMessage.messageID);
    printf("\nsequenceNumber : 0x%x", reqMessage.sequenceNumber);
    printf("\nmessageType : 0x%x", reqMessage.messageType);
printf("\nreserved : 0x%x", reqMessage.reserved);

        memcpy(&reqMessage, buff, sizeof(stRequestMsg));
        printf("\n RESULT of std::copy(&hdr, &hdr + 1, reinterpret_cast<stRequestMsg*>(buff));");
        printf("\nstartTag : 0x%x", reqMessage.startTag);
    printf("\nmessageSize : 0x%x", reqMessage.messageSize);
    printf("\nmessageID : 0x%x", reqMessage.messageID);
    printf("\nsequenceNumber : 0x%x", reqMessage.sequenceNumber);
    printf("\nmessageType : 0x%x", reqMessage.messageType);
printf("\nreserved : 0x%x", reqMessage.reserved);

        reqMessage = *reinterpret_cast<stRequestMsg*>(buff);
        printf("\n RESULT of reqMessage = *reinterpret_cast<stRequestMsg*>(buff);");
        printf("\nstartTag : 0x%x", reqMessage.startTag);
    printf("\nmessageSize : 0x%x", reqMessage.messageSize);
    printf("\nmessageID : 0x%x", reqMessage.messageID);
    printf("\nsequenceNumber : 0x%x", reqMessage.sequenceNumber);
    printf("\nmessageType : 0x%x", reqMessage.messageType);
printf("\nreserved : 0x%x", reqMessage.reserved);

    }

I could see the values that received on the buff are to be correct, but when i tried to map the whole buffer to struct, I could see missing data. I am sure the solution may be simple, but I don't know where is the problem.

The output looks like,

Actual value on buffer
 buff[0] = 0x50
 buff[1] = 0x0
 buff[2] = 0x1e
 buff[3] = 0x0
 buff[4] = 0x0
 buff[5] = 0x0
 buff[6] = 0x31
 buff[7] = 0x0
 buff[8] = 0x1
 buff[9] = 0x0
 buff[10] = 0x2
 buff[11] = 0x0
 buff[12] = 0x1
 buff[13] = 0x0
 buff[14] = 0x0
 buff[15] = 0x0

RESULT of reqMessage = *(stRequestMsg *)buff;
startTag : 0x50
messageSize : 0x310000
messageID : 0x1
sequenceNumber : 0x2
messageType : 0x1
reserved : 0x8a5c6da

RESULT of std::copy(&hdr, &hdr + 1, reinterpret_cast<stRequestMsg*>(buff));
startTag : 0x50
messageSize : 0x310000
messageID : 0x1
sequenceNumber : 0x2
messageType : 0x1
reserved : 0x8a5c6da

RESULT of memcpy(&reqMessage, buff, sizeof(stRequestMsg));
startTag : 0xf7ec
messageSize : 0x1e0050
messageID : 0x0
sequenceNumber : 0x31
messageType : 0x1
reserved : 0x1

RESULT of reqMessage = *reinterpret_cast<stRequestMsg*>(buff);
startTag : 0xf7ec
messageSize : 0x1e0050
messageID : 0x0
sequenceNumber : 0x31
messageType : 0x1
reserved : 0x1

but what I expected is

startTag = 0x5000;
messageSize = 0x1E000000;
messageID = 0x3100;
sequenceNumber = 0x100;
messageType = 0x200;
reserved = 0x1000000;

EDIT: I tried changing the variable size of messageSize and reserved to uint16_t and by doing memcpy I received all correct.

What is `sizeof( struct stRequestMsg )` on both sides of the connection? What is the endianness of both machines? — Andrew Henle, Feb 02 '16 at 18:04
The whole thing is UB: http://stackoverflow.com/a/11283924/4082723 — 2501, Feb 02 '16 at 18:15
@Andrew Henle: Both the machine are same. I am using Windows 7 OS on 64-bit machine. I have two console application one as client and other as server. — kar, Feb 02 '16 at 18:16
@kar Well, your expectations seem to be wrong for a little endian machine. — πάντα ῥεῖ, Feb 02 '16 at 18:25

Daniel Jour · Accepted Answer · 2016-02-02T19:01:24.777

50 00 1E 00 00 00 31 00 01 00 02 00 01 00 00 00

If we believe your first output, then this is what is in your buffer.

struct stRequestMsg {
    uint16_t startTag;
    uint32_t messageSize;
    uint16_t messageID;
    uint16_t sequenceNumber;
    uint16_t messageType;
    uint32_t reserved;
};

You seem to be assuming that this structure definition maps like so onto memory:

50 00|1E 00 00 00|31 00|01 00|02 00|01 00 00 00
start|messagesize|msgID|seqNm|msgTp|reserved...

Which is not the case, in your particular situation it's

50 00|1E 00|00 00 31 00|01 00|02 00|01 00|00 00|GG GG GG GG
start|xx xx|messagesize|msgID|seqNm|msgTp|xx xx|reserved...

Where GG means garbage (as it's outside of your buffer). The reason the structure is laid out like this in memory is structure padding: Fields of structures are laid out in memory according to their alignment requirements. For int (and probably uint32_t) that's often 4 Bytes, meaning such a member will only start at an offset that is a multiple of 4. To achieve this the compiler inserts padding. Your structure thus actually looks like:

struct stRequestMsg {
    uint16_t startTag;
    char xxxxPad1[2];
    uint32_t messageSize;
    uint16_t messageID;
    uint16_t sequenceNumber;
    uint16_t messageType;
    char xxxxPad2[2];
    uint32_t reserved;
};

The other issue you're running into is byte order: Your system seems to use little endian, while the data from the network is coming as big endian (which is also the network byte order, so this is a good thing).

That's why startTag == 0x0050 instead of 0x5000. Being little endian, your system assumes the first byte to hold the lowest bits, not the highest.

To get this right you should look up some resources on serialization and deserialization ...

Deserialization example (in C, but should be easily adjustable to C++. Sorry, wasn't paying attention to the tags when writing, I saw a printf and just assumed C :D):

#include <stdio.h>
#include <stdint.h>

struct stRequestMsg {
    uint16_t startTag;
    uint32_t messageSize;
    uint16_t messageID;
    uint16_t sequenceNumber;
    uint16_t messageType;
    uint32_t reserved;
};

void printIt(struct stRequestMsg m) {
    printf("{\n"
           "  .startTag =       %#x;\n"
           "  .messageSize =    %#x;\n"
           "  .messageID =      %#x;\n"
           "  .sequenceNumber = %#x;\n"
           "  .messageType =    %#x;\n"
           "  .reserved =       %#x;\n"
           "}\n",
           m.startTag, m.messageSize, m.messageID,
           m.sequenceNumber, m.messageType, m.reserved);
}

uint16_t deserialize_uint16(char const * const b) {
    return ((uint16_t) b[0] << 8u) |
           ((uint16_t) b[1]);
}
uint32_t deserialize_uint32(char const * const b) {
    return ((uint16_t) b[0] << 24u) |
           ((uint16_t) b[1] << 16u) |
           ((uint16_t) b[2] << 8u) |
           ((uint16_t) b[3]);
}

struct stRequestMsg deserialize(char const * const b) {
    struct stRequestMsg r;
    r.startTag = deserialize_uint16(b);
    r.messageSize = deserialize_uint32(b + 2);
    r.messageID = deserialize_uint16(b + 6);
    r.sequenceNumber = deserialize_uint16(b + 8);
    r.messageType = deserialize_uint16(b + 10);
    r.reserved = deserialize_uint32(b + 12);
    return r;
}

int main(void) {
  char buff[16];
  buff[0] = 0x50; buff[1] = 0x00;
  buff[2] = 0x1E; buff[3] = 0x00;
  buff[4] = 0x00; buff[5] = 0x00;
  buff[6] = 0x31; buff[7] = 0x00;
  buff[8] = 0x10; buff[9] = 0x00;
  buff[10] = 0x20; buff[11] = 0x00;
  buff[12] = 0x10; buff[13] = 0x00;
  buff[14] = 0x00; buff[15] = 0x00;

  struct stRequestMsg msg = deserialize(&(buff[0]));
  printIt(msg);
  return 0;
}

(Live on ideone)

Thanks for your info. In the above case, say if I create a deserialization logic void deserialize(char *dataBuff) { memcpy(this, dataBuff, sizeof(stRequestMsg)); memset(dataBuff, '\0', sizeof(stRequestMsg)); } Will this resolve. OR should I need to change the structure elements so that it includes padding as well. — kar, Feb 02 '16 at 18:31
No, `memcpy` **won't** work. You need to deserialize it. Normally the networking library has some functions to help you with that, i.e. to deserialize primitve types. But you could also write them yourself, see my edit for a full example. — Daniel Jour, Feb 02 '16 at 18:57

Missing data while copying buffer to struct

1 Answers1

Linked