Can I take a byte array and deserialize it into a struct?

Question

I'm reading a series of bytes from a socket and I need to put each segment of n bytes as a item in a struct.

use std::mem;

#[derive(Debug)]
struct Things {
    x: u8,
    y: u16,
}

fn main() {
    let array = [22 as u8, 76 as u8, 34 as u8];
    let foobar: Things;
    unsafe {
        foobar = mem::transmute::<[u8; 3], Things>(array);
    }

    println!("{:?}", foobar);

}

I'm getting errors that say that foobar is 32 bits when array is 24 bits. Shouldn't foobar be 24 bits (8 + 16 = 24)?

Answer 1

The issue here is that the y field is 16-bit-aligned. So your memory layout is actually

x
padding
y
y

Note that swapping the order of x and y doesn't help, because Rust's memory layout for structs is actually undefined (and thus still 32 bits for no reason but simplicity in the compiler). If you depend on it you will get undefined behavior.

The reasons for alignment are explained in Purpose of memory alignment.

You can prevent alignment from happening by adding the attribute repr(packed) to your struct, but you'll lose performance and the ability to take references of fields:

#[repr(packed)]
struct Things {
    x: u8,
    y: u16,
}

---

The best way would be to not use transmute at all, but to extract the values manually and hope the optimizer makes it fast:

let foobar = Things {
    x: array[0],
    y: ((array[1] as u16) << 8) | (array[2] as u16),
};

A crate like byteorder may simplify the process of reading different sizes and endianness from the bytes.

Answer 2

bincode and serde can do this quit simply.

use bincode::{deserialize};
use serde::{Deserialize};

#[derive(Deserialize, Debug)]
struct Things {
    x: u8,
    y: u16,
}

fn main() {
    let array = [22 as u8, 76 as u8, 34 as u8];
    let foobar: Things = deserialize(&array).unwrap();
    println!("{:?}", foobar);
}

This also works well for serializing a struct into bytes as well.

use bincode::{serialize};
use serde::{Serialize};

#[derive(Serialize, Debug)]
struct Things {
    x: u8,
    y: u16,
}

fn main() {
    let things = Things{
        x: 22,
        y: 8780,
    };
    let baz = serialize(&things).unwrap();
    println!("{:?}", baz);

}

Answer 3

I was having issues using the byteorder crate when dealing with structs that also had char arrays. I couldn't get past the compiler errors. I ended up casting like this:

#[repr(packed)]
struct Things {
    x: u8,
    y: u16,    
}

fn main() {
    let data: [u8; 3] = [0x22, 0x76, 0x34];
    
    unsafe {
        let things_p: *const Things = data.as_ptr() as *const Things;
        let things: &Things = &*things_p;
        
        println!("{:x} {:x}", things.x, things.y);
    }
}

Note that with using packed, you get this warning:

   = warning: this was previously accepted by the compiler but is being phased out; it will become a hard error in a future release!

If you can, change Things to behave like a C struct:

#[repr(C)]
struct Things2 {
    x: u8,
    y: u16,    
}

Then initialize data like this. Note the extra byte for alignment purposes.

let data: [u8; 4] = [0x22, 0, 0x76, 0x34];

Answer 4

use std::mem;

fn main() {
    let bytes = vec!(0u8, 1u8,2u8, 3, 4, 5, 6, 7, 8, 9, 0xffu8, );


    let data_ptr: *const u64 = unsafe { mem::transmute(bytes[0..4].as_ptr()) };

    let data: u64 = unsafe { *data_ptr };

    println!("{:#x}", data);
}