如何使用 serde 和 bincode 映射填充超过 32 字节的 C 结构？

Question

我正在使用 serde 和 bincode 映射二进制结构。

#[macro_use]
extern crate serde_derive;
extern crate serde;
extern crate bincode;

#[derive(Serialize, Deserialize)]
struct Superblock {
    magic: [u8; 16],
    //reserved: [u8; 492],
    crc: u32,
}

事情按预期进行，但我无法映射保留字段。显然，固定大小的数组仅定义为最大 32 字节的大小。

如何注册自定义大小的数组以便对填充进行反序列化？

serde+bincode 是正确的方法吗？我需要控制字节顺序（由 bincode 提供）并且我喜欢声明式风格。

Answer 1

serde_derive 支持 field attributes #[serde(serialize_with="func")]、#[serde(deserialize_with="func")] 和 #[serde(with="module")]，这允许提供自定义 serialization/deserialization 例程：

#[derive(Serialize, Deserialize)]
struct Superblock {
    magic: [u8; 16],
    #[serde(with="array_492")]   // <--
    reserved: [u8; 492],
    crc: u32,
}

mod array_492 {
    use serde::*;

    pub fn serialize<S, T>(array: &[T; 492], ser: S) -> Result<S::Ok, S::Error> 
        where S: Serializer, T: Serialize
    {
        unimplemented!() // fill in yourself.
    }

    pub fn deserialize<'de, D, T>(de: D) -> Result<[T; 492], D::Error> { 
        where D: Deserializer<'de>, T: Deserialize<'de>
    {
        unimplemented!() // fill in yourself.
    }
}

有关实际实施，请参阅此要点：https://gist.github.com/kennytm/21403667b5a17172cfcd11f9df9365e2。请注意，这是 not 针对一次反序列化整个字节数组进行了优化。

Answer 2

作为解决方法，您可以构造一个大小合适的不同对象，然后忽略它。例如：

reserved: ([u64; 32], [u64; 29], u32) // 492 bytes

Answer 3

我建议 implementing Serialize and implementing Deserialize 使用自定义类型。

需要注意的重要一点是您根本不关心数据。这意味着没有理由占用内存！我们可以定义一个类型 Reserved ，它将序列化为一堆字节并从字节反序列化，但实际上并不需要我们的内存结构中的任何 space。

那么，就是填写trait的实现了：

#[macro_use]
extern crate serde_derive;
extern crate serde;
extern crate bincode;

use std::fmt;
use serde::ser::SerializeTuple;

#[derive(Debug, Serialize, Deserialize, PartialEq)]
struct Superblock {
    magic: [u8; 16],
    reserved: Reserved,
    crc: u32,
}

#[derive(Debug, PartialEq)]
struct Reserved;
const RESERVED_LENGTH: usize = 492;

impl serde::Serialize for Reserved {
    fn serialize<S>(&self, serializer: S) -> Result<S::Ok, S::Error>
        where S: serde::Serializer
    {
        let mut tuple = serializer.serialize_tuple(RESERVED_LENGTH)?;
        for _ in 0..RESERVED_LENGTH {
            tuple.serialize_element(&0xA0_u8)?; // Just to see it easily in the output
        }
        tuple.end()
    }
}

impl<'de> serde::Deserialize<'de> for Reserved {
    fn deserialize<D>(deserializer: D) -> Result<Self, D::Error>
        where D: serde::Deserializer<'de>
    {
        struct Visitor;
        impl<'de> serde::de::Visitor<'de> for Visitor {
            type Value = Reserved;

            fn expecting(&self, f: &mut fmt::Formatter) -> fmt::Result {
                write!(f, "{} bytes", RESERVED_LENGTH)
            }

            fn visit_seq<A>(self, mut tuple: A) -> Result<Self::Value, A::Error>
                where A: serde::de::SeqAccess<'de>,
            {
                for _ in 0..RESERVED_LENGTH {
                    tuple.next_element::<u8>()?;
                }
                Ok(Reserved)
            }
        }

        deserializer.deserialize_tuple(RESERVED_LENGTH, Visitor)
    }
}

fn main() {
    let block = Superblock {
        magic: [
            0x00, 0x01, 0x02, 0x03,
            0x04, 0x05, 0x06, 0x07,
            0x08, 0x09, 0x0a, 0x0b,
            0x0c, 0x0d, 0x0e, 0x0f,
        ],
        reserved: Reserved,
        crc: 0xffffffff,
    };

    let ser = bincode::serialize(&block, bincode::Infinite).expect("Couldn't serialize");
    println!("length: {}", ser.len());
    println!("{:?}", ser);

    let block2: Superblock = bincode::deserialize(&ser).expect("Couldn't deserialize");
    assert_eq!(block, block2);
    println!("{:?}", block2);

    println!("Takes: {} bytes", std::mem::size_of::<Superblock>());
    // prints "Takes: 20 bytes"
}

如何使用 serde 和 bincode 映射填充超过 32 字节的 C 结构？

How do I map a C struct with padding over 32 bytes using serde and bincode?

rust

serde