Save/load 结构的工作或失败取决于成员向量长度

Question

我使用 serde 和 bincode 通过自定义 load/save 方法定义了以下结构：

use std::{
    fs::File,
    io::{Read, Seek, SeekFrom, Write},
};

use serde::{Deserialize, Serialize};
#[derive(Clone, Debug, Serialize, Deserialize)]
pub struct Histogram {
    pub bars: Vec<f64>,
}

#[derive(Clone, Debug, Serialize, Deserialize)]
pub struct Histograms {
    pub num_bars: usize,
    pub num_histograms: usize,
    pub vec: Vec<Histogram>,
}

fn main() {
    println!("Hello, world!");
}

impl Histogram {
    pub fn new(num_bars: usize) -> Histogram {
        Histogram {
            bars: vec![0.0; num_bars],
        }
    }
}

impl Histograms {
    pub fn new(num_histograms: usize, num_bars: usize) -> Histograms {
        let histograms = Histograms {
            vec: vec![Histogram::new(num_bars); num_histograms],
            num_histograms,
            num_bars,
        };
        histograms
    }

    pub fn save(&self, filename: &str) {
        let mut file = File::create(format!("{}{}", "path", filename)).unwrap();
        let bin = bincode::serialize(&self).unwrap();
        Write::write_all(&mut file, &bin).unwrap();
    }

    pub fn load(filename: &str) -> Histograms {
        let mut file = File::open(format!("{}{}", "path", filename)).unwrap();
        let mut vec: Vec<u8> = vec![0; file.seek(SeekFrom::End(0)).unwrap() as usize];
        file.seek(SeekFrom::Start(0)).unwrap();
        file.read(&mut vec).unwrap();
        let result: Histograms = bincode::deserialize(&vec).unwrap();
        result
    }
}

现在奇怪的是，下面的测试告诉我，如果 vec（直方图的成员）的长度很小，save/load 可以正常工作，但它失败了（我没有得到任何错误，只是生成的直方图实例是错误的）具有较大的值，例如 10000000。准确地说，我从它不再正确的地方得到一个值 5263431。

mod tests {
    use core::panic;

    use super::*;
    #[test]
    fn save_load_test() {
        let histogramms = Histograms::new(10000000, 50);
        histogramms.save("debug_test");
        let histogramms1 = Histograms::load("debug_test");
        assert_eq!(histogramms.vec.len(), histogramms1.vec.len());
        let mut failed = false;
        for i in 0..histogramms.vec.len() {
            if histogramms.vec[i].bars.len() != histogramms1.vec[i].bars.len() {
                failed = true;
                println!("first i that failed: {}", i);
                println!(
                    "histogramms.vec.bars.len: {} histogramms1.vec.bars.len: {}",
                    histogramms.vec[i].bars.len(),
                    histogramms1.vec[i].bars.len()
                );
                break;
            }
        }
        if failed {
            panic!()
        }
    }
}

知道出了什么问题吗？

Answer 1

Read::read() 函数从源中提取一些字节，但不能保证读取所有字节。您得到的唯一保证是，如果还有剩余，您将获得一些字节（至少如果您的缓冲区长度不为零）。

解决此问题的最简单方法是改用 std::fs::read() 函数：

pub fn load(filename: &str) -> Histograms {
    let vec = std::fs::read(format!("{}{}", "path", filename)).unwrap();
    bincode::deserialize(&vec).unwrap()
}

更好的解决方法是直接从文件反序列化，而不是先将整个文件读入内存：

pub fn save(&self, filename: &str) {
    let mut w = BufWriter::new(File::create(format!("{}{}", "path", filename)).unwrap());
    bincode::serialize_into(&mut w, &self).unwrap();
    w.flush().unwrap();
}

pub fn load(filename: &str) -> Histograms {
    let file = File::open(format!("{}{}", "path", filename)).unwrap();
    bincode::deserialize_from(BufReader::new(file)).unwrap()
}

（当然，您应该为实际代码添加适当的错误处理。）

Save/load 结构的工作或失败取决于成员向量长度

Save/load of struct works or fails depending on member vector length

rust

serde

bincode