de/serializing yaml 或 json 使用 serde 时如何解构枚举?

How to destructure enum when de/serializing yaml or json with serde?

我有一段 serde 代码可以执行我想要的操作,但我不喜欢它的执行方式。我正在寻求有关如何改进它的帮助。

Playground

use std::any::Any;

trait Model: std::fmt::Debug + Any {
    fn as_any(&self) -> &dyn Any;
}

impl Model for Generator {
    fn as_any(&self) -> &dyn Any {
        self
    }
}
impl Model for Connector {
    fn as_any(&self) -> &dyn Any {
        self
    }
}

#[derive(Debug, Clone, serde::Serialize, serde::Deserialize)]
pub struct Generator {
    id: String,
    #[serde(rename = "sourceID")]
    source_id: String,
}

#[derive(Debug, Clone, serde::Serialize, serde::Deserialize)]
pub struct Connector {
    id: String,
    #[serde(rename = "sourceID")]
    source_id: String,
}

#[derive(Debug, Clone, serde::Serialize, serde::Deserialize)]
#[serde(tag = "type")]
enum AllModels {
    Generator(Generator),
    Connector(Connector),
}

fn main() {
    let data = r#"
- sourceID: "generator-01"
  id: "connector-01"
  type: "Generator"
- sourceID: "geneiator-01"
  type: "Connector"
  id: "connector-01"
"#;
    let p: Vec<Box<dyn Model>> = serde_yaml::from_str::<Vec<AllModels>>(&data)
        .unwrap()
        .into_iter()
        .collect();
    println!("{:?}", p);
    let l = serde_yaml::to_string(&p.into_iter().collect::<Vec<AllModels>>());
    println!("{:?}", l);
}

impl std::iter::FromIterator<AllModels> for Vec<Box<dyn Model>> {
    fn from_iter<I: IntoIterator<Item = AllModels>>(iter: I) -> Self {
        let mut v: Vec<Box<dyn Model>> = Vec::new();
        for i in iter {
            match i {
                AllModels::Generator(d) => {
                    v.push(Box::new(d));
                }
                AllModels::Connector(d) => {
                    v.push(Box::new(d));
                }
            }
        }
        v
    }
}

impl std::iter::FromIterator<std::boxed::Box<dyn Model>> for std::vec::Vec<AllModels> {
    fn from_iter<I: IntoIterator<Item = Box<dyn Model>>>(iter: I) -> Self {
        let mut v: Vec<AllModels> = Vec::new();
        for i in iter {
            if let Some(model) = i.as_any().downcast_ref::<Generator>() {
                v.push(AllModels::Generator(model.clone()));
            } else if let Some(model) = i.as_any().downcast_ref::<Connector>() {
                v.push(AllModels::Connector(model.clone()));
            }
        }
        v
    }
}

我想要实现的是 de/serialize yaml 到多个结构之一,根据它解析的 yaml 中 type 字段的值动态选择它应该反序列化到哪个结构。例如

- id: Foo
  source: Bar
  type: Connector

应该解析成struct Connector

我想我可以使用枚举表示来处理这个问题,但是,它会产生不希望的副作用 - 默认情况下遵循 yaml:

- id: Foo
  source: Bar
  type: Connector
- id: Foo
  source: Bar
  type: Generator

将被解析为:

[Connector(Connector{...}), Generator(Generator{...})]

所以我的结构被包裹在枚举变体中。为了“打开它”,我想我可以实现 FromIterator<AllModels> for Vec<Box<dyn Model>> ,多亏了它并键入 conversion/coercion(不确定哪个是正确的词),输出更改为:

[Connector{...}, Generator{...}]

到目前为止还不错。

我在使用此解决方案时遇到的两个问题是:

  1. 代码重复 - 对于每个新结构(连接器,生成器,......)我必须更新 enum AllModelsmatch arm inside FromIterator 实现 - 后者是困扰我最我可能可以用宏来完成,但我还没有学会如何编写它们,在我这样做之前,我想探索其他可能的解决方案
  2. 额外迭代 - 为了从 Vec<enum variant> 转换为 Vec<struct>,我需要执行以下操作:let p: Vec<Box<dyn Model>> = serde_yaml::from_str::<Vec<AllModels>>(&data).unwrap().into_iter().collect();。我更愿意在没有额外迭代的情况下进行转换

我已经考虑了一些选项,但我不知道如何实施它们...

一个。 serde 容器属性 from/into docs

#[serde(from = "FromType")] - 我认为它的工作方式是将我的枚举变量直接强制转换为所需的结构,没有额外的迭代,也没有代码重复。但是,我未能实现它 - Playground.

当我尝试添加 from 属性时

#[derive(Debug, Clone, Serialize, Deserialize)]
#[serde(tag = "type")]
enum AllModels {
  Gene(Generator),
  Connector(Connector),
}

编译器会骂我

 ❯ cargo r
   Compiling serdeissue v0.1.0 (/sandbox/serdeissue)
error[E0277]: the trait bound `Box<dyn Model>: From<AllModels>` is not satisfied
  --> src/main.rs:21:24
   |
21 | #[derive(Debug, Clone, Serialize, Deserialize)]
   |                        ^^^^^^^^^ the trait `From<AllModels>` is not implemented for `Box<dyn Model>`
   |
   = help: the following implementations were found:
             <Box<(dyn StdError + 'a)> as From<E>>
             <Box<(dyn StdError + 'static)> as From<&str>>
             <Box<(dyn StdError + 'static)> as From<Cow<'a, str>>>
             <Box<(dyn StdError + 'static)> as From<std::string::String>>
           and 22 others
   = note: required because of the requirements on the impl of `Into<Box<dyn Model>>` for `AllModels`
   = note: required by `into`
   = note: this error originates in a derive macro (in Nightly builds, run with -Z macro-backtrace for more info)

error[E0277]: the trait bound `dyn Model: Serialize` is not satisfied
   --> src/main.rs:21:24
    |
21  | #[derive(Debug, Clone, Serialize, Deserialize)]
    |                        ^^^^^^^^^ the trait `Serialize` is not implemented for `dyn Model`
    |
   ::: /home/marcin/.cargo/registry/src/github.com-1ecc6299db9ec823/serde-1.0.123/src/ser/mod.rs:247:18
    |
247 |     fn serialize<S>(&self, serializer: S) -> Result<S::Ok, S::Error>
    |                  - required by this bound in `serialize`
    |
    = note: required because of the requirements on the impl of `Serialize` for `Box<dyn Model>`
    = note: this error originates in a derive macro (in Nightly builds, run with -Z macro-backtrace for more info)

error[E0277]: the trait bound `dyn Model: Deserialize<'_>` is not satisfied
   --> src/main.rs:21:35
    |
21  | #[derive(Debug, Clone, Serialize, Deserialize)]
    |                                   ^^^^^^^^^^^ the trait `Deserialize<'_>` is not implemented for `dyn Model`
    |
   ::: /home/marcin/.cargo/registry/src/github.com-1ecc6299db9ec823/serde-1.0.123/src/de/mod.rs:539:12
    |
539 |         D: Deserializer<'de>;
    |            ----------------- required by this bound in `_::_serde::Deserialize::deserialize`
    |
    = note: required because of the requirements on the impl of `Deserialize<'_>` for `Box<dyn Model>`
    = note: this error originates in a derive macro (in Nightly builds, run with -Z macro-backtrace for more info)

error[E0277]: the trait bound `AllModels: From<Box<dyn Model>>` is not satisfied
   --> src/main.rs:21:35
    |
21  | #[derive(Debug, Clone, Serialize, Deserialize)]
    |                                   ^^^^^^^^^^^ the trait `From<Box<dyn Model>>` is not implemented for `AllModels`
    |
   ::: /home/marcin/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/library/core/src/convert/mod.rs:372:1
    |
372 | pub trait From<T>: Sized {
    | ------------------------ required by this bound in `From`
    |
    = note: this error originates in a derive macro (in Nightly builds, run with -Z macro-backtrace for more info)

我的攻击角度如下:使用error msg to copy-pasteroni-dummy-implementoni missing trait bounds:

impl From<AllModels> for Box<dyn Model> {
    fn from(am: AllModels) -> Self {
        Box::new(Generator{id:String::from("arst"),source_id:String::from("arst")})
    }
}
impl Serialize for dyn Model {
    fn serialize(&self) -> Self {
        Box::new(Generator{id:String::from("arst"),source_id:String::from("arst")})
    }
}

impl Deserialize<'_> for dyn Model {}
impl From<Box<dyn Model>> for AllModels {
    fn from(dm: Box<dyn Model>) -> Self {
        AllModels::Gene(Generator{id:String::from("arst"),source_id:String::from("arst")})
    }
}

但随后发生了这种情况:

 ❯ cargo r
   Compiling serdeissue v0.1.0 (/sandbox/serdeissue)
error[E0277]: the size for values of type `(dyn Model + 'static)` cannot be known at compilation time
   --> src/main.rs:75:6
    |
75  | impl Deserialize<'_> for dyn Model {}
    |      ^^^^^^^^^^^^^^^ doesn't have a size known at compile-time
    |
   ::: /home/marcin/.cargo/registry/src/github.com-1ecc6299db9ec823/serde-1.0.123/src/de/mod.rs:530:29
    |
530 | pub trait Deserialize<'de>: Sized {
    |                             ----- required by this bound in `Deserialize`
    |
    = help: the trait `Sized` is not implemented for `(dyn Model + 'static)`

对我来说游戏结束了

乙。 erased-serde

这似乎是完成这项工作的正确工具,但同样,我 运行 在实施它时遇到了问题(难怪 - 我不知道自己在做什么:)

use erased_serde::{Deserializer,  Serializer, serialize_trait_object};
use serde::{Serialize, Deserialize};


#[derive(Debug, Clone, Serialize, Deserialize)]
pub struct Generator {
  id: String,
  source: String,
}

#[derive(Debug, Clone, Serialize, Deserialize)]
pub struct Connector {
  id: String,
  source: String,
}

trait Model: std::fmt::Debug + erased_serde::Serialize {}
erased_serde::serialize_trait_object!(Model);
impl Model for Generator {}
impl Model for Connector {}

#[derive(Debug, Clone, Serialize, Deserialize)]
#[serde(tag = "type", into = "Box<dyn Model>", from = "Box<dyn Model>")]
enum AllModels {
  Generator(Generator),
  Connector(Connector),
}

fn main() {
  let data = r#"
- source: "generator-01"
  id: "g-01"
  type: "Generator"
- source: "connector-01"
  type: "Connector"
  id: "c-01"
"#;
  let p: Vec<Box<dyn Model>> = serde_yaml::from_str(&data).unwrap();
  println!("{:?}", p);
}


impl From<AllModels> for Box<dyn Model> {
    fn from(am: AllModels) -> Self {
        Box::new(Generator{id:String::from("arst"),source_id:String::from("arst")})
    }
}
// impl Serialize for dyn Model {
//     fn serialize(&self) -> Self {
//         Box::new(Generator{id:String::from("arst"),source_id:String::from("arst")})
//     }
// }
impl Deserialize<'_> for dyn Model {}
impl From<Box<dyn Model>> for AllModels {
    fn from(dm: Box<dyn Model>) -> Self {
        AllModels::Generator(Generator{id:String::from("arst"),source_id:String::from("arst")})
    }
}

// impl std::convert::From<AllModels> for Box<dyn Model> {
//     fn from(m: AllModels) -> Self {
//        Box::new(Generator {source_id: String::from("i"), id: String::from("r")})
//     }
//     }

我在编译时遇到这个错误:

 ❯ cargo r
   Compiling serdeissue v0.1.0 (/sandbox/serdeissue)
warning: unused imports: `Deserializer`, `Serializer`, `serialize_trait_object`
 --> src/main.rs:1:20
  |
1 | use erased_serde::{Deserializer,  Serializer, serialize_trait_object};
  |                    ^^^^^^^^^^^^   ^^^^^^^^^^  ^^^^^^^^^^^^^^^^^^^^^^
  |
  = note: `#[warn(unused_imports)]` on by default

error[E0277]: the size for values of type `(dyn Model + 'static)` cannot be known at compilation time
   --> src/main.rs:76:6
    |
76  | impl Deserialize<'_> for dyn Model {}
    |      ^^^^^^^^^^^^^^^ doesn't have a size known at compile-time
    |
   ::: /home/marcin/.cargo/registry/src/github.com-1ecc6299db9ec823/serde-1.0.123/src/de/mod.rs:530:29
    |
530 | pub trait Deserialize<'de>: Sized {
    |                             ----- required by this bound in `Deserialize`
    |
    = help: the trait `Sized` is not implemented for `(dyn Model + 'static)`

我认为 erased_serde 可以提供帮助,也许可以,但我不知道如何实施它。


遗憾的是我不能使用 typetag crate,因为它不支持我需要的 wasm 编译目标。我不考虑对每个枚举变体使用 #[serde(serialize_with = "path")],因为它使我的问题 #1 比现在更糟。

我也知道这个问题 however the code provided by @dtolnay 无法编译,我不知道如何修复它

❯ cargo r
   Compiling serdeissue v0.1.0 (/sandbox/serdeissue)
error[E0603]: module `export` is private
   --> src/main.rs:168:10
    |
168 | #[derive(Serialize)]
    |          ^^^^^^^^^ private module
    |
note: the module `export` is defined here
   --> /home/marcin/.cargo/registry/src/github.com-1ecc6299db9ec823/serde-1.0.123/src/lib.rs:275:5
    |
275 | use self::__private as export;
    |     ^^^^^^^^^^^^^^^^^^^^^^^^^

error[E0603]: module `export` is private
   --> src/main.rs:173:10
    |
173 | #[derive(Serialize)]
    |          ^^^^^^^^^ private module
    |
note: the module `export` is defined here
   --> /home/marcin/.cargo/registry/src/github.com-1ecc6299db9ec823/serde-1.0.123/src/lib.rs:275:5
    |
275 | use self::__private as export;
    |     ^^^^^^^^^^^^^^^^^^^^^^^^^

error[E0603]: module `export` is private
   --> src/main.rs:179:10
    |
179 | #[derive(Serialize)]
    |          ^^^^^^^^^ private module
    |
note: the module `export` is defined here
   --> /home/marcin/.cargo/registry/src/github.com-1ecc6299db9ec823/serde-1.0.123/src/lib.rs:275:5
    |
275 | use self::__private as export;
    |     ^^^^^^^^^^^^^^^^^^^^^^^^^

error[E0603]: module `export` is private
   --> src/main.rs:184:10
    |
184 | #[derive(Serialize)]
    |          ^^^^^^^^^ private module
    |
note: the module `export` is defined here
   --> /home/marcin/.cargo/registry/src/github.com-1ecc6299db9ec823/serde-1.0.123/src/lib.rs:275:5
    |
275 | use self::__private as export;
    |     ^^^^^^^^^^^^^^^^^^^^^^^^^

warning: trait objects without an explicit `dyn` are deprecated
   --> src/main.rs:176:22
    |
176 |     widgets: Vec<Box<WidgetTrait>>,
    |                      ^^^^^^^^^^^ help: use `dyn`: `dyn WidgetTrait`
    |
    = note: `#[warn(bare_trait_objects)]` on by default

看来我正在寻找的功能正在等待在这里实现:https://github.com/serde-rs/serde/issues/1402

还有这个问题https://github.com/serde-rs/serde/issues/1350 which suggests manual Deserializer implementation Playground。游乐场代码表明这可以帮助解决我的问题 #2“额外迭代”,但是代码实现中仍然有一些重复,因此我仍在寻找更好的答案。


编辑:我也在考虑 Enum or trait object,无法确定评估我是否需要其中之一的正确方法。

我找到了一个令我满意的解决方案:

  • 圆滑
  • 无代码重复
  • 没有动态调度,没有额外的迭代
  • 使用 enumserde(tag="...")
    • type 字段在 json/yaml 中可能是乱序的(不必是第一个)
    • 序列化回 json/yaml 时包含
    • type 字段

诀窍是使用 enum_dispatch

use serde::{Serialize, Deserialize};
use enum_dispatch::enum_dispatch;


#[derive(Debug, Clone, Serialize, Deserialize)]
struct Generator {
    source_id: String,
    id: String,
}

#[derive(Debug, Clone, Serialize, Deserialize)]
struct Gate {
    source_id: String,
    id: String,
}

#[enum_dispatch]
trait Model {fn time_advance(self, a:i32,b:i32) -> i32;} 
impl Model for Generator { fn time_advance(self,a:i32,b:i32) -> i32 {a+b} }
impl Model for Gate { fn time_advance(self,a:i32,b:i32) -> i32 {a+b} }

#[enum_dispatch(Model)]
#[derive(Debug, Clone, Serialize, Deserialize)]
#[serde(tag = "type")]
enum ModelTypes {
    Generator(Generator),
    Gate(Gate),
}

fn main() {
    let s = r#"
- source_id: "generator-01"
  id: "connector-01"
  type: "Generator"
- source_id: "geneiator-01"
  type: "Gate"
  id: "connector-01"
"#;
    let data: Vec<ModelTypes> = serde_yaml::from_str(s).unwrap();
    println!("{:?}", serde_yaml::to_string(&data));
    for d in data {
        println!("{:?}", d.time_advance(4, 2));
    }
}
[package]
name = "enum_unpack"
version = "0.1.0"
authors = ["---"]
edition = "2018"

[dependencies]
serde = { version = "1.0.124", features = ["derive"] }
serde_yaml = "0.8.1"
enum_dispatch = "0.3.5"

输出:

> cargo run
   Compiling enum_unpack v0.1.0 (.../enum_unpack)
    Finished dev [unoptimized + debuginfo] target(s) in 0.66s
     Running `target/debug/enum_unpack`
Ok("---\n- type: Generator\n  source_id: generator-01\n  id: connector-01\n- type: Gate\n  source_id: geneiator-01\n  id: connector-01\n")
6
6