在 F# 中反序列化具有可区分联合的数据的另一个失败
Another failure at deserializing data with discriminated unions, in F#
在回答提供序列化/反序列化受歧视联合的有效解决方案的问题之后 ()
我现在有一个失败的实际案例(尽管它适用于更简单的案例)。
这里是测试代码:
open System.Collections.Generic
open Microsoft.FSharpLu.Json
open Newtonsoft.Json
open Newtonsoft.Json.Serialization
// set up the serialization / deserialization based on answer from:
//
let settings =
JsonSerializerSettings(
NullValueHandling = NullValueHandling.Ignore,
Converters = [| CompactUnionJsonConverter(true, true) |]
)
let serialize object =
JsonConvert.SerializeObject(object, settings)
let deserialize<'a> object =
JsonConvert.DeserializeObject<'a>(object, settings)
// define the type used
type BookSide =
| Bid
| Ask
type BookEntry =
{
S : float
P : float
}
type BookSideData =
Dictionary<int, BookEntry>
type BookData =
{
Data: Dictionary<BookSide, BookSideData>
}
static member empty =
{
Data = Dictionary<BookSide, BookSideData>(dict [ (BookSide.Bid, BookSideData()); (BookSide.Ask, BookSideData()) ])
}
// make some sample data
let bookEntry = { S=3.; P=5. }
let bookData = BookData.empty
bookData.Data.[BookSide.Bid].Add(1, bookEntry)
// serialize. This part works
let s = serialize bookData
// deserialize. This part fails
deserialize<BookData> s
序列化测试数据将如下所示:
{"Data":{"Bid":{"1":{"S":3.0,"P":5.0}},"Ask":{}}}
但是反序列化会像这样崩溃:
Could not convert string 'Bid' to dictionary key type 'FSI_0023+BookSide'. Create a TypeConverter to convert from the string to the key type object.
虽然DU的序列化/反序列化是通过FSharpLu进行的,FSharpLu有一个DU转换器。
我试图找到一些自动化解决方案,而不是编写自定义 TypeConverter 的原因(除了我从未做过的事实)是我有很多我无法控制的类型。
这是一个fiddle:
https://dotnetfiddle.net/Sx0k4x
您的基本问题是您正在使用 BookSide
作为字典键——但这是一个 f# 联合,它使它成为一个 复杂键 ——一个不是立即可转换为字符串或从字符串转换为字符串。不幸的是,Json.NET 不支持开箱即用的复杂字典键,如其 Serialization Guide:
中所述
When serializing a dictionary, the keys of the dictionary are converted to strings and used as the JSON object property names. The string written for a key can be customized by either overriding ToString()
for the key type or by implementing a TypeConverter
. A TypeConverter
will also support converting a custom string back again when deserializing a dictionary.
有两种处理此问题的基本方法:
实现 TypeConverter
,如 Not ableTo Serialize Dictionary with Complex key using Json.net.
所示
将字典序列化为 key/value 对对象的数组,例如如Serialize dictionary as array (of key value pairs).
所示
由于您的数据模型包含带有各种键(DU、字符串和整数)的字典,第二种解决方案似乎是唯一的可能性。以下DictionaryConverter
应该有必要的逻辑:
let inline isNull (x:^T when ^T : not struct) = obj.ReferenceEquals (x, null)
type Type with
member t.BaseTypesAndSelf() =
t |> Seq.unfold (fun state -> if isNull state then None else Some(state, state.BaseType))
member t.DictionaryKeyValueTypes() =
t.BaseTypesAndSelf()
|> Seq.filter (fun i -> i.IsGenericType && i.GetGenericTypeDefinition() = typedefof<Dictionary<_,_>>)
|> Seq.map (fun i -> i.GetGenericArguments())
type JsonReader with
member r.ReadAndAssert() =
if not (r.Read()) then raise (JsonReaderException("Unexpected end of JSON stream."))
r
member r.MoveToContentAndAssert() =
if r.TokenType = JsonToken.None then r.ReadAndAssert() |> ignore
while r.TokenType = JsonToken.Comment do r.ReadAndAssert() |> ignore
r
type internal DictionaryReadOnlySurrogate<'TKey, 'TValue>(i : IDictionary<'TKey, 'TValue>) =
interface IReadOnlyDictionary<'TKey, 'TValue> with
member this.ContainsKey(key) = i.ContainsKey(key)
member this.TryGetValue(key, value) = i.TryGetValue(key, &value)
member this.Item with get(index) = i.[index]
member this.Keys = i.Keys :> IEnumerable<'TKey>
member this.Values = i.Values :> IEnumerable<'TValue>
member this.Count = i.Count
member this.GetEnumerator() = i.GetEnumerator()
member this.GetEnumerator() = i.GetEnumerator() :> IEnumerator
type DictionaryConverter () =
// ReadJson adapted from this answer
// To
// By https://whosebug.com/users/3744182/dbc
inherit JsonConverter()
override this.CanConvert(t) = (t.DictionaryKeyValueTypes().Count() = 1) // If ever implemented for IReadOnlyDictionary<'TKey, 'TValue> then reject DictionaryReadOnlySurrogate<'TKey, 'TValue>
member private this.ReadJsonGeneric<'TKey, 'TValue> (reader : JsonReader, t : Type, existingValue : obj, serializer : JsonSerializer) : obj =
let contract = serializer.ContractResolver.ResolveContract(t)
let dict = if (existingValue :? IDictionary<'TKey, 'TValue>) then existingValue :?> IDictionary<'TKey, 'TValue> else contract.DefaultCreator.Invoke() :?> IDictionary<'TKey, 'TValue>
match reader.MoveToContentAndAssert().TokenType with
| JsonToken.StartArray ->
let l = serializer.Deserialize<List<KeyValuePair<'TKey, 'TValue>>>(reader)
for p in l do dict.Add(p)
dict :> obj
| JsonToken.StartObject ->
serializer.Populate(reader, dict)
dict :> obj
| JsonToken.Null -> null // Or throw an exception if you prefer
| _ -> raise (JsonSerializationException(String.Format("Unexpected token {0}", reader.TokenType)))
override this.ReadJson(reader, t, existingValue, serializer) =
let keyValueTypes = t.DictionaryKeyValueTypes().Single(); // Throws an exception if not exactly one.
let m = typeof<DictionaryConverter>.GetMethod("ReadJsonGeneric", BindingFlags.NonPublic ||| BindingFlags.Instance ||| BindingFlags.Public);
m.MakeGenericMethod(keyValueTypes).Invoke(this, [| reader; t; existingValue; serializer |])
member private this.WriteJsonGeneric<'TKey, 'TValue> (writer : JsonWriter, value : obj, serializer : JsonSerializer) =
let dict = value :?> IDictionary<'TKey, 'TValue>
let keyContract = serializer.ContractResolver.ResolveContract(typeof<'Key>)
// Wrap the value in an enumerator or read-only surrogate to prevent infinite recursion.
match keyContract with
| :? JsonPrimitiveContract -> serializer.Serialize(writer, new DictionaryReadOnlySurrogate<'TKey, 'TValue>(dict))
| _ -> serializer.Serialize(writer, seq { yield! dict })
()
override this.WriteJson(writer, value, serializer) =
let keyValueTypes = value.GetType().DictionaryKeyValueTypes().Single(); // Throws an exception if not exactly one.
let m = typeof<DictionaryConverter>.GetMethod("WriteJsonGeneric", BindingFlags.NonPublic ||| BindingFlags.Instance ||| BindingFlags.Public);
m.MakeGenericMethod(keyValueTypes).Invoke(this, [| writer; value; serializer |])
()
您将按如下方式添加到设置中:
let settings =
JsonSerializerSettings(
NullValueHandling = NullValueHandling.Ignore,
Converters = [| CompactUnionJsonConverter(true, true); DictionaryConverter() |]
)
并为您的 bookData
生成以下 JSON:
{
"Data": [
{
"Key": "Bid",
"Value": [
{
"Key": 1,
"Value": {
"S": 3.0,
"P": 5.0
}
}
]
},
{
"Key": "Ask",
"Value": []
}
]
}
备注:
转换器适用于所有 Dictionary<TKey, TValue>
类型(和子类型)。
转换器检测字典键是否将使用原始合约进行序列化,如果是,则将字典紧凑地序列化为 JSON 对象。如果不是,字典将序列化为数组。您可以在上面显示的 JSON 中观察到这一点:Dictionary<BookSide, BookSideData>
字典被序列化为 JSON 数组,而 Dictionary<int, BookEntry>
字典被序列化为 JSON 对象.
在反序列化期间,转换器检测传入的 JSON 值是数组还是对象,并根据需要进行调整。
转换器仅针对可变 .Net Dictionary<TKey, TValue>
类型实现。逻辑需要稍作修改才能反序列化不可变 Map<'Key,'Value>
类型。
演示 fiddle here.
在回答提供序列化/反序列化受歧视联合的有效解决方案的问题之后 (
我现在有一个失败的实际案例(尽管它适用于更简单的案例)。
这里是测试代码:
open System.Collections.Generic
open Microsoft.FSharpLu.Json
open Newtonsoft.Json
open Newtonsoft.Json.Serialization
// set up the serialization / deserialization based on answer from:
//
let settings =
JsonSerializerSettings(
NullValueHandling = NullValueHandling.Ignore,
Converters = [| CompactUnionJsonConverter(true, true) |]
)
let serialize object =
JsonConvert.SerializeObject(object, settings)
let deserialize<'a> object =
JsonConvert.DeserializeObject<'a>(object, settings)
// define the type used
type BookSide =
| Bid
| Ask
type BookEntry =
{
S : float
P : float
}
type BookSideData =
Dictionary<int, BookEntry>
type BookData =
{
Data: Dictionary<BookSide, BookSideData>
}
static member empty =
{
Data = Dictionary<BookSide, BookSideData>(dict [ (BookSide.Bid, BookSideData()); (BookSide.Ask, BookSideData()) ])
}
// make some sample data
let bookEntry = { S=3.; P=5. }
let bookData = BookData.empty
bookData.Data.[BookSide.Bid].Add(1, bookEntry)
// serialize. This part works
let s = serialize bookData
// deserialize. This part fails
deserialize<BookData> s
序列化测试数据将如下所示:
{"Data":{"Bid":{"1":{"S":3.0,"P":5.0}},"Ask":{}}}
但是反序列化会像这样崩溃:
Could not convert string 'Bid' to dictionary key type 'FSI_0023+BookSide'. Create a TypeConverter to convert from the string to the key type object.
虽然DU的序列化/反序列化是通过FSharpLu进行的,FSharpLu有一个DU转换器。
我试图找到一些自动化解决方案,而不是编写自定义 TypeConverter 的原因(除了我从未做过的事实)是我有很多我无法控制的类型。
这是一个fiddle: https://dotnetfiddle.net/Sx0k4x
您的基本问题是您正在使用 BookSide
作为字典键——但这是一个 f# 联合,它使它成为一个 复杂键 ——一个不是立即可转换为字符串或从字符串转换为字符串。不幸的是,Json.NET 不支持开箱即用的复杂字典键,如其 Serialization Guide:
When serializing a dictionary, the keys of the dictionary are converted to strings and used as the JSON object property names. The string written for a key can be customized by either overriding
ToString()
for the key type or by implementing aTypeConverter
. ATypeConverter
will also support converting a custom string back again when deserializing a dictionary.
有两种处理此问题的基本方法:
实现
TypeConverter
,如 Not ableTo Serialize Dictionary with Complex key using Json.net. 所示
将字典序列化为 key/value 对对象的数组,例如如Serialize dictionary as array (of key value pairs).
所示
由于您的数据模型包含带有各种键(DU、字符串和整数)的字典,第二种解决方案似乎是唯一的可能性。以下DictionaryConverter
应该有必要的逻辑:
let inline isNull (x:^T when ^T : not struct) = obj.ReferenceEquals (x, null)
type Type with
member t.BaseTypesAndSelf() =
t |> Seq.unfold (fun state -> if isNull state then None else Some(state, state.BaseType))
member t.DictionaryKeyValueTypes() =
t.BaseTypesAndSelf()
|> Seq.filter (fun i -> i.IsGenericType && i.GetGenericTypeDefinition() = typedefof<Dictionary<_,_>>)
|> Seq.map (fun i -> i.GetGenericArguments())
type JsonReader with
member r.ReadAndAssert() =
if not (r.Read()) then raise (JsonReaderException("Unexpected end of JSON stream."))
r
member r.MoveToContentAndAssert() =
if r.TokenType = JsonToken.None then r.ReadAndAssert() |> ignore
while r.TokenType = JsonToken.Comment do r.ReadAndAssert() |> ignore
r
type internal DictionaryReadOnlySurrogate<'TKey, 'TValue>(i : IDictionary<'TKey, 'TValue>) =
interface IReadOnlyDictionary<'TKey, 'TValue> with
member this.ContainsKey(key) = i.ContainsKey(key)
member this.TryGetValue(key, value) = i.TryGetValue(key, &value)
member this.Item with get(index) = i.[index]
member this.Keys = i.Keys :> IEnumerable<'TKey>
member this.Values = i.Values :> IEnumerable<'TValue>
member this.Count = i.Count
member this.GetEnumerator() = i.GetEnumerator()
member this.GetEnumerator() = i.GetEnumerator() :> IEnumerator
type DictionaryConverter () =
// ReadJson adapted from this answer
// To
// By https://whosebug.com/users/3744182/dbc
inherit JsonConverter()
override this.CanConvert(t) = (t.DictionaryKeyValueTypes().Count() = 1) // If ever implemented for IReadOnlyDictionary<'TKey, 'TValue> then reject DictionaryReadOnlySurrogate<'TKey, 'TValue>
member private this.ReadJsonGeneric<'TKey, 'TValue> (reader : JsonReader, t : Type, existingValue : obj, serializer : JsonSerializer) : obj =
let contract = serializer.ContractResolver.ResolveContract(t)
let dict = if (existingValue :? IDictionary<'TKey, 'TValue>) then existingValue :?> IDictionary<'TKey, 'TValue> else contract.DefaultCreator.Invoke() :?> IDictionary<'TKey, 'TValue>
match reader.MoveToContentAndAssert().TokenType with
| JsonToken.StartArray ->
let l = serializer.Deserialize<List<KeyValuePair<'TKey, 'TValue>>>(reader)
for p in l do dict.Add(p)
dict :> obj
| JsonToken.StartObject ->
serializer.Populate(reader, dict)
dict :> obj
| JsonToken.Null -> null // Or throw an exception if you prefer
| _ -> raise (JsonSerializationException(String.Format("Unexpected token {0}", reader.TokenType)))
override this.ReadJson(reader, t, existingValue, serializer) =
let keyValueTypes = t.DictionaryKeyValueTypes().Single(); // Throws an exception if not exactly one.
let m = typeof<DictionaryConverter>.GetMethod("ReadJsonGeneric", BindingFlags.NonPublic ||| BindingFlags.Instance ||| BindingFlags.Public);
m.MakeGenericMethod(keyValueTypes).Invoke(this, [| reader; t; existingValue; serializer |])
member private this.WriteJsonGeneric<'TKey, 'TValue> (writer : JsonWriter, value : obj, serializer : JsonSerializer) =
let dict = value :?> IDictionary<'TKey, 'TValue>
let keyContract = serializer.ContractResolver.ResolveContract(typeof<'Key>)
// Wrap the value in an enumerator or read-only surrogate to prevent infinite recursion.
match keyContract with
| :? JsonPrimitiveContract -> serializer.Serialize(writer, new DictionaryReadOnlySurrogate<'TKey, 'TValue>(dict))
| _ -> serializer.Serialize(writer, seq { yield! dict })
()
override this.WriteJson(writer, value, serializer) =
let keyValueTypes = value.GetType().DictionaryKeyValueTypes().Single(); // Throws an exception if not exactly one.
let m = typeof<DictionaryConverter>.GetMethod("WriteJsonGeneric", BindingFlags.NonPublic ||| BindingFlags.Instance ||| BindingFlags.Public);
m.MakeGenericMethod(keyValueTypes).Invoke(this, [| writer; value; serializer |])
()
您将按如下方式添加到设置中:
let settings =
JsonSerializerSettings(
NullValueHandling = NullValueHandling.Ignore,
Converters = [| CompactUnionJsonConverter(true, true); DictionaryConverter() |]
)
并为您的 bookData
生成以下 JSON:
{
"Data": [
{
"Key": "Bid",
"Value": [
{
"Key": 1,
"Value": {
"S": 3.0,
"P": 5.0
}
}
]
},
{
"Key": "Ask",
"Value": []
}
]
}
备注:
转换器适用于所有
Dictionary<TKey, TValue>
类型(和子类型)。转换器检测字典键是否将使用原始合约进行序列化,如果是,则将字典紧凑地序列化为 JSON 对象。如果不是,字典将序列化为数组。您可以在上面显示的 JSON 中观察到这一点:
Dictionary<BookSide, BookSideData>
字典被序列化为 JSON 数组,而Dictionary<int, BookEntry>
字典被序列化为 JSON 对象.在反序列化期间,转换器检测传入的 JSON 值是数组还是对象,并根据需要进行调整。
转换器仅针对可变 .Net
Dictionary<TKey, TValue>
类型实现。逻辑需要稍作修改才能反序列化不可变Map<'Key,'Value>
类型。
演示 fiddle here.