C# 类型安全 JSON-行反序列化
C# Type safe JSON-Lines Deserialization
目前我正在使用 Shopify GraphQL Bulk Query。
此查询 returns 一个 JSON Lines 文件。这样的文件可能如下所示:
{"id":"gid:\/\/shopify\/Product\/5860091625632","title":"Levis Jeans","description":"Cool Jeans","vendor":"Levis","status":"ACTIVE"}
{"id":"gid:\/\/shopify\/ProductImage\/20289865679008","__parentId":"gid:\/\/shopify\/Product\/5860091625632"}
{"id":"gid:\/\/shopify\/ProductVariant\/37178118963360","title":"32","position":1,"image":null,"selectedOptions":[{"name":"Size","value":"32"}],"inventoryItem":{},"__parentId":"gid:\/\/shopify\/Product\/5860091625632"}
{"available":10,"location":{"id":"gid:\/\/shopify\/Location\/57510625440"},"__parentId":"gid:\/\/shopify\/ProductVariant\/37178118963360"}
{"id":"gid:\/\/shopify\/ProductVariant\/37178118996128","title":"31","position":2,"image":null,"selectedOptions":[{"name":"Size","value":"31"}],"inventoryItem":{},"__parentId":"gid:\/\/shopify\/Product\/5860091625632"}
{"available":5,"location":{"id":"gid:\/\/shopify\/Location\/57510625440"},"__parentId":"gid:\/\/shopify\/ProductVariant\/37178118996128"}
{"available":3,"location":{"id":"gid:\/\/shopify\/Location\/57951518880"},"__parentId":"gid:\/\/shopify\/ProductVariant\/37178118996128"}
{"id":"gid:\/\/shopify\/ProductVariant\/37178119028896","title":"34","position":3,"image":null,"selectedOptions":[{"name":"Size","value":"34"}],"inventoryItem":{},"__parentId":"gid:\/\/shopify\/Product\/5860091625632"}
{"available":5,"location":{"id":"gid:\/\/shopify\/Location\/57510625440"},"__parentId":"gid:\/\/shopify\/ProductVariant\/37178119028896"}
{"available":15,"location":{"id":"gid:\/\/shopify\/Location\/57951518880"},"__parentId":"gid:\/\/shopify\/ProductVariant\/37178119028896"}
此文件的每一行都是一个有效的 JSON-object 并且这些行通过 __parentId
相互连接。
我的目标是将其反序列化为 C# 类,如下所示:
class Product
{
public string Id { get; set; }
public string Title { get; set; }
public string Description { get; set; }
public IEnumerable<ProductImage> Images { get; set; }
public IEnumerable<ProductVariant> Variants { get; set; }
}
class ProductImage
{
public string Id { get; set; }
}
class ProductVariant
{
public string Id { get; set; }
public IEnumerable<IDictionary<string, string>> SelectedOptions { get; set; }
public IEnumerable<InventoryLevel> Levels { get; set; }
}
class InventoryLevel
{
public int Available { get; set; }
}
以及执行反序列化的潜在函数的输出:
var file = new System.IO.StreamReader(@"c:\test.jsonl");
var products = DeserializeJsonL<IEnumerable<Product>>(file);
Shopify建议反向读取文件。我明白了。
但是我无法想象如何以类型安全的方式反序列化这个文件。我如何确定当前行是 ProductVariant
、ProductImage
还是其他内容?我无法影响 JSONL 输出以包含类型信息。
我很确定没有类型信息我不能安全地反序列化它。但是我应该如何处理这些数据然后插入数据库?
编辑 {"id":"gid:\/\/shopify\/Product\/5860091625632"}
中的类名不能用于确定类型!
我最终通过为每种类型定义一个唯一的字段名来向我的 graphql 查询添加某种类型信息,该字段名可能在生成的 JSON 行文件中的新行上。
为此我使用了 GraphQL 字段别名:
someQuery {
uniqueFieldAlias : fieldName
}
当我阅读文件时,我会在每一行中搜索唯一的字段名。然后我将该行反序列化为相应的 class.
using (var file = new StreamReader(await res.Content.ReadAsStreamAsync()))
{
string line;
while ((line = await file.ReadLineAsync()) != null)
{
if (line.Contains("\"uniqueFieldAlias\""))
{
var product = JsonSerializer.Deserialize<Product>(line);
products.Add(product);
continue;
}
if (line.Contains("\"otherUniqueAlias\""))
{
var somethingElse = JsonSerializer.Deserialize<SomeClass>(line);
products[productIndex].Something.Add(somethingElse);
continue;
}
}
}
这个想法受到@Caius Jard 评论的启发
目前我正在使用 Shopify GraphQL Bulk Query。
此查询 returns 一个 JSON Lines 文件。这样的文件可能如下所示:
{"id":"gid:\/\/shopify\/Product\/5860091625632","title":"Levis Jeans","description":"Cool Jeans","vendor":"Levis","status":"ACTIVE"}
{"id":"gid:\/\/shopify\/ProductImage\/20289865679008","__parentId":"gid:\/\/shopify\/Product\/5860091625632"}
{"id":"gid:\/\/shopify\/ProductVariant\/37178118963360","title":"32","position":1,"image":null,"selectedOptions":[{"name":"Size","value":"32"}],"inventoryItem":{},"__parentId":"gid:\/\/shopify\/Product\/5860091625632"}
{"available":10,"location":{"id":"gid:\/\/shopify\/Location\/57510625440"},"__parentId":"gid:\/\/shopify\/ProductVariant\/37178118963360"}
{"id":"gid:\/\/shopify\/ProductVariant\/37178118996128","title":"31","position":2,"image":null,"selectedOptions":[{"name":"Size","value":"31"}],"inventoryItem":{},"__parentId":"gid:\/\/shopify\/Product\/5860091625632"}
{"available":5,"location":{"id":"gid:\/\/shopify\/Location\/57510625440"},"__parentId":"gid:\/\/shopify\/ProductVariant\/37178118996128"}
{"available":3,"location":{"id":"gid:\/\/shopify\/Location\/57951518880"},"__parentId":"gid:\/\/shopify\/ProductVariant\/37178118996128"}
{"id":"gid:\/\/shopify\/ProductVariant\/37178119028896","title":"34","position":3,"image":null,"selectedOptions":[{"name":"Size","value":"34"}],"inventoryItem":{},"__parentId":"gid:\/\/shopify\/Product\/5860091625632"}
{"available":5,"location":{"id":"gid:\/\/shopify\/Location\/57510625440"},"__parentId":"gid:\/\/shopify\/ProductVariant\/37178119028896"}
{"available":15,"location":{"id":"gid:\/\/shopify\/Location\/57951518880"},"__parentId":"gid:\/\/shopify\/ProductVariant\/37178119028896"}
此文件的每一行都是一个有效的 JSON-object 并且这些行通过 __parentId
相互连接。
我的目标是将其反序列化为 C# 类,如下所示:
class Product
{
public string Id { get; set; }
public string Title { get; set; }
public string Description { get; set; }
public IEnumerable<ProductImage> Images { get; set; }
public IEnumerable<ProductVariant> Variants { get; set; }
}
class ProductImage
{
public string Id { get; set; }
}
class ProductVariant
{
public string Id { get; set; }
public IEnumerable<IDictionary<string, string>> SelectedOptions { get; set; }
public IEnumerable<InventoryLevel> Levels { get; set; }
}
class InventoryLevel
{
public int Available { get; set; }
}
以及执行反序列化的潜在函数的输出:
var file = new System.IO.StreamReader(@"c:\test.jsonl");
var products = DeserializeJsonL<IEnumerable<Product>>(file);
Shopify建议反向读取文件。我明白了。
但是我无法想象如何以类型安全的方式反序列化这个文件。我如何确定当前行是 ProductVariant
、ProductImage
还是其他内容?我无法影响 JSONL 输出以包含类型信息。
我很确定没有类型信息我不能安全地反序列化它。但是我应该如何处理这些数据然后插入数据库?
编辑 {"id":"gid:\/\/shopify\/Product\/5860091625632"}
中的类名不能用于确定类型!
我最终通过为每种类型定义一个唯一的字段名来向我的 graphql 查询添加某种类型信息,该字段名可能在生成的 JSON 行文件中的新行上。
为此我使用了 GraphQL 字段别名:
someQuery {
uniqueFieldAlias : fieldName
}
当我阅读文件时,我会在每一行中搜索唯一的字段名。然后我将该行反序列化为相应的 class.
using (var file = new StreamReader(await res.Content.ReadAsStreamAsync()))
{
string line;
while ((line = await file.ReadLineAsync()) != null)
{
if (line.Contains("\"uniqueFieldAlias\""))
{
var product = JsonSerializer.Deserialize<Product>(line);
products.Add(product);
continue;
}
if (line.Contains("\"otherUniqueAlias\""))
{
var somethingElse = JsonSerializer.Deserialize<SomeClass>(line);
products[productIndex].Something.Add(somethingElse);
continue;
}
}
}
这个想法受到@Caius Jard 评论的启发