将 JArray 转换为 Dynamic[] 以便 NEST 的 IndexMany 工作

Question

问题描述

我需要接收 JSON 个对象数组（几乎任何形状）并使用函数 IndexMany（或一些类似的批量索引函数）将它们存储到 ES 数据库中。我发现了一些笨拙的解决方案，但有一个缺点——它没有在 ES 中正确设置 _id 属性（根据 JSON 对象中的 id 属性）。

而且我还想知道是否有更优雅的方式来实现我的目标，而无需将每个 JArray 项目转换为字符串并返回到 ExpandoObject。

附加信息

弹性搜索数据库 7.5.1
鸟巢 (7.6.1)
Newtonsoft.Json (12.0.3)

TLDR

下面代码有没有优雅的解决方案：

var settings = new ConnectionSettings(new Uri("http://localhost:9200")).DefaultIndex("people");
var client = new ElasticClient(settings);
            
// this string represents incoming JSON message
string json = @"[
    {
        ""id"": ""1"",
        ""name"": ""Andrej"",
        ""surname"": ""Burak"",
        ""dob"": ""1921-11-10T00:00:00+00:00""
    },
    {
        ""id"": ""2"",
        ""name"": ""Franta"",
        ""surname"": ""Dobrota"",
        ""dob"": ""1933-10-05T00:00:00+00:00""
    },
    {
        ""id"": ""3"",
        ""name"": ""Milos"",
        ""surname"": ""Ovcacek"",
        ""dob"": ""1988-05-05T00:00:00+00:00""
    }
]";


JArray jArray = JArray.Parse(json);

foreach (var jtoken in jArray)
{
    var jobj = (JObject)jtoken;
    jobj.Add("DbCreated", JToken.FromObject(DateTime.UtcNow));
    jobj.Add("DbCreatedBy", JToken.FromObject("authors name"));
}

//working, but seems to me a bit too clumsy to convert every item to string and then back to dynamic object
var converter = new ExpandoObjectConverter();
dynamic[] dlst = jArray.Select(t => (dynamic)JsonConvert.DeserializeObject<ExpandoObject>(t.ToString(), converter)).ToArray();

//not working cast 
dynamic[] dlstNW = jArray.ToObject<dynamic[]>();
            
var indexManyResponse = client.IndexMany(dlst); //working partially (but not using ID as index)
var indexManyResponseNotWorking = client.IndexMany(jArray); //not working
var indexManyResponseNotWorking2 = client.IndexMany(dlstNW); //not working


// expected behavior
dynamic[] jsondyn = new dynamic[]
{
    new { Id = "1", Name = "foo" },
    new { Id = "2", Name = "bar" },
    new { Id = "3", Name = "baz" },
};
var indexManyResponseWithIndex = client.IndexMany(jsondyn); //working perfectly, but don't know how to acieve this

Answer 1

I think I have found my answer. Important thing is to build request manually 之后。

var settings = new ConnectionSettings(new Uri("http://localhost:9200")).DefaultIndex("people").DisableDirectStreaming();
var client = new ElasticClient(settings);
            
// this string represents incoming JSON message
string json = @"[
    {
        ""id"": ""1"",
        ""name"": ""Andrej"",
        ""surname"": ""Burak"",
        ""dob"": ""1921-11-10T00:00:00+00:00""
    },
    {
        ""Id"": ""2"",
        ""name"": ""Franta"",
        ""surname"": ""Dobrota"",
        ""dob"": ""1933-10-05T00:00:00+00:00""
    },
    {
        ""Id"": ""3"",
        ""name"": ""Milos"",
        ""surname"": ""Ovcacek"",
        ""dob"": ""1988-05-05T00:00:00+00:00""
    }
]";


JArray jArray = JArray.Parse(json);

// I have to build my request manually
List<string> esRequest = new List<string>();

foreach (var jtoken in jArray)
{
    var jobj = (JObject)jtoken;
    jobj.Add("DbCreated", JToken.FromObject(DateTime.UtcNow));
    jobj.Add("DbCreatedBy", JToken.FromObject("authors name"));

    string id;
    // Ensure that Id is set even if we receive it in lowercase        
    if( jobj["Id"] == null)
    {
        if( jobj["id"] == null)
        {
            throw new Exception("missing Id");
        }

        id = jobj["id"].Value<string>();
    }
    else
    {
        id = jobj["Id"].Value<string>();
    }
                                
    string indexName = "people";

    esRequest.Add($"{{\"index\":{{\"_index\":\"{indexName}\",\"_id\":\"{id}\"}}}}");
    esRequest.Add(jobj.ToString(Formatting.None));
                
}


PostData pd = PostData.MultiJson(esRequest);
var llres = client.LowLevel.Bulk<BulkResponse>(pd);

希望对大家有所帮助。

将 JArray 转换为 Dynamic[] 以便 NEST 的 IndexMany 工作

Casting JArray to Dynamic[] so NEST's IndexMany works

.net

c#

elasticsearch

nest

问题描述

附加信息

TLDR