为什么 ElasticSearch 请求在 10.000 个文档后失败？

Question

我们正在使用 ElasticSearch.NET / NEST 来查询 ElasticSearch 配置。计划是分批获取 1000 个文档并在获取下一个 1000 个文档之前处理它们

然而处理10批后总是失败

ELK Search failed Invalid NEST response built from a unsuccessful (500)

如果我们将批量大小更改为 10,000，它将在 1 批后失败批次大小为 100，它将在 100 批次后失败失败总是在 10.000 个文档之后

代码看起来像这样

        private void ProcessRequest(SearchRequest request)
    {
        request.Size = 1000;
        for (request.From = 0; request.From < 1_000_000; request.From += request.Size)
        {
            Console.WriteLine(request.From);
            var responses = _client.Search<GroupStaticElkDocument>(request);
            foreach (var response in responses)
                _requestCounter.Add(response.ToRequest());
        }
    }

Answer 1

也许你应该试试 paging/scrolling

为什么 ElasticSearch 请求在 10.000 个文档后失败？

Why is ElasticSearch request failing after 10.000 documents?

nest

elasticsearch.net

elk