将数据索引到 elasticsearch 时发生批量 API 错误

Question

我想使用批量 API 将一些数据导入 elasticsearch。这是我使用 Kibana 开发工具创建的映射：

PUT /main-news-test-data
{
  "mappings": {
    "properties": {
      "content": {
        "type": "text"
      },
      "title": {
        "type": "text"
      },
      "lead": {
        "type": "text"
      },
      "agency": {
        "type": "keyword"
      },
      "date_created": {
        "type": "date"
      },
      "url": {
        "type": "keyword"
      },
      "image": {
        "type": "keyword"
      },
      "category": {
        "type": "keyword"
      },
      "id":{
        "type": "keyword"
      }
    }
  }
}

这是我的批量数据：

{ "index" : { "_index" : "main-news-test-data", "_id" : "1" } }
{
  "content":"\u0641\u0647\u06cc\u0645\u0647 \u062d\u0633\u0646\u200c\u0645\u06cc\u0631\u06cc: \u0627\u06af\u0631\u0686\u0647 \u062f\u0631 \u0647\u06cc\u0627\u0647\u0648\u06cc ",
        "title":"\u06a9\u0627\u0631\u0647\u0627\u06cc \u0642\u0627\u0644\u06cc\u0628\u0627\u0641",
        "lead":"\u062c\u0627\u0645\u0639\u0647&nbsp;&gt;&nbsp;\u0634\u0647\u0631\u06cc -.",
        "agency":"13",
        "date_created":1494518193,
        "url":"http://www.khabaronline.ir/(X(1)S(bud4wg3ebzbxv51mj45iwjtp))/detail/663749/society/urban",
        "image":"uploads/2017/05/11/1589793661.jpg",
        "category":"15",
        "id":"2981643"
}
{ "index" : { "_index" : "main-news-test-data", "_id" : "2" } }
{ 
....

但是当我想要 post 数据时，我收到此错误：

{
  "error" : {
    "root_cause" : [
      {
        "type" : "illegal_argument_exception",
        "reason" : "Malformed action/metadata line [3], expected START_OBJECT but found [VALUE_STRING]"
      }
  "status" : 400
}

有什么问题？我在 Kibana 开发工具中同时使用了 PowerShell 和 POST 方法，但我在两者中都收到了相同的错误。

Answer 1

数据应该像这样在一行中指定：

{ "index" : { "_index" : "main-news-test-data", "_id" : "1" } }
{ "content":"\u0641\u0647","title":"\u06a9" }

请参考这个SO answer

试试下面的批量格式 JSON。我也在本地测试了这个批量 API 请求，现在它工作得很好：

{ "index" : { "_index" : "main-news-test-data", "_id" : "1" } }
{"content":"\u0641\u0647\u06cc\u0645\u0647 \u062d\u0633\u0646\u200c\u0645\u06cc\u0631\u06cc: \u0627\u06af\u0631\u0686\u0647 \u062f\u0631 \u0647\u06cc\u0627\u0647\u0648\u06cc ", "title":"\u06a9\u0627\u0631\u0647\u0627\u06cc \u0642\u0627\u0644\u06cc\u0628\u0627\u0641", "lead":"\u062c\u0627\u0645\u0639\u0647&nbsp;&gt;&nbsp;\u0634\u0647\u0631\u06cc -.", "agency":"13", "date_created":1494518193, "url":"http://www.khabaronline.ir/(X(1)S(bud4wg3ebzbxv51mj45iwjtp))/detail/663749/society/urban", "image":"uploads/2017/05/11/1589793661.jpg", "category":"15", "id":"2981643"}

不要忘记在内容末尾添加新行。

将数据索引到 elasticsearch 时发生批量 API 错误

Bulk API error while indexing data into elasticsearch

bulk-load

elasticsearch

elasticsearch-bulk-api