Elasticsearch：按请求中传递的数组过滤文档包含所有文档数组元素

Question

我存储在 elasticsearch 中的文档具有以下结构：

{
  "id": 1,
  "test": "name",
  "rules": [
    {
      "id": 2,
      "name": "rule1",
      "ruleDetails": [
        {
          "id": 3,
          "requiredAnswerId": 1
        },
        {
          "id": 4,
          "requiredAnswerId": 2
        },
        {
          "id": 5,
          "requiredAnswerId": 3
        }
      ]
    }
  ]
}

其中，rules 属性具有 nested 类型。

我需要通过检查在搜索请求（提供的术语）中传递的 requiredAnswerId 数组包含存储在文档中的所有 rules.ruleDetails.requiredAnswerId 来查询文档。

有谁知道我可以使用哪个 elasticsearch 选项来构建这样的特定查询？或者，最好是获取整个文档并在应用程序级别执行过滤。

已更新 添加映射

{
  "my_index": {
    "mappings": {
      "properties": {
        "id": {
          "type": "long"
        },
        "test": {
          "type": "text",
          "fields": {
            "keyword": {
              "type": "keyword"
            }
          }
        },
        "rules": {
          "type": "nested",
          "properties": {
            "id": {
              "type": "long"
            },
            "name": {
              "type": "text",
              "fields": {
                "keyword": {
                  "type": "keyword",
                  "ignore_above": 256
                }
              }
            },
            "ruleDetails": {
              "properties": {
                "id": {
                  "type": "long"
                },
                "requiredAnswerId": {
                  "type": "long"
                }
              }
            }
          }
        }
      }
    }
  }
}

Answer 1

映射：

{
  "index4" : {
    "mappings" : {
      "properties" : {
        "id" : {
          "type" : "integer"
        },
        "rules" : {
          "type" : "nested",
          "properties" : {
            "id" : {
              "type" : "integer"
            },
            "name" : {
              "type" : "text",
              "fields" : {
                "keyword" : {
                  "type" : "keyword"
                }
              }
            },
            "ruleDetails" : {
              "properties" : {
                "id" : {
                  "type" : "long"
                },
                "requiredAnswerId" : {
                  "type" : "long"
                }
              }
            }
          }
        },
        "test" : {
          "type" : "text",
          "fields" : {
            "keyword" : {
              "type" : "keyword"
            }
          }
        }
      }
    }
  }
}

查询： 这需要使用从性能角度看不太好的脚本。我正在遍历所有文档并检查是否存在字段是否传递了参数

{
  "query": {
    "nested": {
      "path": "rules",
      "query": {
        "script": {
          "script": {
            "source": "for(a in doc['rules.ruleDetails.requiredAnswerId']){if(!params.Ids.contains((int)a)) return false; }  return true;",
            "params": {
              "Ids": [
                1,
                2,
                3
              ]
            }
          }
        }
      },
      "inner_hits": {}
    }
  }
}

结果：

  "hits" : [
      {
        "_index" : "index4",
        "_type" : "_doc",
        "_id" : "TxOpvnEBf42mOjxvvLQB",
        "_score" : 4.0,
        "_source" : {
          "id" : 1,
          "test" : "name",
          "rules" : [
            {
              "id" : 2,
              "name" : "rule1",
              "ruleDetails" : [
                {
                  "id" : 3,
                  "requiredAnswerId" : 1
                },
                {
                  "id" : 4,
                  "requiredAnswerId" : 2
                },
                {
                  "id" : 5,
                  "requiredAnswerId" : 3
                }
              ]
            },
            {
              "id" : 3,
              "name" : "rule3",
              "ruleDetails" : [
                {
                  "id" : 3,
                  "requiredAnswerId" : 1
                },
                {
                  "id" : 4,
                  "requiredAnswerId" : 2
                }
              ]
            }
          ]
        },
        "inner_hits" : {
          "rules" : {
            "hits" : {
              "total" : {
                "value" : 1,
                "relation" : "eq"
              },
              "max_score" : 4.0,
              "hits" : [
                {
                  "_index" : "index4",
                  "_type" : "_doc",
                  "_id" : "TxOpvnEBf42mOjxvvLQB",
                  "_nested" : {
                    "field" : "rules",
                    "offset" : 0
                  },
                  "_score" : 4.0,
                  "_source" : {
                    "id" : 2,
                    "name" : "rule1",
                    "ruleDetails" : [
                      {
                        "id" : 3,
                        "requiredAnswerId" : 1
                      },
                      {
                        "id" : 4,
                        "requiredAnswerId" : 2
                      },
                      {
                        "id" : 5,
                        "requiredAnswerId" : 3
                      }
                    ]
                  }
                }
              ]
            }
          }
        }
      }
    ]

编辑 1

Terms_set 可以作为替代。与脚本查询相比会更快

Returns documents that contain a minimum number of exact terms in a provided field.

minimum_should_match_script- 数组大小可用于匹配传递值的最小数量。

查询：

{
  "query": {
    "nested": {
      "path": "rules",
      "query": {
        "bool": {
          "filter": {
            "terms_set": {
              "rules.ruleDetails.requiredAnswerId": {
                "terms": [
                  1,
                  2,
                  3
                ],
                "minimum_should_match_script": {
                  "source": "doc['rules.ruleDetails.requiredAnswerId'].size()"
                }
              }
            }
          }
        }
      },
      "inner_hits": {}
    }
  }
}

Answer 2

在玩了一段时间ES并阅读了它的文档后，我发现你应该记住提供的script应该被编译并应用于文档，因此它会更慢，如果你只知道需要提前匹配的元素个数。

因此，我创建了一个单独的字段 requiredMatches 来存储每个文档的 rules.ruleDetails.requiredAnswerId 元素的数量，并在索引文档之前计算它。然后，我没有在搜索查询中使用 minimum_should_match_script，而是使用 minimum_should_match_field:

{
  "query": {
    "nested": {
      "path": "rules",
      "query": {
        "bool": {
          "filter": {
            "terms_set": {
              "rules.ruleDetails.requiredAnswerId": {
                "terms": [
                  1,
                  2,
                  3
                ],
                "minimum_should_match_field": "requiredMatches"
              }
            }
          }
        }
      },
      "inner_hits": {}
    }
  }
}

我用了，following example，作为参考

Elasticsearch：按请求中传递的数组过滤文档包含所有文档数组元素

Elasticsearch: filter documents by array passed in request contains all document array elements

elasticsearch

elasticsearch-dsl

elasticsearch-aggregation