聚合嵌套 ElasticSearch 中的字典
Aggregating on dictionaries in nest ElasticSearch
所以我有一组索引产品，其中包含一个带有单个键的字典和一个值列表，我试图用它来构建多面搜索。然而我是一个非常有弹性的新手。
Product Product {
Dictionary<string, List<string>> Properties
//extra fields removed for simplicity
}
其中属性可能类似于
["Color":["Blue","Yellow","Red"],"Size":["Small","Medium","Large"]
或
["Material":["Wood"], "Shape":["Circle","Square"], "Size":["Tiny","Medium","Large","Huge"]
我想写几个聚合，它们将 return 键和这些键的值。
即，如果要对上述示例进行索引，则第一个聚合将 return 包含 "Color","Size","Material","Shape"
的存储桶
第二个聚合将 return 4 个桶，每个桶的每个键都有唯一值。
即 Size:["Tiny","small","medium","large","huge"]
等等
我意识到为此我需要一个嵌套聚合，但是 none 我的尝试都带回了桶中的任何东西。任何指针将不胜感激。这是我目前所拥有的。
var ProductsQuery = client.Search<Product>(s => s
.Index("products")
.Query(q => q.MatchAll())
.Aggregations(a => a
.Nested("properties", n => n
.Path(p => p.Properties.Suffix("keyword"))
.Aggregations(a => a
.Terms("property-keys", t => t
.Field(f => f.Properties.Keys.Suffix("keyword"))))));
编辑一些请求的详细信息：
当前属性映射（它似乎正在为每个键创建一个新映射，我不确定这是否是典型的？）我没有将整个对象映射放在这里，因为它相当大。产品有很多领域：
"properties" : {
"properties" : {
"Colour" : {
"type" : "text",
"fields" : {
"keyword" : {
"type" : "keyword",
"ignore_above" : 256
}
}
},
"Equipment" : {
"type" : "text",
"fields" : {
"keyword" : {
"type" : "keyword",
"ignore_above" : 256
}
}
},
"Football Age" : {
"type" : "text",
"fields" : {
"keyword" : {
"type" : "keyword",
"ignore_above" : 256
}
}
},
"Football Size" : {
"type" : "text",
"fields" : {
"keyword" : {
"type" : "keyword",
"ignore_above" : 256
}
}
},
"Frame Weight" : {
"type" : "text",
"fields" : {
"keyword" : {
"type" : "keyword",
"ignore_above" : 256
}
}
},
"Garment" : {
"type" : "text",
"fields" : {
"keyword" : {
"type" : "keyword",
"ignore_above" : 256
}
}
},
"Head Shape" : {
"type" : "text",
"fields" : {
"keyword" : {
"type" : "keyword",
"ignore_above" : 256
}
}
},
"Level" : {
"type" : "text",
"fields" : {
"keyword" : {
"type" : "keyword",
"ignore_above" : 256
}
}
},
"Product" : {
"type" : "text",
"fields" : {
"keyword" : {
"type" : "keyword",
"ignore_above" : 256
}
}
},
"Size" : {
"type" : "text",
"fields" : {
"keyword" : {
"type" : "keyword",
"ignore_above" : 256
}
}
},
"Sport" : {
"type" : "text",
"fields" : {
"keyword" : {
"type" : "keyword",
"ignore_above" : 256
}
}
},
"Surface" : {
"type" : "text",
"fields" : {
"keyword" : {
"type" : "keyword",
"ignore_above" : 256
}
}
},
"Type" : {
"type" : "text",
"fields" : {
"keyword" : {
"type" : "keyword",
"ignore_above" : 256
}
}
},
"Unit" : {
"type" : "text",
"fields" : {
"keyword" : {
"type" : "keyword",
"ignore_above" : 256
}
}
},
"Weight" : {
"type" : "text",
"fields" : {
"keyword" : {
"type" : "keyword",
"ignore_above" : 256
}
}
},
"comparer" : {
"type" : "object"
},
"count" : {
"type" : "integer"
},
"item" : {
"type" : "text",
"fields" : {
"keyword" : {
"type" : "keyword",
"ignore_above" : 256
}
}
},
"keys" : {
"properties" : {
"count" : {
"type" : "integer"
}
}
},
"values" : {
"properties" : {
"count" : {
"type" : "integer"
}
}
}
}
}
和一些索引文档
"hits" : [
{
"_index" : "products-20-01-2021-13-49-08",
"_type" : "_doc",
"_id" : "134550",
"_score" : 1.0,
"_source" : {
"properties" : {
"Type" : [
"Sleds"
],
"Product" : [
"Sleds"
],
"Colour" : [
"Black"
]
}
}
},
{
"_index" : "products-20-01-2021-13-49-08",
"_type" : "_doc",
"_id" : "134566",
"_score" : 1.0,
"_source" : {
"properties" : {
"Sport" : [
"Fitness"
],
"Type" : [
"Corner",
"Edge",
"Middle"
],
"Size" : [
"10mm",
"15mm",
"20mm"
],
"Product" : [
"Floor Matting"
]
}
}
},
{
"_index" : "products-20-01-2021-13-49-08",
"_type" : "_doc",
"_id" : "134576",
"_score" : 1.0,
"_source" : {
"properties" : {
"Sport" : [
"Rugby"
],
"Type" : [
"Skills Training"
],
"Equipment" : [
"Rugby Balls"
],
"Size" : [
"4",
"5"
],
"Level" : [
"Skills"
]
}
}
},
{
"_index" : "products-20-01-2021-13-49-08",
"_type" : "_doc",
"_id" : "134579",
"_score" : 1.0,
"_source" : {
"properties" : {
"Sport" : [
"Rugby"
],
"Type" : [
"Match Union"
],
"Equipment" : [
"Rugby Balls"
],
"Size" : [
"4",
"5"
],
"Level" : [
"Club",
" School"
],
"Unit" : [
"12 Pack",
"Each"
],
"Colour" : [
"Blue",
"Red",
"White"
]
}
}
},
{
"_index" : "products-20-01-2021-13-49-08",
"_type" : "_doc",
"_id" : "134600",
"_score" : 1.0,
"_source" : {
"properties" : {
"Sport" : [
"Rugby"
],
"Size" : [
"Large",
"Medium",
"Small",
"X/Large",
"X/Small",
"XX/Small",
"XXX/Small"
],
"Garment" : [
"Gloves"
],
"Colour" : [
"Red"
]
}
}
},
{
"_index" : "products-20-01-2021-13-49-08",
"_type" : "_doc",
"_id" : "134601",
"_score" : 1.0,
"_source" : {
"properties" : {
"Sport" : [
"Netball"
],
"Size" : [
"Large",
"X/Large",
"X/Small",
"XX/Small",
"XXX/Small"
],
"Garment" : [
"Gloves"
],
"Colour" : [
"Red"
]
}
}
},
{
"_index" : "products-20-01-2021-13-49-08",
"_type" : "_doc",
"_id" : "134609",
"_score" : 1.0,
"_source" : {
"properties" : {
"Sport" : [
"Netball"
],
"Size" : [
"Large",
"Medium",
"Small",
"X/Large",
"X/Small",
"XXX/Small"
],
"Garment" : [
"Gloves"
],
"Colour" : [
"Black",
"Green"
]
}
}
},
{
"_index" : "products-20-01-2021-13-49-08",
"_type" : "_doc",
"_id" : "134617",
"_score" : 1.0,
"_source" : {
"properties" : {
"Sport" : [
"Football"
],
"Type" : [
"Training"
],
"Football Size" : [
"2"
],
"Equipment" : [
"Footballs"
],
"Size" : [
"4",
"5"
],
"Unit" : [
"12 Pack",
"Each"
],
"Weight" : [
"290",
"360"
],
"Surface" : [
"Grass",
" Astroturf"
],
"Football Age" : [
"9-14 years",
" 14+ years"
]
}
}
},
{
"_index" : "products-20-01-2021-13-49-08",
"_type" : "_doc",
"_id" : "134548",
"_score" : 1.0,
"_source" : {
"properties" : {
"Type" : [
"Sleds"
],
"Product" : [
"Sleds"
],
"Colour" : [
"Black",
"Grey"
]
}
}
},
{
"_index" : "products-20-01-2021-13-49-08",
"_type" : "_doc",
"_id" : "134558",
"_score" : 1.0,
"_source" : {
"properties" : {
"Sport" : [
"Squash"
],
"Equipment" : [
"Squash Rackets"
],
"Size" : [
"27\""
],
"Head Shape" : [
"Bridged Closed Throat"
],
"Frame Weight" : [
"Over 160g"
]
}
}
}
]
首先，为了获得这些桶，您可以使用 Query DSL 说以下内容：
POST products-*/_search
{
"size": 0,
"aggs": {
"by_Colour": {
"terms": {
"field": "properties.Colour.keyword"
}
},
"by_Size": {
"terms": {
"field": "properties.Size.keyword"
}
}
}
}
然后需要将其翻译成 NEST 代码——我相信那里有很多例子。
但您的观察是正确的 -- ES 自动创建了很多映射。
我建议不要使用当前的数据格式，而是使用：
{
"properties": [
{
"key": "Type",
"values": ["Sleds"]
},
{
"key": "Product",
"values": ["Sleds"]
},
{
"key": "Colour",
"values": ["Black"]
}
]
}
并制作 properties 类型 nested。
这样您将在共享路径 properties.key 上查询并在 properties.values.keyword 上聚合。
请记住，nested 字段类型需要它们自己的实际 nested mapping -- 如果没有正确的映射，您将无法使用嵌套查询。
查看此 and also 了解更多上下文。
所以我有一组索引产品，其中包含一个带有单个键的字典和一个值列表，我试图用它来构建多面搜索。然而我是一个非常有弹性的新手。
Product Product {
Dictionary<string, List<string>> Properties
//extra fields removed for simplicity
}
其中属性可能类似于
["Color":["Blue","Yellow","Red"],"Size":["Small","Medium","Large"]
或
["Material":["Wood"], "Shape":["Circle","Square"], "Size":["Tiny","Medium","Large","Huge"]
我想写几个聚合，它们将 return 键和这些键的值。
即，如果要对上述示例进行索引，则第一个聚合将 return 包含 "Color","Size","Material","Shape"
第二个聚合将 return 4 个桶，每个桶的每个键都有唯一值。
即 Size:["Tiny","small","medium","large","huge"]
等等
我意识到为此我需要一个嵌套聚合，但是 none 我的尝试都带回了桶中的任何东西。任何指针将不胜感激。这是我目前所拥有的。
var ProductsQuery = client.Search<Product>(s => s
.Index("products")
.Query(q => q.MatchAll())
.Aggregations(a => a
.Nested("properties", n => n
.Path(p => p.Properties.Suffix("keyword"))
.Aggregations(a => a
.Terms("property-keys", t => t
.Field(f => f.Properties.Keys.Suffix("keyword"))))));
编辑一些请求的详细信息：
当前属性映射（它似乎正在为每个键创建一个新映射，我不确定这是否是典型的？）我没有将整个对象映射放在这里，因为它相当大。产品有很多领域：
"properties" : {
"properties" : {
"Colour" : {
"type" : "text",
"fields" : {
"keyword" : {
"type" : "keyword",
"ignore_above" : 256
}
}
},
"Equipment" : {
"type" : "text",
"fields" : {
"keyword" : {
"type" : "keyword",
"ignore_above" : 256
}
}
},
"Football Age" : {
"type" : "text",
"fields" : {
"keyword" : {
"type" : "keyword",
"ignore_above" : 256
}
}
},
"Football Size" : {
"type" : "text",
"fields" : {
"keyword" : {
"type" : "keyword",
"ignore_above" : 256
}
}
},
"Frame Weight" : {
"type" : "text",
"fields" : {
"keyword" : {
"type" : "keyword",
"ignore_above" : 256
}
}
},
"Garment" : {
"type" : "text",
"fields" : {
"keyword" : {
"type" : "keyword",
"ignore_above" : 256
}
}
},
"Head Shape" : {
"type" : "text",
"fields" : {
"keyword" : {
"type" : "keyword",
"ignore_above" : 256
}
}
},
"Level" : {
"type" : "text",
"fields" : {
"keyword" : {
"type" : "keyword",
"ignore_above" : 256
}
}
},
"Product" : {
"type" : "text",
"fields" : {
"keyword" : {
"type" : "keyword",
"ignore_above" : 256
}
}
},
"Size" : {
"type" : "text",
"fields" : {
"keyword" : {
"type" : "keyword",
"ignore_above" : 256
}
}
},
"Sport" : {
"type" : "text",
"fields" : {
"keyword" : {
"type" : "keyword",
"ignore_above" : 256
}
}
},
"Surface" : {
"type" : "text",
"fields" : {
"keyword" : {
"type" : "keyword",
"ignore_above" : 256
}
}
},
"Type" : {
"type" : "text",
"fields" : {
"keyword" : {
"type" : "keyword",
"ignore_above" : 256
}
}
},
"Unit" : {
"type" : "text",
"fields" : {
"keyword" : {
"type" : "keyword",
"ignore_above" : 256
}
}
},
"Weight" : {
"type" : "text",
"fields" : {
"keyword" : {
"type" : "keyword",
"ignore_above" : 256
}
}
},
"comparer" : {
"type" : "object"
},
"count" : {
"type" : "integer"
},
"item" : {
"type" : "text",
"fields" : {
"keyword" : {
"type" : "keyword",
"ignore_above" : 256
}
}
},
"keys" : {
"properties" : {
"count" : {
"type" : "integer"
}
}
},
"values" : {
"properties" : {
"count" : {
"type" : "integer"
}
}
}
}
}
和一些索引文档
"hits" : [
{
"_index" : "products-20-01-2021-13-49-08",
"_type" : "_doc",
"_id" : "134550",
"_score" : 1.0,
"_source" : {
"properties" : {
"Type" : [
"Sleds"
],
"Product" : [
"Sleds"
],
"Colour" : [
"Black"
]
}
}
},
{
"_index" : "products-20-01-2021-13-49-08",
"_type" : "_doc",
"_id" : "134566",
"_score" : 1.0,
"_source" : {
"properties" : {
"Sport" : [
"Fitness"
],
"Type" : [
"Corner",
"Edge",
"Middle"
],
"Size" : [
"10mm",
"15mm",
"20mm"
],
"Product" : [
"Floor Matting"
]
}
}
},
{
"_index" : "products-20-01-2021-13-49-08",
"_type" : "_doc",
"_id" : "134576",
"_score" : 1.0,
"_source" : {
"properties" : {
"Sport" : [
"Rugby"
],
"Type" : [
"Skills Training"
],
"Equipment" : [
"Rugby Balls"
],
"Size" : [
"4",
"5"
],
"Level" : [
"Skills"
]
}
}
},
{
"_index" : "products-20-01-2021-13-49-08",
"_type" : "_doc",
"_id" : "134579",
"_score" : 1.0,
"_source" : {
"properties" : {
"Sport" : [
"Rugby"
],
"Type" : [
"Match Union"
],
"Equipment" : [
"Rugby Balls"
],
"Size" : [
"4",
"5"
],
"Level" : [
"Club",
" School"
],
"Unit" : [
"12 Pack",
"Each"
],
"Colour" : [
"Blue",
"Red",
"White"
]
}
}
},
{
"_index" : "products-20-01-2021-13-49-08",
"_type" : "_doc",
"_id" : "134600",
"_score" : 1.0,
"_source" : {
"properties" : {
"Sport" : [
"Rugby"
],
"Size" : [
"Large",
"Medium",
"Small",
"X/Large",
"X/Small",
"XX/Small",
"XXX/Small"
],
"Garment" : [
"Gloves"
],
"Colour" : [
"Red"
]
}
}
},
{
"_index" : "products-20-01-2021-13-49-08",
"_type" : "_doc",
"_id" : "134601",
"_score" : 1.0,
"_source" : {
"properties" : {
"Sport" : [
"Netball"
],
"Size" : [
"Large",
"X/Large",
"X/Small",
"XX/Small",
"XXX/Small"
],
"Garment" : [
"Gloves"
],
"Colour" : [
"Red"
]
}
}
},
{
"_index" : "products-20-01-2021-13-49-08",
"_type" : "_doc",
"_id" : "134609",
"_score" : 1.0,
"_source" : {
"properties" : {
"Sport" : [
"Netball"
],
"Size" : [
"Large",
"Medium",
"Small",
"X/Large",
"X/Small",
"XXX/Small"
],
"Garment" : [
"Gloves"
],
"Colour" : [
"Black",
"Green"
]
}
}
},
{
"_index" : "products-20-01-2021-13-49-08",
"_type" : "_doc",
"_id" : "134617",
"_score" : 1.0,
"_source" : {
"properties" : {
"Sport" : [
"Football"
],
"Type" : [
"Training"
],
"Football Size" : [
"2"
],
"Equipment" : [
"Footballs"
],
"Size" : [
"4",
"5"
],
"Unit" : [
"12 Pack",
"Each"
],
"Weight" : [
"290",
"360"
],
"Surface" : [
"Grass",
" Astroturf"
],
"Football Age" : [
"9-14 years",
" 14+ years"
]
}
}
},
{
"_index" : "products-20-01-2021-13-49-08",
"_type" : "_doc",
"_id" : "134548",
"_score" : 1.0,
"_source" : {
"properties" : {
"Type" : [
"Sleds"
],
"Product" : [
"Sleds"
],
"Colour" : [
"Black",
"Grey"
]
}
}
},
{
"_index" : "products-20-01-2021-13-49-08",
"_type" : "_doc",
"_id" : "134558",
"_score" : 1.0,
"_source" : {
"properties" : {
"Sport" : [
"Squash"
],
"Equipment" : [
"Squash Rackets"
],
"Size" : [
"27\""
],
"Head Shape" : [
"Bridged Closed Throat"
],
"Frame Weight" : [
"Over 160g"
]
}
}
}
]
首先，为了获得这些桶，您可以使用 Query DSL 说以下内容：
POST products-*/_search
{
"size": 0,
"aggs": {
"by_Colour": {
"terms": {
"field": "properties.Colour.keyword"
}
},
"by_Size": {
"terms": {
"field": "properties.Size.keyword"
}
}
}
}
然后需要将其翻译成 NEST 代码——我相信那里有很多例子。
但您的观察是正确的 -- ES 自动创建了很多映射。 我建议不要使用当前的数据格式，而是使用：
{
"properties": [
{
"key": "Type",
"values": ["Sleds"]
},
{
"key": "Product",
"values": ["Sleds"]
},
{
"key": "Colour",
"values": ["Black"]
}
]
}
并制作 properties 类型 nested。
这样您将在共享路径 properties.key 上查询并在 properties.values.keyword 上聚合。
请记住，nested 字段类型需要它们自己的实际 nested mapping -- 如果没有正确的映射，您将无法使用嵌套查询。
查看此