更新 Elasticsearch _mapping 中的字符串参数

Question

我在Elasticsearch中有这样一个_mapping 6.8:

{
  "grch38_test__wes__grch38__variants__20210222" : {
    "mappings" : {
      "variant" : {
        "_meta" : {
          "gencodeVersion" : "25",
          "hail_version" : "0.2.20",
          "genomeVersion" : "38",
          "sampleType" : "WES",
          "sourceFilePath" : "s3://my_folder/my_vcf.vcf"
        },
    ...

我的目标是在 Kibana 中发出查询以修改 variant._meta.sourceFilePath。以下主题：

我能够提出查询：

PUT /grch38_test__wes__grch38__variants__20210222/_mapping/variant
{
  "properties": {
    "variant": {
      "type": "nested",
      "properties": {
        "_meta": {
          "type": "nested",
          "properties": {
            "type": "text",
            "sourceFilePath": "s3://my_folder/my_vcf.vcf"
          }
        }
      }
    }
  }
}

但是它给我一个错误：

完整的错误信息：

{
  "error": {
    "root_cause": [
      {
        "type": "mapper_parsing_exception",
        "reason": "Expected map for property [fields] on field [type] but got a class java.lang.String"
      }
    ],
    "type": "mapper_parsing_exception",
    "reason": "Expected map for property [fields] on field [type] but got a class java.lang.String"
  },
  "status": 400
}

我也试过：

PUT /grch38_test__wes__grch38__variants__20210222/_mapping/variant
{
  "properties": {
    "variant": {
      "type": "nested",
      "properties": {
        "_meta": {
          "type": "nested",
          "properties": {
            "sourceFilePath": {
              "type": "text",
              "value":"s3://my_folder/my_vcf.vcf"
            }
          }
        }
      }
    }
  }
}

但它告诉我 value 不受支持：

{
  "error": {
    "root_cause": [
      {
        "type": "mapper_parsing_exception",
        "reason": "Mapping definition for [sourceFilePath] has unsupported parameters:  [value : s3://seqr-dp-data--prod/vcf/dev/grch38_test_contracted.vcf]"
      }
    ],
    "type": "mapper_parsing_exception",
    "reason": "Mapping definition for [sourceFilePath] has unsupported parameters:  [value : s3://seqr-dp-data--prod/vcf/dev/grch38_test_contracted.vcf]"
  },
  "status": 400
}

我做错了什么？如何修改字段？

Answer 1

_meta 是 storing application-specific metadata. It's not meant to be searchable and can be only retrieved through the GET Mapping API 的保留字段。

这意味着如果您的 _meta 内容旨在与 _meta 字段的设计目的保持一致，您不能将任何映射应用到它。它是具体值的“最终”散列图，需要在更新映射有效负载的顶层定义：

PUT /grch38_test__wes__grch38__variants__20210222/_mapping/variant
{
  "_meta": {
    "variant": {            <-- shared index-level metadata
      "gencodeVersion": "25",
      "hail_version": "0.2.20",
      "genomeVersion": "38",
      "sampleType": "WES",
      "sourceFilePath": "s3://my_folder/my_vcf.vcf"
    }
  },
  "properties": {
    "some_text_field": {    <-- actual document properties
      "type": "text" 
    }
  }
}

另一方面，如果您的 _meta 字段是一个 不幸的命名巧合 ，您可以像这样为其声明映射：

PUT /grch38_test__wes__grch38__variants__20210222/_mapping/variant
{
  "properties": {
    "_meta": {
      "properties": {
        "variant": {
          "properties": {
            "gencodeVersion": {
              "type": "text"
            },
            "genomeVersion": {
              "type": "text"
            },
            "hail_version": {
              "type": "text"
            },
            "sampleType": {
              "type": "text"
            },
            "sourceFilePath": {
              "type": "text"
            }
          }
        }
      }
    }
  }
}

并提取以下格式的文档：

POST grch38_test__wes__grch38__variants__20210222/variant/_doc
{
  "_meta": {
    "variant": {
      "gencodeVersion": "25",
      "hail_version": "0.2.20",
      "genomeVersion": "38",
      "sampleType": "WES",
      "sourceFilePath": "s3://my_folder/my_vcf.vcf"
    }
  }
}

但是，_meta 内容将是特定于文档的，而不是索引范围的！

顺便说一句，nested 映射只有在处理 arrays of objects 时才有意义，而不是对象的对象。但如果你坚持想要它，你可以这样做：

PUT /grch38_test__wes__grch38__variants__20210222/_mapping/variant?include_type_name
{
  "properties": {
    "_meta": {
      "type": "nested",            <---
      "properties": {
        "variant": {
          "type": "nested",        <---
          "properties": {
            "gencodeVersion": {
              "type": "text"
            },
            "genomeVersion": {
              "type": "text"
            },
            "hail_version": {
              "type": "text"
            },
            "sampleType": {
              "type": "text"
            },
            "sourceFilePath": {
              "type": "text"
            }
          }
        }
      }
    }
  }
}

更新 Elasticsearch _mapping 中的字符串参数

Update a string parameter in Elasticsearch _mapping

mapping

elasticsearch