如何使用 Elasticsearch 映射更改现有索引的字段类型 API
How to change field types for existing index using Elasticsearch Mapping API
我正在使用 ELK
并具有以下文档结构
{
"_index": "prod1-db.log-*",
"_type": "db.log",
"_id": "AVadEaq7",
"_score": null,
"_source": {
"message": "2016-07-08T12:52:42.026+0000 I NETWORK [conn4928242] end connection 192.168.170.62:47530 (31 connections now open)",
"@version": "1",
"@timestamp": "2016-08-18T09:50:54.247Z",
"type": "log",
"input_type": "log",
"count": 1,
"beat": {
"hostname": "prod1",
"name": "prod1"
},
"offset": 1421607236,
"source": "/var/log/db/db.log",
"fields": null,
"host": "prod1",
"tags": [
"beats_input_codec_plain_applied"
]
},
"fields": {
"@timestamp": [
1471513854247
]
},
"sort": [
1471513854247
]
}
我想将 message
字段更改为 not_analyzed
。我想知道如何使用 Elasticsedarch Mapping API
来实现这一点?例如,如何使用PUT Mapping API
向现有索引添加新类型?
我正在使用 Kibana 4.5
和 Elasticsearch 2.3
。
更新
在 logstash
、
中尝试了以下 template.json
1 {
2 "template": "logstash-*",
3 "mappings": {
4 "_default_": {
5 "properties": {
6 "message" : {
7 "type" : "string",
8 "index" : "not_analyzed"
9 }
10 }
11 }
12 }
13 }
启动时出现以下错误 logstash
,
logstash_1 | {:timestamp=>"2016-08-24T11:00:26.097000+0000", :message=>"Invalid setting for elasticsearch output plugin:\n\n output {\n elasticsearch {\n # This setting must be a path\n # File does not exist or cannot be opened /home/dw/docker-elk/logstash/core_mapping_template.json\n template => \"/home/dw/docker-elk/logstash/core_mapping_template.json\"\n ...\n }\n }", :level=>:error}
logstash_1 | {:timestamp=>"2016-08-24T11:00:26.153000+0000", :message=>"Pipeline aborted due to error", :exception=>#<LogStash::ConfigurationError: Something is wrong with your configuration.>, :backtrace=>["/opt/logstash/vendor/bundle/jruby/1.9/gems/logstash-core-2.3.4-java/lib/logstash/config/mixin.rb:134:in `config_init'", "/opt/logstash/vendor/bundle/jruby/1.9/gems/logstash-core-2.3.4-java/lib/logstash/outputs/base.rb:63:in `initialize'", "/opt/logstash/vendor/bundle/jruby/1.9/gems/logstash-core-2.3.4-java/lib/logstash/output_delegator.rb:74:in `register'", "/opt/logstash/vendor/bundle/jruby/1.9/gems/logstash-core-2.3.4-java/lib/logstash/pipeline.rb:181:in `start_workers'", "org/jruby/RubyArray.java:1613:in `each'", "/opt/logstash/vendor/bundle/jruby/1.9/gems/logstash-core-2.3.4-java/lib/logstash/pipeline.rb:181:in `start_workers'", "/opt/logstash/vendor/bundle/jruby/1.9/gems/logstash-core-2.3.4-java/lib/logstash/pipeline.rb:136:in `run'", "/opt/logstash/vendor/bundle/jruby/1.9/gems/logstash-core-2.3.4-java/lib/logstash/agent.rb:473:in `start_pipeline'"], :level=>:error}
logstash_1 | {:timestamp=>"2016-08-24T11:00:29.168000+0000", :message=>"stopping pipeline", :id=>"main"}
您不能更改已经存在的索引映射,除非您创建新字段到对象或多字段。
如果您想为此使用映射 API,您的请求将如下所示:
PUT /prod1-db.log-*/_mapping/log
{
"properties": {
"message": {
"type": "string",
"index": "not_analyzed"
}
}
}
不过我建议您使用映射创建一个 JSON 文件并将其添加到您的 logstash 配置中。
模板文件可能如下所示(您需要对其进行自定义):
{
"template": "logstash-*",
"mappings": {
"_default_": {
"properties": {
"action" : {
"type" : "string",
"fields" : {
"raw" : {
"index" : "not_analyzed",
"type" : "string"
}
}
},
"ad_domain" : {
"type" : "string"
},
"auth" : {
"type" : "long"
},
"authtime" : {
"type" : "long"
},
"avscantime" : {
"type" : "long"
},
"cached" : {
"type" : "boolean"
}
}
}
}
}
并且您的 Logstash 配置中的 elasticsearch
条目如下所示:
elasticsearch {
template => "/etc/logstash/template/template.json"
template_overwrite => true
}
如果您在创建索引时没有为您的字段指定任何映射,那么在您第一次将文档索引到索引中时,elastic search 会根据提供的数据自动为每个字段选择最佳映射.查看您在问题中提供的文档,elasticsearch 应该已经为字段 message
分配了一个分析器。一旦分配,您将无法更改它。唯一的方法是创建一个新索引。
我正在使用 ELK
并具有以下文档结构
{
"_index": "prod1-db.log-*",
"_type": "db.log",
"_id": "AVadEaq7",
"_score": null,
"_source": {
"message": "2016-07-08T12:52:42.026+0000 I NETWORK [conn4928242] end connection 192.168.170.62:47530 (31 connections now open)",
"@version": "1",
"@timestamp": "2016-08-18T09:50:54.247Z",
"type": "log",
"input_type": "log",
"count": 1,
"beat": {
"hostname": "prod1",
"name": "prod1"
},
"offset": 1421607236,
"source": "/var/log/db/db.log",
"fields": null,
"host": "prod1",
"tags": [
"beats_input_codec_plain_applied"
]
},
"fields": {
"@timestamp": [
1471513854247
]
},
"sort": [
1471513854247
]
}
我想将 message
字段更改为 not_analyzed
。我想知道如何使用 Elasticsedarch Mapping API
来实现这一点?例如,如何使用PUT Mapping API
向现有索引添加新类型?
我正在使用 Kibana 4.5
和 Elasticsearch 2.3
。
更新
在 logstash
、
template.json
1 {
2 "template": "logstash-*",
3 "mappings": {
4 "_default_": {
5 "properties": {
6 "message" : {
7 "type" : "string",
8 "index" : "not_analyzed"
9 }
10 }
11 }
12 }
13 }
启动时出现以下错误 logstash
,
logstash_1 | {:timestamp=>"2016-08-24T11:00:26.097000+0000", :message=>"Invalid setting for elasticsearch output plugin:\n\n output {\n elasticsearch {\n # This setting must be a path\n # File does not exist or cannot be opened /home/dw/docker-elk/logstash/core_mapping_template.json\n template => \"/home/dw/docker-elk/logstash/core_mapping_template.json\"\n ...\n }\n }", :level=>:error}
logstash_1 | {:timestamp=>"2016-08-24T11:00:26.153000+0000", :message=>"Pipeline aborted due to error", :exception=>#<LogStash::ConfigurationError: Something is wrong with your configuration.>, :backtrace=>["/opt/logstash/vendor/bundle/jruby/1.9/gems/logstash-core-2.3.4-java/lib/logstash/config/mixin.rb:134:in `config_init'", "/opt/logstash/vendor/bundle/jruby/1.9/gems/logstash-core-2.3.4-java/lib/logstash/outputs/base.rb:63:in `initialize'", "/opt/logstash/vendor/bundle/jruby/1.9/gems/logstash-core-2.3.4-java/lib/logstash/output_delegator.rb:74:in `register'", "/opt/logstash/vendor/bundle/jruby/1.9/gems/logstash-core-2.3.4-java/lib/logstash/pipeline.rb:181:in `start_workers'", "org/jruby/RubyArray.java:1613:in `each'", "/opt/logstash/vendor/bundle/jruby/1.9/gems/logstash-core-2.3.4-java/lib/logstash/pipeline.rb:181:in `start_workers'", "/opt/logstash/vendor/bundle/jruby/1.9/gems/logstash-core-2.3.4-java/lib/logstash/pipeline.rb:136:in `run'", "/opt/logstash/vendor/bundle/jruby/1.9/gems/logstash-core-2.3.4-java/lib/logstash/agent.rb:473:in `start_pipeline'"], :level=>:error}
logstash_1 | {:timestamp=>"2016-08-24T11:00:29.168000+0000", :message=>"stopping pipeline", :id=>"main"}
您不能更改已经存在的索引映射,除非您创建新字段到对象或多字段。
如果您想为此使用映射 API,您的请求将如下所示:
PUT /prod1-db.log-*/_mapping/log
{
"properties": {
"message": {
"type": "string",
"index": "not_analyzed"
}
}
}
不过我建议您使用映射创建一个 JSON 文件并将其添加到您的 logstash 配置中。
模板文件可能如下所示(您需要对其进行自定义):
{
"template": "logstash-*",
"mappings": {
"_default_": {
"properties": {
"action" : {
"type" : "string",
"fields" : {
"raw" : {
"index" : "not_analyzed",
"type" : "string"
}
}
},
"ad_domain" : {
"type" : "string"
},
"auth" : {
"type" : "long"
},
"authtime" : {
"type" : "long"
},
"avscantime" : {
"type" : "long"
},
"cached" : {
"type" : "boolean"
}
}
}
}
}
并且您的 Logstash 配置中的 elasticsearch
条目如下所示:
elasticsearch {
template => "/etc/logstash/template/template.json"
template_overwrite => true
}
如果您在创建索引时没有为您的字段指定任何映射,那么在您第一次将文档索引到索引中时,elastic search 会根据提供的数据自动为每个字段选择最佳映射.查看您在问题中提供的文档,elasticsearch 应该已经为字段 message
分配了一个分析器。一旦分配,您将无法更改它。唯一的方法是创建一个新索引。