通过 daemonset 的 Fluentd pods 运行 因 google 容器引擎上的警告而终止
Fluentd pods running via daemonset getting terminated with warnings on google container engine
我正在尝试 运行 作为 kubernetes 集群 (GKE) 上的守护进程流利。配置被成功解析，然后插件收到关闭信号，警告消息很少。没有错误消息。我尝试增加详细级别，以下是 pods:
的输出
fluentd-7przp fluentd 2018-09-08 11:02:46 +0000 [info]: #0 fluent/log.rb:322:info: starting fluentd worker pid=9 ppid=1 worker=0
fluentd-7przp fluentd 2018-09-08 11:02:46 +0000 [info]: #0 fluent/log.rb:322:info: fluentd worker is now running worker=0
fluentd-sr764 fluentd 2018-09-08 11:02:50 +0000 [warn]: #0 fluent/log.rb:342:warn: dump an error event: error_class=NoMethodError error="undefined method `[]' for nil:NilClass" location="/fluentd/vendor/bundle/ruby/2.3.0/gems/fluent-plugin-elasticsearch-2.10.1/lib/fluent/plugin/out_elasticsearch_dynamic.rb:268:in `eval'" tag="fluent.info" time=2018-09-08 11:02:45.151774166 +0000 record={"worker"=>0, "message"=>"fluentd worker is now running worker=0"}
fluentd-dhwnx fluentd 2018-09-08 11:02:51 +0000 [warn]: #0 fluent/log.rb:342:warn: dump an error event: error_class=NoMethodError error="undefined method `[]' for nil:NilClass" location="/fluentd/vendor/bundle/ruby/2.3.0/gems/fluent-plugin-elasticsearch-2.10.1/lib/fluent/plugin/out_elasticsearch_dynamic.rb:268:in `eval'" tag="fluent.info" time=2018-09-08 11:02:46.029522363 +0000 record={"worker"=>0, "message"=>"fluentd worker is now running worker=0"}
fluentd-7przp fluentd 2018-09-08 11:02:51 +0000 [warn]: #0 fluent/log.rb:342:warn: dump an error event: error_class=NoMethodError error="undefined method `[]' for nil:NilClass" location="/fluentd/vendor/bundle/ruby/2.3.0/gems/fluent-plugin-elasticsearch-2.10.1/lib/fluent/plugin/out_elasticsearch_dynamic.rb:268:in `eval'" tag="fluent.info" time=2018-09-08 11:02:46.538377182 +0000 record={"worker"=>0, "message"=>"fluentd worker is now running worker=0"}
fluentd-sr764 fluentd 2018-09-08 11:02:55 +0000 [warn]: #0 fluent/log.rb:342:warn: dump an error event: error_class=NoMethodError error="undefined method `[]' for nil:NilClass" location="/fluentd/vendor/bundle/ruby/2.3.0/gems/fluent-plugin-elasticsearch-2.10.1/lib/fluent/plugin/out_elasticsearch_dynamic.rb:268:in `eval'" tag="fluent.warn" time=2018-09-08 11:02:50.153922217 +0000 record={"error"=>"#<NoMethodError: undefined method `[]' for nil:NilClass>", "location"=>"/fluentd/vendor/bundle/ruby/2.3.0/gems/fluent-plugin-elasticsearch-2.10.1/lib/fluent/plugin/out_elasticsearch_dynamic.rb:268:in `eval'", "tag"=>"fluent.info", "time"=>2018-09-08 11:02:45.151774166 +0000, "record"=>{"worker"=>0, "message"=>"fluentd worker is now running worker=0"}, "message"=>"dump an error event: error_class=NoMethodError error=\"undefined method `[]' for nil:NilClass\" location=\"/fluentd/vendor/bundle/ruby/2.3.0/gems/fluent-plugin-elasticsearch-2.10.1/lib/fluent/plugin/out_elasticsearch_dynamic.rb:268:in `eval'\" tag=\"fluent.info\" time=2018-09-08 11:02:45.151774166 +0000 record={\"worker\"=>0, \"message\"=>\"fluentd worker is now running worker=0\"}"}
fluentd-sr764 fluentd 2018-09-08 11:03:10 +0000 [warn]: #0 fluent/log.rb:342:warn: dump an error event: error_class=NoMethodError error="undefined method `[]' for nil:NilClass" location="/fluentd/vendor/bundle/ruby/2.3.0/gems/fluent-plugin-elasticsearch-2.10.1/lib/fluent/plugin/out_elasticsearch_dynamic.rb:268:in `eval'" tag="fluent.warn" time=2018-09-08 11:03:05.168427649 +0000 record={"error"=>"#<NoMethodError: undefined method `[]' for nil:NilClass>", "location"=>"/fluentd/vendor/bundle/ruby/2.3.0/gems/fluent-plugin-elasticsearch-2.10.1/lib/fluent/plugin/out_elasticsearch_dynamic.rb:268:in `eval'", "tag"=>"fluent.warn", "time"=>2018-09-08 11:03:00.165843014 +0000, "record"=>{"error"=>"#<NoMethodError: undefined method `[]' for nil:NilClass>", "location"=>"/fluentd/vendor/bundle/ruby/2.3.0/gems/fluent-plugin-elasticsearch-2.10.1/lib/fluent/plugin/out_elasticsearch_dynamic.rb:268:in `eval'", "tag"=>"fluent.warn", "time"=>2018-09-08 11:02:55.156840516 +0000, "record"=>{"error"=>"#<NoMethodError: undefined method `[]' for nil:NilClass>", "location"=>"/fluentd/vendor/bundle/ruby/2.3.0/gems/fluent-plugin-elasticsearch-2.10.1/lib/fluent/plugin/out_elasticsearch_dynamic.rb:268:in `eval'", "tag"=>"fluent.warn", "time"=>2018-09-08 11:02:50.153922217 +0000, "record"=>{"error"=>"#<NoMethodError: undefined method `[]' for nil:NilClass>", "location"=>"/fluentd/vendor/bundle/ruby/2.3.0/gems/fluent-plugin-elasticsearch-2.10.1/lib/fluent/plugin/out_elasticsearch_dynamic.rb:268:in `eval'", "tag"=>"fluent.info", "time"=>2018-09-08 11:02:45.151774166 +0000, "record"=>{"worker"=>0, "message"=>"fluentd worker is now running worker=0"}, "message"=>"dump an error event: error_class=NoMethodError error=\"undefined method `[]' for nil:NilClass\" location=\"/fluentd/vendor/bundle/ruby/2.3.0/gems/fluent-plugin-elasticsearch-2.10.1/lib/fluent/plugin/out_elasticsearch_dynamic.rb:268:in `eval'\" tag=\"fluent.info\" time=2018-09-08 11:02:45.151774166 +0000 record={\"worker\"=>0, \"message\"=>\"fluentd worker is now running worker=0\"}"}, "message"=>"dump an error event: error_class=NoMethodError error=\"undefined method `[]' for nil:NilClass\" location=\"/fluentd/vendor/bundle/ruby/2.3.0/gems/fluent-plugin-elasticsearch-2.10.1/lib/fluent/plugin/out_elasticsearch_dynamic.rb:268:in `eval'\" tag=\"fluent.warn\" time=2018-09-08 11:02:50.153922217 +0000 record={\"error\"=>\"#<NoMethodError: undefined method `[]' for nil:NilClass>\", \"location\"=>\"/fluentd/vendor/bundle/ruby/2.3.0/gems/fluent-plugin-elasticsearch-2.10.1/lib/fluent/plugin/out_elasticsearch_dynamic.rb:268:in `eval'\", \"tag\"=>\"fluent.info\", \"time\"=>2018-09-08 11:02:45.151774166 +0000, \"record\"=>{\"worker\"=>0, \"message\"=>\"fluentd worker is now running worker=0\"}, \"message\"=>\"dump an error event: error_class=NoMethodError error=\\"undefined method `[]' for nil:NilClass\\" location=\\"/fluentd/vendor/bundle/ruby/2.3.0/gems/fluent-plugin-elasticsearch-2.10.1/lib/fluent/plugin/out_elasticsearch_dynamic.rb:268:in `eval'\\" tag=\\"fluent.info\\" time=2018-09-08 11:02:45.151774166 +0000 record={\\"worker\\"=>0, \\"message\\"=>\\"fluentd worker is now running worker=0\\"}\"}"}, "message"=>"dump an error event: error_class=NoMethodError error=\"undefined method `[]' for nil:NilClass\" location=\"/fluentd/vendor/bundle/ruby/2.3.0/gems/fluent-plugin-elasticsearch-2.10.1/lib/fluent/plugin/out_elasticsearch_dynamic.rb:268:in `eval'\" tag=\"fluent.warn\" time=2018-09-08 11:02:55.156840516 +0000 record={\"error\"=>\"#<NoMethodError: undefined method `[]' for nil:NilClass>\", \"location\"=>\"/fluentd/vendor/bundle/ruby/2.3.0/gems/fluent-plugin-elasticsearch-2.10.1/lib/fluent/plugin/out_elasticsearch_dynamic.rb:268:in `eval'\", \"tag\"=>\"fluent.warn\", \"time\"=>2018-09-08 11:02:50.153922217 +0000, \"record\"=>{\"error\"=>\"#<NoMethodError: undefined method `[]' for nil:NilClass>\", \"location\"=>\"/fluentd/vendor/bundle/ruby/2.3.0/gems/fluent-plugin-elasticsearch-2.10.1/lib/fluent/plugin/out_elasticsearch_dynamic.rb:268:in `eval'\", \"tag\"=>\"fluent.info\", \"time\"=>2018-09-08 11:02:45.151774166 +0000, \"record\"=>{\"worker\"=>0, \"message\"=>\"fluentd worker is now running worker=0\"}, \"message\"=>\"dump an error event: error_class=NoMethodError error=\\"undefined method `[]' for nil:NilClass\\" location=\\"/fluentd/vendor/bundle/ruby/2.3.0/gems/fluent-plugin-elasticsearch-2.10.1/lib/fluent/plugin/out_elasticsearch_dynamic.rb:268:in `eval'\\" tag=\\"fluent.info\\" time=2018-09-08 11:02:45.151774166 +0000 record={\\"worker\\"=>0, \\"message\\"=>\\"fluentd worker is now running worker=0\\"}\"}, \"message\"=>\"dump an error event: error_class=NoMethodError error=\\"undefined method `[]' for nil:NilClass\\" location=\\"/fluentd/vendor/bundle/ruby/2.3.0/gems/fluent-plugin-elasticsearch-2.10.1/lib/fluent/plugin/out_elasticsearch_dynamic.rb:268:in `eval'\\" tag=\\"fluent.warn\\" time=2018-09-08 11:02:50.153922217 +0000 record={\\"error\\"=>\\"#<NoMethodError: undefined method `[]' for nil:NilClass>\\", \\"location\\"=>\\"/fluentd/vendor/bundle/ruby/2.3.0/gems/fluent-plugin-elasticsearch-2.10.1/lib/fluent/plugin/out_elasticsearch_dynamic.rb:268:in `eval'\\", \\"tag\\"=>\\"fluent.info\\", \\"time\\"=>2018-09-08 11:02:45.151774166 +0000, \\"record\\"=>{\\"worker\\"=>0, \\"message\\"=>\\"fluentd worker is now running worker=0\\"}, \\"message\\"=>\\"dump an error event: error_class=NoMethodError error=\\\\"undefined method `[]' for nil:NilClass\\\\" location=\\\\"/fluentd/vendor/bundle/ruby/2.3.0/gems/fluent-plugin-elasticsearch-2.10.1/lib/fluent/plugin/out_elasticsearch_dynamic.rb:268:in `eval'\\\\" tag=\\\\"fluent.info\\\\" time=2018-09-08 11:02:45.151774166 +0000 record={\\\\"worker\\\\"=>0, \\\\"message\\\\"=>\\\\"fluentd worker is now running worker=0\\\\"}\\"}\"}"}, "message"=>"dump an error event: error_class=NoMethodError error=\"undefined method `[]' for nil:NilClass\" location=\"/fluentd/vendor/bundle/ruby/2.3.0/gems/fluent-plugin-elasticsearch-2.10.1/lib/fluent/plugin/out_elasticsearch_dynamic.rb:268:in `eval'\" tag=\"fluent.warn\" time=2018-09-08 11:03:00.165843014 +0000 record={\"error\"=>\"#<NoMethodError: undefined method `[]' for nil:NilClass>\", \"location\"=>\"/fluentd/vendor/bundle/ruby/2.3.0/gems/fluent-plugin-elasticsearch-2.10.1/lib/fluent/plugin/out_elasticsearch_dynamic.rb:268:in `eval'\", \"tag\"=>\"fluent.warn\", \"time\"=>2018-09-08 11:02:55.156840516 +0000, \"record\"=>{\"error\"=>\"#<NoMethodError: undefined method `[]' for nil:NilClass>\", \"location\"=>\"/fluentd/vendor/bundle/ruby/2.3.0/gems/fluent-plugin-elasticsearch-2.10.1/lib/fluent/plugin/out_elasticsearch_dynamic.rb:268:in `eval'\", \"tag\"=>\"fluent.warn\", \"time\"=>2018-09-08 11:02:50.153922217 +0000, \"record\"=>{\"error\"=>\"#<NoMethodError: undefined method `[]' for nil:NilClass>\", \"location\"=>\"/fluentd/vendor/bundle/ruby/2.3.0/gems/fluent-plugin-elasticsearch-2.10.1/lib/fluent/plugin/out_elasticsearch_dynamic.rb:268:in `eval'\", \"tag\"=>\"fluent.info\", \"time\"=>2018-09-08 11:02:45.151774166 +0000, \"record\"=>{\"worker\"=>0, \"message\"=>\"fluentd worker is now running worker=0\"}, \"message\"=>\"dump an error event: error_class=NoMethodError error=\\"undefined method `[]' for nil:NilClass\\" location=\\"/fluentd/vendor/bundle/ruby/2.3.0/gems/fluent-plugin-elasticsearch-2.10.1/lib/fluent/plugin/out_elasticsearch_dynamic.rb:268:in `eval'\\" tag=\\"fluent.info\\" time=2018-09-08 11:02:45.151774166 +0000 record={\\"worker\\"=>0, \\"message\\"=>\\"fluentd worker is now running worker=0\\"}\"}, \"message\"=>\"dump an error event: error_class=NoMethodError error=\\"undefined method `[]' for nil:NilClass\\" location=\\"/fluentd/vendor/bundle/ruby/2.3.0/gems/fluent-plugin-elasticsearch-2.10.1/lib/fluent/plugin/out_elasticsearch_dynamic.rb:268:in `eval'\\" tag=\\"fluent.warn\\" time=2018-09-08 11:02:50.153922217 +0000 record={\\"error\\"=>\\"#<NoMethodError: undefined method `[]' for nil:NilClass>\\", \\"location\\"=>\\"/fluentd/vendor/bundle/ruby/2.3.0/gems/fluent-plugin-elasticsearch-2.10.1/lib/fluent/plugin/out_elasticsearch_dynamic.rb:268:in `eval'\\", \\"tag\\"=>\\"fluent.info\\", \\"time\\"=>2018-09-08 11:02:45.151774166 +0000, \\"record\\"=>{\\"worker\\"=>0, \\"message\\"=>\\"fluentd worker is now running worker=0\\"}, \\"message\\"=>\\"dump an error event: error_class=NoMethodError error=\\\\"undefined method `[]' for nil:NilClass\\\\" location=\\\\"/fluentd/vendor/bundle/ruby/2.3.0/gems/fluent-plugin-elasticsearch-2.10.1/lib/fluent/plugin/out_elasticsearch_dynamic.rb:268:in `eval'\\\\" tag=\\\\"fluent.info\\\\" time=2018-09-08 11:02:45.151774166 +0000 record={\\\\"worker\\\\"=>0, \\\\"message\\\\"=>\\\\"fluentd worker is now running worker=0\\\\"}\\"}\"}, \"message\"=>\"dump an error event: error_class=NoMethodError error=\\"undefined method `[]' for nil:NilClass\\" location=\\"/fluentd/vendor/bundle/ruby/2.3.0/gems/fluent-plugin-elasticsearch-2.10.1/lib/fluent/plugin/out_elasticsearch_dynamic.rb:268:in `eval'\\" tag=\\"fluent.warn\\" time=2018-09-08 11:02:55.156840516 +0000 record={\\"error\\"=>\\"#<NoMethodError: undefined method `[]' for nil:NilClass>\\", \\"location\\"=>\\"/fluentd/vendor/bundle/ruby/2.3.0/gems/fluent-plugin-elasticsearch-2.10.1/lib/fluent/plugin/out_elasticsearch_dynamic.rb:268:in `eval'\\", \\"tag\\"=>\\"fluent.warn\\", \\"time\\"=>2018-09-08 11:02:50.153922217 +0000, \\"record\\"=>{\\"error\\"=>\\"#<NoMethodError: undefined method `[]' for nil:NilClass>\\", \\"location\\"=>\\"/fluentd/vendor/bundle/ruby/2.3.0/gems/fluent-plugin-elasticsearch-2.10.1/lib/fluent/plugin/out_elasticsearch_dynamic.rb:268:in `eval'\\", \\"tag\\"=>\\"fluent.info\\", \\"time\\"=>2018-09-08 11:02:45.151774166 +0000, \\"record\\"=>{\\"worker\\"=>0, \\"message\\"=>\\"fluentd worker is now running worker=0\\"}, \\"message\\"=>\\"dump an error event: error_class=NoMethodError error=\\\\"undefined method `[]' for nil:NilClass\\\\" location=\\\\"/fluentd/vendor/bundle/ruby/2.3.0/gems/fluent-plugin-elasticsearch-2.10.1/lib/fluent/plugin/out_elasticsearch_dynamic.rb:268:in `eval'\\\\" tag=\\\\"fluent.info\\\\" time=2018-09-08 11:02:45.151774166 +0000 record={\\\\"worker\\\\"=>0, \\\\"message\\\\"=>\\\\"fluentd worker is now running worker=0\\\\"}\\"}, \\"message\\"=>\\"dump an error event: error_class=NoMethodError error=\\\\"undefined method `[]' for nil:NilClass\\\\" location=\\\\"/fluentd/vendor/bundle/ruby/2.3.0/gems/fluent-plugin-elasticsearch-2.10.1/lib/fluent/plugin/out_elasticsearch_dynamic.rb:268:in `eval'\\\\" tag=\\\\"fluent.warn\\\\" time=2018-09-08 11:02:50.153922217 +0000 record={\\\\"error\\\\"=>\\\\"#<NoMethodError: undefined method `[]' for nil:NilClass>\\\\", \\\\"location\\\\"=>\\\\"/fluentd/vendor/bundle/ruby/2.3.0/gems/fluent-plugin-elasticsearch-2.10.1/lib/fluent/plugin/out_elasticsearch_dynamic.rb:268:in `eval'\\\\", \\\\"tag\\\\"=>\\\\"fluent.info\\\\", \\\\"time\\\\"=>2018-09-08 11:02:45.151774166 +0000, \\\\"record\\\\"=>{\\\\"worker\\\\"=>0, \\\\"message\\\\"=>\\\\"fluentd worker is now running worker=0\\\\"}, \\\\"message\\\\"=>\\\\"dump an error event: error_class=NoMethodError error=\\\\\\\\"undefined method `[]' for nil:NilClass\\\\\\\\" location=\\\\\\\\"/fluentd/vendor/bundle/ruby/2.3.0/gems/fluent-plugin-elasticsearch-2.10.1/lib/fluent/plugin/out_elasticsearch_dynamic.rb:268:in `eval'\\\\\\\\" tag=\\\\\\\\"fluent.info\\\\\\\\" time=2018-09-08 11:02:45.151774166 +0000 record={\\\\\\\\"worker\\\\\\\\"=>0, \\\\\\\\"message\\\\\\\\"=>\\\\\\\\"fluentd worker is now running worker=0\\\\\\\\"}\\\\"}\\"}\"}"}
fluentd-7przp fluentd 2018-09-08 11:03:24 +0000 [debug]: #0 fluent/log.rb:302:debug: preparing shutdown output plugin type=:elasticsearch_dynamic plugin_id="kubelet_out_es"
fluentd-7przp fluentd 2018-09-08 11:03:24 +0000 [info]: #0 fluent/log.rb:322:info: shutting down output plugin type=:elasticsearch_dynamic plugin_id="kubelet_out_es"
fluentd-dhwnx fluentd 2018-09-08 11:03:25 +0000 [warn]: #0 fluent/log.rb:342:warn: dump an error event: error_class=NoMethodError error="undefined method `[]' for nil:NilClass" location="/fluentd/vendor/bundle/ruby/2.3.0/gems/fluent-plugin-elasticsearch-2.10.1/lib/fluent/plugin/out_elasticsearch_dynamic.rb:268:in `eval'" tag="fluent.debug" time=2018-09-08 11:03:24.151685730 +0000 record={"message"=>"fluentd main process get SIGTERM"}
fluentd-sr764 fluentd-dhwnx fluentd-7przp fluentd fluentd 2018-09-08 11:03:25 +0000 [debug]: #0 fluent/log.rb:302:debug: calling terminate on filter plugin type=:parser plugin_id="myapp_filter"
fluentd 2018-09-08 11:03:25 +0000 [debug]: #0 fluent/log.rb:302:debug: calling terminate on filter plugin type=:parser plugin_id="myapp_filter"
fluentd-dhwnxfluentd-sr764 fluentd 2018-09-08 11:03:25 +0000 [info]: fluent/log.rb:322:info: Worker 0 finished with status 0
fluentd 2018-09-08 11:03:25 +0000 [info]: fluent/log.rb:322:info: Worker 0 finished with status 0
2018-09-08 11:03:25 +0000 [debug]: #0 fluent/log.rb:302:debug: calling terminate on output plugin type=:elasticsearch_dynamic plugin_id="kubelet_out_es"
fluentd-7przp fluentd 2018-09-08 11:03:25 +0000 [debug]: #0 fluent/log.rb:302:debug: calling terminate on output plugin type=:elasticsearch_dynamic plugin_id="apiserver_out_es"
fluentd-7przp fluentd 2018-09-08 11:03:25 +0000 [debug]: #0 fluent/log.rb:302:debug: calling terminate on output plugin type=:elasticsearch_dynamic plugin_id="out_es"
fluentd-7przp fluentd 2018-09-08 11:03:25 +0000 [debug]: #0 fluent/log.rb:302:debug: calling terminate on filter plugin type=:parser plugin_id="myapp_filter"
fluentd-7przp fluentd 2018-09-08 11:03:26 +0000 [info]: fluent/log.rb:322:info: Worker 0 finished with status 0
fluentd.conf:
@include systemd.conf
@include kubernetes.conf
# Start of fluent.conf
<filter kubernetes.var.log.containers.myapp-provider**.log>
@type parser
@id myapp_filter
key_name log
reserve_data true
remove_key_name_field true
<parse>
@type multiline
format_firstline /^[A-Z]/
format1 /^(?<level>[^ ]*)[ \t]+\[(?<time>[^\]]*)\] \[(?<thread>[^\]]*)\] \[(?<request>[^\]]*)\] (?<class>[^ ]*): (?<message>.*)$/
time_format %Y-%m-%d %H:%M:%S,%L %z
</parse>
</filter>
<match kubelet>
@type elasticsearch_dynamic
@id kubelet_out_es
log_level info
include_tag_key true
host "#{ENV['FLUENT_ELASTICSEARCH_HOST']}"
port "#{ENV['FLUENT_ELASTICSEARCH_PORT']}"
scheme "#{ENV['FLUENT_ELASTICSEARCH_SCHEME'] || 'http'}"
ssl_verify "#{ENV['FLUENT_ELASTICSEARCH_SSL_VERIFY'] || 'true'}"
#user "#{ENV['FLUENT_ELASTICSEARCH_USER']}"
#password "#{ENV['FLUENT_ELASTICSEARCH_PASSWORD']}"
reload_connections "#{ENV['FLUENT_ELASTICSEARCH_RELOAD_CONNECTIONS'] || 'true'}"
index_name fluentd-${tag_parts[0]+ "-" + Time.at(time).getlocal("+05:30").strftime(@logstash_dateformat)}
include_timestamp true
#logstash_prefix "#{ENV['FLUENT_ELASTICSEARCH_LOGSTASH_PREFIX'] || 'logstash'}"
#logstash_format true
<buffer>
flush_thread_count 8
flush_interval 5s
chunk_limit_size 2M
queue_limit_length 32
retry_max_interval 30
retry_forever true
</buffer>
</match>
<match kube-apiserver>
@type elasticsearch_dynamic
@id apiserver_out_es
log_level info
include_tag_key true
host "#{ENV['FLUENT_ELASTICSEARCH_HOST']}"
port "#{ENV['FLUENT_ELASTICSEARCH_PORT']}"
scheme "#{ENV['FLUENT_ELASTICSEARCH_SCHEME'] || 'http'}"
ssl_verify "#{ENV['FLUENT_ELASTICSEARCH_SSL_VERIFY'] || 'true'}"
#user "#{ENV['FLUENT_ELASTICSEARCH_USER']}"
#password "#{ENV['FLUENT_ELASTICSEARCH_PASSWORD']}"
reload_connections "#{ENV['FLUENT_ELASTICSEARCH_RELOAD_CONNECTIONS'] || 'true'}"
index_name fluentd-${tag_parts[0]+ "-" + Time.at(time).getlocal("+05:30").strftime(@logstash_dateformat)}
include_timestamp true
#logstash_prefix "#{ENV['FLUENT_ELASTICSEARCH_LOGSTASH_PREFIX'] || 'logstash'}"
#logstash_format true
<buffer>
flush_thread_count 8
flush_interval 5s
chunk_limit_size 2M
queue_limit_length 32
retry_max_interval 30
retry_forever true
</buffer>
</match>
<match **>
@type elasticsearch_dynamic
@id out_es
log_level info
include_tag_key true
host "#{ENV['FLUENT_ELASTICSEARCH_HOST']}"
port "#{ENV['FLUENT_ELASTICSEARCH_PORT']}"
scheme "#{ENV['FLUENT_ELASTICSEARCH_SCHEME'] || 'http'}"
ssl_verify "#{ENV['FLUENT_ELASTICSEARCH_SSL_VERIFY'] || 'true'}"
#user "#{ENV['FLUENT_ELASTICSEARCH_USER']}"
#password "#{ENV['FLUENT_ELASTICSEARCH_PASSWORD']}"
reload_connections "#{ENV['FLUENT_ELASTICSEARCH_RELOAD_CONNECTIONS'] || 'true'}"
index_name fluentd-${record['kubernetes']['container_name']}-${Time.at(time).getlocal("+05:30").strftime(@logstash_dateformat)}
include_timestamp true
#logstash_prefix "#{ENV['FLUENT_ELASTICSEARCH_LOGSTASH_PREFIX'] || 'logstash'}"
#logstash_format true
<buffer>
flush_thread_count 8
flush_interval 5s
chunk_limit_size 2M
queue_limit_length 32
retry_max_interval 30
retry_forever true
</buffer>
</match>
谁能帮忙调试一下。提前致谢！
我将从这个命令开始，以检查 pods 频繁重启的情况：
kubectl get pods --all-namespaces
然后 运行 下面的命令是为了收集更多关于容器最后已知状态的信息以及相关的 exit code. Docker exit codes are based on the chroot standard。
kubectl describe pods [podname]
执行 "kubectl describe" 应该可以为您提供有关 pods 终止原因的更多信息。
这可能是此处提到的 kubernetes 错误 - https://github.com/kubernetes/kubernetes/issues/51376
尽管此问题目前已关闭，但我建议您在 default 命名空间中尝试 运行 相同的 daemonset。
即使我在 kube-system 命名空间中尝试了 运行 但它失败了并且在 default 命名空间中成功 运行。
我正在尝试 运行 作为 kubernetes 集群 (GKE) 上的守护进程流利。配置被成功解析，然后插件收到关闭信号，警告消息很少。没有错误消息。我尝试增加详细级别，以下是 pods:
的输出fluentd-7przp fluentd 2018-09-08 11:02:46 +0000 [info]: #0 fluent/log.rb:322:info: starting fluentd worker pid=9 ppid=1 worker=0
fluentd-7przp fluentd 2018-09-08 11:02:46 +0000 [info]: #0 fluent/log.rb:322:info: fluentd worker is now running worker=0
fluentd-sr764 fluentd 2018-09-08 11:02:50 +0000 [warn]: #0 fluent/log.rb:342:warn: dump an error event: error_class=NoMethodError error="undefined method `[]' for nil:NilClass" location="/fluentd/vendor/bundle/ruby/2.3.0/gems/fluent-plugin-elasticsearch-2.10.1/lib/fluent/plugin/out_elasticsearch_dynamic.rb:268:in `eval'" tag="fluent.info" time=2018-09-08 11:02:45.151774166 +0000 record={"worker"=>0, "message"=>"fluentd worker is now running worker=0"}
fluentd-dhwnx fluentd 2018-09-08 11:02:51 +0000 [warn]: #0 fluent/log.rb:342:warn: dump an error event: error_class=NoMethodError error="undefined method `[]' for nil:NilClass" location="/fluentd/vendor/bundle/ruby/2.3.0/gems/fluent-plugin-elasticsearch-2.10.1/lib/fluent/plugin/out_elasticsearch_dynamic.rb:268:in `eval'" tag="fluent.info" time=2018-09-08 11:02:46.029522363 +0000 record={"worker"=>0, "message"=>"fluentd worker is now running worker=0"}
fluentd-7przp fluentd 2018-09-08 11:02:51 +0000 [warn]: #0 fluent/log.rb:342:warn: dump an error event: error_class=NoMethodError error="undefined method `[]' for nil:NilClass" location="/fluentd/vendor/bundle/ruby/2.3.0/gems/fluent-plugin-elasticsearch-2.10.1/lib/fluent/plugin/out_elasticsearch_dynamic.rb:268:in `eval'" tag="fluent.info" time=2018-09-08 11:02:46.538377182 +0000 record={"worker"=>0, "message"=>"fluentd worker is now running worker=0"}
fluentd-sr764 fluentd 2018-09-08 11:02:55 +0000 [warn]: #0 fluent/log.rb:342:warn: dump an error event: error_class=NoMethodError error="undefined method `[]' for nil:NilClass" location="/fluentd/vendor/bundle/ruby/2.3.0/gems/fluent-plugin-elasticsearch-2.10.1/lib/fluent/plugin/out_elasticsearch_dynamic.rb:268:in `eval'" tag="fluent.warn" time=2018-09-08 11:02:50.153922217 +0000 record={"error"=>"#<NoMethodError: undefined method `[]' for nil:NilClass>", "location"=>"/fluentd/vendor/bundle/ruby/2.3.0/gems/fluent-plugin-elasticsearch-2.10.1/lib/fluent/plugin/out_elasticsearch_dynamic.rb:268:in `eval'", "tag"=>"fluent.info", "time"=>2018-09-08 11:02:45.151774166 +0000, "record"=>{"worker"=>0, "message"=>"fluentd worker is now running worker=0"}, "message"=>"dump an error event: error_class=NoMethodError error=\"undefined method `[]' for nil:NilClass\" location=\"/fluentd/vendor/bundle/ruby/2.3.0/gems/fluent-plugin-elasticsearch-2.10.1/lib/fluent/plugin/out_elasticsearch_dynamic.rb:268:in `eval'\" tag=\"fluent.info\" time=2018-09-08 11:02:45.151774166 +0000 record={\"worker\"=>0, \"message\"=>\"fluentd worker is now running worker=0\"}"}
fluentd-sr764 fluentd 2018-09-08 11:03:10 +0000 [warn]: #0 fluent/log.rb:342:warn: dump an error event: error_class=NoMethodError error="undefined method `[]' for nil:NilClass" location="/fluentd/vendor/bundle/ruby/2.3.0/gems/fluent-plugin-elasticsearch-2.10.1/lib/fluent/plugin/out_elasticsearch_dynamic.rb:268:in `eval'" tag="fluent.warn" time=2018-09-08 11:03:05.168427649 +0000 record={"error"=>"#<NoMethodError: undefined method `[]' for nil:NilClass>", "location"=>"/fluentd/vendor/bundle/ruby/2.3.0/gems/fluent-plugin-elasticsearch-2.10.1/lib/fluent/plugin/out_elasticsearch_dynamic.rb:268:in `eval'", "tag"=>"fluent.warn", "time"=>2018-09-08 11:03:00.165843014 +0000, "record"=>{"error"=>"#<NoMethodError: undefined method `[]' for nil:NilClass>", "location"=>"/fluentd/vendor/bundle/ruby/2.3.0/gems/fluent-plugin-elasticsearch-2.10.1/lib/fluent/plugin/out_elasticsearch_dynamic.rb:268:in `eval'", "tag"=>"fluent.warn", "time"=>2018-09-08 11:02:55.156840516 +0000, "record"=>{"error"=>"#<NoMethodError: undefined method `[]' for nil:NilClass>", "location"=>"/fluentd/vendor/bundle/ruby/2.3.0/gems/fluent-plugin-elasticsearch-2.10.1/lib/fluent/plugin/out_elasticsearch_dynamic.rb:268:in `eval'", "tag"=>"fluent.warn", "time"=>2018-09-08 11:02:50.153922217 +0000, "record"=>{"error"=>"#<NoMethodError: undefined method `[]' for nil:NilClass>", "location"=>"/fluentd/vendor/bundle/ruby/2.3.0/gems/fluent-plugin-elasticsearch-2.10.1/lib/fluent/plugin/out_elasticsearch_dynamic.rb:268:in `eval'", "tag"=>"fluent.info", "time"=>2018-09-08 11:02:45.151774166 +0000, "record"=>{"worker"=>0, "message"=>"fluentd worker is now running worker=0"}, "message"=>"dump an error event: error_class=NoMethodError error=\"undefined method `[]' for nil:NilClass\" location=\"/fluentd/vendor/bundle/ruby/2.3.0/gems/fluent-plugin-elasticsearch-2.10.1/lib/fluent/plugin/out_elasticsearch_dynamic.rb:268:in `eval'\" tag=\"fluent.info\" time=2018-09-08 11:02:45.151774166 +0000 record={\"worker\"=>0, \"message\"=>\"fluentd worker is now running worker=0\"}"}, "message"=>"dump an error event: error_class=NoMethodError error=\"undefined method `[]' for nil:NilClass\" location=\"/fluentd/vendor/bundle/ruby/2.3.0/gems/fluent-plugin-elasticsearch-2.10.1/lib/fluent/plugin/out_elasticsearch_dynamic.rb:268:in `eval'\" tag=\"fluent.warn\" time=2018-09-08 11:02:50.153922217 +0000 record={\"error\"=>\"#<NoMethodError: undefined method `[]' for nil:NilClass>\", \"location\"=>\"/fluentd/vendor/bundle/ruby/2.3.0/gems/fluent-plugin-elasticsearch-2.10.1/lib/fluent/plugin/out_elasticsearch_dynamic.rb:268:in `eval'\", \"tag\"=>\"fluent.info\", \"time\"=>2018-09-08 11:02:45.151774166 +0000, \"record\"=>{\"worker\"=>0, \"message\"=>\"fluentd worker is now running worker=0\"}, \"message\"=>\"dump an error event: error_class=NoMethodError error=\\"undefined method `[]' for nil:NilClass\\" location=\\"/fluentd/vendor/bundle/ruby/2.3.0/gems/fluent-plugin-elasticsearch-2.10.1/lib/fluent/plugin/out_elasticsearch_dynamic.rb:268:in `eval'\\" tag=\\"fluent.info\\" time=2018-09-08 11:02:45.151774166 +0000 record={\\"worker\\"=>0, \\"message\\"=>\\"fluentd worker is now running worker=0\\"}\"}"}, "message"=>"dump an error event: error_class=NoMethodError error=\"undefined method `[]' for nil:NilClass\" location=\"/fluentd/vendor/bundle/ruby/2.3.0/gems/fluent-plugin-elasticsearch-2.10.1/lib/fluent/plugin/out_elasticsearch_dynamic.rb:268:in `eval'\" tag=\"fluent.warn\" time=2018-09-08 11:02:55.156840516 +0000 record={\"error\"=>\"#<NoMethodError: undefined method `[]' for nil:NilClass>\", \"location\"=>\"/fluentd/vendor/bundle/ruby/2.3.0/gems/fluent-plugin-elasticsearch-2.10.1/lib/fluent/plugin/out_elasticsearch_dynamic.rb:268:in `eval'\", \"tag\"=>\"fluent.warn\", \"time\"=>2018-09-08 11:02:50.153922217 +0000, \"record\"=>{\"error\"=>\"#<NoMethodError: undefined method `[]' for nil:NilClass>\", \"location\"=>\"/fluentd/vendor/bundle/ruby/2.3.0/gems/fluent-plugin-elasticsearch-2.10.1/lib/fluent/plugin/out_elasticsearch_dynamic.rb:268:in `eval'\", \"tag\"=>\"fluent.info\", \"time\"=>2018-09-08 11:02:45.151774166 +0000, \"record\"=>{\"worker\"=>0, \"message\"=>\"fluentd worker is now running worker=0\"}, \"message\"=>\"dump an error event: error_class=NoMethodError error=\\"undefined method `[]' for nil:NilClass\\" location=\\"/fluentd/vendor/bundle/ruby/2.3.0/gems/fluent-plugin-elasticsearch-2.10.1/lib/fluent/plugin/out_elasticsearch_dynamic.rb:268:in `eval'\\" tag=\\"fluent.info\\" time=2018-09-08 11:02:45.151774166 +0000 record={\\"worker\\"=>0, \\"message\\"=>\\"fluentd worker is now running worker=0\\"}\"}, \"message\"=>\"dump an error event: error_class=NoMethodError error=\\"undefined method `[]' for nil:NilClass\\" location=\\"/fluentd/vendor/bundle/ruby/2.3.0/gems/fluent-plugin-elasticsearch-2.10.1/lib/fluent/plugin/out_elasticsearch_dynamic.rb:268:in `eval'\\" tag=\\"fluent.warn\\" time=2018-09-08 11:02:50.153922217 +0000 record={\\"error\\"=>\\"#<NoMethodError: undefined method `[]' for nil:NilClass>\\", \\"location\\"=>\\"/fluentd/vendor/bundle/ruby/2.3.0/gems/fluent-plugin-elasticsearch-2.10.1/lib/fluent/plugin/out_elasticsearch_dynamic.rb:268:in `eval'\\", \\"tag\\"=>\\"fluent.info\\", \\"time\\"=>2018-09-08 11:02:45.151774166 +0000, \\"record\\"=>{\\"worker\\"=>0, \\"message\\"=>\\"fluentd worker is now running worker=0\\"}, \\"message\\"=>\\"dump an error event: error_class=NoMethodError error=\\\\"undefined method `[]' for nil:NilClass\\\\" location=\\\\"/fluentd/vendor/bundle/ruby/2.3.0/gems/fluent-plugin-elasticsearch-2.10.1/lib/fluent/plugin/out_elasticsearch_dynamic.rb:268:in `eval'\\\\" tag=\\\\"fluent.info\\\\" time=2018-09-08 11:02:45.151774166 +0000 record={\\\\"worker\\\\"=>0, \\\\"message\\\\"=>\\\\"fluentd worker is now running worker=0\\\\"}\\"}\"}"}, "message"=>"dump an error event: error_class=NoMethodError error=\"undefined method `[]' for nil:NilClass\" location=\"/fluentd/vendor/bundle/ruby/2.3.0/gems/fluent-plugin-elasticsearch-2.10.1/lib/fluent/plugin/out_elasticsearch_dynamic.rb:268:in `eval'\" tag=\"fluent.warn\" time=2018-09-08 11:03:00.165843014 +0000 record={\"error\"=>\"#<NoMethodError: undefined method `[]' for nil:NilClass>\", \"location\"=>\"/fluentd/vendor/bundle/ruby/2.3.0/gems/fluent-plugin-elasticsearch-2.10.1/lib/fluent/plugin/out_elasticsearch_dynamic.rb:268:in `eval'\", \"tag\"=>\"fluent.warn\", \"time\"=>2018-09-08 11:02:55.156840516 +0000, \"record\"=>{\"error\"=>\"#<NoMethodError: undefined method `[]' for nil:NilClass>\", \"location\"=>\"/fluentd/vendor/bundle/ruby/2.3.0/gems/fluent-plugin-elasticsearch-2.10.1/lib/fluent/plugin/out_elasticsearch_dynamic.rb:268:in `eval'\", \"tag\"=>\"fluent.warn\", \"time\"=>2018-09-08 11:02:50.153922217 +0000, \"record\"=>{\"error\"=>\"#<NoMethodError: undefined method `[]' for nil:NilClass>\", \"location\"=>\"/fluentd/vendor/bundle/ruby/2.3.0/gems/fluent-plugin-elasticsearch-2.10.1/lib/fluent/plugin/out_elasticsearch_dynamic.rb:268:in `eval'\", \"tag\"=>\"fluent.info\", \"time\"=>2018-09-08 11:02:45.151774166 +0000, \"record\"=>{\"worker\"=>0, \"message\"=>\"fluentd worker is now running worker=0\"}, \"message\"=>\"dump an error event: error_class=NoMethodError error=\\"undefined method `[]' for nil:NilClass\\" location=\\"/fluentd/vendor/bundle/ruby/2.3.0/gems/fluent-plugin-elasticsearch-2.10.1/lib/fluent/plugin/out_elasticsearch_dynamic.rb:268:in `eval'\\" tag=\\"fluent.info\\" time=2018-09-08 11:02:45.151774166 +0000 record={\\"worker\\"=>0, \\"message\\"=>\\"fluentd worker is now running worker=0\\"}\"}, \"message\"=>\"dump an error event: error_class=NoMethodError error=\\"undefined method `[]' for nil:NilClass\\" location=\\"/fluentd/vendor/bundle/ruby/2.3.0/gems/fluent-plugin-elasticsearch-2.10.1/lib/fluent/plugin/out_elasticsearch_dynamic.rb:268:in `eval'\\" tag=\\"fluent.warn\\" time=2018-09-08 11:02:50.153922217 +0000 record={\\"error\\"=>\\"#<NoMethodError: undefined method `[]' for nil:NilClass>\\", \\"location\\"=>\\"/fluentd/vendor/bundle/ruby/2.3.0/gems/fluent-plugin-elasticsearch-2.10.1/lib/fluent/plugin/out_elasticsearch_dynamic.rb:268:in `eval'\\", \\"tag\\"=>\\"fluent.info\\", \\"time\\"=>2018-09-08 11:02:45.151774166 +0000, \\"record\\"=>{\\"worker\\"=>0, \\"message\\"=>\\"fluentd worker is now running worker=0\\"}, \\"message\\"=>\\"dump an error event: error_class=NoMethodError error=\\\\"undefined method `[]' for nil:NilClass\\\\" location=\\\\"/fluentd/vendor/bundle/ruby/2.3.0/gems/fluent-plugin-elasticsearch-2.10.1/lib/fluent/plugin/out_elasticsearch_dynamic.rb:268:in `eval'\\\\" tag=\\\\"fluent.info\\\\" time=2018-09-08 11:02:45.151774166 +0000 record={\\\\"worker\\\\"=>0, \\\\"message\\\\"=>\\\\"fluentd worker is now running worker=0\\\\"}\\"}\"}, \"message\"=>\"dump an error event: error_class=NoMethodError error=\\"undefined method `[]' for nil:NilClass\\" location=\\"/fluentd/vendor/bundle/ruby/2.3.0/gems/fluent-plugin-elasticsearch-2.10.1/lib/fluent/plugin/out_elasticsearch_dynamic.rb:268:in `eval'\\" tag=\\"fluent.warn\\" time=2018-09-08 11:02:55.156840516 +0000 record={\\"error\\"=>\\"#<NoMethodError: undefined method `[]' for nil:NilClass>\\", \\"location\\"=>\\"/fluentd/vendor/bundle/ruby/2.3.0/gems/fluent-plugin-elasticsearch-2.10.1/lib/fluent/plugin/out_elasticsearch_dynamic.rb:268:in `eval'\\", \\"tag\\"=>\\"fluent.warn\\", \\"time\\"=>2018-09-08 11:02:50.153922217 +0000, \\"record\\"=>{\\"error\\"=>\\"#<NoMethodError: undefined method `[]' for nil:NilClass>\\", \\"location\\"=>\\"/fluentd/vendor/bundle/ruby/2.3.0/gems/fluent-plugin-elasticsearch-2.10.1/lib/fluent/plugin/out_elasticsearch_dynamic.rb:268:in `eval'\\", \\"tag\\"=>\\"fluent.info\\", \\"time\\"=>2018-09-08 11:02:45.151774166 +0000, \\"record\\"=>{\\"worker\\"=>0, \\"message\\"=>\\"fluentd worker is now running worker=0\\"}, \\"message\\"=>\\"dump an error event: error_class=NoMethodError error=\\\\"undefined method `[]' for nil:NilClass\\\\" location=\\\\"/fluentd/vendor/bundle/ruby/2.3.0/gems/fluent-plugin-elasticsearch-2.10.1/lib/fluent/plugin/out_elasticsearch_dynamic.rb:268:in `eval'\\\\" tag=\\\\"fluent.info\\\\" time=2018-09-08 11:02:45.151774166 +0000 record={\\\\"worker\\\\"=>0, \\\\"message\\\\"=>\\\\"fluentd worker is now running worker=0\\\\"}\\"}, \\"message\\"=>\\"dump an error event: error_class=NoMethodError error=\\\\"undefined method `[]' for nil:NilClass\\\\" location=\\\\"/fluentd/vendor/bundle/ruby/2.3.0/gems/fluent-plugin-elasticsearch-2.10.1/lib/fluent/plugin/out_elasticsearch_dynamic.rb:268:in `eval'\\\\" tag=\\\\"fluent.warn\\\\" time=2018-09-08 11:02:50.153922217 +0000 record={\\\\"error\\\\"=>\\\\"#<NoMethodError: undefined method `[]' for nil:NilClass>\\\\", \\\\"location\\\\"=>\\\\"/fluentd/vendor/bundle/ruby/2.3.0/gems/fluent-plugin-elasticsearch-2.10.1/lib/fluent/plugin/out_elasticsearch_dynamic.rb:268:in `eval'\\\\", \\\\"tag\\\\"=>\\\\"fluent.info\\\\", \\\\"time\\\\"=>2018-09-08 11:02:45.151774166 +0000, \\\\"record\\\\"=>{\\\\"worker\\\\"=>0, \\\\"message\\\\"=>\\\\"fluentd worker is now running worker=0\\\\"}, \\\\"message\\\\"=>\\\\"dump an error event: error_class=NoMethodError error=\\\\\\\\"undefined method `[]' for nil:NilClass\\\\\\\\" location=\\\\\\\\"/fluentd/vendor/bundle/ruby/2.3.0/gems/fluent-plugin-elasticsearch-2.10.1/lib/fluent/plugin/out_elasticsearch_dynamic.rb:268:in `eval'\\\\\\\\" tag=\\\\\\\\"fluent.info\\\\\\\\" time=2018-09-08 11:02:45.151774166 +0000 record={\\\\\\\\"worker\\\\\\\\"=>0, \\\\\\\\"message\\\\\\\\"=>\\\\\\\\"fluentd worker is now running worker=0\\\\\\\\"}\\\\"}\\"}\"}"}
fluentd-7przp fluentd 2018-09-08 11:03:24 +0000 [debug]: #0 fluent/log.rb:302:debug: preparing shutdown output plugin type=:elasticsearch_dynamic plugin_id="kubelet_out_es"
fluentd-7przp fluentd 2018-09-08 11:03:24 +0000 [info]: #0 fluent/log.rb:322:info: shutting down output plugin type=:elasticsearch_dynamic plugin_id="kubelet_out_es"
fluentd-dhwnx fluentd 2018-09-08 11:03:25 +0000 [warn]: #0 fluent/log.rb:342:warn: dump an error event: error_class=NoMethodError error="undefined method `[]' for nil:NilClass" location="/fluentd/vendor/bundle/ruby/2.3.0/gems/fluent-plugin-elasticsearch-2.10.1/lib/fluent/plugin/out_elasticsearch_dynamic.rb:268:in `eval'" tag="fluent.debug" time=2018-09-08 11:03:24.151685730 +0000 record={"message"=>"fluentd main process get SIGTERM"}
fluentd-sr764 fluentd-dhwnx fluentd-7przp fluentd fluentd 2018-09-08 11:03:25 +0000 [debug]: #0 fluent/log.rb:302:debug: calling terminate on filter plugin type=:parser plugin_id="myapp_filter"
fluentd 2018-09-08 11:03:25 +0000 [debug]: #0 fluent/log.rb:302:debug: calling terminate on filter plugin type=:parser plugin_id="myapp_filter"
fluentd-dhwnxfluentd-sr764 fluentd 2018-09-08 11:03:25 +0000 [info]: fluent/log.rb:322:info: Worker 0 finished with status 0
fluentd 2018-09-08 11:03:25 +0000 [info]: fluent/log.rb:322:info: Worker 0 finished with status 0
2018-09-08 11:03:25 +0000 [debug]: #0 fluent/log.rb:302:debug: calling terminate on output plugin type=:elasticsearch_dynamic plugin_id="kubelet_out_es"
fluentd-7przp fluentd 2018-09-08 11:03:25 +0000 [debug]: #0 fluent/log.rb:302:debug: calling terminate on output plugin type=:elasticsearch_dynamic plugin_id="apiserver_out_es"
fluentd-7przp fluentd 2018-09-08 11:03:25 +0000 [debug]: #0 fluent/log.rb:302:debug: calling terminate on output plugin type=:elasticsearch_dynamic plugin_id="out_es"
fluentd-7przp fluentd 2018-09-08 11:03:25 +0000 [debug]: #0 fluent/log.rb:302:debug: calling terminate on filter plugin type=:parser plugin_id="myapp_filter"
fluentd-7przp fluentd 2018-09-08 11:03:26 +0000 [info]: fluent/log.rb:322:info: Worker 0 finished with status 0
fluentd.conf:
@include systemd.conf
@include kubernetes.conf
# Start of fluent.conf
<filter kubernetes.var.log.containers.myapp-provider**.log>
@type parser
@id myapp_filter
key_name log
reserve_data true
remove_key_name_field true
<parse>
@type multiline
format_firstline /^[A-Z]/
format1 /^(?<level>[^ ]*)[ \t]+\[(?<time>[^\]]*)\] \[(?<thread>[^\]]*)\] \[(?<request>[^\]]*)\] (?<class>[^ ]*): (?<message>.*)$/
time_format %Y-%m-%d %H:%M:%S,%L %z
</parse>
</filter>
<match kubelet>
@type elasticsearch_dynamic
@id kubelet_out_es
log_level info
include_tag_key true
host "#{ENV['FLUENT_ELASTICSEARCH_HOST']}"
port "#{ENV['FLUENT_ELASTICSEARCH_PORT']}"
scheme "#{ENV['FLUENT_ELASTICSEARCH_SCHEME'] || 'http'}"
ssl_verify "#{ENV['FLUENT_ELASTICSEARCH_SSL_VERIFY'] || 'true'}"
#user "#{ENV['FLUENT_ELASTICSEARCH_USER']}"
#password "#{ENV['FLUENT_ELASTICSEARCH_PASSWORD']}"
reload_connections "#{ENV['FLUENT_ELASTICSEARCH_RELOAD_CONNECTIONS'] || 'true'}"
index_name fluentd-${tag_parts[0]+ "-" + Time.at(time).getlocal("+05:30").strftime(@logstash_dateformat)}
include_timestamp true
#logstash_prefix "#{ENV['FLUENT_ELASTICSEARCH_LOGSTASH_PREFIX'] || 'logstash'}"
#logstash_format true
<buffer>
flush_thread_count 8
flush_interval 5s
chunk_limit_size 2M
queue_limit_length 32
retry_max_interval 30
retry_forever true
</buffer>
</match>
<match kube-apiserver>
@type elasticsearch_dynamic
@id apiserver_out_es
log_level info
include_tag_key true
host "#{ENV['FLUENT_ELASTICSEARCH_HOST']}"
port "#{ENV['FLUENT_ELASTICSEARCH_PORT']}"
scheme "#{ENV['FLUENT_ELASTICSEARCH_SCHEME'] || 'http'}"
ssl_verify "#{ENV['FLUENT_ELASTICSEARCH_SSL_VERIFY'] || 'true'}"
#user "#{ENV['FLUENT_ELASTICSEARCH_USER']}"
#password "#{ENV['FLUENT_ELASTICSEARCH_PASSWORD']}"
reload_connections "#{ENV['FLUENT_ELASTICSEARCH_RELOAD_CONNECTIONS'] || 'true'}"
index_name fluentd-${tag_parts[0]+ "-" + Time.at(time).getlocal("+05:30").strftime(@logstash_dateformat)}
include_timestamp true
#logstash_prefix "#{ENV['FLUENT_ELASTICSEARCH_LOGSTASH_PREFIX'] || 'logstash'}"
#logstash_format true
<buffer>
flush_thread_count 8
flush_interval 5s
chunk_limit_size 2M
queue_limit_length 32
retry_max_interval 30
retry_forever true
</buffer>
</match>
<match **>
@type elasticsearch_dynamic
@id out_es
log_level info
include_tag_key true
host "#{ENV['FLUENT_ELASTICSEARCH_HOST']}"
port "#{ENV['FLUENT_ELASTICSEARCH_PORT']}"
scheme "#{ENV['FLUENT_ELASTICSEARCH_SCHEME'] || 'http'}"
ssl_verify "#{ENV['FLUENT_ELASTICSEARCH_SSL_VERIFY'] || 'true'}"
#user "#{ENV['FLUENT_ELASTICSEARCH_USER']}"
#password "#{ENV['FLUENT_ELASTICSEARCH_PASSWORD']}"
reload_connections "#{ENV['FLUENT_ELASTICSEARCH_RELOAD_CONNECTIONS'] || 'true'}"
index_name fluentd-${record['kubernetes']['container_name']}-${Time.at(time).getlocal("+05:30").strftime(@logstash_dateformat)}
include_timestamp true
#logstash_prefix "#{ENV['FLUENT_ELASTICSEARCH_LOGSTASH_PREFIX'] || 'logstash'}"
#logstash_format true
<buffer>
flush_thread_count 8
flush_interval 5s
chunk_limit_size 2M
queue_limit_length 32
retry_max_interval 30
retry_forever true
</buffer>
</match>
谁能帮忙调试一下。提前致谢！
我将从这个命令开始，以检查 pods 频繁重启的情况：
kubectl get pods --all-namespaces
然后 运行 下面的命令是为了收集更多关于容器最后已知状态的信息以及相关的 exit code. Docker exit codes are based on the chroot standard。
kubectl describe pods [podname]
执行 "kubectl describe" 应该可以为您提供有关 pods 终止原因的更多信息。
这可能是此处提到的 kubernetes 错误 - https://github.com/kubernetes/kubernetes/issues/51376
尽管此问题目前已关闭，但我建议您在 default 命名空间中尝试 运行 相同的 daemonset。
即使我在 kube-system 命名空间中尝试了 运行 但它失败了并且在 default 命名空间中成功 运行。