我们如何在 filebeat kubernetes 中过滤名称空间?
How we can filter namespace in filebeat kubernetes?
我正在设置管道以将 kubernetes pods 日志发送到弹性集群。我在我的集群中将 filebeat 作为 deamonset(stream:stdout)安装并将输出连接到 logstash。 Beats 与 logstash 连接没有问题,现在我想要来自应用程序命名空间的日志,而不是来自集群中所有命名空间的日志。有人可以指导我如何在 beat 中过滤此信息,以及如何在 es 中查看来自 json 的源消息?
这是我的配置:
data:
kubernetes.yml: |-
- type: docker
containers:
path: "/var/lib/docker/containers"
stream: "stdout"
ids: "*"
multiline.pattern: '^\s'
multiline.match: after
fields:
logtype: container
multiline:
pattern: '^[0-9]{4}-[0-9]{2}-[0-9]{2}'
negate: true
match: after
ignore_older: 1h
processors:
- add_kubernetes_metadata:
in_cluster: true
- decode_json_fields:
fields: ["log"]
overwrite_keys: true
target: ""
kibana 中的输出:
{
"_index": "filebeat-6.8.4-2020.03.06",
"_type": "doc",
"_id": "vHkzsHABJ57Tsdxxxxx",
"_version": 1,
"_score": null,
"_source": {
"log": {
"file": {
"path": "/var/lib/docker/containers/aa54562be9448183d69d8d2e1953e74560309176f044aed23484ac9e3260982c/sdnksdsdlsdnfsdlfslfnsdslfnsnlnflksdnflkdsfnsdflsdfndslffndslf-json.log"
}
},
"tags": [
"beats_input_codec_plain_applied",
"_grokparsefailure"
],
"input": {
"type": "docker"
},
"@version": "1",
"prospector": {
"type": "docker"
},
"beat": {
"version": "6.8.4",
"name": "filebeat-vtp2f",
"hostname": "filebeat-vtp2f"
},
"host": {
"name": "filebeat-vtp2f"
},
"offset": 5798785,
"stream": "stdout",
"fields": {
"logtype": "container"
},
"kubernetes": {
"node": {
"name": "k8-test-22313607-0"
},
"labels": {
"version": "v1",
"kubernetes": {
"io/cluster-service": "true"
},
"controller-revision-hash": "6b56cfcb69",
"pod-template-generation": "1",
"k8s-app": "fluent"
},
"container": {
"name": "fluentd"
},
"pod": {
"uid": "72c50b54-5ef0-11ea-83e1-26018882335d",
"name": "fluent-4lft2"
},
"namespace": "fluentd"
},
"source": "/var/lib/docker/containers/aa54562be9448183d69d8d2e1953e74560309176f044aed23484ac9e3260982c/aa54562be9448183d69d8d2e1953e74560309176f044aed23484ac9e3260982c-json.log",
"@timestamp": "2020-03-06T14:15:18.561Z"
},
"fields": {
"@timestamp": [
"2020-03-06T14:15:18.561Z"
]
},
"highlight": {
"prospector.type": [
"@kibana-highlighted-field@docker@/kibana-highlighted-field@"
]
},
"sort": [
1583504118561
]
}
我不知道如何过滤 filebeat(或者即使可能),但您可以使用 conditionals 在 logstash 配置的输出部分过滤字段:
output {
if [kubernetes][namespace] == "fluentd" {
...
Send to Elasticsearch
...
} else {
...
}
}
通过这种方式,您可以根据 kubernetes.namespace 字段的值选择对每封邮件执行不同的操作。
如果您希望 Filebeat 仅从您使用条件的特定名称空间中获取日志:
filebeat.yml
:
logging.level: error
logging.json: true
filebeat.config:
inputs:
# Mounted `filebeat-inputs` configmap:
path: ${path.config}/inputs.d/*.yml
# Reload inputs configs as they change:
reload.enabled: false
modules:
path: ${path.config}/modules.d/*.yml
# Reload module configs as they change:
reload.enabled: false
filebeat.autodiscover:
providers:
- type: kubernetes
templates:
- condition:
equals:
kubernetes.namespace: stage
config:
- type: container
paths:
- /var/log/containers/*${data.kubernetes.container.id}.log
multiline.pattern: '^[[:space:]]'
multiline.negate: false
multiline.match: after
include_lines: ['^{']
注意这部分:
templates:
- condition:
equals:
kubernetes.namespace: stage
我 运行 在每个命名空间中将 Filebeat 作为 Daemonset。这有点额外的开销,但 Filebeat 可能很挑剔,所以它确实帮助我们首先解决其他逻辑环境中的问题。
如何删除一些名称空间,我在此处记录:https://ezyforanykey.blogspot.com/2020/11/filebeat-exclude-kubernetes-namespace.html
示例如下:
- type: container
paths:
- /var/log/containers/*.log
exclude_files:
- /var/log/containers/java.*
processors:
- add_kubernetes_metadata:
host: ${NODE_NAME}
matchers:
- logs_path:
logs_path: "/var/log/containers/"
- drop_event.when:
or:
- equals:
kubernetes.namespace: "kube-system"
- equals:
kubernetes.namespace: "calico-system"
我正在设置管道以将 kubernetes pods 日志发送到弹性集群。我在我的集群中将 filebeat 作为 deamonset(stream:stdout)安装并将输出连接到 logstash。 Beats 与 logstash 连接没有问题,现在我想要来自应用程序命名空间的日志,而不是来自集群中所有命名空间的日志。有人可以指导我如何在 beat 中过滤此信息,以及如何在 es 中查看来自 json 的源消息?
这是我的配置:
data:
kubernetes.yml: |-
- type: docker
containers:
path: "/var/lib/docker/containers"
stream: "stdout"
ids: "*"
multiline.pattern: '^\s'
multiline.match: after
fields:
logtype: container
multiline:
pattern: '^[0-9]{4}-[0-9]{2}-[0-9]{2}'
negate: true
match: after
ignore_older: 1h
processors:
- add_kubernetes_metadata:
in_cluster: true
- decode_json_fields:
fields: ["log"]
overwrite_keys: true
target: ""
kibana 中的输出:
{
"_index": "filebeat-6.8.4-2020.03.06",
"_type": "doc",
"_id": "vHkzsHABJ57Tsdxxxxx",
"_version": 1,
"_score": null,
"_source": {
"log": {
"file": {
"path": "/var/lib/docker/containers/aa54562be9448183d69d8d2e1953e74560309176f044aed23484ac9e3260982c/sdnksdsdlsdnfsdlfslfnsdslfnsnlnflksdnflkdsfnsdflsdfndslffndslf-json.log"
}
},
"tags": [
"beats_input_codec_plain_applied",
"_grokparsefailure"
],
"input": {
"type": "docker"
},
"@version": "1",
"prospector": {
"type": "docker"
},
"beat": {
"version": "6.8.4",
"name": "filebeat-vtp2f",
"hostname": "filebeat-vtp2f"
},
"host": {
"name": "filebeat-vtp2f"
},
"offset": 5798785,
"stream": "stdout",
"fields": {
"logtype": "container"
},
"kubernetes": {
"node": {
"name": "k8-test-22313607-0"
},
"labels": {
"version": "v1",
"kubernetes": {
"io/cluster-service": "true"
},
"controller-revision-hash": "6b56cfcb69",
"pod-template-generation": "1",
"k8s-app": "fluent"
},
"container": {
"name": "fluentd"
},
"pod": {
"uid": "72c50b54-5ef0-11ea-83e1-26018882335d",
"name": "fluent-4lft2"
},
"namespace": "fluentd"
},
"source": "/var/lib/docker/containers/aa54562be9448183d69d8d2e1953e74560309176f044aed23484ac9e3260982c/aa54562be9448183d69d8d2e1953e74560309176f044aed23484ac9e3260982c-json.log",
"@timestamp": "2020-03-06T14:15:18.561Z"
},
"fields": {
"@timestamp": [
"2020-03-06T14:15:18.561Z"
]
},
"highlight": {
"prospector.type": [
"@kibana-highlighted-field@docker@/kibana-highlighted-field@"
]
},
"sort": [
1583504118561
]
}
我不知道如何过滤 filebeat(或者即使可能),但您可以使用 conditionals 在 logstash 配置的输出部分过滤字段:
output {
if [kubernetes][namespace] == "fluentd" {
...
Send to Elasticsearch
...
} else {
...
}
}
通过这种方式,您可以根据 kubernetes.namespace 字段的值选择对每封邮件执行不同的操作。
如果您希望 Filebeat 仅从您使用条件的特定名称空间中获取日志:
filebeat.yml
:
logging.level: error
logging.json: true
filebeat.config:
inputs:
# Mounted `filebeat-inputs` configmap:
path: ${path.config}/inputs.d/*.yml
# Reload inputs configs as they change:
reload.enabled: false
modules:
path: ${path.config}/modules.d/*.yml
# Reload module configs as they change:
reload.enabled: false
filebeat.autodiscover:
providers:
- type: kubernetes
templates:
- condition:
equals:
kubernetes.namespace: stage
config:
- type: container
paths:
- /var/log/containers/*${data.kubernetes.container.id}.log
multiline.pattern: '^[[:space:]]'
multiline.negate: false
multiline.match: after
include_lines: ['^{']
注意这部分:
templates:
- condition:
equals:
kubernetes.namespace: stage
我 运行 在每个命名空间中将 Filebeat 作为 Daemonset。这有点额外的开销,但 Filebeat 可能很挑剔,所以它确实帮助我们首先解决其他逻辑环境中的问题。
如何删除一些名称空间,我在此处记录:https://ezyforanykey.blogspot.com/2020/11/filebeat-exclude-kubernetes-namespace.html
示例如下:
- type: container
paths:
- /var/log/containers/*.log
exclude_files:
- /var/log/containers/java.*
processors:
- add_kubernetes_metadata:
host: ${NODE_NAME}
matchers:
- logs_path:
logs_path: "/var/log/containers/"
- drop_event.when:
or:
- equals:
kubernetes.namespace: "kube-system"
- equals:
kubernetes.namespace: "calico-system"