重命名 json 输出中的子项

Question

是否可以重命名 JSON 中的子项或添加新的子项？

例如，我有这个日志输出：

{
    "kubernetes": {
        "pod_name": "kube-apiserver-tst",
        "namespace_name": "kube-system",
        "pod_id": "93a2b43a-46e6-4539-8674-06dede2619fa",
        "labels": {
            "component": "kube-apiserver",
            "tier": "control-plane"
        }
}

我知道 record_transformer 我可以添加一个新密钥：

<record>
  pod_labels "something ..."
</record>

但是好像只能在JSON的根目录下新建key:

{
    "kubernetes": {
        "pod_name": "kube-apiserver-tst",
        "namespace_name": "kube-system",
        "pod_id": "93a2b43a-46e6-4539-8674-06dede2619fa",
        "labels": {
            "component": "kube-apiserver",
            "tier": "control-plane"
        }
    "pod_labels": "something ..."
}

但是我可以让它看起来像这样吗？

{
    "kubernetes": {
        "pod_name": "kube-apiserver-tst",
        "namespace_name": "kube-system",
        "pod_id": "93a2b43a-46e6-4539-8674-06dede2619fa",
        "labels": {
            "component": "kube-apiserver",
            "tier": "control-plane"
        }
        "pod_labels": {
            "component": "kube-apiserver",
            "tier": "control-plane"
        }
}

或者这个：

{
    "kubernetes": {
        "pod_name": "kube-apiserver-tst",
        "namespace_name": "kube-system",
        "pod_id": "93a2b43a-46e6-4539-8674-06dede2619fa",
        "pod_labels": {
            "component": "kube-apiserver",
            "tier": "control-plane"
        }
}

Answer 1

JSON 无效。缺少右大括号。

这是有效的 JSON:

{
    "kubernetes": {
        "pod_name": "kube-apiserver-tst",
        "namespace_name": "kube-system",
        "pod_id": "93a2b43a-46e6-4539-8674-06dede2619fa",
        "labels": {
            "component": "kube-apiserver",
            "tier": "control-plane"
        }
    }
}

缩小 JSON (echo '{JSON}' | jq -c .):

{"kubernetes":{"pod_name":"kube-apiserver-tst","namespace_name":"kube-system","pod_id":"93a2b43a-46e6-4539-8674-06dede2619fa","labels":{"component":"kube-apiserver","tier":"control-plane"}}}

record_transformer filter plugin can be used with Ruby support (via enable_ruby option) to manipulate an existing key and then the unwanted keys can be removed with its remove_keys 选项。

这是示例配置：

<filter debug.test>
  @type         record_transformer
  enable_ruby   true
  <record>
    temp        ${ l = record["kubernetes"]["labels"]; record["kubernetes"]["pod_labels"] = l; nil; }
  </record>
  remove_keys   temp, $.kubernetes.labels
</filter>

这是完整的测试：

fluent.conf

<source>
  @type       forward
</source>

<filter debug.test>
  @type         record_transformer
  enable_ruby   true
  <record>
    temp        ${ l = record["kubernetes"]["labels"]; record["kubernetes"]["pod_labels"] = l; nil; }
  </record>
  remove_keys   temp, $.kubernetes.labels
</filter>

<match debug.test>
  @type       stdout
</match>

使用此配置启动 fluentd：

fluentd -c fluent.conf

在另一个终端上，发送带有 fluent-cat (echo '{JSON}' | fluent-cat debug.test) 的事件：

echo '{"kubernetes":{"pod_name":"kube-apiserver-tst","namespace_name":"kube-system","pod_id":"93a2b43a-46e6-4539-8674-06dede2619fa","labels":{"component":"kube-apiserver","tier":"control-plane"}}}' | fluent-cat debug.test

在 fluentd 日志中，您应该会看到所需的输出：

2022-02-16 23:08:25.919967225 +0500 debug.test: {"kubernetes":{"pod_name":"kube-apiserver-tst","namespace_name":"kube-system","pod_id":"93a2b43a-46e6-4539-8674-06dede2619fa","pod_labels":{"component":"kube-apiserver","tier":"control-plane"}}}

带 jq (echo '{JSON}' | jq .) 的格式化输出：

echo '{"kubernetes":{"pod_name":"kube-apiserver-tst","namespace_name":"kube-system","pod_id":"93a2b43a-46e6-4539-8674-06dede2619fa","pod_labels":{"component":"kube-apiserver","tier":"control-plane"}}}' | jq .

输出：

{
  "kubernetes": {
    "pod_name": "kube-apiserver-tst",
    "namespace_name": "kube-system",
    "pod_id": "93a2b43a-46e6-4539-8674-06dede2619fa",
    "pod_labels": {
      "component": "kube-apiserver",
      "tier": "control-plane"
    }
  }
}

重命名 json 输出中的子项

Rename subkey in json output

fluentd