JOLT 将子数组移动到父对象并修改内容

Question

我正在尝试将两个子数组的内容移动到它们在 JOLT 中的父对象中，以便在 NiFi 中使用。输入 JSON如下：

{
  "url": "RETURNED URL",
  "repository_url": "RETURNED URL",
  "labels_url": "RETURNED URL",
  "comments_url": "RETURNED URL",
  "events_url": "RETURNED URL",
  "html_url": "RETURNED URL",
  "id": RETURNED_ID,
  "node_id": "RETURNED id",
  "number": 10,
  "title": RETURNED TITLE,
  "user": {
    "login": "xxxx",
    "id": xxxx,
    "node_id": "xxxx",
    "avatar_url": "RETURNED URL",
    "gravatar_id": "",
    "url": "RETURNED URL",
    "html_url": "RETURNED URL",
    "followers_url": "RETURNED URL",
    "following_url": "RETURNED URL",
    "gists_url": "RETURNED URL",
    "starred_url": "RETURNED URL",
    "subscriptions_url": "RETURNED URL",
    "organizations_url": "RETURNED URL",
    "repos_url": "RETURNED URL",
    "events_url": "RETURNED URL",
    "received_events_url": "RETURNED URL",
    "type": "xxxx",
    "site_admin": false
  },
  "labels": [
    {
      "id": 123456,
      "node_id": "RETURNED URL",
      "url": "RETURNED URL",
      "name": xxxxx,
      "color": "53516b",
      "default": false,
      "description": "xxxxx"
    },
    {
      "id": 2345678,
      "node_id": "RETURNED URL",
      "url": "RETURNED URL",
      "name": "xxxxx",
      "color": "B5D1D9",
      "default": false,
      "description": "xxxxxx"
    },
   ...
  ],
  "state": "xxxx",
  "locked": false,
  "assignee": {
    "login": "xxxx",
    "id": 12345,
    "node_id": "RETURNED URL",
    "avatar_url": "RETURNED URL",
    "gravatar_id": "",
    "url": "RETURNED URL",
    "html_url": "RETURNED URL",
    "followers_url": "RETURNED URL",
    "following_url": "RETURNED URL",
    "gists_url": "RETURNED URL",
    "starred_url": "RETURNED URL",
    "subscriptions_url": "RETURNED URL",
    "organizations_url": "RETURNED URL",
    "repos_url": "RETURNED URL",
    "events_url": "RETURNED URL",
    "received_events_url": "RETURNED URL",
    "type": "xxxx",
    "site_admin": false
  },
  "assignees": [
    {
    "login": "xxxx",
    "id": 12345,
    "node_id": "RETURNED URL",
    "avatar_url": "RETURNED URL",
    "gravatar_id": "",
    "url": "RETURNED URL",
    "html_url": "RETURNED URL",
    "followers_url": "RETURNED URL",
    "following_url": "RETURNED URL",
    "gists_url": "RETURNED URL",
    "starred_url": "RETURNED URL",
    "subscriptions_url": "RETURNED URL",
    "organizations_url": "RETURNED URL",
    "repos_url": "RETURNED URL",
    "events_url": "RETURNED URL",
    "received_events_url": "RETURNED URL",
    "type": "xxxx",
    "site_admin": false
  },
    {
    "login": "xxxx",
    "id": 35678,
    "node_id": "RETURNED URL",
    "avatar_url": "RETURNED URL",
    "gravatar_id": "",
    "url": "RETURNED URL",
    "html_url": "RETURNED URL",
    "followers_url": "RETURNED URL",
    "following_url": "RETURNED URL",
    "gists_url": "RETURNED URL",
    "starred_url": "RETURNED URL",
    "subscriptions_url": "RETURNED URL",
    "organizations_url": "RETURNED URL",
    "repos_url": "RETURNED URL",
    "events_url": "RETURNED URL",
    "received_events_url": "RETURNED URL",
    "type": "xxxx",
    "site_admin": false
  },
{
    "login": "xxxx",
    "id": 5785456,
    "node_id": "RETURNED URL",
    "avatar_url": "RETURNED URL",
    "gravatar_id": "",
    "url": "RETURNED URL",
    "html_url": "RETURNED URL",
    "followers_url": "RETURNED URL",
    "following_url": "RETURNED URL",
    "gists_url": "RETURNED URL",
    "starred_url": "RETURNED URL",
    "subscriptions_url": "RETURNED URL",
    "organizations_url": "RETURNED URL",
    "repos_url": "RETURNED URL",
    "events_url": "RETURNED URL",
    "received_events_url": "RETURNED URL",
    "type": "xxxx",
    "site_admin": false
  },
 ...
  ],
  "milestone": null,
  "comments": 1,
  "created_at": "xxxxx",
  "updated_at": "xxxxx",
  "closed_at": null,
  "author_association": "xxxx",
  "active_lock_reason": null,
  "body": "RETURNED_TEXT",
  "performed_via_github_app": null
}

我想要的输出:

{
    "id": RETURNED_ID,
    "number": 10,
    "title": RETURNED TITLE,
    "creator": "xxxx",
    "creator_id": "xxxx",
    "creator_admin_stat": false,
    "number_of_comments": 1,
    "created_at": "xxxxx",
    "updated_at": "xxxxx",
    "closed_at": null,
    "issue_body": "xxxx",
    "label_1": {
        "id": 123456
            "name": "xxxxx",
        "description": "xxxxx"
    },
    "label_2": {
        "id": 2345678,
        "name": "xxxxx",
        "description": "xxxxxx"
    },
    // (rest of labels at this same level)
    "user_1": {
        "user_login": "xxxx",
        "user_id": 12345,
        "user_admin_stat": false
    },
    "user_2": {
        "user_login": "xxxx",
        "user_id": 35678,
        "user_admin_stat": false
    },
    "user_3": {
        "user_login": "xxxx",
        "user_id": 578546,
        "user_admin_stat": false
    }
    // (rest of users at this same level)
}

规格我目前有：

[
  {
    "operation": "shift",
    "spec": {
      "id": "repo_id",
      "number": "repo_number",
      "title": "repo_title",
      "user": {
        "login": "creator",
        "id": "creator_id",
        "site_admin": "creator_admin_stat"
      },
      "comments": "number_of_comments",
      "created_at": "created_at",
      "updated_at": "updated_at",
      "closed_at": "closed_at",
      "body": "issue_body",
      "labels": {
        "*": {
          "id": "label_ids",
          "name": "label_names"
        }
      },
      "assignees": {
        "*": {
          "id": "user_ids",
          "login": "usernames"
        }
      }
    }
    }

]

我的当前输出：

{
  "repo_id" : 4366853,
  "repo_number" : 10,
  "repo_title" : "xxxx",
  "repo_creator" : "xxxx",
  "repo_creator_id" : 12345,
  "repo_creator_admin_stat" : false,
  "number_of_comments" : 1,
  "created_at" : "xxxx",
  "updated_at" : "xxxx",
  "closed_at" : null,
  "issue_body" : "xxxx",
  "label_ids" : [ // list of label ids],
  "label_names" : [ // list of label names],
  "user_ids" : [ // list of user id numbers ],
  "usernames" : [ // list of usernames ],
  "user_admin_stat" : [ // list of user admin stats ]
}

所有输入 JSON 都至少有一个标签，但有些可能根本没有任何受让人。我构建了一个流来添加空用户数据（如果它不存在），因此我可以将该部分放入该流中，但我的主要 objective 试图将标签和用户放入他们自己的子对象中在数组中。

如果可能的话，我还想增加 label/user ids/names，例如“label_（此处的标签计数）”。

这些都将转到 Kibana，据我所知，Kibana 不能很好地（或根本不？）使用值列表，因此我在这里挣扎。

Answer 1

您可以使用以下规范，除了一些缺失或不符合要求的键值对，并从 zero[=16 开始索引数组 labels 和 assignees =]

[
  {
    "operation": "shift",
    "spec": {
      "id": "&",
      "number": "&",
      "title": "&",
      "created_at": "&",
      "labels": {
        "*": {
          "id": "&2_&1.&",
          "name": "&2_&1.&",
          "description": "&2_&1.&"
        }
      },
      "assignees": {
        "*": {
          "login": "user_&1.user_&",
          "id": "user_&1.user_&",
          "site_admin": "user_&1.user_admin_stat"
        }
      }
    }
  }
]

产量输出

{
  "id" : "RETURNED_ID",
  "number" : 10,
  "title" : "RETURNED TITLE",
  "created_at" : "xxxxx",
  "labels_0" : {
    "id" : 123456,
    "name" : "xxxxx",
    "description" : "xxxxx"
  },
  "labels_1" : {
    "id" : 2345678,
    "name" : "xxxxx",
    "description" : "xxxxxx"
  },
  "user_0" : {
    "user_login" : "xxxx",
    "user_id" : 12345,
    "user_admin_stat" : false
  },
  "user_1" : {
    "user_login" : "xxxx",
    "user_id" : 35678,
    "user_admin_stat" : false
  },
  "user_2" : {
    "user_login" : "xxxx",
    "user_id" : 5785456,
    "user_admin_stat" : false
  }
}

JOLT 将子数组移动到父对象并修改内容

JOLT move subarray to parent object and modify contents

json

kibana

jolt

apache-nifi