jq - 将嵌套的 JSON 提取到多个数组中

Question

我在这个 post 的底部有一些（编辑过的）JSON，这是 AWS 中运行 ec2-describe-instances 的结果。示例 JSON 有 2 个数组，但是这可能更大。我以前使用过 jq，但我无法提取深深嵌套在 JSON 中的值。我特别感兴趣的是：

InstanceId ["Reservations": [ "Instances": [ "InstanceId":
设备名称["Reservations":["Instances":["BlockDeviceMappings":["DeviceName":
VolumeId ["Reservations": [ "Instances": [ "BlockDeviceMappings": [ "Ebs": { "VolumeId":
标签 1 Value/Key
标签 2 Value/Key

总结一下，我想要一个可以迭代的 JSON 数组，它会给我一个 InstanceID、该实例上的标签和一个单个卷。您会注意到每个实例都附加了 3 个卷（可能更多，也可能更少）。我希望能够单独迭代每一个。我希望最终结果是这样的：

[
{
"InstanceId": "i-11111111",
"DeviceName": "/dev/sda1",
"VolumeId": "vol-1111111a",
"TAG1": "VALUE1",
"TAG2": "VALUE2"
},
{
"InstanceId": "i-11111111",
"DeviceName": "xvdf",
"VolumeId": "vol-1111111b",
"TAG1": "VALUE1",
"TAG2": "VALUE2"
},
{
"InstanceId": "i-11111111",
"DeviceName": "xvdg",
"VolumeId": "vol-1111111c",
"TAG1": "VALUE1",
"TAG2": "VALUE2"
},
{
"InstanceId": "i-22222222",
"DeviceName": "/dev/sda1",
"VolumeId": "vol-2222222a",
"TAG1": "VALUE1",
"TAG2": "VALUE2"
},
{
"InstanceId": "i-22222222",
"DeviceName": "/dev/sdb",
"VolumeId": "vol-2222222b",
"TAG1": "VALUE1",
"TAG2": "VALUE2"
},
{
"InstanceId": "i-22222222",
"DeviceName": "/dev/sdc",
"VolumeId": "vol-2222222c",
"TAG1": "VALUE1",
"TAG2": "VALUE2"
}
]

这与我 post 编辑的另一个问题类似。我尝试了很多不同的组合，但无法让它发挥作用。非常感谢任何帮助。

这是JSON：

{
  "Reservations": [
    {
      "OwnerId": "xx",
      "ReservationId": "xx",
      "Groups": [],
      "Instances": [
        {
          "Monitoring": {
            "State": "xx"
          },
          "PublicDnsName": "",
          "Platform": "xx",
          "State": {
            "Code": xx,
            "Name": "xx"
          },
          "EbsOptimized": xx,
          "LaunchTime": "xx",
          "PrivateIpAddress": "xx",
          "ProductCodes": [],
          "VpcId": "xx",
          "StateTransitionReason": "",
          "InstanceId": "i-11111111",
          "ImageId": "xx",
          "PrivateDnsName": "xx",
          "KeyName": "xx",
          "SecurityGroups": [
            {
              "GroupName": "xx",
              "GroupId": "xx"
            },
            {
              "GroupName": "xx",
              "GroupId": "xx"
            }
          ],
          "ClientToken": "xx",
          "SubnetId": "xx",
          "InstanceType": "xx",
          "NetworkInterfaces": [
            {
              "Status": "xx",
              "MacAddress": "xx",
              "SourceDestCheck": xx,
              "VpcId": "xx",
              "Description": "xx",
              "NetworkInterfaceId": "xx",
              "PrivateIpAddresses": [
                {
                  "PrivateDnsName": "xx",
                  "Primary": xx,
                  "PrivateIpAddress": "xx"
                }
              ],
              "PrivateDnsName": "xx",
              "Attachment": {
                "Status": "xx",
                "DeviceIndex": xx,
                "DeleteOnTermination": xx,
                "AttachmentId": "xx",
                "AttachTime": "xx"
              },
              "Groups": [
                {
                  "GroupName": "xx",
                  "GroupId": "xx"
                },
                {
                  "GroupName": "xx",
                  "GroupId": "xx"
                }
              ],
              "SubnetId": "xx",
              "OwnerId": "xx",
              "PrivateIpAddress": "xx"
            }
          ],
          "SourceDestCheck": xx,
          "Placement": {
            "Tenancy": "xx",
            "GroupName": "xx",
            "AvailabilityZone": "xx"
          },
          "Hypervisor": "xx",
          "BlockDeviceMappings": [
            {
              "DeviceName": "/dev/sda1",
              "Ebs": {
                "Status": "attached",
                "DeleteOnTermination": xx,
                "VolumeId": "vol-1111111a",
                "AttachTime": "xx"
              }
            },
            {
              "DeviceName": "xvdf",
              "Ebs": {
                "Status": "attached",
                "DeleteOnTermination": xx,
                "VolumeId": "vol-1111111b",
                "AttachTime": "xx"
              }
            },
            {
              "DeviceName": "xvdg",
              "Ebs": {
                "Status": "attached",
                "DeleteOnTermination": xx,
                "VolumeId": "vol-11111111c",
                "AttachTime": "xx"
              }
            }
          ],
          "Architecture": "xx",
          "RootDeviceType": "xx",
          "RootDeviceName": "xx",
          "VirtualizationType": "hvm",
          "Tags": [
            {
              "Value": "TAG1",
              "Key": "VALUE1"
            },
            {
              "Value": "TAG2",
              "Key": "VALUE2"
            },
            {
              "Value": "TAG3",
              "Key": "VALUE3"
            },
            {
              "Value": "TAG4",
              "Key": "VALUE4"
            },
            {
              "Value": "TAG5",
              "Key": "VALUE5"
            },
            {
              "Value": "TAG6",
              "Key": "Value6"
            }
          ],
          "AmiLaunchIndex": xx
        }
      ]
    },
    {
      "OwnerId": "xx",
      "ReservationId": "xx",
      "Groups": [],
      "Instances": [
        {
          "Monitoring": {
            "State": "xx"
          },
          "PublicDnsName": "",
          "Platform": "xx",
          "State": {
            "Code": xx,
            "Name": "xx"
          },
          "EbsOptimized": xx,
          "LaunchTime": "xx",
          "PrivateIpAddress": "xx",
          "ProductCodes": [],
          "VpcId": "xx",
          "StateTransitionReason": "",
          "InstanceId": "i-22222222",
          "ImageId": "xx",
          "PrivateDnsName": "xx",
          "KeyName": "xx",
          "SecurityGroups": [
            {
              "GroupName": "xx",
              "GroupId": "xx"
            },
            {
              "GroupName": "xx",
              "GroupId": "xx"
            }
          ],
          "ClientToken": "xx",
          "SubnetId": "xx",
          "InstanceType": "xx",
          "NetworkInterfaces": [
            {
              "Status": "xx",
              "MacAddress": "xx",
              "SourceDestCheck": xx,
              "VpcId": "xx",
              "Description": "xx",
              "NetworkInterfaceId": "xx",
              "PrivateIpAddresses": [
                {
                  "PrivateDnsName": "xx",
                  "Primary": xx,
                  "PrivateIpAddress": "xx"
                }
              ],
              "PrivateDnsName": "xx",
              "Attachment": {
                "Status": "xx",
                "DeviceIndex": xx,
                "DeleteOnTermination": xx,
                "AttachmentId": "xx",
                "AttachTime": "xx"
              },
              "Groups": [
                {
                  "GroupName": "xx",
                  "GroupId": "xx"
                },
                {
                  "GroupName": "xx",
                  "GroupId": "xx"
                }
              ],
              "SubnetId": "xx",
              "OwnerId": "xx",
              "PrivateIpAddress": "xx"
            }
          ],
          "SourceDestCheck": xx,
          "Placement": {
            "Tenancy": "xx",
            "GroupName": "xx",
            "AvailabilityZone": "xx"
          },
          "Hypervisor": "xx",
          "BlockDeviceMappings": [
            {
              "DeviceName": "/dev/sda1",
              "Ebs": {
                "Status": "attached",
                "DeleteOnTermination": xx,
                "VolumeId": "vol-2222222a",
                "AttachTime": "xx"
              }
            },
            {
              "DeviceName": "/dev/sdb",
              "Ebs": {
                "Status": "attached",
                "DeleteOnTermination": xx,
                "VolumeId": "vol-2222222b",
                "AttachTime": "xx"
              }
            },
            {
              "DeviceName": "/dev/sdc",
              "Ebs": {
                "Status": "attached",
                "DeleteOnTermination": xx,
                "VolumeId": "vol-2222222c",
                "AttachTime": "xx"
              }
            }
          ],
          "Architecture": "xx",
          "RootDeviceType": "xx",
          "RootDeviceName": "xx",
          "VirtualizationType": "hvm",
          "Tags": [
            {
              "Value": "TAG1",
              "Key": "VALUE1"
            },
            {
              "Value": "TAG2",
              "Key": "VALUE2"
            },
            {
              "Value": "TAG3",
              "Key": "VALUE3"
            },
            {
              "Value": "TAG4",
              "Key": "VALUE4"
            },
            {
              "Value": "TAG5",
              "Key": "VALUE5"
            },
            {
              "Value": "TAG6",
              "Key": "Value6"
            }
          ],
          "AmiLaunchIndex": xx
        }
      ]
    }
]

编辑：我尝试过的一个例子（为了简单起见，这个不是试图获取标签：

jq -r '.Reservations.Instances | map(({ InstanceId } + (.BlockDeviceMappings | add) + (.BlockDeviceMappings.Ebs | Add))| { InstanceId, DeviceName, VolumeId })' <(echo "$json_array_windows")

jq: error: Add/0 is not defined at <top-level>, line 1:
.Reservations.Instances | map(({ InstanceId } + (.BlockDeviceMappings | add) + (.BlockDeviceMappings.Ebs | Add))| { InstanceId, DeviceName, VolumeId })                                      
jq: 1 compile error

Answer 1

以下 jq 过滤器根据请求生成 JSON 个对象流。如果您确实需要 JSON 数组，则只需将整个表达式括在方括号中即可。

.Reservations[] | .Instances[]
| { InstanceId }
  + (.BlockDeviceMappings[]
     | { DeviceName,
         "VolumeId": .Ebs.VolumeId } )
  + (.Tags
     | { "TAG1": ( map( select(.Value == "TAG1"))[] | .Key),
         "TAG2": ( map( select(.Value == "TAG2"))[] | .Key) } )

这是输出（一旦输入 JSON 已更正）：

{
  "InstanceId": "i-11111111",
  "DeviceName": "/dev/sda1",
  "VolumeId": "vol-1111111a",
  "TAG1": "VALUE1",
  "TAG2": "VALUE2"
}
{
  "InstanceId": "i-11111111",
  "DeviceName": "xvdf",
  "VolumeId": "vol-1111111b",
  "TAG1": "VALUE1",
  "TAG2": "VALUE2"
}
{
  "InstanceId": "i-11111111",
  "DeviceName": "xvdg",
  "VolumeId": "vol-11111111c",
  "TAG1": "VALUE1",
  "TAG2": "VALUE2"
}
{
  "InstanceId": "i-22222222",
  "DeviceName": "/dev/sda1",
  "VolumeId": "vol-2222222a",
  "TAG1": "VALUE1",
  "TAG2": "VALUE2"
}
{
  "InstanceId": "i-22222222",
  "DeviceName": "/dev/sdb",
  "VolumeId": "vol-2222222b",
  "TAG1": "VALUE1",
  "TAG2": "VALUE2"
}
{
  "InstanceId": "i-22222222",
  "DeviceName": "/dev/sdc",
  "VolumeId": "vol-2222222c",
  "TAG1": "VALUE1",
  "TAG2": "VALUE2"
}

jq - 将嵌套的 JSON 提取到多个数组中

jq - extract nested JSON into multiple arrays

json

jq