将 JSON 转换为垂直 table

Question

我有以下有效载荷，我想要生成的是这样的水平列输出，条目之间有一个换行符。有人知道如何实现吗？要么直接在 jq 中，要么与一些有趣的 bash。 :

StackId              : arn:aws:cloudformation:us-east-1:882038671278:stack/cbongiorno-30800-bb-lambda/97b14e40-75a3-11ea-bb77-0e8a861a6983
EventId              : 97b2fbf0-75a3-11ea-bb77-0e8a861a6983
StackName            : cbongiorno-30800-bb-lambda
LogicalResourceId    : cbongiorno-30800-bb-lambda
PhysicalResourceId   : arn:aws:cloudformation:us-east-1:882038671278:stack/cbongiorno-30800-bb-lambda/97b14e40-75a3-11ea-bb77-0e8a861a6983
ResourceType         : AWS::CloudFormation::Stack
Timestamp            : 2020-04-03T12:06:47.501Z
ResourceStatus       : CREATE_IN_PROGRESS
ResourceStatusReason : User Initiated

EventId              : BBPassword-CREATE_IN_PROGRESS-2020-04-03T12:06:51.336Z
StackName            : cbongiorno-30800-bb-lambda
LogicalResourceId    : BBPassword
PhysicalResourceId   : 
ResourceType         : AWS::SSM::Parameter
Timestamp            : 2020-04-03T12:06:51.336Z
ResourceStatus       : CREATE_IN_PROGRESS

这是我用来生成输出的 2 个命令，但都不理想。

我删除了一个通常用 JSON 填充的密钥，它只会把一切都搞砸了。
第一个示例我插入了一个定界符，我希望以后可以使用它来删除
第二个例子给我一个错误xargs: unterminated quote:
在这两种情况下，我都对格式长度进行了硬编码。但出于好奇，可以这样做：jq -re '.StackEvents | map(to_entries | map(.key | length) | max) | max'

jq -re '.StackEvents | .[] | del(.ResourceProperties) | . * {"entry":"---"} | to_entries | .[] | "\(.key) \"\(.value?)\""' bin/logs/3.json  | xargs -n 2 printf "%-21s: %s\n"

jq -re '.StackEvents | .[] | del(.ResourceProperties) | . * {"":"\n"} | to_entries | .[] | "\(.key) \"\(.value?)\""' bin/logs/3.json  | xargs -n 2 printf "%-21s: %s\n"

这是有效载荷：

{
    "StackEvents": [
        {
            "StackId": "arn:aws:cloudformation:us-east-1:882038671278:stack/cbongiorno-30800-bb-lambda/97b14e40-75a3-11ea-bb77-0e8a861a6983",
            "EventId": "BBWebhookLogGroup-CREATE_IN_PROGRESS-2020-04-03T12:06:51.884Z",
            "StackName": "cbongiorno-30800-bb-lambda",
            "LogicalResourceId": "BBWebhookLogGroup",
            "PhysicalResourceId": "cbongiorno-30800-bb-lambda",
            "ResourceType": "AWS::Logs::LogGroup",
            "Timestamp": "2020-04-03T12:06:51.884Z",
            "ResourceStatus": "CREATE_IN_PROGRESS",
            "ResourceStatusReason": "Resource creation Initiated",
            "ResourceProperties": "{\"RetentionInDays\":\"7\",\"LogGroupName\":\"cbongiorno-30800-bb-lambda\"}"
        },
        {
            "StackId": "arn:aws:cloudformation:us-east-1:882038671278:stack/cbongiorno-30800-bb-lambda/97b14e40-75a3-11ea-bb77-0e8a861a6983",
            "EventId": "BBUserName-CREATE_IN_PROGRESS-2020-04-03T12:06:51.509Z",
            "StackName": "cbongiorno-30800-bb-lambda",
            "LogicalResourceId": "BBUserName",
            "PhysicalResourceId": "",
            "ResourceType": "AWS::SSM::Parameter",
            "Timestamp": "2020-04-03T12:06:51.509Z",
            "ResourceStatus": "CREATE_IN_PROGRESS",
            "ResourceProperties": "{\"Type\":\"String\",\"Description\":\"The username for this lambda to operate under\",\"Value\":\"chb0bitbucket\",\"Name\":\"/bb-webhooks/authorization/username\"}"
        },
        {
            "StackId": "arn:aws:cloudformation:us-east-1:882038671278:stack/cbongiorno-30800-bb-lambda/97b14e40-75a3-11ea-bb77-0e8a861a6983",
            "EventId": "BBWebhookLogGroup-CREATE_IN_PROGRESS-2020-04-03T12:06:51.409Z",
            "StackName": "cbongiorno-30800-bb-lambda",
            "LogicalResourceId": "BBWebhookLogGroup",
            "PhysicalResourceId": "",
            "ResourceType": "AWS::Logs::LogGroup",
            "Timestamp": "2020-04-03T12:06:51.409Z",
            "ResourceStatus": "CREATE_IN_PROGRESS",
            "ResourceProperties": "{\"RetentionInDays\":\"7\",\"LogGroupName\":\"cbongiorno-30800-bb-lambda\"}"
        },
        {
            "StackId": "arn:aws:cloudformation:us-east-1:882038671278:stack/cbongiorno-30800-bb-lambda/97b14e40-75a3-11ea-bb77-0e8a861a6983",
            "EventId": "BBPassword-CREATE_IN_PROGRESS-2020-04-03T12:06:51.336Z",
            "StackName": "cbongiorno-30800-bb-lambda",
            "LogicalResourceId": "BBPassword",
            "PhysicalResourceId": "",
            "ResourceType": "AWS::SSM::Parameter",
            "Timestamp": "2020-04-03T12:06:51.336Z",
            "ResourceStatus": "CREATE_IN_PROGRESS",
            "ResourceProperties": "{\"Type\":\"String\",\"Description\":\"The password for this lambda to operate under with BB. Unfortunately, using an encrypted password is currently not possible\",\"Value\":\"****\",\"Name\":\"/bb-webhooks/authorization/password\"}"
        },
        {
            "StackId": "arn:aws:cloudformation:us-east-1:882038671278:stack/cbongiorno-30800-bb-lambda/97b14e40-75a3-11ea-bb77-0e8a861a6983",
            "EventId": "97b2fbf0-75a3-11ea-bb77-0e8a861a6983",
            "StackName": "cbongiorno-30800-bb-lambda",
            "LogicalResourceId": "cbongiorno-30800-bb-lambda",
            "PhysicalResourceId": "arn:aws:cloudformation:us-east-1:882038671278:stack/cbongiorno-30800-bb-lambda/97b14e40-75a3-11ea-bb77-0e8a861a6983",
            "ResourceType": "AWS::CloudFormation::Stack",
            "Timestamp": "2020-04-03T12:06:47.501Z",
            "ResourceStatus": "CREATE_IN_PROGRESS",
            "ResourceStatusReason": "User Initiated"
        }
    ]
}

根据其他人的意见，我整理了一个简单的 bash 脚本来说明一个小异常（列宽不统一）：

#!/usr/bin/env bash
set -e
set -o pipefail

fileCount=$(( $( ls -1 logs/*.json | wc -l) - 1))
for i in $(seq 1 $fileCount); do
    jq -rs '
      def width:      map(keys_unsorted | map(length) | max) | max ;
      def pad($w):    . + (($w-length)*" ") ;

      .[1].StackEvents - .[0].StackEvents | sort_by (.Timestamp) 
      | width as $w | map(to_entries | map("\(.key|pad($w)) : \(.value)"), [""]) 
      | .[][]
    ' "logs/$((i - 1)).json" "logs/$i.json"

done

产量：

StackId              : arn:aws:cloudformation:us-east-1:882038671278:stack/cbongiorno-30800-bb-lambda/97b14e40-75a3-11ea-bb77-0e8a861a6983
EventId              : ApiKey-CREATE_COMPLETE-2020-04-03T12:07:47.382Z
StackName            : cbongiorno-30800-bb-lambda
LogicalResourceId    : ApiKey
PhysicalResourceId   : KYgzCNAzPw5Tsy3dKBdoTaHlxywijTSrb1d2UIQ2
ResourceType         : AWS::ApiGateway::ApiKey
Timestamp            : 2020-04-03T12:07:47.382Z
ResourceStatus       : CREATE_COMPLETE
ResourceProperties   : {"StageKeys":[{"StageName":"beta","RestApiId":"8n6tijwaib"}]}

StackId            : arn:aws:cloudformation:us-east-1:882038671278:stack/cbongiorno-30800-bb-lambda/97b14e40-75a3-11ea-bb77-0e8a861a6983
EventId            : bc9371c0-75a3-11ea-b442-1217092af407
StackName          : cbongiorno-30800-bb-lambda
LogicalResourceId  : cbongiorno-30800-bb-lambda
PhysicalResourceId : arn:aws:cloudformation:us-east-1:882038671278:stack/cbongiorno-30800-bb-lambda/97b14e40-75a3-11ea-bb77-0e8a861a6983
ResourceType       : AWS::CloudFormation::Stack
Timestamp          : 2020-04-03T12:07:49.203Z
ResourceStatus     : CREATE_COMPLETE

Answer 1

JQ 没有用于填充字符串的内置函数，但实现该功能并不难。给定命令行上的 -r/--raw-output 选项，下面的脚本将产生您想要的输出。

.StackEvents
| map(del(.ResourceProperties))
| ( [ .[] | keys_unsorted[] ]
    | map(length)
    | max + 1
  ) as $max
| .[]
  | ( keys_unsorted as $keys
    | [ $keys,
      ( $keys
        | map(length)
        | map($max - .)
        | map(. * " " + ": ")
      ),
      map(.)
    ]
    | transpose[]
    | add
  ),
  ""

^{Online demo}

Answer 2

这是一个解决方案，其中包含一些可以推广用于其他用途的辅助函数。

def width:      map(keys | map(length) | max) | max ;
def pad($w):    . + (($w-length)*" ") ;

  .StackEvents
| width as $w
| map(del(.ResourceProperties) | to_entries | map("\(.key|pad($w)) : \(.value)"), [""])
| .[][]

如果传递 jq，它应该会产生所需的输出 -r

Try it online!

编辑：正如 peak and oguz ismail 在评论中指出的那样，可以使用 keys_unsorted 改进此解决方案，并且应该从宽度计算中排除 .ResourceProperties。
这是具有这些改进的版本：

def width:      map(keys_unsorted | map(length) | max) | max ;
def pad($w):    . + (($w-length)*" ") ;

  .StackEvents
| map(del(.ResourceProperties))
| width as $w
| map(to_entries | map("\(.key|pad($w)) : \(.value)"), [""])
| .[][]

Try it online!

Answer 3

这是一个解决方案：

使用 max/1 提高效率
解决了一些问题计算 Unicode 字符串的 "width"，例如如果我们想要 "width" 个：

"J̲o̲s̲é̲"计算为4

注意这里定义的jq过滤器grapheme_length忽略控制字符和零宽度空格的问题。

泛型函数

def max(stream):
  reduce stream as $x (null; if . == null then $x elif $x > . then $x else . end);

# Grapheme Length ignoring issues with control characters
# Mn = non-spacing mark
# Mc = combining
# Cf = soft-hyphen, bidi control characters, and language tag characters
def grapheme_length:
  gsub("\p{Mn}";"") | gsub("\p{Mc}";"") | gsub("\p{Cf}";"")
  | length;

def pad($w): tostring + (($w - grapheme_length)*" ") ;

主程序

.StackEvents
| max(.[]
      | keys_unsorted[] 
      | select(. != "ResourceProperties") 
      | grapheme_length) as $w
| map(del(.ResourceProperties)
      | to_entries
      | map("\(.key|pad($w)) : \(.value)"), [""])
| .[][]

将 JSON 转换为垂直 table

Convert JSON to vertical table

json

bash

width

jq

tabular-form

泛型函数

主程序