如何遍历字典列表并根据 Bash 中的键对输出进行排序？

Question

我们正在从以下字典格式列表中的 API 调用中获取输出，

[
{
        "Key": "/builder-deployer/test2/services/serviceA/ip",
        "Val": "10.1.2.1"
    },
    {
        "Key": "/builder-deployer/test2/services/serviceA/port",
        "Val": "2"
    },
    {
        "Key": "/builder-deployer/test2/services/serviceA/url",
        "Val": "serviceA.abc.com"
    },
    {
        "Key": "/builder-deployer/test2/services/serviceA/username",
        "Val": "jenkins"
    },
    {
        "Key": "/builder-deployer/test2/services/serviceB/ip",
        "Val": "10.1.2.2"
    },
    {
        "Key": "/builder-deployer/test2/services/serviceB/port",
        "Val": "3"
    },
    {
        "Key": "/builder-deployer/test2/services/serviceB/url",
        "Val": "serviceB.abc.com"
    },
    {
        "Key": "/builder-deployer/test2/services/serviceB/username",
        "Val": "jenkins"
    },
    {
        "Key": "/builder-deployer/test2/services/serviceC/ip",
        "Val": "10.1.2.2"
    },
    {
        "Key": "/builder-deployer/test2/services/serviceC/port",
        "Val": "4"
    },
    {
        "Key": "/builder-deployer/test2/services/serviceC/url",
        "Val": "serviceC.abc.com"
    },
    {
        "Key": "/builder-deployer/test2/services/serviceC/username",
        "Val": "jenkins"
    }
]

我需要从列表中以 username@ip 的形式获取唯一的用户名和 IP 组合，以安装一些依赖项。即在上面的字典列表中，我们有 2 个独特的用户名@ip 组合。所以我需要分别在jenkins@10.1.2.1、jenkins@10.1.2.2机器上安装包。

我在 python 脚本中有以下代码，它执行相同的操作，但我们的大部分代码都在 Bash 中，我们也想将其转换为 Bash。任何帮助都会很有帮助

ip=[]
for username in outputs:
    for ip in outputs:
        if username['Key'].split('/')[-2] == ip['Key'].split('/')[-2] and username['Key'].split('/')[-1] in ['username'] and ip['Key'].split('/')[-1] in ['ip']:
            ip.append(username['Val']+'@'+ip['Val'])
print(set(ip))

Answer 1

评论中关于将您的解决方案留在 python 中的建议（根据 Gordon Davisson）也是我的偏好。但是，如果您确实需要 bash 解决方案，则以下是一种选择。请记住，如果您的 API JSON 格式随时间变化，它将变得脆弱。它也不是非常有效，因为它依赖于多次阅读源代码 JSON。此外，还利用了 grep 和 sort：

#!/bin/bash

input_json="${1:-/tmp/bar/t.json}"

while read -r line ; do
    search_str=$(grep -oB 1 "${line}" "${input_json}" \
     | awk 'NR==1 {gsub(/^"/,"", ); \
     gsub(/ip",$/,"username", ); print }')
    
    user=$(grep -oA 1 "${search_str}" "${input_json}" | \
     awk 'NR==2 {gsub(/"/,"", ); print }')
    
    echo "${user}@${line}"

# get unique ip addresses from source json
done < <(grep -Eo "([0-9]{1,3}[\.]){3}[0-9]{1,3}" "${input_json}" | sort -u)

示例数据：

[
{
        "Key": "/builder-deployer/test2/services/serviceA/ip",
        "Val": "10.1.2.1"
    },
    {
        "Key": "/builder-deployer/test2/services/serviceA/port",
        "Val": "2"
    },
    {
        "Key": "/builder-deployer/test2/services/serviceA/url",
        "Val": "serviceA.abc.com"
    },
    {
        "Key": "/builder-deployer/test2/services/serviceA/username",
        "Val": "jenkins"
    },
    {
        "Key": "/builder-deployer/test2/services/serviceB/ip",
        "Val": "10.1.2.2"
    },
    {
        "Key": "/builder-deployer/test2/services/serviceB/port",
        "Val": "3"
    },
    {
        "Key": "/builder-deployer/test2/services/serviceB/url",
        "Val": "serviceB.abc.com"
    },
    {
        "Key": "/builder-deployer/test2/services/serviceB/username",
        "Val": "jenkins"
    },
    {
        "Key": "/builder-deployer/test2/services/serviceC/ip",
        "Val": "10.1.2.2"
    },
    {
        "Key": "/builder-deployer/test2/services/serviceC/port",
        "Val": "4"
    },
    {
        "Key": "/builder-deployer/test2/services/serviceC/url",
        "Val": "serviceC.abc.com"
    },
    {
        "Key": "/builder-deployer/test2/services/serviceC/username",
        "Val": "jenkins"
    }
]

示例输出：

$ ./t.sh
jenkins@10.1.2.1
jenkins@10.1.2.2

Answer 2

jq

$ FILE="file.json"
$ paste -d@ \
        <(jq -r '.[]|select(.Key|contains("/username")).Val' "$FILE") \
        <(jq -r '.[]|select(.Key|contains("/ip")).Val' "$FILE") | \
        sort -u

jenkins@10.1.2.1
jenkins@10.1.2.2

awk

awk -F\" '/\/ip/{getline;ip=}/username/{getline;[=11=]="@"ip;print|"sort -u"}' "$FILE"

jenkins@10.1.2.1
jenkins@10.1.2.2

Answer 3

一次使用jq，不需要临时文件

$ cat test.json | jq -r '.[]|select(.Key|test("(username|ip)$")).Val' \
  | paste -d '@' - - | sort -u
10.1.2.1@jenkins
10.1.2.2@jenkins

如何遍历字典列表并根据 Bash 中的键对输出进行排序？

How to iterate through list of dictionaries and sort the output based on the key in Bash?

bash

ubuntu