解析来自 ec2instances.info 的 EC2 按需定价

Question

我正在尝试使用 curl 和 jq 来解析 AWS EC2 按需定价并构建适合在 Terraform 模块中使用的 JSON 映射。

我想出的脚本看起来像这样，但似乎不正确：

curl --silent --show-error 'https://raw.githubusercontent.com/powdahound/ec2instances.info/master/www/instances.json' |
jq '.[]
    | .instance_type as $instance_type
    | (.pricing | keys) as $keys 
    | [.pricing[].linux.ondemand | .] as $values
    | reduce range(0; $keys|length) as $i 
        ({}; . + { ($keys[$i] + "|" + $instance_type): $values[$i] })'

我做错了什么？这是一个较小的代码示例来说明问题：

curl --silent --show-error 'https://gist.githubusercontent.com/joshuaspence/0904a6ce25f8830d9ae2eac8fc44fc7a/raw/b24600ab2e536556a74f4dbb45e2ddaa432d430e/sample.json' |
jq '.[]
    | .instance_type as $instance_type
    | (.pricing | keys) as $keys
    | [.pricing[].linux.ondemand | .] as $values
    | reduce range(0; $keys|length) as $i
        ({}; . + { ($keys[$i] + "|" + $instance_type): $values[$i] })'

上述命令的预期输出是：

{
  "ap-south-1|m1.small": "N/A",
  "us-east-1|m1.small": "0.061",
  "sa-east-1|m1.small": "0.058",
  "ap-northeast-2|m1.small": "0.058",
  "ap-southeast-2|m1.small": "0.058",
  "us-west-2|m1.small": "0.044",
  "us-gov-west-1|m1.small": "0.053",
  "us-west-1|m1.small": "0.047",
  "eu-central-1|m1.small": "N/A",
  "eu-west-1|m1.small": "0.047"
}
{
  "ap-south-1|m1.medium": "N/A",
  "us-east-1|m1.medium": "0.087",
  "ap-northeast-1|m1.medium": "0.122",
  "sa-east-1|m1.medium": "0.117",
  "ap-northeast-2|m1.medium": "N/A",
  "ap-southeast-1|m1.medium": "0.117",
  "ap-southeast-2|m1.medium": "0.117",
  "us-west-2|m1.medium": "0.087",
  "us-gov-west-1|m1.medium": "0.106",
  "us-west-1|m1.medium": "0.095",
  "us-central-1|m1.medium": "N/A",
  "us-west-1|m1.medium": "0.095"
}

实际输出为：

{
  "ap-northeast-2|m1.small": "N/A",
  "ap-south-1|m1.small": "0.061",
  "ap-southeast-2|m1.small": "0.058",
  "eu-central-1|m1.small": "0.058",
  "eu-west-1|m1.small": "0.058",
  "sa-east-1|m1.small": "0.044",
  "us-east-1|m1.small": "0.053",
  "us-gov-west-1|m1.small": "0.047",
  "us-west-1|m1.small": "N/A",
  "us-west-2|m1.small": "0.047"
}
{
  "ap-northeast-1|m1.medium": "N/A",
  "ap-northeast-2|m1.medium": "0.087",
  "ap-south-1|m1.medium": "0.122",
  "ap-southeast-1|m1.medium": "0.117",
  "ap-southeast-2|m1.medium": "N/A",
  "eu-central-1|m1.medium": "0.117",
  "eu-west-1|m1.medium": "0.117",
  "sa-east-1|m1.medium": "0.087",
  "us-east-1|m1.medium": "0.106",
  "us-gov-west-1|m1.medium": "0.095",
  "us-west-1|m1.medium": "N/A",
  "us-west-2|m1.medium": "0.095"
}

Answer 1

您的脚本提供错误输出的原因是 JSON 对象的键没有特定的顺序，并且 jq 内置的顺序不稳定.这意味着当您执行 (.pricing | keys) 和 [.pricing[].linux.ondemand | .] 时，前者中键的顺序与后者中值的顺序不匹配。

你的 jq 程序的一个简化的工作版本如下：

jq '.[] | .instance_type as $it | .pricing | with_entries(.key |= "\(.)|\($it)" | .value |= .linux.ondemand)'

这个jq程序使用with_entries将一个JSON对象转换为一个JSON数组{key, value}对象，并在重新组装之前对键值对进行转换原始对象。

Answer 2

这是一个解决方案，它使用 keys_unsorted 来保留原始 .pricing 对象中键的顺序。

    .[]
  | .instance_type as $instance_type
  | .pricing
  | [
        keys_unsorted[] as $k
      | .[$k].linux.ondemand
      | {("\($k)|\($instance_type)"): .}
    ]
  | add

解析来自 ec2instances.info 的 EC2 按需定价

Parsing EC2 on-demand pricing from ec2instances.info

jq