从异常 JSON 中提取数据

Question

有没有办法用下面的 JSON 代码制作一个漂亮的 CSV 文件？

{
    "cod:e1!!@23" : {
        "typeA" : {
            "lsk:d##fjd": {
                "title" : "slkdfjlkdjfd",
                "year" : "2014"
            },
        "sdfdsfsd" : {
            "title" : "slkdfjlkdjfddewfsdfd",
            "year" : "2015"
            }
        },
        "Ct@ype" : {
            "sd$!!fs:$dfds" : {
                "title" : "slkdfjsdfsdfdsfsd",
                "year" : "2012"
            }
        }
    }
}

这是我在 jq 中尝试的方法：

jq -rc 'keys[] as $x 
  | .[]|keys[] as $y
  | .[]|keys[] as $z
  |.[]
  |[$x,$y,$z,.year] | @csv'

jq -rc 'keys_unsorted[] as $x
  | .[]|keys_unsorted[] as $y
  | .[]|keys_unsorted[] as $z
  | .[]|[$x,$y,$z,.year] | @csv'

但输出不正确，因为如果有多个这样的记录，则键会按排序和排列顺序出现。我也试过 keys_unsorted，但没有解决问题。

目前无法修复原始 JSON 生成，因此我们将不胜感激。

理想情况下，我会得到：

"cod:e1!!@23","typeA","lsk:d##fjd","slkdfjlkdjfd","2014"
"cod:e1!!@23","typeA","sdfdsfsd","slkdfjlkdjfddewfsdfd","2015"
"cod:e1!!@23","Ct@ype","sd$!!fs:$dfds","slkdfjsdfsdfdsfsd","2012"

Answer 1

您可以将“https://github.com/zemirco/json2csv”与 flatten 选项一起使用。这将生成像 cod:e1!!@23.typeA.lsk:d##fjd.title 这样的列。

cat input.json | json2csv -F >> output.csv

编辑：这不是您想要的。

Answer 2

这是一个 jq 脚本，它遍历输入中的 "leaf elements" 并根据它经过的每个键生成一个 CSV 列：

jq -r 'leaf_paths as $path | $path + [getpath($path)] | @csv'

请注意，这不是您要查找的内容：

"cod:e1!!@23","typeA","lsk:d##fjd","title","slkdfjlkdjfd"
"cod:e1!!@23","typeA","lsk:d##fjd","year","2014"
"cod:e1!!@23","typeA","sdfdsfsd","title","slkdfjlkdjfddewfsdfd"
"cod:e1!!@23","typeA","sdfdsfsd","year","2015"
"cod:e1!!@23","Ct@ype","sd$!!fs:$dfds","title","slkdfjsdfsdfdsfsd"
"cod:e1!!@23","Ct@ype","sd$!!fs:$dfds","year","2012"

Answer 3

对您在初始 post 中提供的脚本稍作修改即可使其正常工作。我没有使用 .[]，而是按从 keys_unsorted 保存为变量的特定键进行索引。为了方便起见，我还在 CSV 中添加了一个 header：

jq -r '["x", "y", "z", "title", "year"],
  (keys_unsorted[] as $x
   | .[$x] | keys_unsorted[] as $y
   | .[$y] | keys_unsorted[] as $z
   | .[$z] | [$x, $y, $z, .title, .year]) | @csv'

这确实提供了您正在寻找的输出（带有 header）：

"x","y","z","title","year"
"cod:e1!!@23","typeA","lsk:d##fjd","slkdfjlkdjfd","2014"
"cod:e1!!@23","typeA","sdfdsfsd","slkdfjlkdjfddewfsdfd","2015"
"cod:e1!!@23","Ct@ype","sd$!!fs:$dfds","slkdfjsdfsdfdsfsd","2012"

Answer 4

下面提供了规则结构的通用解决方案嵌套对象（笼统地说，它们可以被认为是 "babushka objects"，就像嵌套的娃娃）；此外，对象中的键可以以任何方式排序。

关键概念是 "scalar objects" -- 所有的对象键具有标量值。

要从“标量”中提取信息的模板对象”作为 'emit' 过滤器的参数提供并用于确保在生产时保持适当的顺序 CSV 行。

def emit(template):

  def is_scalar_object:
    def is_scalar: type | ((. != "object") and (. != "array"));
    . as $in | (type == "object") and all($in[] | is_scalar);

  . as $in
  | paths as $path
  | select(getpath($path) | is_scalar_object)
  | $path + [ template + ($in | getpath($path)) | .[]]
  ;


data | emit( {title,  year} ) | @csv

用法：

 jq -r emit.jq input.json

输出：

"cod:e1!!@23","typeA","lsk:d##fjd","slkdfjlkdjfd","2014"
"cod:e1!!@23","typeA","sdfdsfsd","slkdfjlkdjfddewfsdfd","2015"
"cod:e1!!@23","Ct@ype","sd$!!fs:$dfds","slkdfjsdfsdfdsfsd","2012"

从异常 JSON 中提取数据

Extracting data from an unusual JSON

json

csv

hierarchical-data

jq