如何使用 JSON_EXTRACT 或 JSON_EXTRACTSCALAR 在 Bigquery 中读取多级 JSON 数据

Question

我正在尝试使用 Bigquery 中的 JSON_EXTRACT 读取 Bigquery 中的以下 JSON 结构。它说不受支持的运算符路径“*”

在 BigQuery 中尝试了所有可能的方法并请求您的帮助

错误： JSON路径中不支持的运算符：*

****JSON 数据：** 只是具有多个值并且在阅读时有问题的部分。需要阅读下面的所有 4 "id" 值，例如并且还需要阅读所有其他专栏在组合部分下，它产生 4 行不同的 ID、类型等。**

"Combos": [
  {
    "Id": "1111",
    "Type": 0,
    "Description": "ABCD",
    "ComboDuration": {
      "StartDate": "2009-10-26T08:00:00",
      "EndDate": "2009-10-29T08:00:00"
    }
  },
  {
    "Id": "2222",
    "Type": 1,
    "Description": "XYZ",
    "ComboDuration": {
      "StartDate": "2019-10-26T08:00:00",
      "EndDate": "2019-10-29T08:00:00"
    }
  },
  {
    "Id": "39933",
    "Type": 3,
    "Description": "General",
    "ComboDuration": {
      "StartDate": "2019-10-26T08:00:00",
      "EndDate": "2019-10-29T08:00:00"
    }
  },
  {
    "Id": "39934",
    "Type": 2,
    "Description": "ABCDXYZ",
    "ComboDuration": {
      "StartDate": "2019-10-26T08:00:00",
      "EndDate": "2019-10-29T08:00:00"
    }
  },

]

****Code:** P.S - conv_column is a string column where my JSON structure stored**

SELECT 
JSON_EXTRACT(conv_column,"$.Combos.*.Id") as combo_id
from lz.json_file

SELECT JSON_EXTRACT(conv_column,"$.Combos[*].Id") as combo_id
from lz.json_file

SELECT JSON_EXTRACT(conv_column,"$.Combos[?@.Id]") as combo_id
from lz.json_file

Answer 1

下面是标准的 BigQuery 示例 SQL

#standardSQL
CREATE TEMP FUNCTION jsonparse(input STRING)
RETURNS ARRAY<STRING>
LANGUAGE js AS """
  return JSON.parse(input).map(x=>JSON.stringify(x));
"""; 
WITH `project.lz.json_file` AS (
  SELECT '''{
  "Combos": [  {
    "Id": "1111",
    "Type": 0,
    "Description": "ABCD",
    "ComboDuration": {
      "StartDate": "2009-10-26T08:00:00",
      "EndDate": "2009-10-29T08:00:00"
    }  },  {
    "Id": "2222",
    "Type": 1,
    "Description": "XYZ",
    "ComboDuration": {
      "StartDate": "2019-10-26T08:00:00",
      "EndDate": "2019-10-29T08:00:00"
    }  },  {
    "Id": "39933",
    "Type": 3,
    "Description": "General",
    "ComboDuration": {
      "StartDate": "2019-10-26T08:00:00",
      "EndDate": "2019-10-29T08:00:00"
    }  },  {
    "Id": "39934",
    "Type": 2,
    "Description": "ABCDXYZ",
    "ComboDuration": {
      "StartDate": "2019-10-26T08:00:00",
      "EndDate": "2019-10-29T08:00:00"
    }  }]}  ''' AS conv_column
)
SELECT
  JSON_EXTRACT_SCALAR(combo, '$.Id') AS Id,
  JSON_EXTRACT_SCALAR(combo, '$.Type') AS Type,
  JSON_EXTRACT_SCALAR(combo, '$.Description') AS Description,
  JSON_EXTRACT_SCALAR(combo, '$.ComboDuration.StartDate') AS StartDate,
  JSON_EXTRACT_SCALAR(combo, '$.ComboDuration.EndDate') AS EndDate
FROM `project.lz.json_file`,
UNNEST(jsonparse(JSON_EXTRACT(conv_column, '$.Combos'))) combo

输出

Row Id      Type    Description StartDate           EndDate  
1   1111    0       ABCD        2009-10-26T08:00:00 2009-10-29T08:00:00  
2   2222    1       XYZ         2019-10-26T08:00:00 2019-10-29T08:00:00  
3   39933   3       General     2019-10-26T08:00:00 2019-10-29T08:00:00  
4   39934   2       ABCDXYZ     2019-10-26T08:00:00 2019-10-29T08:00:00

如何使用 JSON_EXTRACT 或 JSON_EXTRACTSCALAR 在 Bigquery 中读取多级 JSON 数据

how to read multiple levels of JSON data in Big Query using JSON_EXTRACT or JSON_EXTRACT_SCALAR

json

jsonpath

google-bigquery