将 name/value 对提取到列和值中 - Bigquery 中的 Shopify 数据

Question

我们使用 FiveTran 从 shopify 中提取数据并将其存储在 BigQuery 中。 order_line table 中的字段“属性”包含看起来像 key/value 对数组的内容。在这种情况下 name/value。字段类型为string 这里以内容为例

order_line_id   properties
9956058529877   [{"name":"_order_bump_rule_id","value":"4afx7cbw6"},{"name":"_order_bump_bump_id","value":"769d1996-b6fb-4bc3-8d41-c4d7125768c5"},{"name":"_source","value":"order-bump"}]
4467731660885   [{"name":"shipping_interval_unit_type","value":null},{"name":"charge_delay","value":null},{"name":"charge_on_day_of_week","value":null},{"name":"charge_interval_frequency","value":null},{"name":"charge_on_day_of_month","value":null},{"name":"shipping_interval_frequency","value":null},{"name":"number_charges_until_expiration","value":null}]
4467738738773   [{"name":"shipping_interval_unit_type","value":null},{"name":"charge_delay","value":null},{"name":"charge_on_day_of_week","value":null},{"name":"charge_interval_frequency","value":null},{"name":"charge_on_day_of_month","value":null},{"name":"shipping_interval_frequency","value":null},{"name":"number_charges_until_expiration","value":null}]
4578798600277   [{"name":"shipping_interval_unit_type","value":null},{"name":"charge_interval_frequency","value":null},{"name":"shipping_interval_frequency","value":null}]

我正在尝试编写一个查询，为每条记录生成一行，其中每个名称值都有一列：

shipping_interval_unit_type
charge_on_day_of_week
charge_interval_frequency
charge_on_day_of_month
subscription_id
number_charges_until_expiration
shipping_interval_frequency

和对应的“值”。这个“属性”字段可以包含许多不同的“名称”值，并且每次它们的顺序都不同。上面提到的“名称”值并不总是出现在“属性”字段中。

我试过 json 函数，但它的格式似乎不适合 json。我试过取消嵌套但失败了，因为它是一个字符串。

Answer 1

考虑以下方法

select * from (
  select order_line_id, 
    json_extract_scalar(property, '$.name') name,
    json_extract_scalar(property, '$.value') value
  from your_table, unnest(json_extract_array(properties)) property
)
pivot (min(value) for name in (
  'shipping_interval_unit_type',
  'charge_on_day_of_week',
  'charge_interval_frequency',
  'charge_on_day_of_month',
  'subscription_id',
  'number_charges_until_expiration',
  'shipping_interval_frequency'
))

将 name/value 对提取到列和值中 - Bigquery 中的 Shopify 数据

extracting name/value pairs into columns and values - Shopify Data In Bigquery

arrays

json

google-bigquery

shopify

fivetran