如何对 BigQuery 中的重复字段进行分组

Question

在 BigQuery 中，我创建了一个具有以下架构的 table

id                  INTEGER NULLABLE    
visits              INTEGER NULLABLE    
dimensions          RECORD  REPEATED    
dimensions.value    STRING  
dimensions.key      STRING

如何通过对设备和状态值进行分组来获取总和（访问次数）？

示例数据：

{"id": 1, visits: 100, "dimensions": [{"key":"device","value":"mobile"}, {"key":"state","value":"CA"}]}
{"id": 1, visits: 500, "dimensions": [{"key":"device","value":"desktop"}, {"key":"state","value":"CA"}]}
{"id": 1, visits: 200, "dimensions": [{"key":"device","value":"mobile"}, {"key":"state","value":"NY"}]}
{"id": 2, visits: 100, "dimensions": [{"key":"device","value":"mobile"}, {"key":"state","value":"CA"}]}
{"id": 2, visits: 500, "dimensions": [{"key":"device","value":"desktop"}, {"key":"state","value":"CA"}]}
{"id": 2, visits: 200, "dimensions": [{"key":"device","value":"mobile"}, {"key":"state","value":"NY"}]}
{"id": 2, visits: 780, "dimensions": [{"key":"device","value":"desktop"}, {"key":"state","value":"NY"}]}

我想要输出中的 id、device、state、sum(visits)。

我可以通过使用带有以下查询的单个维度来进行分组，但不知道如何针对多个维度进行分组。

SELECT id,d.value, sum(visits) FROM dataset.tabe_name,UNNEST(dimensions) as d where d.key = "device" group by id, d.value LIMIT 1000

当键值事先未知时，是否可以编写通用查询？

Answer 1

以下适用于 BigQuery 标准 SQL

#standardSQL
SELECT 
  id,
  (SELECT value FROM UNNEST(dimensions) WHERE key = "device") AS device,
  (SELECT value FROM UNNEST(dimensions) WHERE key = "state") AS state,
  SUM(visits) AS visits
FROM `dataset.tabe_name`  
GROUP BY id, device, state
LIMIT 1000

您可以尝试/使用示例中的虚拟数据进行播放，如下所示

#standardSQL
WITH data AS (
  SELECT 1 AS id, 100 AS visits, ARRAY<STRUCT<key STRING, value STRING>>[("device", "mobile"), ("state", "CA")] AS dimensions UNION ALL
  SELECT 1, 100, [STRUCT<key STRING, value STRING>("device", "mobile"), ("state", "CA")] UNION ALL
  SELECT 1, 500, [STRUCT<key STRING, value STRING>("device", "desktop"), ("state", "CA")] UNION ALL
  SELECT 1, 200, [STRUCT<key STRING, value STRING>("device", "mobile"), ("state", "NY")] UNION ALL
  SELECT 2, 100, [STRUCT<key STRING, value STRING>("device", "mobile"), ("state", "CA")] UNION ALL
  SELECT 2, 500, [STRUCT<key STRING, value STRING>("device", "desktop"), ("state", "CA")] UNION ALL
  SELECT 2, 200, [STRUCT<key STRING, value STRING>("device", "mobile"), ("state", "NY")] UNION ALL
  SELECT 2, 780, [STRUCT<key STRING, value STRING>("device", "desktop"), ("state", "NY")] 
)
SELECT 
  id,
  (SELECT value FROM UNNEST(dimensions) WHERE key = "device") AS device,
  (SELECT value FROM UNNEST(dimensions) WHERE key = "state") AS state,
  SUM(visits) AS visits
FROM data  
GROUP BY id, device, state
-- ORDER BY id, device, state

如何对 BigQuery 中的重复字段进行分组

How to do group by on repeated field in BigQuery

google-bigquery

bigquery-standard-sql