bigquery 转置和连接每条记录

Question

我想实现如下改造。我已经 last_name 存储在重复记录中，如下所示。

data before transformation

我想实现以下目标。 data after transformation

创建示例数据的示例。

create or replace table t1.cte1 as
WITH t1 AS (
  SELECT 1 as id,'eren' AS last_name UNION ALL
  SELECT 1 as id,'yilmaz' AS last_name UNION ALL
  SELECT 1 as id,'kaya' AS last_name
)
SELECT id,ARRAY_AGG(STRUCT(last_name)) AS last_name_rec
FROM t1
GROUP BY id;

with test as (
select x.id, x.lname_agg,y.last_name  from
(
select id, STRING_AGG(h.last_name,' ') lname_agg FROM
  t1.cte1
  LEFT JOIN
  UNNEST(last_name_rec) AS h
  group by id
  ) x,
  (select id,h.last_name last_name  FROM
  t1.cte1
  LEFT JOIN
  UNNEST(last_name_rec) AS h
  group by last_name,id) y
) select id ,sp.string_flatten_dedup( lname_agg,' ') concat_last_name, last_name from test;

我也不确定是否应该将其存储为数组而不是串联字段，但最好知道如何实现这两者。 storing the concat_last_name as an array

我已经实现了如下的第一个转换，但是我不得不用我写的函数对连接的字段进行去重。我相信有更好的方法可以实现这一目标。

       with test as (
select x.id id, x.lname_agg,y.last_name  from 
(
select id, STRING_AGG(h.last_name,' ') lname_agg FROM
  small_test
  LEFT JOIN
  UNNEST(last_name_rec) AS h
  group by id
  ) x,
  (select id,h.last_name last_name  FROM
  small_test
  LEFT JOIN
  UNNEST(last_name_rec) AS h group by last_name,id) y
) select id ,sp.string_flatten_dedup( lname_agg,' ') concat_last_name, last_name from test;

函数。 string_flatten_dedup

CREATE OR REPLACE FUNCTION
sp.string_flatten_dedup(string_value string,
    delim string) AS
(
                            ARRAY_TO_STRING
                                (ARRAY(SELECT distinct string_value
                                       FROM UNNEST(SPLIT(string_value, delim)) AS string_value
                                       order by string_value desc, string_value),
                                 delim)
);

在使用函数之前。 intermediate results.

应用去重功能后的最终结果。 final output

更新了 table 结构。 t1.ccte1

你的工作正常，但我第一次发布时 table 结构不正确。

create or replace table t1.cte2 as
with your_table as (
  select 1 id, ['brown', 'smith', 'jones'] last_name union all 
  select 2, ['ryan', 'murphy']
) select id, ln as last_name,
   array_to_string(last_name, ',') as concat_last_name,
from your_table, unnest(last_name) ln;

select id, ln as last_name,
   array_to_string(last_name, ',') as concat_last_name,
from t1.cte2, unnest(last_name) ln;

--fails as its not the structure I thought it was cte1 is different then cte2
select id, ln.last_name
   --array_to_string(last_name, ',') as concat_last_name,
from t1.cte1, unnest(last_name_rec) ln;

Answer 1

考虑以下方法

select id, ln as last_name, 
   array_to_string(last_name, ',') as concat_last_name,
from your_table, unnest(last_name) ln

如果应用于您问题中的示例数据data before transformation

with your_table as (
  select 1 id, ['brown', 'smith', 'jones'] last_name union all 
  select 2, ['ryan', 'murphy']
)

输出为

如果您想要将姓氏作为一个数组 - 您已经有了这个数组 - 请参阅下文了解如何使用它

select id, ln as last_name, 
  last_name as concat_last_name,
from your_table, unnest(last_name) ln

有输出

bigquery 转置和连接每条记录

bigquery transpose and concatenate for each record

sql

arrays

transpose

concatenation

google-bigquery