Google BigQuery APPROX_QUANTILES 并获得真正的四分位数

Question

根据 docs:

Returns the approximate boundaries for a group of expression values, where number represents the number of quantiles to create. This function returns an array of number + 1 elements, where the first element is the approximate minimum and the last element is the approximate maximum.

听起来如果我想要真正的 quartiles，我需要使用 APPROX_QUANTILES(values, 4) 这将 return [minvalue, 1st quartile, 2nd quartile, 3rd quartile, maxvalue]

根据 https://en.wikipedia.org/wiki/Quartile，四分位数集包含 3 个数据点 - none 其中是数据的 min/max 值。

我的假设正确吗？ APPROX_QUANTILES(values, 4) 会达到 return 真正的四分位数吗？

Answer 1

作为基准，这是未经任何修改的输出，使用 1 到 100 之间的数字输入：

SELECT APPROX_QUANTILES(x, 4) AS output
FROM UNNEST(GENERATE_ARRAY(1, 100)) AS x;
+----------------------------+
|           output           |
+----------------------------+
| ["1","25","50","75","100"] |
+----------------------------+

输出包括最小值 (1) 和最大值 (100)。如果您只想要四分位数，则需要将它们从数组中删除。为了 readability/composability，最好使用临时 SQL UDF 来执行此操作。在这里，我使用 INT64 作为元素类型，但您可以使用不同的元素类型，或者：

CREATE TEMP FUNCTION StripFirstLast(arr ARRAY<INT64>) AS (
  ARRAY(SELECT x FROM UNNEST(arr) AS x WITH OFFSET
  WHERE OFFSET BETWEEN 1 AND ARRAY_LENGTH(arr) - 2)
);

SELECT
  APPROX_QUANTILES(x, 4) AS output,
  StripFirstLast(APPROX_QUANTILES(x, 4)) AS quartiles
FROM UNNEST(GENERATE_ARRAY(1, 100)) AS x;
+----------------------------+------------------+
|           output           |    quartiles     |
+----------------------------+------------------+
| ["1","25","50","75","100"] | ["25","50","75"] |
+----------------------------+------------------+

您可以看到 quartiles 数组仅包含所需的值。

Google BigQuery APPROX_QUANTILES 并获得真正的四分位数

Google BigQuery APPROX_QUANTILES and getting true quartiles

statistics

percentile

quantile

google-bigquery

quartile