GROUP_BYs 的两个 LEFT JOIN 的 GROUP_CONCAT 的奇怪重复行为
Strange duplicate behavior from GROUP_CONCAT of two LEFT JOINs of GROUP_BYs
Here 是我所有表的结构和查询 （请关注下面附加的 last 查询） .正如您在 fiddle 中看到的，这里是 当前输出：
+---------+-----------+-------+------------+--------------+
| user_id | user_name | score | reputation | top_two_tags |
+---------+-----------+-------+------------+--------------+
| 1 | Jack | 0 | 18 | css,mysql |
| 4 | James | 1 | 5 | html |
| 2 | Peter | 0 | 0 | null |
| 3 | Ali | 0 | 0 | null |
+---------+-----------+-------+------------+--------------+
正确无误。
现在我又多了一个存在"category"。每个 post 罐头只有一个类别。而且我还想为每个用户获得前两个类别。 here 是我的新查询。正如您在结果中看到的，发生了一些重复：
+---------+-----------+-------+------------+--------------+------------------------+
| user_id | user_name | score | reputation | top_two_tags | top_two_categories |
+---------+-----------+-------+------------+--------------+------------------------+
| 1 | Jack | 0 | 18 | css,css | technology,technology |
| 4 | James | 1 | 5 | html | political |
| 2 | Peter | 0 | 0 | null | null |
| 3 | Ali | 0 | 0 | null | null |
+---------+-----------+-------+------------+--------------+------------------------+
看到了吗？ css,css、technology, technology。为什么这些是重复的？我刚刚为 categories 添加了一个 LEFT JOIN，与 tags 完全一样。但它没有按预期工作，甚至影响了标签。
无论如何，这是预期的结果：
+---------+-----------+-------+------------+--------------+------------------------+
| user_id | user_name | score | reputation | top_two_tags | category |
+---------+-----------+-------+------------+--------------+------------------------+
| 1 | Jack | 0 | 18 | css,mysql | technology,social |
| 4 | James | 1 | 5 | html | political |
| 2 | Peter | 0 | 0 | null | null |
| 3 | Ali | 0 | 0 | null | null |
+---------+-----------+-------+------------+--------------+------------------------+
有人知道我该怎么做吗？
CREATE TABLE users(id integer PRIMARY KEY, user_name varchar(5));
CREATE TABLE tags(id integer NOT NULL PRIMARY KEY, tag varchar(5));
CREATE TABLE reputations(
id integer PRIMARY KEY,
post_id integer /* REFERENCES posts(id) */,
user_id integer REFERENCES users(id),
score integer,
reputation integer,
date_time integer);
CREATE TABLE post_tag(
post_id integer /* REFERENCES posts(id) */,
tag_id integer REFERENCES tags(id),
PRIMARY KEY (post_id, tag_id));
CREATE TABLE categories(id INTEGER NOT NULL PRIMARY KEY, category varchar(10) NOT NULL);
CREATE TABLE post_category(
post_id INTEGER NOT NULL /* REFERENCES posts(id) */,
category_id INTEGER NOT NULL REFERENCES categories(id),
PRIMARY KEY(post_id, category_id)) ;
SELECT
q1.user_id, q1.user_name, q1.score, q1.reputation,
substring_index(group_concat(q2.tag ORDER BY q2.tag_reputation DESC SEPARATOR ','), ',', 2) AS top_two_tags,
substring_index(group_concat(q3.category ORDER BY q3.category_reputation DESC SEPARATOR ','), ',', 2) AS category
FROM
(SELECT
u.id AS user_Id,
u.user_name,
coalesce(sum(r.score), 0) as score,
coalesce(sum(r.reputation), 0) as reputation
FROM
users u
LEFT JOIN reputations r
ON r.user_id = u.id
AND r.date_time > 1500584821 /* unix_timestamp(DATE_SUB(now(), INTERVAL 1 WEEK)) */
GROUP BY
u.id, u.user_name
) AS q1
LEFT JOIN
(
SELECT
r.user_id AS user_id, t.tag, sum(r.reputation) AS tag_reputation
FROM
reputations r
JOIN post_tag pt ON pt.post_id = r.post_id
JOIN tags t ON t.id = pt.tag_id
WHERE
r.date_time > 1500584821 /* unix_timestamp(DATE_SUB(now(), INTERVAL 1 WEEK)) */
GROUP BY
user_id, t.tag
) AS q2
ON q2.user_id = q1.user_id
LEFT JOIN
(
SELECT
r.user_id AS user_id, c.category, sum(r.reputation) AS category_reputation
FROM
reputations r
JOIN post_category ct ON ct.post_id = r.post_id
JOIN categories c ON c.id = ct.category_id
WHERE
r.date_time > 1500584821 /* unix_timestamp(DATE_SUB(now(), INTERVAL 1 WEEK)) */
GROUP BY
user_id, c.category
) AS q3
ON q3.user_id = q1.user_id
GROUP BY
q1.user_id, q1.user_name, q1.score, q1.reputation
ORDER BY
q1.reputation DESC, q1.score DESC ;
您的第二个查询的格式为：
q1 -- PK user_id
LEFT JOIN (...
GROUP BY user_id, t.tag
) AS q2
ON q2.user_id = q1.user_id
LEFT JOIN (...
GROUP BY user_id, c.category
) AS q3
ON q3.user_id = q1.user_id
GROUP BY -- group_concats
内部 GROUP BY 导致 (user_id, t.tag) 和 (user_id, c.category) 为 keys/UNIQUEs。除此之外，我不会解决那些 GROUP BY。
TL;DR 当您将 (q1 JOIN q2) 加入 q3 时，它不在其中一个 key/UNIQUE 上，因此对于每个 user_id对于标签和类别的每种可能组合，您都会得到一行。因此，最终的 GROUP BY 输入每个（user_id，标签）和每个（user_id，类别）重复，并且不恰当地 GROUP_CONCATs 每个 user_id 重复标签和类别。正确的是 (q1 JOIN q2 GROUP BY) JOIN (q1 JOIN q3 GROUP BY)，其中所有连接都是公共的 key/UNIQUE (user_id) 并且没有虚假聚合。尽管有时您可以撤消此类虚假聚合。
正确的对称 INNER JOIN 方法：LEFT JOIN q1 & q2--1:many--然后 GROUP BY & GROUP_CONCAT（这是你的第一个查询所做的）；然后分别类似地 LEFT JOIN q1 & q3--1:many--然后 GROUP BY & GROUP_CONCAT;然后 INNER JOIN user_id--1:1.
上的两个结果
正确的对称标量子查询方法：SELECT 来自 q1 的 GROUP_CONCATs 作为 每个都有一个 GROUP BY。
一个正确的累积LEFT JOIN方法：LEFT JOIN q1 & q2--1:many--then GROUP BY & GROUP_CONCAT;然后 LEFT JOIN that & q3--1:many--然后 GROUP BY & GROUP_CONCAT.
像您的第二个查询这样的正确方法：您首先 LEFT JOIN q1 & q2--1:many。然后你 LEFT JOIN that & q3--many:1:many。它为出现 user_id 的标签和类别的每种可能组合提供一行。然后在你 GROUP BY 之后 GROUP_CONCAT--重复 (user_id, tag) 对和重复 (user_id, category) 对。这就是为什么你有重复的列表元素。但是将 DISTINCT 添加到 GROUP_CONCAT 会给出正确的结果。 （根据 wchiquito 的评论。）
与往常一样，您更喜欢根据实际 data/usage/statistics 查询计划和时间进行工程权衡。预期重复量的输入和统计数据）、实际查询的时间等。一个问题是 many:1:many JOIN 方法的额外行是否抵消了它对 GROUP BY 的保存。
-- cumulative LEFT JOIN approach
SELECT
q1.user_id, q1.user_name, q1.score, q1.reputation,
top_two_tags,
substring_index(group_concat(q3.category ORDER BY q3.category_reputation DESC SEPARATOR ','), ',', 2) AS category
FROM
-- your 1st query (less ORDER BY) AS q1
(SELECT
q1.user_id, q1.user_name, q1.score, q1.reputation,
substring_index(group_concat(q2.tag ORDER BY q2.tag_reputation DESC SEPARATOR ','), ',', 2) AS top_two_tags
FROM
(SELECT
u.id AS user_Id,
u.user_name,
coalesce(sum(r.score), 0) as score,
coalesce(sum(r.reputation), 0) as reputation
FROM
users u
LEFT JOIN reputations r
ON r.user_id = u.id
AND r.date_time > 1500584821 /* unix_timestamp(DATE_SUB(now(), INTERVAL 1 WEEK)) */
GROUP BY
u.id, u.user_name
) AS q1
LEFT JOIN
(
SELECT
r.user_id AS user_id, t.tag, sum(r.reputation) AS tag_reputation
FROM
reputations r
JOIN post_tag pt ON pt.post_id = r.post_id
JOIN tags t ON t.id = pt.tag_id
WHERE
r.date_time > 1500584821 /* unix_timestamp(DATE_SUB(now(), INTERVAL 1 WEEK)) */
GROUP BY
user_id, t.tag
) AS q2
ON q2.user_id = q1.user_id
GROUP BY
q1.user_id, q1.user_name, q1.score, q1.reputation
) AS q1
-- finish like your 2nd query
LEFT JOIN
(
SELECT
r.user_id AS user_id, c.category, sum(r.reputation) AS category_reputation
FROM
reputations r
JOIN post_category ct ON ct.post_id = r.post_id
JOIN categories c ON c.id = ct.category_id
WHERE
r.date_time > 1500584821 /* unix_timestamp(DATE_SUB(now(), INTERVAL 1 WEEK)) */
GROUP BY
user_id, c.category
) AS q3
ON q3.user_id = q1.user_id
GROUP BY
q1.user_id, q1.user_name, q1.score, q1.reputation
ORDER BY
q1.reputation DESC, q1.score DESC ;
Here 是我所有表的结构和查询 （请关注下面附加的 last 查询） .正如您在 fiddle 中看到的，这里是 当前输出：
+---------+-----------+-------+------------+--------------+
| user_id | user_name | score | reputation | top_two_tags |
+---------+-----------+-------+------------+--------------+
| 1 | Jack | 0 | 18 | css,mysql |
| 4 | James | 1 | 5 | html |
| 2 | Peter | 0 | 0 | null |
| 3 | Ali | 0 | 0 | null |
+---------+-----------+-------+------------+--------------+
正确无误。
现在我又多了一个存在"category"。每个 post 罐头只有一个类别。而且我还想为每个用户获得前两个类别。 here 是我的新查询。正如您在结果中看到的，发生了一些重复：
+---------+-----------+-------+------------+--------------+------------------------+
| user_id | user_name | score | reputation | top_two_tags | top_two_categories |
+---------+-----------+-------+------------+--------------+------------------------+
| 1 | Jack | 0 | 18 | css,css | technology,technology |
| 4 | James | 1 | 5 | html | political |
| 2 | Peter | 0 | 0 | null | null |
| 3 | Ali | 0 | 0 | null | null |
+---------+-----------+-------+------------+--------------+------------------------+
看到了吗？ css,css、technology, technology。为什么这些是重复的？我刚刚为 categories 添加了一个 LEFT JOIN，与 tags 完全一样。但它没有按预期工作，甚至影响了标签。
无论如何，这是预期的结果：
+---------+-----------+-------+------------+--------------+------------------------+
| user_id | user_name | score | reputation | top_two_tags | category |
+---------+-----------+-------+------------+--------------+------------------------+
| 1 | Jack | 0 | 18 | css,mysql | technology,social |
| 4 | James | 1 | 5 | html | political |
| 2 | Peter | 0 | 0 | null | null |
| 3 | Ali | 0 | 0 | null | null |
+---------+-----------+-------+------------+--------------+------------------------+
有人知道我该怎么做吗？
CREATE TABLE users(id integer PRIMARY KEY, user_name varchar(5));
CREATE TABLE tags(id integer NOT NULL PRIMARY KEY, tag varchar(5));
CREATE TABLE reputations(
id integer PRIMARY KEY,
post_id integer /* REFERENCES posts(id) */,
user_id integer REFERENCES users(id),
score integer,
reputation integer,
date_time integer);
CREATE TABLE post_tag(
post_id integer /* REFERENCES posts(id) */,
tag_id integer REFERENCES tags(id),
PRIMARY KEY (post_id, tag_id));
CREATE TABLE categories(id INTEGER NOT NULL PRIMARY KEY, category varchar(10) NOT NULL);
CREATE TABLE post_category(
post_id INTEGER NOT NULL /* REFERENCES posts(id) */,
category_id INTEGER NOT NULL REFERENCES categories(id),
PRIMARY KEY(post_id, category_id)) ;
SELECT
q1.user_id, q1.user_name, q1.score, q1.reputation,
substring_index(group_concat(q2.tag ORDER BY q2.tag_reputation DESC SEPARATOR ','), ',', 2) AS top_two_tags,
substring_index(group_concat(q3.category ORDER BY q3.category_reputation DESC SEPARATOR ','), ',', 2) AS category
FROM
(SELECT
u.id AS user_Id,
u.user_name,
coalesce(sum(r.score), 0) as score,
coalesce(sum(r.reputation), 0) as reputation
FROM
users u
LEFT JOIN reputations r
ON r.user_id = u.id
AND r.date_time > 1500584821 /* unix_timestamp(DATE_SUB(now(), INTERVAL 1 WEEK)) */
GROUP BY
u.id, u.user_name
) AS q1
LEFT JOIN
(
SELECT
r.user_id AS user_id, t.tag, sum(r.reputation) AS tag_reputation
FROM
reputations r
JOIN post_tag pt ON pt.post_id = r.post_id
JOIN tags t ON t.id = pt.tag_id
WHERE
r.date_time > 1500584821 /* unix_timestamp(DATE_SUB(now(), INTERVAL 1 WEEK)) */
GROUP BY
user_id, t.tag
) AS q2
ON q2.user_id = q1.user_id
LEFT JOIN
(
SELECT
r.user_id AS user_id, c.category, sum(r.reputation) AS category_reputation
FROM
reputations r
JOIN post_category ct ON ct.post_id = r.post_id
JOIN categories c ON c.id = ct.category_id
WHERE
r.date_time > 1500584821 /* unix_timestamp(DATE_SUB(now(), INTERVAL 1 WEEK)) */
GROUP BY
user_id, c.category
) AS q3
ON q3.user_id = q1.user_id
GROUP BY
q1.user_id, q1.user_name, q1.score, q1.reputation
ORDER BY
q1.reputation DESC, q1.score DESC ;
您的第二个查询的格式为：
q1 -- PK user_id
LEFT JOIN (...
GROUP BY user_id, t.tag
) AS q2
ON q2.user_id = q1.user_id
LEFT JOIN (...
GROUP BY user_id, c.category
) AS q3
ON q3.user_id = q1.user_id
GROUP BY -- group_concats
内部 GROUP BY 导致 (user_id, t.tag) 和 (user_id, c.category) 为 keys/UNIQUEs。除此之外，我不会解决那些 GROUP BY。
TL;DR 当您将 (q1 JOIN q2) 加入 q3 时，它不在其中一个 key/UNIQUE 上，因此对于每个 user_id对于标签和类别的每种可能组合，您都会得到一行。因此，最终的 GROUP BY 输入每个（user_id，标签）和每个（user_id，类别）重复，并且不恰当地 GROUP_CONCATs 每个 user_id 重复标签和类别。正确的是 (q1 JOIN q2 GROUP BY) JOIN (q1 JOIN q3 GROUP BY)，其中所有连接都是公共的 key/UNIQUE (user_id) 并且没有虚假聚合。尽管有时您可以撤消此类虚假聚合。
正确的对称 INNER JOIN 方法：LEFT JOIN q1 & q2--1:many--然后 GROUP BY & GROUP_CONCAT（这是你的第一个查询所做的）；然后分别类似地 LEFT JOIN q1 & q3--1:many--然后 GROUP BY & GROUP_CONCAT;然后 INNER JOIN user_id--1:1.
上的两个结果正确的对称标量子查询方法：SELECT 来自 q1 的 GROUP_CONCATs 作为
一个正确的累积LEFT JOIN方法：LEFT JOIN q1 & q2--1:many--then GROUP BY & GROUP_CONCAT;然后 LEFT JOIN that & q3--1:many--然后 GROUP BY & GROUP_CONCAT.
像您的第二个查询这样的正确方法：您首先 LEFT JOIN q1 & q2--1:many。然后你 LEFT JOIN that & q3--many:1:many。它为出现 user_id 的标签和类别的每种可能组合提供一行。然后在你 GROUP BY 之后 GROUP_CONCAT--重复 (user_id, tag) 对和重复 (user_id, category) 对。这就是为什么你有重复的列表元素。但是将 DISTINCT 添加到 GROUP_CONCAT 会给出正确的结果。 （根据 wchiquito 的评论。）
与往常一样，您更喜欢根据实际 data/usage/statistics 查询计划和时间进行工程权衡。预期重复量的输入和统计数据）、实际查询的时间等。一个问题是 many:1:many JOIN 方法的额外行是否抵消了它对 GROUP BY 的保存。
-- cumulative LEFT JOIN approach
SELECT
q1.user_id, q1.user_name, q1.score, q1.reputation,
top_two_tags,
substring_index(group_concat(q3.category ORDER BY q3.category_reputation DESC SEPARATOR ','), ',', 2) AS category
FROM
-- your 1st query (less ORDER BY) AS q1
(SELECT
q1.user_id, q1.user_name, q1.score, q1.reputation,
substring_index(group_concat(q2.tag ORDER BY q2.tag_reputation DESC SEPARATOR ','), ',', 2) AS top_two_tags
FROM
(SELECT
u.id AS user_Id,
u.user_name,
coalesce(sum(r.score), 0) as score,
coalesce(sum(r.reputation), 0) as reputation
FROM
users u
LEFT JOIN reputations r
ON r.user_id = u.id
AND r.date_time > 1500584821 /* unix_timestamp(DATE_SUB(now(), INTERVAL 1 WEEK)) */
GROUP BY
u.id, u.user_name
) AS q1
LEFT JOIN
(
SELECT
r.user_id AS user_id, t.tag, sum(r.reputation) AS tag_reputation
FROM
reputations r
JOIN post_tag pt ON pt.post_id = r.post_id
JOIN tags t ON t.id = pt.tag_id
WHERE
r.date_time > 1500584821 /* unix_timestamp(DATE_SUB(now(), INTERVAL 1 WEEK)) */
GROUP BY
user_id, t.tag
) AS q2
ON q2.user_id = q1.user_id
GROUP BY
q1.user_id, q1.user_name, q1.score, q1.reputation
) AS q1
-- finish like your 2nd query
LEFT JOIN
(
SELECT
r.user_id AS user_id, c.category, sum(r.reputation) AS category_reputation
FROM
reputations r
JOIN post_category ct ON ct.post_id = r.post_id
JOIN categories c ON c.id = ct.category_id
WHERE
r.date_time > 1500584821 /* unix_timestamp(DATE_SUB(now(), INTERVAL 1 WEEK)) */
GROUP BY
user_id, c.category
) AS q3
ON q3.user_id = q1.user_id
GROUP BY
q1.user_id, q1.user_name, q1.score, q1.reputation
ORDER BY
q1.reputation DESC, q1.score DESC ;