为什么这个 "Mysql" 查询有时会挂起？
Why does this "Mysql" query hangs once a while?
我正在使用复杂的 MySQL query 通过 Key,Value table 获取数据。为了实现这一点，我们正在做乘法 self joins。一开始我们认为问题是 MySQL 中的 bit-wise 操作，因为它们不能用 indexes 完成。 但是 没有使用bit-wise 操作，问题仍然存在。即，如果将所有 bit-wise OR 更改为 equal (==)
，也会发生同样的情况
查询
以下query通常运行s在500毫秒左右。但是 有一次 MySQL 运行 这个查询超过 10 秒！！ 甚至更多..
SELECT DISTINCT sometable1_.id AS col_0_0_,
sometable1_.hebrewName AS col_1_0_,
sometable1_.urlBestBigPhoto AS col_2_0_
FROM sometable1 sometable1_
INNER JOIN sometable2 characters1_ ON sometable1_.id=characters1_.term_id
INNER JOIN sometable2 characters2_ ON sometable1_.id=characters2_.term_id
INNER JOIN sometable2 characters3_ ON sometable1_.id=characters3_.term_id
INNER JOIN sometable2 characters4_ ON sometable1_.id=characters4_.term_id
INNER JOIN sometable2 characters5_ ON sometable1_.id=characters5_.term_id
INNER JOIN sometable2 characters6_ ON sometable1_.id=characters6_.term_id
INNER JOIN sometable2 characters7_ ON sometable1_.id=characters7_.term_id
INNER JOIN sometable2 characters8_ ON sometable1_.id=characters8_.term_id
INNER JOIN sometable2 characters9_ ON sometable1_.id=characters9_.term_id
WHERE sometable1_.category="ABC"
AND (characters2_.value = 1<>0
OR characters2_.value=0)
AND characters2_.name="somevalue2"
AND (characters3_.value & 2<>0
OR characters3_.value=0)
AND characters3_.name="somevalue3"
AND (characters4_.value = 128<>0
OR characters4_.value=0)
AND characters4_.name="somevalue4"
AND (characters5_.value = 1<>0
OR characters5_.value=0)
AND characters5_.name="somevalue5"
AND (characters6_.value = 16392<>0
OR characters6_.value=0)
AND characters6_.name="somevalue6"
AND (characters7_.value = 1<>0 OR characters7_.value=0)
AND characters7_.name="somevalue7"
AND (characters8_.value = 256<>0 OR characters8_.value=0)
AND characters8_.name="somevalue8"
AND (characters9_.value = 124<>0 OR characters9_.value=0)
AND characters9_.name="somevalue9"
AND (characters1_.name="somevalue10" OR characters1_.name="somevalue11")
GROUP BY sometable1_.id,
characters9_.term_id,
characters8_.term_id,
characters7_.term_id,
characters6_.term_id,
characters5_.term_id,
characters4_.term_id,
characters3_.term_id,
characters2_.term_id
ORDER BY sum(characters1_.value) DESC, sometable1_.text ASC LIMIT 10
Mysql解释
使用 explain 后我们得到
╔════╦═════════════╦═══════════════╦════════╦═════════════════════╦═════════╦═════════╦══════════════════════╦══════╦═════════════════════════════════════════════════════════════════════╗
║ id ║ select_type ║ table ║ type ║ possible_keys ║ key ║ key_len ║ ref ║ rows ║ Extra ║
╠════╬═════════════╬═══════════════╬════════╬═════════════════════╬═════════╬═════════╬══════════════════════╬══════╬═════════════════════════════════════════════════════════════════════╣
║ 1 ║ SIMPLE ║ characters1_ ║ range ║ FK_47,name ║ name ║ 77 ║ NULL ║ 2146 ║ Using index condition; Using where; Using temporary; Using filesort ║
║ 1 ║ SIMPLE ║ sometable1 ║ eq_ref ║ PRIMARY,category,id ║ PRIMARY ║ 8 ║ characters1_.item_id ║ 1 ║ Using where ║
║ 1 ║ SIMPLE ║ characters5_ ║ ref ║ FK_47,name,value ║ FK_47 ║ 9 ║ characters1_.item_id ║ 9 ║ Using where; Distinct ║
║ 1 ║ SIMPLE ║ characters7_ ║ ref ║ FK_47,name,value ║ FK_47 ║ 9 ║ characters1_.item_id ║ 9 ║ Using where; Distinct ║
║ 1 ║ SIMPLE ║ characters2_ ║ ref ║ FK_47,name,value ║ FK_47 ║ 9 ║ characters1_.item_id ║ 9 ║ Using where; Distinct ║
║ 1 ║ SIMPLE ║ characters3_ ║ ref ║ FK_47,name,value ║ FK_47 ║ 9 ║ characters1_.item_id ║ 9 ║ Using where; Distinct ║
║ 1 ║ SIMPLE ║ characters9_ ║ ref ║ FK_47,name,value ║ FK_47 ║ 9 ║ characters1_.item_id ║ 9 ║ Using where; Distinct ║
║ 1 ║ SIMPLE ║ characters8_ ║ ref ║ FK_47,name,value ║ FK_47 ║ 9 ║ characters1_.item_id ║ 9 ║ Using where; Distinct ║
║ 1 ║ SIMPLE ║ characters6_ ║ ref ║ FK_47,name,value ║ FK_47 ║ 9 ║ characters1_.item_id ║ 9 ║ Using where; Distinct ║
║ 1 ║ SIMPLE ║ characters4_ ║ ref ║ FK_47,name,value ║ FK_47 ║ 9 ║ characters1_.item_id ║ 9 ║ Using where; Distinct ║
╚════╩═════════════╩═══════════════╩════════╩═════════════════════╩═════════╩═════════╩══════════════════════╩══════╩═════════════════════════════════════════════════════════════════════╝
一些想法
我认为这个问题与 group by 和 sort by 有关。 但 与 bit-wise 情况相同，当 group by 和 sort by 从查询中删除时 运行-time 仍然可以超过6秒！
知道导致性能不佳的原因是什么吗？
谢谢，
橡树
p.s
注意：这不能是 cache issue，因为第一个 运行 可能非常快，只有少数 运行 之后查询会执行得非常糟糕。
好的，我花了一段时间才弄明白。似乎由于某些 duplicates keys 查询真的很慢。即如果有 table
id, key ,value
并且 key 不能出现两次相同的 id。然后确保您在 id,key 上具有唯一性，即
ALTER TABLE key_value_table ADD UNIQUE( key, id);
这样做的好处可见一斑（当然及时）explain查询：
range 不再使用，现在 MySQL 对所有 select 个句子
做 ref
kvtable中不需要id；你确实需要 PRIMARY KEY(term_id, key).
InnoDB 将是一个奖励；那么每个 term_id 的 key-values 将是 'clustered' 在一起。
我正在使用复杂的 MySQL query 通过 Key,Value table 获取数据。为了实现这一点，我们正在做乘法 self joins。一开始我们认为问题是 MySQL 中的 bit-wise 操作，因为它们不能用 indexes 完成。 但是 没有使用bit-wise 操作，问题仍然存在。即，如果将所有 bit-wise OR 更改为 equal (==)
查询
以下query通常运行s在500毫秒左右。但是 有一次 MySQL 运行 这个查询超过 10 秒！！ 甚至更多..
SELECT DISTINCT sometable1_.id AS col_0_0_,
sometable1_.hebrewName AS col_1_0_,
sometable1_.urlBestBigPhoto AS col_2_0_
FROM sometable1 sometable1_
INNER JOIN sometable2 characters1_ ON sometable1_.id=characters1_.term_id
INNER JOIN sometable2 characters2_ ON sometable1_.id=characters2_.term_id
INNER JOIN sometable2 characters3_ ON sometable1_.id=characters3_.term_id
INNER JOIN sometable2 characters4_ ON sometable1_.id=characters4_.term_id
INNER JOIN sometable2 characters5_ ON sometable1_.id=characters5_.term_id
INNER JOIN sometable2 characters6_ ON sometable1_.id=characters6_.term_id
INNER JOIN sometable2 characters7_ ON sometable1_.id=characters7_.term_id
INNER JOIN sometable2 characters8_ ON sometable1_.id=characters8_.term_id
INNER JOIN sometable2 characters9_ ON sometable1_.id=characters9_.term_id
WHERE sometable1_.category="ABC"
AND (characters2_.value = 1<>0
OR characters2_.value=0)
AND characters2_.name="somevalue2"
AND (characters3_.value & 2<>0
OR characters3_.value=0)
AND characters3_.name="somevalue3"
AND (characters4_.value = 128<>0
OR characters4_.value=0)
AND characters4_.name="somevalue4"
AND (characters5_.value = 1<>0
OR characters5_.value=0)
AND characters5_.name="somevalue5"
AND (characters6_.value = 16392<>0
OR characters6_.value=0)
AND characters6_.name="somevalue6"
AND (characters7_.value = 1<>0 OR characters7_.value=0)
AND characters7_.name="somevalue7"
AND (characters8_.value = 256<>0 OR characters8_.value=0)
AND characters8_.name="somevalue8"
AND (characters9_.value = 124<>0 OR characters9_.value=0)
AND characters9_.name="somevalue9"
AND (characters1_.name="somevalue10" OR characters1_.name="somevalue11")
GROUP BY sometable1_.id,
characters9_.term_id,
characters8_.term_id,
characters7_.term_id,
characters6_.term_id,
characters5_.term_id,
characters4_.term_id,
characters3_.term_id,
characters2_.term_id
ORDER BY sum(characters1_.value) DESC, sometable1_.text ASC LIMIT 10
Mysql解释
使用 explain 后我们得到
╔════╦═════════════╦═══════════════╦════════╦═════════════════════╦═════════╦═════════╦══════════════════════╦══════╦═════════════════════════════════════════════════════════════════════╗
║ id ║ select_type ║ table ║ type ║ possible_keys ║ key ║ key_len ║ ref ║ rows ║ Extra ║
╠════╬═════════════╬═══════════════╬════════╬═════════════════════╬═════════╬═════════╬══════════════════════╬══════╬═════════════════════════════════════════════════════════════════════╣
║ 1 ║ SIMPLE ║ characters1_ ║ range ║ FK_47,name ║ name ║ 77 ║ NULL ║ 2146 ║ Using index condition; Using where; Using temporary; Using filesort ║
║ 1 ║ SIMPLE ║ sometable1 ║ eq_ref ║ PRIMARY,category,id ║ PRIMARY ║ 8 ║ characters1_.item_id ║ 1 ║ Using where ║
║ 1 ║ SIMPLE ║ characters5_ ║ ref ║ FK_47,name,value ║ FK_47 ║ 9 ║ characters1_.item_id ║ 9 ║ Using where; Distinct ║
║ 1 ║ SIMPLE ║ characters7_ ║ ref ║ FK_47,name,value ║ FK_47 ║ 9 ║ characters1_.item_id ║ 9 ║ Using where; Distinct ║
║ 1 ║ SIMPLE ║ characters2_ ║ ref ║ FK_47,name,value ║ FK_47 ║ 9 ║ characters1_.item_id ║ 9 ║ Using where; Distinct ║
║ 1 ║ SIMPLE ║ characters3_ ║ ref ║ FK_47,name,value ║ FK_47 ║ 9 ║ characters1_.item_id ║ 9 ║ Using where; Distinct ║
║ 1 ║ SIMPLE ║ characters9_ ║ ref ║ FK_47,name,value ║ FK_47 ║ 9 ║ characters1_.item_id ║ 9 ║ Using where; Distinct ║
║ 1 ║ SIMPLE ║ characters8_ ║ ref ║ FK_47,name,value ║ FK_47 ║ 9 ║ characters1_.item_id ║ 9 ║ Using where; Distinct ║
║ 1 ║ SIMPLE ║ characters6_ ║ ref ║ FK_47,name,value ║ FK_47 ║ 9 ║ characters1_.item_id ║ 9 ║ Using where; Distinct ║
║ 1 ║ SIMPLE ║ characters4_ ║ ref ║ FK_47,name,value ║ FK_47 ║ 9 ║ characters1_.item_id ║ 9 ║ Using where; Distinct ║
╚════╩═════════════╩═══════════════╩════════╩═════════════════════╩═════════╩═════════╩══════════════════════╩══════╩═════════════════════════════════════════════════════════════════════╝
一些想法
我认为这个问题与 group by 和 sort by 有关。 但 与 bit-wise 情况相同，当 group by 和 sort by 从查询中删除时 运行-time 仍然可以超过6秒！
知道导致性能不佳的原因是什么吗？
谢谢， 橡树
p.s
注意：这不能是 cache issue，因为第一个 运行 可能非常快，只有少数 运行 之后查询会执行得非常糟糕。
好的，我花了一段时间才弄明白。似乎由于某些 duplicates keys 查询真的很慢。即如果有 table
id, key ,value
并且 key 不能出现两次相同的 id。然后确保您在 id,key 上具有唯一性，即
ALTER TABLE key_value_table ADD UNIQUE( key, id);
这样做的好处可见一斑（当然及时）explain查询：
range 不再使用，现在 MySQL 对所有 select 个句子
ref
kvtable中不需要id；你确实需要 PRIMARY KEY(term_id, key).
InnoDB 将是一个奖励；那么每个 term_id 的 key-values 将是 'clustered' 在一起。