MySQL 上的 0.07 秒查询在 MariaDB 上需要 11.68 秒?

0.07s query on MySQL takes 11.68s on MariaDB?

涉及不同的硬件(MySQL 在我的笔记本电脑上,MariaDB 在服务器上)但通常差异最多是 2 倍而不是 166 倍!

这些表在每个实例上包含相同的数据(_cache_card 中有 18,000 行,card_legality 中有 157,000 行)。

查询

SELECT * FROM _cache_card AS c 
WHERE c.id IN (SELECT card_id FROM card_legality WHERE format_id = 35);

解释

MariaDB:

+------+--------------+---------------+------+---------------------------------+-----------+---------+-------+-------+-------------------------------------------------+
| id   | select_type  | table         | type | possible_keys                   | key       | key_len | ref   | rows  | Extra                                           |
+------+--------------+---------------+------+---------------------------------+-----------+---------+-------+-------+-------------------------------------------------+
|    1 | PRIMARY      | <subquery2>   | ALL  | distinct_key                    | NULL      | NULL    | NULL  |  9414 |                                                 |
|    1 | PRIMARY      | c             | ALL  | NULL                            | NULL      | NULL    | NULL  | 18567 | Using where; Using join buffer (flat, BNL join) |
|    2 | MATERIALIZED | card_legality | ref  | format_id,idx_card_id_format_id | format_id | 4       | const |  9414 |                                                 |
+------+--------------+---------------+------+---------------------------------+-----------+---------+-------+-------+-------------------------------------------------+

MySQL:

+----+--------------+---------------+------------+--------+---------------------------------+------------+---------+------------+-------+----------+-------------+
| id | select_type  | table         | partitions | type   | possible_keys                   | key        | key_len | ref        | rows  | filtered | Extra       |
+----+--------------+---------------+------------+--------+---------------------------------+------------+---------+------------+-------+----------+-------------+
|  1 | SIMPLE       | c             | NULL       | ALL    | NULL                            | NULL       | NULL    | NULL       | 18055 |   100.00 | Using where |
|  1 | SIMPLE       | <subquery2>   | NULL       | eq_ref | <auto_key>                      | <auto_key> | 4       | cards.c.id |     1 |   100.00 | NULL        |
|  2 | MATERIALIZED | card_legality | NULL       | ref    | format_id,idx_card_id_format_id | format_id  | 4       | const      | 37828 |   100.00 | NULL        |
+----+--------------+---------------+------------+--------+---------------------------------+------------+---------+------------+-------+----------+-------------+

创建TABLE

两者:

CREATE TABLE `card_legality` (
  `id` int(11) NOT NULL AUTO_INCREMENT,
  `card_id` int(11) NOT NULL,
  `format_id` int(11) NOT NULL,
  `legality` varchar(190) COLLATE utf8mb4_unicode_ci DEFAULT NULL,
  PRIMARY KEY (`id`),
  KEY `format_id` (`format_id`),
  KEY `idx_card_id_format_id` (`card_id`,`format_id`,`legality`),
  CONSTRAINT `card_legality_ibfk_1` FOREIGN KEY (`card_id`) REFERENCES `card` (`id`),
  CONSTRAINT `card_legality_ibfk_2` FOREIGN KEY (`format_id`) REFERENCES `format` (`id`)
) ENGINE=InnoDB AUTO_INCREMENT=1190863 DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_unicode_ci

(此处的输出是逐个字符相同的。)

MariaDB:

CREATE TABLE `_cache_card` (
  `id` int(11) NOT NULL DEFAULT 0,
  `layout` varchar(190) CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci NOT NULL,
  `face_id` int(11) NOT NULL DEFAULT 0,
  `name` longtext CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci DEFAULT NULL,
  `mana_cost` mediumtext CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci DEFAULT NULL,
  `cmc` double DEFAULT NULL,
  `power` mediumtext CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci DEFAULT NULL,
  `toughness` mediumtext CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci DEFAULT NULL,
  `loyalty` mediumtext CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci DEFAULT NULL,
  `type` longtext CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci DEFAULT NULL,
  `text` mediumtext CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci DEFAULT NULL,
  `search_text` mediumtext CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci DEFAULT NULL,
  `image_name` mediumtext CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci DEFAULT NULL,
  `hand` mediumtext DEFAULT NULL,
  `life` mediumtext DEFAULT NULL,
  `starter` mediumtext DEFAULT NULL,
  `position` mediumtext CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci DEFAULT NULL,
  `name_ascii` longtext CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci DEFAULT NULL,
  `card_id` mediumtext DEFAULT NULL,
  `names` mediumtext CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci DEFAULT NULL,
  `legalities` mediumtext CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci DEFAULT NULL,
  `pd_legal` int(1) DEFAULT NULL,
  `bugs` mediumtext CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci DEFAULT NULL,
  KEY `idx_name_name` (`name`(142))
) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4

MySQL:

CREATE TABLE `_cache_card` (
  `id` int(11) NOT NULL DEFAULT '0',
  `layout` varchar(190) COLLATE utf8mb4_unicode_ci NOT NULL,
  `face_id` int(11) NOT NULL DEFAULT '0',
  `name` longtext COLLATE utf8mb4_unicode_ci,
  `mana_cost` mediumtext COLLATE utf8mb4_unicode_ci,
  `cmc` double DEFAULT NULL,
  `power` mediumtext COLLATE utf8mb4_unicode_ci,
  `toughness` mediumtext COLLATE utf8mb4_unicode_ci,
  `loyalty` mediumtext COLLATE utf8mb4_unicode_ci,
  `type` longtext COLLATE utf8mb4_unicode_ci,
  `text` mediumtext COLLATE utf8mb4_unicode_ci,
  `search_text` mediumtext COLLATE utf8mb4_unicode_ci,
  `image_name` mediumtext COLLATE utf8mb4_unicode_ci,
  `hand` mediumtext CHARACTER SET utf8mb4,
  `life` mediumtext CHARACTER SET utf8mb4,
  `starter` mediumtext CHARACTER SET utf8mb4,
  `position` mediumtext COLLATE utf8mb4_unicode_ci,
  `name_ascii` longtext COLLATE utf8mb4_unicode_ci,
  `card_id` mediumtext CHARACTER SET utf8mb4,
  `names` mediumtext COLLATE utf8mb4_unicode_ci,
  `legalities` mediumtext COLLATE utf8mb4_unicode_ci,
  `pd_legal` int(1) DEFAULT NULL,
  `bugs` mediumtext COLLATE utf8mb4_unicode_ci,
  KEY `idx_name_name` (`name`(142))
) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_unicode_ci

其他详细信息

MariaDB:

version=10.2.16-MariaDB
optimizer_switch=index_merge=on,index_merge_union=on,index_merge_sort_union=on,index_merge_intersection=on,index_merge_sort_intersection=off,engine_condition_pushdown=off,index_condition_pushdown=on,derived_merge=on,derived_with_keys=on,firstmatch=on,loosescan=on,materialization=on,in_to_exists=on,semijoin=on,partial_match_rowid_merge=on,partial_match_table_scan=on,subquery_cache=on,mrr=off,mrr_cost_based=off,mrr_sort_keys=off,outer_join_with_cache=on,semijoin_with_cache=on,join_cache_incremental=on,join_cache_hashed=on,join_cache_bka=on,optimize_join_buffer_size=off,table_elimination=on,extended_keys=on,exists_to_in=on,orderby_uses_equalities=on,condition_pushdown_for_derived=on

MySQL:

version=5.7.17
optimizer_switch=index_merge=on,index_merge_union=on,index_merge_sort_union=on,index_merge_intersection=on,engine_condition_pushdown=on,index_condition_pushdown=on,mrr=on,mrr_cost_based=on,block_nested_loop=on,batched_key_access=off,materialization=on,semijoin=on,loosescan=on,firstmatch=on,duplicateweedout=on,subquery_materialization_cost_based=on,use_index_extensions=on,condition_fanout_filter=on,derived_merge=on

在 card_legality(format_id,card_id)

上添加索引

这不知何故使事情变得变慢

 CREATE INDEX idx_format_id_card_id ON card_legality(format_id,card_id); 

现在超过 15 秒。

EXPLAIN(在 MariaDB 上)说:

+------+--------------+---------------+------+---------------------------------------------+-----------------------+---------+-------+-------+-------------------------------------------------+
| id   | select_type  | table         | type | possible_keys                               | key                   | key_len | ref   | rows  | Extra                                           |
+------+--------------+---------------+------+---------------------------------------------+-----------------------+---------+-------+-------+-------------------------------------------------+
|    1 | PRIMARY      | <subquery2>   | ALL  | distinct_key                                | NULL                  | NULL    | NULL  | 16942 |                                                 |
|    1 | PRIMARY      | c             | ALL  | NULL                                        | NULL                  | NULL    | NULL  | 17653 | Using where; Using join buffer (flat, BNL join) |
|    2 | MATERIALIZED | card_legality | ref  | idx_card_id_format_id,idx_format_id_card_id | idx_format_id_card_id | 4       | const | 16942 | Using index                                     |
+------+--------------+---------------+------+---------------------------------------------+-----------------------+---------+-------+-------+-------------------------------------------------+

我要添加索引:

CREATE INDEX idx_name ON card_legality(format_id, card_id);

DBFiddle Demo


SELECT * 
FROM _cache_card AS c 
WHERE c.id IN (SELECT card_id FROM card_legality WHERE format_id = 35);
               -- covering index created before

IN 从优化的角度来看可能很棘手。我想知道这是否在两个系统上效果更好:

SELECT c.* 
FROM _cache_card  c 
WHERE EXISTS (SELECT 1
              FROM card_legality cl
              WHERE cl.card_id = c.id AND format_id = 35
             );

目前还不清楚为什么 MariaDB 目前使用这个次优的执行计划。它可能假设您的数据分布有问题(尽管我不确定在哪种情况下这将是最佳计划)。使用 optimize table card_legality, _cache_card; 修复统计数据可能会有所帮助。

如果没有,并且由于我们在评论中确定 (card_id,format_id) 是唯一的,我会尝试添加以下索引

CREATE UNIQUE INDEX uidx_card_legality ON card_legality(format_id, card_id)

并使用

SELECT c.* 
FROM _cache_card AS c
JOIN card_legality l FORCE INDEX (uidx_card_legality)
ON l.card_id = c.id AND l.format_id = 35;

这基本上是 MySQL 当前执行您的查询的方式(同时动态创建此索引),尽管您似乎已经尝试过使用 Lukasz 的答案创建该索引但没有成功。

您应该删除 force index(这只是为了绝对确保 MariaDB 没有回旋余地来做其他事情)并检查 MariaDB/MySQL 是否仍在使用它。还要测试 format_id 的其他值会发生什么,因为 35 可能是异常值(例如,您可能只有少数该格式的条目),为此优化执行计划可能会减慢查询所有其他值。当然,请确保您正在比较相似的结果集,就好像 MariaDB 有例如格式 35 有 10k 个条目,而 MySQL 有 none,这不是一场公平的战斗。

差异的原因始于 MySQL 5.6 和 MariaDB 10.0 的出现——它们 分别 开发了几个改进的优化。您遇到了一个涉及 IN 的构造,其中一个做出了另一个没有(可能还没有)拾取的重大改进。

只要 JOIN 可行,就避免 IN ( SELECT ... )

EXISTS( SELECT 1 ... ) 是另一个可以试验的结构。

索引:

PRIMARY KEY on every table !
card_legality:  INDEX(format_id, card_id) -- in this order
_cache_card:  (id)  -- This seems like a serious omission !

一些会影响性能的事情:使用 *TEXT 而较小的 VARCHAR 就足够了。

计时时,运行查询两次。第一个将内容复制到 RAM (buffer_pool);第二个比较现实。

多少内存?每个 innodb_buffer_pool_size 的值是多少?