为什么这个 SQL 查询非常慢?

Why is this SQL query extremely slow?

更新:似乎查询缓存未激活,否则第二次执行同一查询时性能会好得多,因为 MySQL 应该已经缓存了它。我有 MariaDB 版本:10.2.41- 协议版本:10,确实从 MariaDB 10.1.7 开始默认禁用查询缓存。 在我的 VPS 配置上启用它是个好主意吗?或者我最好先升级它?

我有一个 MySQL 查询在具有 1GB RAM 和 1 CPU 核心的 CentOS VPS 上执行得非常慢。

查询需要 400 - 900 毫秒 来执行 (!!),我想弄清楚如何优化。

它超过了几个 table,其中一些相当大(例如 Wordpress 的 wp_postmeta)。 但显然所有涉及的列都已正确编入索引,所以我想就如何操作寻求建议,或者唯一可能的优化是升级 VPS.

提前致谢!

这是查询和 EXPLAIN 输出

EXPLAIN SELECT c.post_id AS wc_variation_id,v.original_price_rand,v.price,c.meta_value AS color,s.meta_value AS size,bp.tier,t.discount_percent,MAX(IFNULL(tag.extra_price_tag,0)) AS extra_price_tag,MAX(IFNULL(lab.extra_price_label,0)) AS extra_price_label,

i.uri,i.branded,i.design_cost,ib.uri AS uri_back,ib.branded AS branded_back,ib.design_cost AS design_cost_back
FROM wp_postmeta c
JOIN brandly_tiers_bp_balances bp
JOIN brandly_tiers t ON t.role_name=bp.tier
JOIN wix_images i
JOIN wix_images_back ib
JOIN wix_imported_product_variations v ON v.wc_id=c.post_id
LEFT JOIN brandly_reseller_tags rt ON rt.user_id=bp.user_id
LEFT JOIN brandly_reseller_branded_packaging_sizes tag ON tag.size_label=rt.size_label
LEFT JOIN brandly_reseller_labels rl ON rl.user_id=bp.user_id
LEFT JOIN brandly_reseller_branded_packaging_sizes lab ON lab.size_label=rl.size_label
JOIN wp_postmeta s ON s.post_id=c.post_id
WHERE c.post_id IN (SELECT DISTINCT(wc_id) FROM wix_imported_product_variations WHERE wc_product_id = 181189)
AND s.meta_key = 'attribute_pa_size'
AND c.meta_key = 'attribute_pa_colour'
AND i.wix_variation_id=v.wix_id
AND ib.wix_variation_id=v.wix_id
AND v.store_id=36
AND bp.user_id=11
GROUP BY c.post_id
ORDER BY color,size

输出:

id select_type table type possible_keys key key_len ref rows Extra
1 PRIMARY bp const PRIMARY,tier PRIMARY 8 const 1 Using temporary; Using filesort
1 PRIMARY t const PRIMARY PRIMARY 52 const 1
1 PRIMARY wix_imported_product_variations ref wc_product_id,wc_id wc_product_id 8 const 3 Start temporary
1 PRIMARY rt ref PRIMARY PRIMARY 8 const 5 Using index
1 PRIMARY tag eq_ref PRIMARY PRIMARY 32 brandly_xvfkg.rt.size_label 1 Using where
1 PRIMARY c ref post_id,meta_key post_id 8 brandly_xvfkg.wix_imported_product_variations.wc_id 14 Using index condition; Using where; End temporary
1 PRIMARY v ref PRIMARY,store_id,wc_id wc_id 8 brandly_xvfkg.c.post_id 1 Using index condition; Using where
1 PRIMARY ib ref wix_variation_id wix_variation_id 103 brandly_xvfkg.v.wix_id 3
1 PRIMARY i ref wix_variation_id wix_variation_id 103 brandly_xvfkg.v.wix_id 3
1 PRIMARY s ref post_id,meta_key post_id 8 brandly_xvfkg.c.post_id 14 Using where
1 PRIMARY rl ref PRIMARY PRIMARY 8 const 5 Using index
1 PRIMARY lab eq_ref PRIMARY PRIMARY 32 brandly_xvfkg.rl.size_label 1 Using where

编辑:子查询需要 0.0005 秒,所以它不是罪魁祸首

EDIT2:wp_postmeta table 有 1709642 条记录,它是 MyISAM:

SHOW CREATE TABLE `wp_postmeta`;
CREATE TABLE `wp_postmeta` (
 `meta_id` bigint(20) unsigned NOT NULL AUTO_INCREMENT,
 `post_id` bigint(20) unsigned NOT NULL DEFAULT 0,
 `meta_key` varchar(255) DEFAULT NULL,
 `meta_value` longtext DEFAULT NULL,
 PRIMARY KEY (`meta_id`),
 KEY `post_id` (`post_id`),
 KEY `meta_key` (`meta_key`(191))
) ENGINE=MyISAM AUTO_INCREMENT=3945402 DEFAULT CHARSET=utf8

EDIT3:实际上所有 table 都是 MyISAM,除了 wix_images、wix_images_back、wix_imported_product_variations 是 innoDB

EDIT4:刚刚将所有 table 转换为 innoDB。现在性能更好,需要 200-400 毫秒。不过还是慢。

EDIT5:在将 table 切换到 innoDB 之后,EXPLAIN 输出现在发生了一些变化。参见 https://codebeautify.org/htmlviewer/y225302bb

EDIT6:这里是table涉及的定义

CREATE TABLE `wp_postmeta` (
 `meta_id` bigint(20) unsigned NOT NULL AUTO_INCREMENT,
 `post_id` bigint(20) unsigned NOT NULL DEFAULT 0,
 `meta_key` varchar(255) DEFAULT NULL,
 `meta_value` longtext DEFAULT NULL,
 PRIMARY KEY (`meta_id`),
 KEY `post_id` (`post_id`),
 KEY `meta_key` (`meta_key`(191))
) ENGINE=InnoDB AUTO_INCREMENT=3945402 DEFAULT CHARSET=utf8

CREATE TABLE `brandly_tiers_bp_balances` (
 `user_id` bigint(20) NOT NULL,
 `bp_balance` bigint(20) NOT NULL DEFAULT 0,
 `bp_pending_balance` bigint(20) NOT NULL DEFAULT 0,
 `tier` varchar(50) COLLATE latin1_general_ci NOT NULL DEFAULT 'Earth',
 `tier_change_ts` bigint(20) DEFAULT NULL,
 PRIMARY KEY (`user_id`),
 KEY `tier` (`tier`)
) ENGINE=InnoDB DEFAULT CHARSET=latin1 COLLATE=latin1_general_ci

CREATE TABLE `brandly_tiers` (
 `role_name` varchar(50) COLLATE latin1_general_ci NOT NULL,
 `bp_threshold` int(11) NOT NULL DEFAULT 0,
 `discount_percent` smallint(6) NOT NULL,
 `bp_percent_branded_orders` smallint(6) NOT NULL,
 `bp_percent_bulk_orders` smallint(6) NOT NULL,
 `bp_percent_courier` smallint(6) NOT NULL,
 `bp_percent_bulk_branded_packaging` smallint(6) NOT NULL,
 `bp_percent_monthly_subscription_fee` smallint(6) NOT NULL,
 `bulk_order_minimum_quantity` int(11) NOT NULL DEFAULT 10,
 `bp_bulk_order_cap` int(11) NOT NULL DEFAULT 7500,
 `tier_monthly_subscription_price` int(11) NOT NULL DEFAULT 0,
 PRIMARY KEY (`role_name`),
 UNIQUE KEY `bp_threshold` (`bp_threshold`)
) ENGINE=InnoDB DEFAULT CHARSET=latin1 COLLATE=latin1_general_ci

CREATE TABLE `wix_images` (
 `id` bigint(20) NOT NULL AUTO_INCREMENT,
 `filename` text COLLATE latin1_general_ci NOT NULL,
 `uri` longtext COLLATE latin1_general_ci NOT NULL,
 `background_image_uri` longtext COLLATE latin1_general_ci NOT NULL,
 `branded` tinyint(1) DEFAULT 0,
 `wix_id` varchar(100) COLLATE latin1_general_ci DEFAULT NULL,
 `wix_product_id` varchar(100) COLLATE latin1_general_ci DEFAULT NULL,
 `wix_variation_id` varchar(100) COLLATE latin1_general_ci DEFAULT NULL,
 `design_cost` decimal(18,2) DEFAULT NULL,
 PRIMARY KEY (`id`),
 KEY `wix_id` (`wix_id`),
 KEY `wix_product_id` (`wix_product_id`),
 KEY `wix_variation_id` (`wix_variation_id`),
 KEY `branded` (`branded`),
 KEY `background_image_uri` (`background_image_uri`(8))
) ENGINE=InnoDB AUTO_INCREMENT=10680 DEFAULT CHARSET=latin1 COLLATE=latin1_general_ci

CREATE TABLE `wix_images_back` (
 `id` bigint(20) NOT NULL AUTO_INCREMENT,
 `filename` text COLLATE latin1_general_ci NOT NULL,
 `uri` longtext COLLATE latin1_general_ci NOT NULL,
 `background_image_uri` longtext COLLATE latin1_general_ci NOT NULL,
 `branded` tinyint(1) DEFAULT 0,
 `wix_id` varchar(100) COLLATE latin1_general_ci DEFAULT NULL,
 `wix_product_id` varchar(100) COLLATE latin1_general_ci DEFAULT NULL,
 `wix_variation_id` varchar(100) COLLATE latin1_general_ci DEFAULT NULL,
 `design_cost` decimal(18,2) DEFAULT NULL,
 PRIMARY KEY (`id`),
 KEY `wix_id` (`wix_id`),
 KEY `wix_product_id` (`wix_product_id`),
 KEY `wix_variation_id` (`wix_variation_id`),
 KEY `branded` (`branded`),
 KEY `background_image_uri` (`background_image_uri`(8))
) ENGINE=InnoDB AUTO_INCREMENT=7733 DEFAULT CHARSET=latin1 COLLATE=latin1_general_ci

CREATE TABLE `wix_imported_product_variations` (
 `wix_id` varchar(100) COLLATE latin1_general_ci NOT NULL,
 `store_id` int(11) NOT NULL,
 `wc_id` bigint(20) NOT NULL,
 `wc_product_id` bigint(20) NOT NULL,
 `sku` varchar(100) COLLATE latin1_general_ci NOT NULL,
 `price` decimal(18,2) NOT NULL,
 `original_price_rand` decimal(18,2) NOT NULL,
 `has_markup` tinyint(1) DEFAULT 1,
 `wix_image_id` varchar(1000) COLLATE latin1_general_ci NOT NULL,
 `imported_at` datetime DEFAULT NULL,
 `tmp_design_image` text COLLATE latin1_general_ci DEFAULT NULL,
 `tmp_branded` tinyint(1) DEFAULT NULL,
 `tmp_design_cost` decimal(18,2) DEFAULT NULL,
 `tmp_design_image_back` text COLLATE latin1_general_ci DEFAULT NULL,
 `tmp_branded_back` tinyint(1) DEFAULT NULL,
 `tmp_design_cost_back` decimal(18,2) DEFAULT NULL,
 PRIMARY KEY (`wix_id`),
 KEY `sku` (`sku`),
 KEY `store_id` (`store_id`),
 KEY `has_markup` (`has_markup`),
 KEY `imported_at` (`imported_at`),
 KEY `wc_product_id` (`wc_product_id`),
 KEY `wc_id` (`wc_id`)
) ENGINE=InnoDB DEFAULT CHARSET=latin1 COLLATE=latin1_general_ci

CREATE TABLE `brandly_reseller_tags` (
 `user_id` bigint(20) NOT NULL,
 `size_label` varchar(30) COLLATE latin1_general_ci NOT NULL,
 `image_front1` mediumtext COLLATE latin1_general_ci NOT NULL,
 `image_back1` mediumtext COLLATE latin1_general_ci NOT NULL,
 `image_front2` mediumtext COLLATE latin1_general_ci NOT NULL,
 `image_back2` mediumtext COLLATE latin1_general_ci NOT NULL,
 `image_front3` mediumtext COLLATE latin1_general_ci NOT NULL,
 `image_back3` mediumtext COLLATE latin1_general_ci NOT NULL,
 `default_index` int(11) DEFAULT NULL,
 `design_cost` decimal(18,2) DEFAULT NULL,
 PRIMARY KEY (`user_id`,`size_label`)
) ENGINE=InnoDB DEFAULT CHARSET=latin1 COLLATE=latin1_general_ci

CREATE TABLE `brandly_reseller_labels` (
 `user_id` bigint(20) NOT NULL,
 `size_label` varchar(30) COLLATE latin1_general_ci NOT NULL,
 `image_front1` mediumtext COLLATE latin1_general_ci NOT NULL,
 `image_back1` mediumtext COLLATE latin1_general_ci NOT NULL,
 `image_front2` mediumtext COLLATE latin1_general_ci NOT NULL,
 `image_back2` mediumtext COLLATE latin1_general_ci NOT NULL,
 `image_front3` mediumtext COLLATE latin1_general_ci NOT NULL,
 `image_back3` mediumtext COLLATE latin1_general_ci NOT NULL,
 `default_index` int(11) DEFAULT NULL,
 `design_cost` decimal(18,2) DEFAULT NULL,
 PRIMARY KEY (`user_id`,`size_label`)
) ENGINE=InnoDB DEFAULT CHARSET=latin1 COLLATE=latin1_general_ci

    
CREATE TABLE `brandly_reseller_branded_packaging_sizes` (
 `size_label` varchar(30) COLLATE latin1_general_ci NOT NULL,
 `size1_cm` int(11) NOT NULL,
 `size2_cm` int(11) NOT NULL,
 `size3_cm` int(11) NOT NULL,
 `is_bag` tinyint(1) DEFAULT NULL,
 `max_weight_kg` decimal(18,2) NOT NULL,
 `extra_price` decimal(18,2) NOT NULL,
 `extra_price_gift` decimal(18,2) NOT NULL,
 `extra_price_tag` decimal(18,2) NOT NULL,
 `extra_price_label` decimal(18,2) NOT NULL,
 PRIMARY KEY (`size_label`),
 KEY `size1_cm` (`size1_cm`),
 KEY `size2_cm` (`size2_cm`),
 KEY `size3_cm` (`size3_cm`)
) ENGINE=InnoDB DEFAULT CHARSET=latin1 COLLATE=latin1_general_ci

编辑7: 将 ON 条件从 JOIN 移动到 WHERE。执行时间 200-400ms 和以前一样。

EXPLAIN SELECT c.post_id AS wc_variation_id,v.original_price_rand,v.price,c.meta_value AS color,s.meta_value AS size,bp.tier,t.discount_percent,MAX(IFNULL(tag.extra_price_tag,0)) AS extra_price_tag,MAX(IFNULL(lab.extra_price_label,0)) AS extra_price_label,

i.uri,i.branded,i.design_cost,ib.uri AS uri_back,ib.branded AS branded_back,ib.design_cost AS design_cost_back
FROM wp_postmeta c
JOIN brandly_tiers_bp_balances bp
JOIN brandly_tiers t
JOIN wix_images i
JOIN wix_images_back ib
JOIN wix_imported_product_variations v 
LEFT JOIN brandly_reseller_tags rt ON rt.user_id=bp.user_id
LEFT JOIN brandly_reseller_branded_packaging_sizes tag ON tag.size_label=rt.size_label
LEFT JOIN brandly_reseller_labels rl ON rl.user_id=bp.user_id
LEFT JOIN brandly_reseller_branded_packaging_sizes lab ON lab.size_label=rl.size_label
JOIN wp_postmeta s 
WHERE c.post_id IN (SELECT DISTINCT(wc_id) FROM wix_imported_product_variations WHERE wc_product_id = 181189)
AND s.meta_key = 'attribute_pa_size'
AND c.meta_key = 'attribute_pa_colour'
AND i.wix_variation_id=v.wix_id
AND ib.wix_variation_id=v.wix_id
AND v.store_id=36
AND bp.user_id=11
AND s.post_id=c.post_id
AND t.role_name=bp.tier
AND v.wc_id=c.post_id
GROUP BY c.post_id
ORDER BY color,size

解释输出

id select_type table type possible_keys key key_len ref rows Extra
1 PRIMARY bp const PRIMARY,tier PRIMARY 8 const 1 Using temporary; Using filesort
1 PRIMARY t const PRIMARY PRIMARY 52 const 1
1 PRIMARY wix_imported_product_variations ref wc_product_id,wc_id wc_product_id 8 const 3 Start temporary
1 PRIMARY rt ref PRIMARY PRIMARY 8 const 5 Using index
1 PRIMARY tag eq_ref PRIMARY PRIMARY 32 brandly_xvfkg.rt.size_label 1 Using where
1 PRIMARY c ref post_id,meta_key post_id 8 brandly_xvfkg.wix_imported_product_variations.wc_id 8 Using index condition; Using where; End temporary
1 PRIMARY v ref PRIMARY,store_id,wc_id wc_id 8 brandly_xvfkg.c.post_id 1 Using index condition; Using where
1 PRIMARY ib ref wix_variation_id wix_variation_id 103 brandly_xvfkg.v.wix_id 3
1 PRIMARY i ref wix_variation_id wix_variation_id 103 brandly_xvfkg.v.wix_id 3
1 PRIMARY s ref post_id,meta_key post_id 8 brandly_xvfkg.v.wc_id 8 Using index condition; Using where
1 PRIMARY rl ref PRIMARY PRIMARY 8 const 5 Using index
1 PRIMARY lab eq_ref PRIMARY PRIMARY 32 brandly_xvfkg.rl.size_label 1 Using where
  • 在更大的服务器上花更多的钱不太可能对你有多大好处。

  • 200-400 毫秒这样的查询还不错。可能没有 SQL 魔法可以让它低于 100 毫秒;它做了很多。 极慢 一词通常指的是需要几分钟或更长时间的查询。

  • 只要 column 的所有值都是正数,
  • MAX(IFNULL(column, 0)) 就会产生与 MAX(column) 完全相同的结果。聚合函数会跳过空值。您也许可以简化使用该表达式的两个地方。

  • 您的查询很难推理。如果这是我的查询,我会重写它以摆脱短 table 别名,而是将它们拼写出来。而且,我会重写它,以便每个 JOIN(包括 LEFT 和普通)都有一个 ON-clause 。注意 ON-clauses 可以是复合的:

    FROM a
    JOIN wp_postmeta   ON a.id = wp_postmeta.postid
                       AND wp_postmeta.meta_key = 'attribute_pa_size'
    

    这是我们 WordPress 黑客立即识别的查询模式。

    对于普通 JOIN,将 ON-clauses 移动到 WHERE-clauses 对执行计划没有影响。

  • 您的计划中的连接类型都是“const”、“eq-ref”或“ref”。太好了。

  • WordPress 的 postmeta table 索引有 field-proven 返工。您的 postmeta table 足够大——几乎有四兆行——您可能可以使用它。它将主键更改为满足查询模式要求的复合键。你的是 WooCommerce 安装中的常见模式。对于 MySQL 5.7 及更高版本,它是 this

    ALTER TABLE wp_postmeta
       ADD UNIQUE KEY meta_id (meta_id),
       DROP PRIMARY KEY,
       ADD PRIMARY KEY (post_id, meta_key, meta_id),
      DROP KEY meta_key,
       ADD KEY meta_key (meta_key, meta_value(32), post_id, meta_id),
       ADD KEY meta_value (meta_value(32), meta_id),
      DROP KEY post_id;
    

    Index WP MySQL For Speed 插件旨在为您安装这套索引,并在 MySQL / MariaDB 5.5 及更高版本上发挥最大作用。

  • 我没有发现在您的 non-WordPress table 上建立索引有任何大问题。 可能有一些机会,但同样,您的查询很难阅读。

95% 的系统在关闭查询缓存后效果会更好。请记住,对 table 的 任何 修改会导致 QC 清除 table 的 所有 条目。如果 table 有很多写入,那么可能需要很多 CPU 周期来清除。

如果你只有1GB的RAM,QC的空间不大;改用 InnoDB 的 buffer_pool 空间。当您从 MyISAM 更改时,您是否降低了 key_buffer_size 并提高了 innodb_buffer_pool_size

其中一些复合索引可能会有所帮助:

c:  INDEX(post_id, meta_key,  meta_value)
v:  INDEX(wix_id, store_id, wc_id,  original_price_rand, price)
s:  INDEX(meta_key, post_id,  meta_value)
bp:  INDEX(user_id, tier)
t:  INDEX(role_name,  discount_percent)
tag:  INDEX(size_label,  extra_price_tag)
lab:  INDEX(size_label,  extra_price_label)
i:  INDEX(wix_variation_id,  uri, branded, design_cost)
ib:  INDEX(wix_variation_id,  uri, branded, design_cost)
rt:  INDEX(user_id,  size_label)
rl:  INDEX(user_id,  size_label)

添加复合索引时,删除具有相同前导列的索引。 也就是说,当你同时拥有INDEX(a)和INDEX(a,b)时,把前者扔掉。