Mysql 临时文件排序问题
Mysql temporary, filesort issue
根据我的研究,由于分组依据和排序依据,我什至不确定是否可以通过以下查询避免临时/文件排序。我也不是一个不自己做大量研究就问的人。但是,如果有人可以帮助弄清楚如何避免文件排序或为我指明正确的方向——即使它完全重写了查询或提出了一些代码方面的建议,我将不胜感激。我完全碰壁试图解决这个问题。底部解释的输出有一个 link。
explain select CONCAT(scfs.name, ' ',scfs.state,' ',scfs.zip_code) as scfname, scfs.zip_code, IF(date(s.scan_datetime) <= date(NOW()),date(s.scan_datetime),null) as scandate, count(*) as total,
sum(case when s.delivery_status = 1 then 1 else 0 end) as final
from order_addresses oa
left join pkg_data_unique s
on oa.trace_code = s.pkg_trace_code
inner join scf_zip_codes z
on SUBSTR(oa.zip,1,3) = z.zip_code
inner join scfs scfs
on z.scf_zip_code = scfs.zip_code
where oa.order_id = 160387
group by 1,2,3
order by scfs.zip_code, scandate
创建订单地址
CREATE TABLE order_addresses (
id int(10) unsigned NOT NULL AUTO_INCREMENT,
order_id int(11) NOT NULL,
name varchar(100) COLLATE utf8_unicode_ci NOT NULL,
address varchar(100) COLLATE utf8_unicode_ci NOT NULL,
address2 varchar(100) COLLATE utf8_unicode_ci NOT NULL,
city varchar(50) COLLATE utf8_unicode_ci NOT NULL,
state varchar(15) COLLATE utf8_unicode_ci NOT NULL,
zip char(5) COLLATE utf8_unicode_ci NOT NULL,
zip4 int(11) NOT NULL,
imb_digits char(31) COLLATE utf8_unicode_ci NOT NULL,
trace_code char(20) COLLATE utf8_unicode_ci NOT NULL,
created_at timestamp NOT NULL DEFAULT '0000-00-00 00:00:00',
updated_at timestamp NOT NULL DEFAULT '0000-00-00 00:00:00',
deleted_at timestamp NULL DEFAULT NULL,
PRIMARY KEY (id),
UNIQUE KEY order_address_unique
(order_id,imb_digits,name,address,address2,city,state,zip),
KEY order_addresses_order_id_index (order_id),
KEY order_addresses_name_index (name),
KEY order_addresses_address_index (address),
KEY order_addresses_city_index (city),
KEY order_addresses_state_index (state),
KEY order_addresses_zip_index (zip),
KEY order_addresses_imb_digits_index (imb_digits),
KEY order_addresses_trace_code_index (trace_code),
KEY order_id_trace_code (order_id,trace_code) USING BTREE
) ENGINE=InnoDB AUTO_INCREMENT=487714542 DEFAULT CHARSET=utf8
COLLATE=utf8_unicode_ci ROW_FORMAT=COMPACT
PKG 数据唯一 TABLE
CREATE TABLE pkg_data_unique (
id int(10) unsigned NOT NULL AUTO_INCREMENT,
zip char(5) COLLATE utf8_unicode_ci NOT NULL,
opcode int(11) NOT NULL,
pkg_trace_code char(20) COLLATE utf8_unicode_ci NOT NULL,
scan_datetime datetime NOT NULL,
original_scan_datetime datetime NOT NULL,
delivery_status int(11) NOT NULL,
created_at timestamp NOT NULL DEFAULT '0000-00-00 00:00:00',
updated_at timestamp NOT NULL DEFAULT '0000-00-00 00:00:00',
PRIMARY KEY (id),
UNIQUE KEY pkg_data_unique_pkg_trace_code_index (pkg_trace_code) USING BTREE,
KEY pkg_data_unique_zip_index (zip),
KEY pkg_data_unique_opcode_index (opcode),
KEY pkg_data_unique_scan_datetime_index (scan_datetime),
KEY pkg_data_unique_delivery_status_index (delivery_status),
KEY pkg_data_unique_original_scan_datetime (original_scan_datetime)
) ENGINE=InnoDB AUTO_INCREMENT=490667214 DEFAULT CHARSET=utf8 COLLATE=utf8_unicode_ci ROW_FORMAT=COMPACT
SCF 邮政编码
CREATE TABLE scf_zip_codes (
id int(10) unsigned NOT NULL AUTO_INCREMENT,
scf_zip_code varchar(3) COLLATE utf8_unicode_ci NOT NULL,
zip_code varchar(3) COLLATE utf8_unicode_ci NOT NULL,
created_at timestamp NOT NULL DEFAULT '0000-00-00 00:00:00',
updated_at timestamp NOT NULL DEFAULT '0000-00-00 00:00:00',
PRIMARY KEY (id),
KEY scf_zip_codes_scf_zip_code_index (scf_zip_code),
KEY scf_zip_codes_zip_code_index (zip_code)
) ENGINE=InnoDB AUTO_INCREMENT=916 DEFAULT CHARSET=utf8 COLLATE=utf8_unicode_ci ROW_FORMAT=COMPACT
SCFS
CREATE TABLE scfs (
id int(10) unsigned NOT NULL AUTO_INCREMENT,
label_id bigint(20) unsigned NOT NULL,
zip_code varchar(5) COLLATE utf8_unicode_ci NOT NULL,
name varchar(255) COLLATE utf8_unicode_ci NOT NULL,
state varchar(255) COLLATE utf8_unicode_ci NOT NULL,
locale_key varchar(255) COLLATE utf8_unicode_ci NOT NULL,
created_at timestamp NOT NULL DEFAULT '0000-00-00 00:00:00',
updated_at timestamp NOT NULL DEFAULT '0000-00-00 00:00:00',
PRIMARY KEY (id),
UNIQUE KEY scfs_label_id_unique (label_id),
UNIQUE KEY scfs_zip_code_unique (zip_code),
KEY scfs_name_index (name),
KEY scfs_state_index (state)
) ENGINE=InnoDB AUTO_INCREMENT=196 DEFAULT CHARSET=utf8 COLLATE=utf8_unicode_ci ROW_FORMAT=COMPACT
Image of Explain Output
figure out how to avoid the filesort or point me in the correct direction
备选方案涉及更改查询的意图。
group by 1,2,3
order by scfs.zip_code, scandate
GROUP BY
可能涉及文件排序和临时 table。
ORDER BY
肯定会涉及(另一个)filesort和temptable,因为它不匹配GROUP BY
。不,改组到 GROUP BY 2,1,3
(因为第 2 列是邮政编码)不会有帮助。
如果您可以接受这一点,GROUP BY 2,1,3 ORDER BY 2,1,3
,那么您可以摆脱一种文件排序并获得大致相同的输出。
您的查询是"slow"吗? 'Filesort' 不是主要的性能杀手。
有一些东西可以加快查询速度:
- 邮政编码(和其他一些东西)不需要 utf8;将列更改为
CHARACTER SET ascii
.
- 二级键,使用时需要向下钻取
PRIMARY KEY
BTree来查找行。考虑哪些(如果有的话)二级索引可以提升为 PRIMARY KEY
.
date(s.scan_datetime) <= date(NOW())
--> s.scan_datetime <= CURDATE()
sum(case when s.delivery_status = 1 then 1 else 0 end)
=> sum(case when s.delivery_status = 1)
因为布尔表达式的计算结果为 1 或 0。
INDEX(a), INDEX(a,b)
--> 第一个可以 DROPped
而不会丢失功能。
scfs
有 3 个唯一键; id
好像没用。提升zipcode
成为PK,使其成为CHAR(5) CHARACTER SET ascii
。它将是 5 个字节而不是 id 的 4 个字节。
- 不要盲目使用
(255)
;选择一个较小(但安全)的值。
根据我的研究,由于分组依据和排序依据,我什至不确定是否可以通过以下查询避免临时/文件排序。我也不是一个不自己做大量研究就问的人。但是,如果有人可以帮助弄清楚如何避免文件排序或为我指明正确的方向——即使它完全重写了查询或提出了一些代码方面的建议,我将不胜感激。我完全碰壁试图解决这个问题。底部解释的输出有一个 link。
explain select CONCAT(scfs.name, ' ',scfs.state,' ',scfs.zip_code) as scfname, scfs.zip_code, IF(date(s.scan_datetime) <= date(NOW()),date(s.scan_datetime),null) as scandate, count(*) as total,
sum(case when s.delivery_status = 1 then 1 else 0 end) as final
from order_addresses oa
left join pkg_data_unique s
on oa.trace_code = s.pkg_trace_code
inner join scf_zip_codes z
on SUBSTR(oa.zip,1,3) = z.zip_code
inner join scfs scfs
on z.scf_zip_code = scfs.zip_code
where oa.order_id = 160387
group by 1,2,3
order by scfs.zip_code, scandate
创建订单地址
CREATE TABLE order_addresses (
id int(10) unsigned NOT NULL AUTO_INCREMENT,
order_id int(11) NOT NULL,
name varchar(100) COLLATE utf8_unicode_ci NOT NULL,
address varchar(100) COLLATE utf8_unicode_ci NOT NULL,
address2 varchar(100) COLLATE utf8_unicode_ci NOT NULL,
city varchar(50) COLLATE utf8_unicode_ci NOT NULL,
state varchar(15) COLLATE utf8_unicode_ci NOT NULL,
zip char(5) COLLATE utf8_unicode_ci NOT NULL,
zip4 int(11) NOT NULL,
imb_digits char(31) COLLATE utf8_unicode_ci NOT NULL,
trace_code char(20) COLLATE utf8_unicode_ci NOT NULL,
created_at timestamp NOT NULL DEFAULT '0000-00-00 00:00:00',
updated_at timestamp NOT NULL DEFAULT '0000-00-00 00:00:00',
deleted_at timestamp NULL DEFAULT NULL,
PRIMARY KEY (id),
UNIQUE KEY order_address_unique
(order_id,imb_digits,name,address,address2,city,state,zip),
KEY order_addresses_order_id_index (order_id),
KEY order_addresses_name_index (name),
KEY order_addresses_address_index (address),
KEY order_addresses_city_index (city),
KEY order_addresses_state_index (state),
KEY order_addresses_zip_index (zip),
KEY order_addresses_imb_digits_index (imb_digits),
KEY order_addresses_trace_code_index (trace_code),
KEY order_id_trace_code (order_id,trace_code) USING BTREE
) ENGINE=InnoDB AUTO_INCREMENT=487714542 DEFAULT CHARSET=utf8
COLLATE=utf8_unicode_ci ROW_FORMAT=COMPACT
PKG 数据唯一 TABLE
CREATE TABLE pkg_data_unique (
id int(10) unsigned NOT NULL AUTO_INCREMENT,
zip char(5) COLLATE utf8_unicode_ci NOT NULL,
opcode int(11) NOT NULL,
pkg_trace_code char(20) COLLATE utf8_unicode_ci NOT NULL,
scan_datetime datetime NOT NULL,
original_scan_datetime datetime NOT NULL,
delivery_status int(11) NOT NULL,
created_at timestamp NOT NULL DEFAULT '0000-00-00 00:00:00',
updated_at timestamp NOT NULL DEFAULT '0000-00-00 00:00:00',
PRIMARY KEY (id),
UNIQUE KEY pkg_data_unique_pkg_trace_code_index (pkg_trace_code) USING BTREE,
KEY pkg_data_unique_zip_index (zip),
KEY pkg_data_unique_opcode_index (opcode),
KEY pkg_data_unique_scan_datetime_index (scan_datetime),
KEY pkg_data_unique_delivery_status_index (delivery_status),
KEY pkg_data_unique_original_scan_datetime (original_scan_datetime)
) ENGINE=InnoDB AUTO_INCREMENT=490667214 DEFAULT CHARSET=utf8 COLLATE=utf8_unicode_ci ROW_FORMAT=COMPACT
SCF 邮政编码
CREATE TABLE scf_zip_codes (
id int(10) unsigned NOT NULL AUTO_INCREMENT,
scf_zip_code varchar(3) COLLATE utf8_unicode_ci NOT NULL,
zip_code varchar(3) COLLATE utf8_unicode_ci NOT NULL,
created_at timestamp NOT NULL DEFAULT '0000-00-00 00:00:00',
updated_at timestamp NOT NULL DEFAULT '0000-00-00 00:00:00',
PRIMARY KEY (id),
KEY scf_zip_codes_scf_zip_code_index (scf_zip_code),
KEY scf_zip_codes_zip_code_index (zip_code)
) ENGINE=InnoDB AUTO_INCREMENT=916 DEFAULT CHARSET=utf8 COLLATE=utf8_unicode_ci ROW_FORMAT=COMPACT
SCFS
CREATE TABLE scfs (
id int(10) unsigned NOT NULL AUTO_INCREMENT,
label_id bigint(20) unsigned NOT NULL,
zip_code varchar(5) COLLATE utf8_unicode_ci NOT NULL,
name varchar(255) COLLATE utf8_unicode_ci NOT NULL,
state varchar(255) COLLATE utf8_unicode_ci NOT NULL,
locale_key varchar(255) COLLATE utf8_unicode_ci NOT NULL,
created_at timestamp NOT NULL DEFAULT '0000-00-00 00:00:00',
updated_at timestamp NOT NULL DEFAULT '0000-00-00 00:00:00',
PRIMARY KEY (id),
UNIQUE KEY scfs_label_id_unique (label_id),
UNIQUE KEY scfs_zip_code_unique (zip_code),
KEY scfs_name_index (name),
KEY scfs_state_index (state)
) ENGINE=InnoDB AUTO_INCREMENT=196 DEFAULT CHARSET=utf8 COLLATE=utf8_unicode_ci ROW_FORMAT=COMPACT
Image of Explain Output
figure out how to avoid the filesort or point me in the correct direction
备选方案涉及更改查询的意图。
group by 1,2,3
order by scfs.zip_code, scandate
GROUP BY
可能涉及文件排序和临时 table。
ORDER BY
肯定会涉及(另一个)filesort和temptable,因为它不匹配GROUP BY
。不,改组到 GROUP BY 2,1,3
(因为第 2 列是邮政编码)不会有帮助。
如果您可以接受这一点,GROUP BY 2,1,3 ORDER BY 2,1,3
,那么您可以摆脱一种文件排序并获得大致相同的输出。
您的查询是"slow"吗? 'Filesort' 不是主要的性能杀手。
有一些东西可以加快查询速度:
- 邮政编码(和其他一些东西)不需要 utf8;将列更改为
CHARACTER SET ascii
. - 二级键,使用时需要向下钻取
PRIMARY KEY
BTree来查找行。考虑哪些(如果有的话)二级索引可以提升为PRIMARY KEY
. date(s.scan_datetime) <= date(NOW())
-->s.scan_datetime <= CURDATE()
sum(case when s.delivery_status = 1 then 1 else 0 end)
=>sum(case when s.delivery_status = 1)
因为布尔表达式的计算结果为 1 或 0。INDEX(a), INDEX(a,b)
--> 第一个可以DROPped
而不会丢失功能。scfs
有 3 个唯一键;id
好像没用。提升zipcode
成为PK,使其成为CHAR(5) CHARACTER SET ascii
。它将是 5 个字节而不是 id 的 4 个字节。- 不要盲目使用
(255)
;选择一个较小(但安全)的值。