与 MySQL 5.7.30 相比,MariaDB 10.4.13 性能较慢
MariaDB 10.4.13 slow performance compared to MySQL 5.7.30
在将 大型(3+ GB)数据库从MySQL 数据迁移到MariaDB 后,遇到特定查询性能问题,它是64 位版本。数据库被分析、优化、重建。下面是 MariaDB 的配置、数据库方案和有问题的查询。
非常感谢 what/how/where/when 提出的解决此问题的建议。
机器参数为:Intel Core i5 CPU @3.6GHz, 16GB RAM, Sandisk 512GB SSD, using Windows 10 v.1909.
SQL 性能低下的查询(10 秒,过去在 MySQL 5.7 上约为 1 秒):
SELECT * FROM (
SELECT
'#AT&T' AS instrument,
(SELECT '2020-05-21 09:30' AS report_period) report_period,
#Average price
(SELECT AVG(avg_price.avg_price) AS avg_price FROM
(
SELECT AVG(t.CLOSE_PRICE) AS avg_price
FROM mt4_trades t
WHERE t.CLOSE_TIME BETWEEN '2020-05-21 09:30' AND DATE_ADD('2020-05-21 09:30', INTERVAL 119 SECOND) AND t.OPEN_TIME > '2012-08-26'
AND t.SYMBOL LIKE '#AT&T%' AND t.CMD IN (0,1)
UNION ALL
SELECT AVG(t.OPEN_PRICE) AS avg_price
FROM mt4_trades t
WHERE t.OPEN_TIME BETWEEN '2020-05-21 09:30' AND DATE_ADD('2020-05-21 09:30', INTERVAL 119 SECOND)
AND t.SYMBOL LIKE '#AT&T%' AND t.CMD IN (0,1)
) avg_price) avg_price,
#Total deals value
(
SELECT SUM(total_deals_value.total_deals_value) AS total_deals_value FROM (
SELECT SUM(t.VOLUME/100.0 * 1 * t.CLOSE_PRICE ) AS total_deals_value
FROM mt4_trades t
WHERE t.CLOSE_TIME BETWEEN '2020-05-21 09:30' AND DATE_ADD('2020-05-21 09:30', INTERVAL 119 SECOND) AND t.OPEN_TIME > '2012-08-26'
AND t.SYMBOL LIKE '#AT&T%' AND t.CMD IN (0,1)
UNION ALL
SELECT SUM(t.VOLUME/100.0 * 1 * t.OPEN_PRICE ) AS total_deals_value
FROM mt4_trades t
WHERE t.OPEN_TIME BETWEEN '2020-05-21 09:30' AND DATE_ADD('2020-05-21 09:30', INTERVAL 119 SECOND)
AND t.SYMBOL LIKE '#AT&T%' AND t.CMD IN (0,1)
) total_deals_value) AS total_deals_value) result
LEFT OUTER JOIN
(SELECT '#AT&T' AS instrument, @fd_time0 AS fd_time, @fd_price0 AS fd_price,
(@fd_volume0/100.0 * 1 * @fd_price0 ) AS fd_volume
FROM (
SELECT @fd_time0 := fd_time AS fd_time, @fd_volume0 := VOLUME AS VOLUME, @fd_price0 := PRICE AS PRICE
FROM
(SELECT MIN(t.CLOSE_TIME) AS fd_time, t.VOLUME, t.CLOSE_PRICE AS PRICE FROM mt4_trades t WHERE t.CLOSE_TIME BETWEEN
DATE_ADD('2020-05-21 09:30', INTERVAL 119 SECOND) AND '2020-05-21 11:30' AND t.OPEN_TIME > '2012-08-26'
AND t.SYMBOL LIKE '#AT&T%'
UNION ALL
SELECT MIN(t.OPEN_TIME) AS fd_time, t.VOLUME, t.OPEN_PRICE AS PRICE FROM mt4_trades t WHERE t.OPEN_TIME BETWEEN
DATE_ADD('2020-05-21 09:30', INTERVAL 119 SECOND) AND '2020-05-21 11:30'
AND t.SYMBOL LIKE '#AT&T%'
ORDER BY fd_time) first_deal WHERE first_deal.fd_time IS NOT NULL ORDER BY first_deal.fd_time ASC LIMIT 1
) AS first_deal) temp_result ON temp_result.instrument = result.instrument
SQL 查询的解释:
为 table 创建 SQL:
CREATE TABLE `mt4_trades` (
`TICKET` INT(11) UNSIGNED NOT NULL,
`LOGIN` INT(11) UNSIGNED NOT NULL,
`SYMBOL` VARCHAR(16) NOT NULL DEFAULT '' COLLATE 'utf8_general_ci',
`DIGITS` TINYINT(3) UNSIGNED NOT NULL,
`CMD` TINYINT(3) UNSIGNED NOT NULL,
`VOLUME` MEDIUMINT(8) UNSIGNED NOT NULL,
`OPEN_TIME` DATETIME NOT NULL,
`OPEN_PRICE` FLOAT(12,0) NOT NULL,
`SL` FLOAT(12,0) NOT NULL,
`TP` FLOAT(12,0) NOT NULL,
`CLOSE_TIME` DATETIME NOT NULL,
`EXPIRATION` DATETIME NOT NULL,
`REASON` TINYINT(3) UNSIGNED NOT NULL DEFAULT '0',
`CONV_RATE1` FLOAT(12,0) NOT NULL,
`CONV_RATE2` FLOAT(12,0) NOT NULL,
`COMMISSION` FLOAT(12,0) NOT NULL,
`COMMISSION_AGENT` FLOAT(12,0) NOT NULL,
`SWAPS` FLOAT(12,0) NOT NULL,
`CLOSE_PRICE` FLOAT(12,0) NOT NULL,
`PROFIT` FLOAT(12,0) NOT NULL,
`TAXES` FLOAT(12,0) NOT NULL,
`COMMENT` VARCHAR(32) NOT NULL DEFAULT '' COLLATE 'utf8_general_ci',
`INTERNAL_ID` INT(11) NOT NULL,
`MARGIN_RATE` FLOAT(12,0) NOT NULL,
`TIMESTAMP` INT(11) UNSIGNED NOT NULL,
`MAGIC` INT(11) NOT NULL DEFAULT '0',
`GW_VOLUME` INT(11) NOT NULL DEFAULT '0',
`GW_OPEN_PRICE` INT(11) NOT NULL DEFAULT '0',
`GW_CLOSE_PRICE` INT(11) NOT NULL DEFAULT '0',
`MODIFY_TIME` DATETIME NOT NULL,
PRIMARY KEY (`TICKET`) USING BTREE,
INDEX `INDEX_STAMP` (`TIMESTAMP`, `COMMENT`) USING BTREE,
INDEX `CMD` (`CMD`, `OPEN_TIME`, `CLOSE_TIME`, `LOGIN`, `VOLUME`, `SYMBOL`, `CLOSE_PRICE`) USING
BTREE
)
COLLATE='utf8_general_ci'
;
MariaDB 的 my.ini
[mysqld]
port= 3306
socket = "C:/xampp/mysql/mysql.sock"
basedir = "C:/xampp/mysql"
tmpdir = "C:/xampp/tmp"
datadir = "C:/xampp/mysql/data"
log_error = "mysql_error.log"
pid_file = "mysql.pid"
collation_server=utf8_general_ci
character_set_server=utf8
## CUSTOM EDIT
sql-mode=NO_AUTO_CREATE_USER,NO_ENGINE_SUBSTITUTION,NO_FIELD_OPTIONS,NO_KEY_OPTIONS,NO_TABLE_OPTIONS,STRICT_TRANS_TABLES
skip_external_locking
skip_name_resolve
max_connections = 200
table_open_cache = 10000
table_definition_cache = 2000
open_files_limit = 20000
##MyISAM setting
key_buffer = 512M
myisam_sort_buffer_size = 2M
#
max_allowed_packet = 16M
max_sort_length = 16384
sort_buffer_size = 1M
net_buffer_length = 64K
read_buffer_size = 256K
read_rnd_buffer_size = 512K
#INNO DB settings
innodb_file_per_table = 1
innodb_buffer_pool_size = 4G
innodb_sort_buffer_size = 16M
## Set .._log_file_size to 25 % of buffer pool size
innodb_log_file_size = 1024M
innodb_log_buffer_size = 32M
innodb_flush_log_at_trx_commit = 2
innodb_stats_on_metadata = 0
innodb_lock_wait_timeout = 600
innodb_flush_method = normal
#A minor optimization when writing blocks to disk. Use 0 for SSD drives; 1 for HDD.
innodb_flush_neighbors = 0
innodb_io_capacity = 2000
#
innodb_buffer_pool_instances = 3
innodb_thread_concurrency = 12
innodb_autoextend_increment = 64
innodb_read_io_threads = 16
innodb_write_io_threads = 16
concurrent_insert = 2
thread_stack = 512K
interactive_timeout = 600
wait_timeout = 600
query_cache_type = 2
query_cache_limit = 64M
query_cache_min_res_unit = 1
query_cache_size = 16M
thread_cache_size = 128
low_priority_updates
tmp_table_size = 4M
max_heap_table_size = 4M
bulk_insert_buffer_size = 256M
group_concat_max_len = 512K
# Define which query should be considered as slow, in seconds
long_query_time = 6
join_cache_level = 8
# Size limit for the whole join
#join_buffer_space_limit = 512M
join_buffer_size = 4M
# Optimizer switches
optimizer_switch ='orderby_uses_equalities=on'
optimizer_switch ='mrr=on,mrr_sort_keys=on'
optimizer_switch ='index_merge_sort_intersection=on'
optimizer_switch ='optimize_join_buffer_size=on'
optimizer_switch ='join_cache_bka=on'
optimizer_switch ='join_cache_hashed=on'
optimizer_switch='in_to_exists=on'
optimizer_switch='join_cache_incremental=on'
#optimizer_switch='loosescan=on'
# Where do all the plugins live
plugin_dir = "C:/xampp/mysql/lib/plugin/"
server-id = 1
在最近的 MariaDB 和错误查询中看到了这种行为(抱歉,我不会粉饰它 - 像这样有很多子选择的查询只是错误的查询),我要出去在这里做一个猜测(因为你不能提供来自 MySQL 5.7 的 EXPLAIN 计划):
在您的 optimizer_switch
设置中切换 semijoin=off
,看看它是否选择了一个不太糟糕的执行计划。
我也不得不注意到您正在切换很多配置设置 - 没有人需要触及其中的绝大多数,所以我建议您从仅设置 innodb_buffer_pool_size
的干净配置开始适合您的内存大小。
这是一个很好的查询。我相信您需要对其进行分解才能了解其性能。
在我看来,您有两个子查询模式。这是一种模式
SELECT something_or_other
FROM mt4_trades t
WHERE t.CLOSE_TIME BETWEEN '2020-05-21 09:30'
AND DATE_ADD('2020-05-21 09:30', INTERVAL 119 SECOND)
AND t.OPEN_TIME > '2012-08-26'
AND t.SYMBOL LIKE '#AT&T%'
AND t.CMD IN (0,1)
这是另一个
SELECT something_or_other
FROM mt4_trades t
WHERE t.OPEN_TIME BETWEEN '2020-05-21 09:30'
AND DATE_ADD('2020-05-21 09:30', INTERVAL 119 SECOND)
AND t.SYMBOL LIKE '#AT&T%'
AND t.CMD IN (0,1)
不幸的是,对于利用索引,您在这些查询模式中没有相等过滤器 (WHERE col=val
)。索引范围扫描可能非常有效,但当它们处理多个相等过滤器然后处理一个范围过滤器时,它们的效果最好。 (time BETWEEN this AND that
)
因此,为了优化,我们需要从具有最大选择性的列开始您的多列索引。我们需要 compound covering indexes 作为您的查询模式。
我认为你应该为你的第一个模式尝试这个索引。
CREATE INDEX closedex ON mt4_trades
(CLOSE_TIME, CMD, OPEN_TIME, SYMBOL, VOLUME, CLOSE_PRICE, LOGIN)
对于你的第二个模式,它有点简单
CREATE INDEX opendex ON mt4_trades
(OPEN_TIME, CMD, SYMBOL, VOLUME, CLOSE_PRICE, LOGIN)
您需要两个索引,因为(我猜)您最有选择性的列是 CLOSE_TIME
和 OPEN_TIME
。您还应该尝试将 CMD
放在这些索引的第一位;也许 MariaDB 知道如何有效地使用 CMD IN (0,1)
的索引。
重点是让查询规划器能够单独满足来自索引的查询,而不必跳回到table。
如果您可以将 SYMBOL LIKE 'value%'
更改为 SYMBOL = 'value'
并且您的应用程序仍能正常工作,请执行此操作。然后将 SYMBOL
放在索引的第一位;那是平等匹配。
(重要提示:在您的查询行中
SELECT MIN(t.CLOSE_TIME) AS fd_time, t.VOLUME, t.CLOSE_PRICE AS PRICE
你得到了 VOLUME 和 CLOSE_PRICE 的不可预测的table 值。
(如果这是我为我的雇主处理其他人的钱的查询,我会花几个小时来分析它的正确性。)
- 不要在
FLOAT
或 DOUBLE
的末尾使用 (m,n)。既然你有 ,0
,你不妨使用一些 INT
变体。
- INDEX
CMD
(CMD
, OPEN_TIME
, CLOSE_TIME
, LOGIN
, VOLUME
, SYMBOL
, CLOSE_PRICE
) -- 可能列太多了。
- 我希望你的表是 InnoDB。
- 除非您有其他应用占用 RAM,否则您可以增加
innodb_buffer_pool_size
。
- 这种模式通常效率低下:
FROM ( SELECT ... ) JOIN ( SELECT ... )
@fd_time0
-- 使用@变量有风险;优化器可能会导致意外;也就是说,您不能依赖于何时设置和使用它们。 (他们最终将被禁止。)
UNION ALLs
好像刚好生成两行,那你把这两个值加起来?而不是将 UNION
送入 SUM
,而是这样做:
( SELECT ( SELECT ... ) + ( SELECT ... ) )
这个 WHERE
有 4 个范围。只有一个人可以使用索引:
WHERE t.CLOSE_TIME BETWEEN ...
AND t.OPEN_TIME > '2012-08-26'
AND t.SYMBOL LIKE '#AT&T%'
AND t.CMD IN (0,1)
我建议您给优化器 3 个选择:
INDEX(close_time)
INDEX(OPEN_TIME)
INDEX(SYMBOL)
如果你不需要符号末尾的通配符,那么这些索引会更好:
INDEX(SYMBOL, close_time)
INDEX(SYMBOL, OPEN_TIME)
(我的偏好)而不是
t.OPEN_TIME BETWEEN '2020-05-21 09:30'
AND DATE_ADD('2020-05-21 09:30', INTERVAL 119 SECOND
我比较喜欢
t.OPEN_TIME >= '2020-05-21 09:30'
AND t.OPEN_TIME < '2020-05-21 09:30' + INTERVAL 2 MINUTE
SELECT MIN(t.CLOSE_TIME) AS fd_time, t.VOLUME ...
会给出错误的数据!!!。查看标签 [groupwise-maximum]
.
在将 大型(3+ GB)数据库从MySQL 数据迁移到MariaDB 后,遇到特定查询性能问题,它是64 位版本。数据库被分析、优化、重建。下面是 MariaDB 的配置、数据库方案和有问题的查询。
非常感谢 what/how/where/when 提出的解决此问题的建议。
机器参数为:Intel Core i5 CPU @3.6GHz, 16GB RAM, Sandisk 512GB SSD, using Windows 10 v.1909.
SQL 性能低下的查询(10 秒,过去在 MySQL 5.7 上约为 1 秒):
SELECT * FROM (
SELECT
'#AT&T' AS instrument,
(SELECT '2020-05-21 09:30' AS report_period) report_period,
#Average price
(SELECT AVG(avg_price.avg_price) AS avg_price FROM
(
SELECT AVG(t.CLOSE_PRICE) AS avg_price
FROM mt4_trades t
WHERE t.CLOSE_TIME BETWEEN '2020-05-21 09:30' AND DATE_ADD('2020-05-21 09:30', INTERVAL 119 SECOND) AND t.OPEN_TIME > '2012-08-26'
AND t.SYMBOL LIKE '#AT&T%' AND t.CMD IN (0,1)
UNION ALL
SELECT AVG(t.OPEN_PRICE) AS avg_price
FROM mt4_trades t
WHERE t.OPEN_TIME BETWEEN '2020-05-21 09:30' AND DATE_ADD('2020-05-21 09:30', INTERVAL 119 SECOND)
AND t.SYMBOL LIKE '#AT&T%' AND t.CMD IN (0,1)
) avg_price) avg_price,
#Total deals value
(
SELECT SUM(total_deals_value.total_deals_value) AS total_deals_value FROM (
SELECT SUM(t.VOLUME/100.0 * 1 * t.CLOSE_PRICE ) AS total_deals_value
FROM mt4_trades t
WHERE t.CLOSE_TIME BETWEEN '2020-05-21 09:30' AND DATE_ADD('2020-05-21 09:30', INTERVAL 119 SECOND) AND t.OPEN_TIME > '2012-08-26'
AND t.SYMBOL LIKE '#AT&T%' AND t.CMD IN (0,1)
UNION ALL
SELECT SUM(t.VOLUME/100.0 * 1 * t.OPEN_PRICE ) AS total_deals_value
FROM mt4_trades t
WHERE t.OPEN_TIME BETWEEN '2020-05-21 09:30' AND DATE_ADD('2020-05-21 09:30', INTERVAL 119 SECOND)
AND t.SYMBOL LIKE '#AT&T%' AND t.CMD IN (0,1)
) total_deals_value) AS total_deals_value) result
LEFT OUTER JOIN
(SELECT '#AT&T' AS instrument, @fd_time0 AS fd_time, @fd_price0 AS fd_price,
(@fd_volume0/100.0 * 1 * @fd_price0 ) AS fd_volume
FROM (
SELECT @fd_time0 := fd_time AS fd_time, @fd_volume0 := VOLUME AS VOLUME, @fd_price0 := PRICE AS PRICE
FROM
(SELECT MIN(t.CLOSE_TIME) AS fd_time, t.VOLUME, t.CLOSE_PRICE AS PRICE FROM mt4_trades t WHERE t.CLOSE_TIME BETWEEN
DATE_ADD('2020-05-21 09:30', INTERVAL 119 SECOND) AND '2020-05-21 11:30' AND t.OPEN_TIME > '2012-08-26'
AND t.SYMBOL LIKE '#AT&T%'
UNION ALL
SELECT MIN(t.OPEN_TIME) AS fd_time, t.VOLUME, t.OPEN_PRICE AS PRICE FROM mt4_trades t WHERE t.OPEN_TIME BETWEEN
DATE_ADD('2020-05-21 09:30', INTERVAL 119 SECOND) AND '2020-05-21 11:30'
AND t.SYMBOL LIKE '#AT&T%'
ORDER BY fd_time) first_deal WHERE first_deal.fd_time IS NOT NULL ORDER BY first_deal.fd_time ASC LIMIT 1
) AS first_deal) temp_result ON temp_result.instrument = result.instrument
SQL 查询的解释:
为 table 创建 SQL:
CREATE TABLE `mt4_trades` (
`TICKET` INT(11) UNSIGNED NOT NULL,
`LOGIN` INT(11) UNSIGNED NOT NULL,
`SYMBOL` VARCHAR(16) NOT NULL DEFAULT '' COLLATE 'utf8_general_ci',
`DIGITS` TINYINT(3) UNSIGNED NOT NULL,
`CMD` TINYINT(3) UNSIGNED NOT NULL,
`VOLUME` MEDIUMINT(8) UNSIGNED NOT NULL,
`OPEN_TIME` DATETIME NOT NULL,
`OPEN_PRICE` FLOAT(12,0) NOT NULL,
`SL` FLOAT(12,0) NOT NULL,
`TP` FLOAT(12,0) NOT NULL,
`CLOSE_TIME` DATETIME NOT NULL,
`EXPIRATION` DATETIME NOT NULL,
`REASON` TINYINT(3) UNSIGNED NOT NULL DEFAULT '0',
`CONV_RATE1` FLOAT(12,0) NOT NULL,
`CONV_RATE2` FLOAT(12,0) NOT NULL,
`COMMISSION` FLOAT(12,0) NOT NULL,
`COMMISSION_AGENT` FLOAT(12,0) NOT NULL,
`SWAPS` FLOAT(12,0) NOT NULL,
`CLOSE_PRICE` FLOAT(12,0) NOT NULL,
`PROFIT` FLOAT(12,0) NOT NULL,
`TAXES` FLOAT(12,0) NOT NULL,
`COMMENT` VARCHAR(32) NOT NULL DEFAULT '' COLLATE 'utf8_general_ci',
`INTERNAL_ID` INT(11) NOT NULL,
`MARGIN_RATE` FLOAT(12,0) NOT NULL,
`TIMESTAMP` INT(11) UNSIGNED NOT NULL,
`MAGIC` INT(11) NOT NULL DEFAULT '0',
`GW_VOLUME` INT(11) NOT NULL DEFAULT '0',
`GW_OPEN_PRICE` INT(11) NOT NULL DEFAULT '0',
`GW_CLOSE_PRICE` INT(11) NOT NULL DEFAULT '0',
`MODIFY_TIME` DATETIME NOT NULL,
PRIMARY KEY (`TICKET`) USING BTREE,
INDEX `INDEX_STAMP` (`TIMESTAMP`, `COMMENT`) USING BTREE,
INDEX `CMD` (`CMD`, `OPEN_TIME`, `CLOSE_TIME`, `LOGIN`, `VOLUME`, `SYMBOL`, `CLOSE_PRICE`) USING
BTREE
)
COLLATE='utf8_general_ci'
;
MariaDB 的 my.ini
[mysqld]
port= 3306
socket = "C:/xampp/mysql/mysql.sock"
basedir = "C:/xampp/mysql"
tmpdir = "C:/xampp/tmp"
datadir = "C:/xampp/mysql/data"
log_error = "mysql_error.log"
pid_file = "mysql.pid"
collation_server=utf8_general_ci
character_set_server=utf8
## CUSTOM EDIT
sql-mode=NO_AUTO_CREATE_USER,NO_ENGINE_SUBSTITUTION,NO_FIELD_OPTIONS,NO_KEY_OPTIONS,NO_TABLE_OPTIONS,STRICT_TRANS_TABLES
skip_external_locking
skip_name_resolve
max_connections = 200
table_open_cache = 10000
table_definition_cache = 2000
open_files_limit = 20000
##MyISAM setting
key_buffer = 512M
myisam_sort_buffer_size = 2M
#
max_allowed_packet = 16M
max_sort_length = 16384
sort_buffer_size = 1M
net_buffer_length = 64K
read_buffer_size = 256K
read_rnd_buffer_size = 512K
#INNO DB settings
innodb_file_per_table = 1
innodb_buffer_pool_size = 4G
innodb_sort_buffer_size = 16M
## Set .._log_file_size to 25 % of buffer pool size
innodb_log_file_size = 1024M
innodb_log_buffer_size = 32M
innodb_flush_log_at_trx_commit = 2
innodb_stats_on_metadata = 0
innodb_lock_wait_timeout = 600
innodb_flush_method = normal
#A minor optimization when writing blocks to disk. Use 0 for SSD drives; 1 for HDD.
innodb_flush_neighbors = 0
innodb_io_capacity = 2000
#
innodb_buffer_pool_instances = 3
innodb_thread_concurrency = 12
innodb_autoextend_increment = 64
innodb_read_io_threads = 16
innodb_write_io_threads = 16
concurrent_insert = 2
thread_stack = 512K
interactive_timeout = 600
wait_timeout = 600
query_cache_type = 2
query_cache_limit = 64M
query_cache_min_res_unit = 1
query_cache_size = 16M
thread_cache_size = 128
low_priority_updates
tmp_table_size = 4M
max_heap_table_size = 4M
bulk_insert_buffer_size = 256M
group_concat_max_len = 512K
# Define which query should be considered as slow, in seconds
long_query_time = 6
join_cache_level = 8
# Size limit for the whole join
#join_buffer_space_limit = 512M
join_buffer_size = 4M
# Optimizer switches
optimizer_switch ='orderby_uses_equalities=on'
optimizer_switch ='mrr=on,mrr_sort_keys=on'
optimizer_switch ='index_merge_sort_intersection=on'
optimizer_switch ='optimize_join_buffer_size=on'
optimizer_switch ='join_cache_bka=on'
optimizer_switch ='join_cache_hashed=on'
optimizer_switch='in_to_exists=on'
optimizer_switch='join_cache_incremental=on'
#optimizer_switch='loosescan=on'
# Where do all the plugins live
plugin_dir = "C:/xampp/mysql/lib/plugin/"
server-id = 1
在最近的 MariaDB 和错误查询中看到了这种行为(抱歉,我不会粉饰它 - 像这样有很多子选择的查询只是错误的查询),我要出去在这里做一个猜测(因为你不能提供来自 MySQL 5.7 的 EXPLAIN 计划):
在您的 optimizer_switch
设置中切换 semijoin=off
,看看它是否选择了一个不太糟糕的执行计划。
我也不得不注意到您正在切换很多配置设置 - 没有人需要触及其中的绝大多数,所以我建议您从仅设置 innodb_buffer_pool_size
的干净配置开始适合您的内存大小。
这是一个很好的查询。我相信您需要对其进行分解才能了解其性能。
在我看来,您有两个子查询模式。这是一种模式
SELECT something_or_other
FROM mt4_trades t
WHERE t.CLOSE_TIME BETWEEN '2020-05-21 09:30'
AND DATE_ADD('2020-05-21 09:30', INTERVAL 119 SECOND)
AND t.OPEN_TIME > '2012-08-26'
AND t.SYMBOL LIKE '#AT&T%'
AND t.CMD IN (0,1)
这是另一个
SELECT something_or_other
FROM mt4_trades t
WHERE t.OPEN_TIME BETWEEN '2020-05-21 09:30'
AND DATE_ADD('2020-05-21 09:30', INTERVAL 119 SECOND)
AND t.SYMBOL LIKE '#AT&T%'
AND t.CMD IN (0,1)
不幸的是,对于利用索引,您在这些查询模式中没有相等过滤器 (WHERE col=val
)。索引范围扫描可能非常有效,但当它们处理多个相等过滤器然后处理一个范围过滤器时,它们的效果最好。 (time BETWEEN this AND that
)
因此,为了优化,我们需要从具有最大选择性的列开始您的多列索引。我们需要 compound covering indexes 作为您的查询模式。
我认为你应该为你的第一个模式尝试这个索引。
CREATE INDEX closedex ON mt4_trades
(CLOSE_TIME, CMD, OPEN_TIME, SYMBOL, VOLUME, CLOSE_PRICE, LOGIN)
对于你的第二个模式,它有点简单
CREATE INDEX opendex ON mt4_trades
(OPEN_TIME, CMD, SYMBOL, VOLUME, CLOSE_PRICE, LOGIN)
您需要两个索引,因为(我猜)您最有选择性的列是 CLOSE_TIME
和 OPEN_TIME
。您还应该尝试将 CMD
放在这些索引的第一位;也许 MariaDB 知道如何有效地使用 CMD IN (0,1)
的索引。
重点是让查询规划器能够单独满足来自索引的查询,而不必跳回到table。
如果您可以将 SYMBOL LIKE 'value%'
更改为 SYMBOL = 'value'
并且您的应用程序仍能正常工作,请执行此操作。然后将 SYMBOL
放在索引的第一位;那是平等匹配。
(重要提示:在您的查询行中
SELECT MIN(t.CLOSE_TIME) AS fd_time, t.VOLUME, t.CLOSE_PRICE AS PRICE
你得到了 VOLUME 和 CLOSE_PRICE 的不可预测的table 值。
(如果这是我为我的雇主处理其他人的钱的查询,我会花几个小时来分析它的正确性。)
- 不要在
FLOAT
或DOUBLE
的末尾使用 (m,n)。既然你有,0
,你不妨使用一些INT
变体。 - INDEX
CMD
(CMD
,OPEN_TIME
,CLOSE_TIME
,LOGIN
,VOLUME
,SYMBOL
,CLOSE_PRICE
) -- 可能列太多了。 - 我希望你的表是 InnoDB。
- 除非您有其他应用占用 RAM,否则您可以增加
innodb_buffer_pool_size
。 - 这种模式通常效率低下:
FROM ( SELECT ... ) JOIN ( SELECT ... )
@fd_time0
-- 使用@变量有风险;优化器可能会导致意外;也就是说,您不能依赖于何时设置和使用它们。 (他们最终将被禁止。)UNION ALLs
好像刚好生成两行,那你把这两个值加起来?而不是将UNION
送入SUM
,而是这样做:( SELECT ( SELECT ... ) + ( SELECT ... ) )
这个
WHERE
有 4 个范围。只有一个人可以使用索引:WHERE t.CLOSE_TIME BETWEEN ... AND t.OPEN_TIME > '2012-08-26' AND t.SYMBOL LIKE '#AT&T%' AND t.CMD IN (0,1)
我建议您给优化器 3 个选择:
INDEX(close_time)
INDEX(OPEN_TIME)
INDEX(SYMBOL)
如果你不需要符号末尾的通配符,那么这些索引会更好:
INDEX(SYMBOL, close_time) INDEX(SYMBOL, OPEN_TIME)
(我的偏好)而不是
t.OPEN_TIME BETWEEN '2020-05-21 09:30' AND DATE_ADD('2020-05-21 09:30', INTERVAL 119 SECOND
我比较喜欢
t.OPEN_TIME >= '2020-05-21 09:30'
AND t.OPEN_TIME < '2020-05-21 09:30' + INTERVAL 2 MINUTE
SELECT MIN(t.CLOSE_TIME) AS fd_time, t.VOLUME ...
会给出错误的数据!!!。查看标签[groupwise-maximum]
.