MySQL 中加速 LEFT JOIN 的建议

Suggestions for speeding up LEFT JOIN in MySQL

我有一个名为 usr_data 的 tabled,其中有 40,000 条左右的记录,我想通过 select 在加入,显然这真的很慢(最多150秒)。

我想知道是否有任何方法可以加快速度?不幸的是,这是连接中子 select 的最快查询。

我的查询

    SELECT  usr_data.usr_id,usr_data.login,orgus.title FROM usr_data
    LEFT JOIN (
        SELECT object_reference.ref_id,rbac_ua.usr_id,object_data.obj_id,object_data.title
        FROM rbac_ua
        JOIN rbac_fa ON rbac_fa.rol_id = rbac_ua.rol_id
        JOIN object_reference ON rbac_fa.parent = object_reference.ref_id
        JOIN object_data ON object_data.obj_id = object_reference.obj_id
        JOIN object_data role ON role.obj_id = rbac_ua.rol_id
        WHERE object_data.type = 'orgu') as orgus on orgus.usr_id = usr_data.usr_id
    WHERE usr_data.usr_id > 0 AND usr_data.login <> "anonymous" 

查询描述

+------+-------------+------------------+--------+-----------------------+---------+---------+--------------------------------+------+--------------------------+
| id   | select_type | table            | type   | possible_keys         | key     | key_len | ref                            | rows | Extra                    |
+------+-------------+------------------+--------+-----------------------+---------+---------+--------------------------------+------+--------------------------+
|    1 | SIMPLE      | usr_data         | range  | PRIMARY,i1_idx        | PRIMARY | 4       | NULL                           | 8148 | Using where              |
|    1 | SIMPLE      | rbac_ua          | ref    | PRIMARY,i1_idx,i2_idx | PRIMARY | 4       | ildCPC.usr_data.usr_id         |    2 | Using where; Using index |
|    1 | SIMPLE      | rbac_fa          | ref    | PRIMARY,i1_idx        | PRIMARY | 4       | ildCPC.rbac_ua.rol_id          |    1 | Using where; Using index |
|    1 | SIMPLE      | role             | eq_ref | PRIMARY               | PRIMARY | 4       | ildCPC.rbac_ua.rol_id          |    1 | Using index              |
|    1 | SIMPLE      | object_reference | eq_ref | PRIMARY,i1_idx        | PRIMARY | 4       | ildCPC.rbac_fa.parent          |    1 | Using where              |
|    1 | SIMPLE      | object_data      | eq_ref | PRIMARY,i1_idx        | PRIMARY | 4       | ildCPC.object_reference.obj_id |    1 | Using where              |
+------+-------------+------------------+--------+-----------------------+---------+---------+--------------------------------+------+--------------------------+

usr_data table

+----------------------+---------------+------+-----+---------+-------+
| Field                | Type          | Null | Key | Default | Extra |
+----------------------+---------------+------+-----+---------+-------+
| usr_id               | int(11)       | NO   | PRI | 0       |       |
| login                | varchar(80)   | YES  | MUL | NULL    |       |
| passwd               | varchar(80)   | YES  |     | NULL    |       |
| firstname            | varchar(32)   | YES  |     | NULL    |       |
| lastname             | varchar(32)   | YES  |     | NULL    |       |
| title                | varchar(32)   | YES  |     | NULL    |       |
| gender               | char(1)       | YES  |     | m       |       |
| email                | varchar(80)   | YES  |     | NULL    |       |

usr_data 索引

+----------+------------+----------+--------------+-------------+-----------+-------------+----------+--------+------+------------+---------+---------------+
| Table    | Non_unique | Key_name | Seq_in_index | Column_name | Collation | Cardinality | Sub_part | Packed | Null | Index_type | Comment | Index_comment |
+----------+------------+----------+--------------+-------------+-----------+-------------+----------+--------+------+------------+---------+---------------+
| usr_data |          0 | PRIMARY  |            1 | usr_id      | A         |       29354 |     NULL | NULL   |      | BTREE      |         |               |
| usr_data |          1 | i1_idx   |            1 | login       | A         |       29354 |     NULL | NULL   | YES  | BTREE      |         |               |
| usr_data |          1 | i1_idx   |            2 | passwd      | A         |       29354 |     NULL | NULL   | YES  | BTREE      |         |               |
| usr_data |          1 | i2_idx   |            1 | ext_account | A         |           2 |     NULL | NULL   | YES  | BTREE      |         |               |
| usr_data |          1 | i2_idx   |            2 | auth_mode   | A         |           4 |     NULL | NULL   | YES  | BTREE      |         |               |
+----------+------------+----------+--------------+-------------+-----------+-------------+----------+--------+------+------------+---------+---------------+

我尝试在 LEFT JOIN 中为 SELECT 创建一个临时 table,但它并没有真正加快速度,目前查询最多需要 150 秒,通过正确的连接,它可以将其降低到大约 1 秒。 (这是较小的table)。

select * from object_data where type = 'orgu' returns 1058 rows.

显示创建 TABLE rbac_ua

+---------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| Table   | Create Table                                                                                                                                                                                                                                                                                                                                                |
+---------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| rbac_ua | CREATE TABLE `rbac_ua` (
  `usr_id` int(11) NOT NULL DEFAULT '0',
  `rol_id` int(11) NOT NULL DEFAULT '0',
  PRIMARY KEY (`usr_id`,`rol_id`),
  KEY `i1_idx` (`usr_id`),
  KEY `i2_idx` (`rol_id`),
  KEY `rol_id` (`rol_id`,`usr_id`),
  KEY `rol_usr` (`rol_id`,`usr_id`)
) ENGINE=InnoDB DEFAULT CHARSET=utf8 COLLATE=utf8_unicode_ci ROW_FORMAT=DYNAMIC |
+---------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+

显示创建 TABLE rbac_fa

+---------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| Table   | Create Table                                                                                                                                                                                                                                                                                                                                                                                                    |
+---------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| rbac_fa | CREATE TABLE `rbac_fa` (
  `rol_id` int(11) NOT NULL DEFAULT '0',
  `parent` int(11) NOT NULL DEFAULT '0',
  `assign` char(1) COLLATE utf8_unicode_ci DEFAULT NULL,
  `protected` char(1) COLLATE utf8_unicode_ci DEFAULT 'n',
  PRIMARY KEY (`rol_id`,`parent`),
  KEY `i1_idx` (`parent`),
  KEY `parent` (`parent`,`rol_id`)
) ENGINE=InnoDB DEFAULT CHARSET=utf8 COLLATE=utf8_unicode_ci ROW_FORMAT=DYNAMIC |
+---------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+

显示创建 TABLE object_data

+-------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| Table       | Create Table                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                           |
+-------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| object_data | CREATE TABLE `object_data` (
  `obj_id` int(11) NOT NULL DEFAULT '0',
  `type` char(4) COLLATE utf8_unicode_ci DEFAULT 'none',
  `title` char(255) COLLATE utf8_unicode_ci DEFAULT NULL,
  `description` char(128) COLLATE utf8_unicode_ci DEFAULT NULL,
  `owner` int(11) NOT NULL DEFAULT '0',
  `create_date` datetime DEFAULT NULL,
  `last_update` datetime DEFAULT NULL,
  `import_id` char(50) COLLATE utf8_unicode_ci DEFAULT NULL,
  PRIMARY KEY (`obj_id`),
  KEY `i1_idx` (`type`),
  KEY `i2_idx` (`title`),
  KEY `i4_idx` (`import_id`),
  FULLTEXT KEY `i3_idx` (`title`,`description`)
) ENGINE=InnoDB DEFAULT CHARSET=utf8 COLLATE=utf8_unicode_ci ROW_FORMAT=DYNAMIC |
+-------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

显示创建 TABLE object_reference

+------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| Table            | Create Table                                                                                                                                                                                                                                                                                                                                               |
+------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| object_reference | CREATE TABLE `object_reference` (
  `ref_id` int(11) NOT NULL DEFAULT '0',
  `obj_id` int(11) NOT NULL DEFAULT '0',
  `deleted` datetime DEFAULT NULL,
  PRIMARY KEY (`ref_id`),
  KEY `i1_idx` (`obj_id`),
  KEY `i2_idx` (`deleted`),
  KEY `obj_id` (`obj_id`,`ref_id`)
) ENGINE=InnoDB DEFAULT CHARSET=utf8 COLLATE=utf8_unicode_ci ROW_FORMAT=DYNAMIC |
+------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+

如果我遗漏了任何有用的信息,请告诉我。

提前致谢。

通过删除子选择和使用连接,您应该能够提高性能。首先处理子选择,生成一个临时文件 table,这是一个相当大的性能损失。

SELECT  
    usr_data.usr_id,
    usr_data.login,
    object_data.title,
    object_reference.ref_id,
    rbac_ua.usr_id,
    object_data.obj_id,
    object_data.title
FROM usr_data
LEFT JOIN rbac_ua
ON rbac_ua.usr_id = usr_data.usr_id
JOIN rbac_fa ON rbac_fa.rol_id = rbac_ua.rol_id
JOIN object_reference ON rbac_fa.parent = object_reference.ref_id
JOIN object_data ON object_data.obj_id = object_reference.obj_id
JOIN object_data role ON role.obj_id = rbac_ua.rol_id
WHERE 
    object_data.type = 'orgu' AND 
    usr_data.usr_id > 0 AND 
    usr_data.login <> "anonymous"

通过使用 LEFT JOIN,您需要一个大型结果集,其中包含与您的 WHERE usr_data whatever 子句匹配的每个项目的行。即使查询规划器做得很好,这也需要时间推送到您的客户端软件。

usr_data.usr_id > 0 似乎是多余的,因为您的 usr_id 列似乎是一个自动增量列。所有值都大于零吗?

子选择绝对是一个性能消耗者。

先生Thrasher 很接近,我想。问题是在 WHERE 子句中提到 LEFT JOINed table 的列:将 LEFT JOIN 转换为直接的 JOIN.

试试这个看看你是否得到合适的结果。我不明白你的架构,所以这里有一些猜测。

     SELECT usr_data.usr_id, usr_data.login, object_data.title 
       FROM usr_data
  LEFT JOIN rbac_ua ON rbac_ua.usr_id = usr_data.usr_id
  LEFT JOIN rbac_fa ON rbac_fa.rol_id = rbac_ua.rol_id
  LEFT JOIN object_reference ON rbac_fa.parent = object_reference.ref_id
  LEFT JOIN object_data   ON object_data.obj_id = object_reference.obj_id
                         AND object_data.type = 'orgu'
  LEFT JOIN object_data role ON role.obj_id = rbac_ua.rol_id
      WHERE usr_data.login <> 'anonymous'

注意 object_data.type = 'orgu' 如何在 ON 子句中结束。 (是的,ON 子句可以包含与 WHERE 子句相同的内容!)这使得它不会将 LEFT JOIN 变成直接的 JOIN.

我不确定这行的重点。

  LEFT JOIN object_data role ON role.obj_id = rbac_ua.rol_id

table 似乎对您的结果集没有贡献。

EXPLAIN 中,我看到优化程序认为“LEFT”没有影响。所以它删除了它。然后它决定 "derived table" 可以变成 JOIN。你本可以做到这一切的。

但真正的性能问题是过滤的重要部分在 type = 'orgu' 上,但被埋在 'last' table.

(此后我猜测是因为 每个 table 都缺少 SHOW CREATE TABLE。而 DESCRIBE 不像描述性广告 SHOW CREATE.)

让我们看看这个:

SELECT  ud.usr_id, ud.login, od.title
    FROM  object_data AS od
    JOIN  object_reference AS r  ON od.obj_id = r.obj_id
    JOIN  rbac_fa AS rfa  ON rfa.parent = r.ref_id
    JOIN  rbac_ua AS rua  ON rfa.rol_id = rua.rol_id
    JOIN  usr_data AS ud  ON ud.usr_id = rua.usr_id
     -- unnec?:  JOIN  object_data role  ON role.obj_id = rua.rol_id
    WHERE  od.type = 'orgu'
      AND  ud.usr_id > 0
      AND  ud.login <> "anonymous" 

我认为在 添加下面的索引后效率更高。如果 type = 'orgu' 具有足够的选择性,则可能会出现这种情况。

请注意,role 除了验证给定的 obj_id.

是否存在角色外,不向查询添加任何内容

需要索引以便它可以 start with type = 'orgu':

object_data: INDEX(type)
object_reference:  INDEX(obj_id, ref_id)  -- (covering, too)
rbac_fa:  INDEX(parent, rol_id)  -- (covering, too)
rbac_ua:  INDEX(rol_id, usr_id)  -- (covering, too)

如果没有帮助,请提供 EXPLAIN SELECT ... 结果。并告诉我们有多少行 object_data 有 type=orgu.