MySQL 加入限制动态查询优化

Question

这里的架构：-

Student_Details
ID | Student_id | Name          | Value
1  | 2          | City          | NewYork
2  | 2          | Height        | 5'11'
3  | 2          | Class         | B
4  | 2          | RollNo        | 265454
5  | 2          | Credit        | 800
6  | 3          | City          | Manila
7  | 3          | Height        | 5'10'
8  | 3          | Type          | International
9  | 3          | RollNo        | 2653
10 | 1          | Address       | Main Street
11 | 1          | Height        | 5'9'
12 | 1          | TST           | 21.54
13 | 1          | CCA           | 242
13 | 1          | SEX           | Male

Students 
ID | Name   | Age | AddedOn
1  | Alpha  | 27  | 20 Jan 2019
2  | Beta   | 15  | 19 Jan 2019
3  | Gamma  | 18  | 20 Jan 2019 

Student_Dep_Map
ID |DEP_ID | Student_id
 1 | 4 | 2
 2 | 2 | 1

Department 
ID| Name 
1 | Science
2 | Physics
3 | Chemistry 
4 | Psychology

所以搜索围绕学生展开，我们需要使用各种搜索条件搜索学生，例如 'City' ，'Height' 或 'TST' ，学生在哪个部门，添加时间，年龄，等等..，基本上所有这些 tables。动态搜索。此外，输出搜索还应包含每个 tables 中的一些固定详细信息。所以我用 pivot 做了这个查询。

select distinct s.*
FROM  students s
LEFT JOIN ( SELECT s.ID as s_id, group_concat(distinct d.name SEPARATOR ', ') as ASSIGNED_DEPT FROM students s   
          JOIN  Student_Dep_Map sdp JOIN  Department d on sdp.DEP_ID = d.ID ON sdp.Student_id=s.id  GROUP BY s.id  ) aa  ON aa.s_id = s.ID 
LEFT JOIN ( SELECT DISTINCT  student_id, 
            MAX(if(Name = 'TST', value, null)) as TST,   
                        MAX(if(Name = 'Height', value, null)) as HEIGHT, 
            MAX(if(Name = 'CITY', value, null)) as CITY, 
            MAX(if(Name = 'SEX',  value, null)) as SEX,
            MAX(if(Name = 'RollNo',  value, null)) as ROLLNO 
            FROM student_detials tip  GROUP BY  student_id ) sd ON sd.student_id = s.ID
WHERE 1=1 AND SD.SEX = 'Male' LIMIT 0,100 ;

AND 条件是通过查询生成器动态添加的。

挑战是！这个查询太慢了，随着 tables 中记录的增加，查询变得非常慢。我确实使用 explain 来了解查询计划。如果我离开索引并只谈论优化查询计划。我注意到当我使用 pivot 时，即使搜索条件来自 main table student，查询计划仍然会在旋转学生详细信息时遍历几乎所有记录。有什么方法可以限制加入，直到找到 100 条学生记录。一种方法是为每个 table 编写子查询到 LIMIT 结果，但是有没有更好的方法来处理这个问题？我需要限制连接，而不仅仅是连接后的结果，因此查询会导致更快的获取。

注意：抱歉，我无法 post 真实 tables ，它只是模拟数据。

Answer 1

我建议使用 exists 而不是聚合：

SELECT s.*,
       (SELECT GROUP_CONCAT(d.name SEPARATOR ', ')
        FROM Student_Dep_Map sdp JOIN
             Department d 
             ON sdp.DEP_ID = d.ID 
        WHERE sdp.Student_id = s.id
       ) as ASSIGNED_DEPT
FROM students s 
WHERE EXISTS (SELECT 1
              FROM student_details sd
              WHERE sd.student_id = s.id AND
                    sd.Name = 'SEX' AND
                    sd.Value = 'Male'
             )
LIMIT 0, 100 ;

为了提高性能，您需要索引：

Student_Dep_Map(student_id, dep_id)
Departments(id, name)（可能存在因为id应该是主键）
student_details(student_id, name, value).

要添加更多条件，请添加更多 EXISTS 个子查询。

Answer 2

如果您有兴趣从 student_details 中选择某些字段，则可以采用另一种可能的方法。

SELECT s.*

FROM students s

LEFT OUTER JOIN
  (
    SELECT student_id 
    , MAX(if(Name = 'TST', value, null)) as TST   
    , MAX(if(Name = 'Height', value, null)) as HEIGHT 
    , MAX(if(Name = 'CITY', value, null)) as CITY 
    , MAX(if(Name = 'SEX',  value, null)) as SEX
    , MAX(if(Name = 'RollNo',  value, null)) as ROLLNO 

    FROM student_details tip  

    GROUP BY  student_id
  ) sd
  ON s.ID = sd.student_ID
   AND sd.SEX = 'Male'

LEFT OUTER JOIN 
  (
    SELECT sdp.ID as s_id
    , group_concat(d.name SEPARATOR ', ') as ASSIGNED_DEPT

    FROM Student_Dep_Map sdp
    INNER JOIN Department d ON sdp.DEP_ID = d.ID

    GROUP BY sdp.id
  ) aa
  ON aa.s_id = s.ID
;

您的原始查询也有 LEFT JOIN(...) WHERE sd.sex = 'Male'，这将消除整个 LEFT JOIN 部分。您可以通过以下方式简化它：

SELECT s.ID
, s.Name
, s.Age
, s.AddedOn
, group_concat(d.name SEPARATOR ', ') as ASSIGNED_DEPT
, sd.City

FROM students s
INNER JOIN student_dep_map sdp  ON s.ID = sdp.ID
INNER JOIN Department d     ON sdp.DEP_ID = d.ID
INNER JOIN 
  (
    SELECT student_id 
    , MAX(if(Name = 'TST', value, null)) as TST   
    , MAX(if(Name = 'Height', value, null)) as HEIGHT 
    , MAX(if(Name = 'CITY', value, null)) as CITY 
    , MAX(if(Name = 'SEX',  value, null)) as SEX
    , MAX(if(Name = 'RollNo',  value, null)) as ROLLNO 

    FROM student_details tip  

    GROUP BY  student_id
  ) sd
  ON s.ID = sd.student_ID

WHERE sd.SEX = 'Male'

GROUP BY s.ID
, s.Name
, s.Age
, s.AddedOn
, sd.City;

MySQL 加入限制动态查询优化

MySQL JOIN LIMIT DYNAMIC QUERY OPTIMIZATION

mysql

sql

query-optimization

entity-attribute-value