从 5.6.1 升级到 6.5 的数据库迁移异常缓慢

Question

我正在尝试将 5.6.1 sonarqube 实例升级到 6.5，它在 linux 上针对 Oracle 数据库运行。

我添加了以下插件；

groovy1.5
ldap 2.2.0.608

oracle驱动是ojdbc6-11.2.0.2.0 jdk 是 1.8。0_144

进程启动，但网络进程挂起。我添加了 TRACE 日志记录并查看

2017.09.07 22:07:02 TRACE web[][o.s.s.p.Platform] level1 started | time=2286ms
2017.09.07 22:07:02 TRACE web[][o.s.s.p.Platform] level2 configured | time=33ms

然后什么都没有，直到

2017.09.07 23:47:53 TRACE web[][o.s.c.p.ComponentContainer] org.sonar.server.platform.db.migration.history.MigrationHistoryTableImpl started | time=2139938ms
2017.09.08 00:27:00 TRACE web[][o.s.c.p.ComponentContainer] org.sonar.server.platform.db.migration.history.MigrationHistoryImpl started | time=2347061ms
2017.09.08 00:27:00 TRACE web[][sql] time=4ms | sql=select version from schema_migrations
2017.09.08 00:27:00 TRACE web[][sql] time=2ms | sql=select version from schema_migrations
2017.09.08 00:27:00 WARN  web[][o.s.s.p.DatabaseServerCompatibility] Database must be upgraded. Please backup database and browse /setup

例如，一旦迁移开始，同样的缓慢会继续

2017.09.08 17:06:45 INFO  web[][o.s.s.p.d.m.v.v.DropRelatedDashboardTables] Removing tables [widget_properties, widgets, active_dashboards, dashboards]
2017.09.08 17:06:45 TRACE web[][sql] time=35ms | sql=BEGIN   EXECUTE IMMEDIATE 'DROP SEQUENCE widget_properties_seq'; EXCEPTION   WHEN OTHERS THEN     IF SQLCODE != -2289 THEN       RAISE;     END IF; END;
2017.09.08 17:06:45 TRACE web[][sql] time=19ms | sql=BEGIN   EXECUTE IMMEDIATE 'DROP TRIGGER widget_properties_idt'; EXCEPTION   WHEN OTHERS THEN     IF SQLCODE != -4080 THEN       RAISE;     END IF; END;
2017.09.08 17:06:45 TRACE web[][sql] time=87ms | sql=DROP TABLE widget_properties
2017.09.08 17:06:45 TRACE web[][sql] time=12ms | sql=BEGIN   EXECUTE IMMEDIATE 'DROP SEQUENCE widgets_seq'; EXCEPTION   WHEN OTHERS THEN     IF SQLCODE != -2289 THEN       RAISE;     END IF; END;
2017.09.08 17:06:45 TRACE web[][sql] time=16ms | sql=BEGIN   EXECUTE IMMEDIATE 'DROP TRIGGER widgets_idt'; EXCEPTION   WHEN OTHERS THEN     IF SQLCODE != -4080 THEN       RAISE;     END IF; END;
2017.09.08 17:06:45 TRACE web[][sql] time=43ms | sql=DROP TABLE widgets
2017.09.08 17:06:45 TRACE web[][sql] time=11ms | sql=BEGIN   EXECUTE IMMEDIATE 'DROP SEQUENCE active_dashboards_seq'; EXCEPTION   WHEN OTHERS THEN     IF SQLCODE != -2289 THEN       RAISE;     END IF; END;
2017.09.08 17:06:45 TRACE web[][sql] time=27ms | sql=BEGIN   EXECUTE IMMEDIATE 'DROP TRIGGER active_dashboards_idt'; EXCEPTION   WHEN OTHERS THEN     IF SQLCODE != -4080 THEN       RAISE;     END IF; END;
2017.09.08 17:06:45 TRACE web[][sql] time=64ms | sql=DROP TABLE active_dashboards
2017.09.08 17:06:45 TRACE web[][sql] time=12ms | sql=BEGIN   EXECUTE IMMEDIATE 'DROP SEQUENCE dashboards_seq'; EXCEPTION   WHEN OTHERS THEN     IF SQLCODE != -2289 THEN       RAISE;     END IF; END;
2017.09.08 17:06:45 TRACE web[][sql] time=19ms | sql=BEGIN   EXECUTE IMMEDIATE 'DROP TRIGGER dashboards_idt'; EXCEPTION   WHEN OTHERS THEN     IF SQLCODE != -4080 THEN       RAISE;     END IF; END;
2017.09.08 17:06:45 TRACE web[][sql] time=37ms | sql=DROP TABLE dashboards
2017.09.08 17:06:45 TRACE web[][sql] time=6ms | sql=insert into schema_migrations(version) values (?) | params=1420
2017.09.08 17:06:45 INFO  web[][DbMigrations] #1420 'Drop tables related to dashboards': success | time=8515237ms
2017.09.08 17:06:45 INFO  web[][DbMigrations] #1421 'Drop tables related to measure filters'...

即大量操作似乎需要数小时才能完成

什么会导致性能如此缓慢？由于实际查询执行时间似乎很短，因此日志中没有任何提示可以说明它在做什么。

Answer 1

这个问题的根本原因原来是数据字典查询执行不当，一个例子是这样的查询

SELECT NULL AS table_cat, 
     o.owner AS table_schem, 
     o.object_name AS table_name, 
     o.object_type AS table_type, 
     NULL AS remarks 
FROM all_objects o 
WHERE     o.owner LIKE :1 ESCAPE '/' 
     AND o.object_name LIKE :2 ESCAPE '/' 
     AND o.object_type IN ('xxx', 'TABLE') 
ORDER BY table_type, table_schem, table_name

这在 sonarqube 5.6 中没有出现。

从 5.6.1 升级到 6.5 的数据库迁移异常缓慢

Database migration in a 5.6.1 to 6.5 upgrade is exceptionally slow

sonarqube

sonarqube-ops