SonarQube 5.6.7 无法搜索问题（ES错误）
SonarQube 5.6.7 unable to search for issues (ES error)
我可以进行分析，。现在我在浏览问题时面临更多问题。我们的基础架构跟不上 Elastic Search 的要求，SECCOMP 没有编译到内核中，所以我将这个选项添加到 sonar.properties sonar.search.javaAdditionalOpts=-Dbootstrap.system_call_filter=false
我还提供了大量的内存，
sonar.search.javaOpts=-Xmx2G -Xms1G -Xss1m -Djava.net.preferIPv4Stack=true \
-XX:+UseParNewGC -XX:+UseConcMarkSweepGC -XX:CMSInitiatingOccupancyFraction=75 \
-XX:+UseCMSInitiatingOccupancyOnly -XX:+HeapDumpOnOutOfMemoryError
系统 64 位，16 GM RAM 和 2 核；运行 RHEL 6.7
并且 ES 出现故障并显示以下日志。
2018.03.01 14:32:34 WARN es[o.e.indices.breaker] [sonar-1519909671773] [FIELDDATA] New used memory 1285367970 [1.1gb] from field [key] would be larger than configured breaker: 1278030643 [1.1gb], breaking
2018.03.01 15:09:43 WARN es[o.e.indices.breaker] [sonar-1519909671773] [FIELDDATA] New used memory 1285374084 [1.1gb] from field [key] would be larger than configured breaker: 1278030643 [1.1gb], breaking
2018.03.01 15:10:04 WARN es[o.e.indices.breaker] [sonar-1519909671773] [FIELDDATA] New used memory 1285374084 [1.1gb] from field [key] would be larger than configured breaker: 1278030643 [1.1gb], breaking
2018.03.01 15:20:15 WARN es[o.e.indices.breaker] [sonar-1519909671773] [FIELDDATA] New used memory 1285379720 [1.1gb] from field [key] would be larger than configured breaker: 1278030643 [1.1gb], breaking
2018.03.01 15:20:47 WARN es[o.e.indices.breaker] [sonar-1519909671773] [FIELDDATA] New used memory 1285379720 [1.1gb] from field [key] would be larger than configured breaker: 1278030643 [1.1gb], breaking
2018.03.01 15:21:30 WARN es[o.e.indices.breaker] [sonar-1519909671773] [FIELDDATA] New used memory 1285379720 [1.1gb] from field [key] would be larger than configured breaker: 1278030643 [1.1gb], breaking
2018.03.01 15:21:39 WARN es[o.e.indices.breaker] [sonar-1519909671773] [FIELDDATA] New used memory 1287374978 [1.1gb] from field [key] would be larger than configured breaker: 1278030643 [1.1gb], breaking
2018.03.01 15:22:02 WARN es[o.e.indices.breaker] [sonar-1519909671773] [FIELDDATA] New used memory 1287374978 [1.1gb] from field [key] would be larger than configured breaker: 1278030643 [1.1gb], breaking
2018.03.01 15:22:12 WARN es[o.e.indices.breaker] [sonar-1519909671773] [FIELDDATA] New used memory 1279785543 [1.1gb] from field [issueCreatedAt] would be larger than configured breaker: 1278030643 [1.1gb], breaking
2018.03.01 15:22:12 WARN es[o.e.indices.breaker] [sonar-1519909671773] [FIELDDATA] New used memory 1279798446 [1.1gb] from field [issueCreatedAt] would be larger than configured breaker: 1278030643 [1.1gb], breaking
2018.03.01 15:22:12 WARN es[o.e.indices.breaker] [sonar-1519909671773] [FIELDDATA] New used memory 1279787446 [1.1gb] from field [issueCreatedAt] would be larger than configured breaker: 1278030643 [1.1gb], breaking
2018.03.01 15:22:12 WARN es[o.e.indices.breaker] [sonar-1519909671773] [FIELDDATA] New used memory 1279771440 [1.1gb] from field [issueCreatedAt] would be larger than configured breaker: 1278030643 [1.1gb], breaking
2018.03.01 15:22:12 WARN es[o.e.indices.breaker] [sonar-1519909671773] [FIELDDATA] New used memory 1279800560 [1.1gb] from field [issueCreatedAt] would be larger than configured breaker: 1278030643 [1.1gb], breaking
2018.03.01 15:22:13 ERROR web[o.s.s.w.WebServiceEngine] Fail to process request http://172.17.33.86:9091/sonar/api/issues/search?p=1&ps=50&s=FILE_LINE&asc=true&additionalFields=_all&facets=types%2Cresolutions%2CcreatedAt&resolved=false&types=VULNERABILITY&sinceLeakPeriod=true&componentUuids=AWHS0koMv8wI-iczVKXw
java.lang.IllegalStateException: Fail to execute ES search request '{"from":0,"size":50,"query":{"filtered":{"query":{"match_all":{}},"filter":{"bool":{"must":[{"terms":{"modulePath":["AWHS0koMv8wI-iczVKXw"]}},{"missing":{"field":"resolution"}},{"terms":{"type":["VULNERABILITY"]}},{"range":{"issueCreatedAt":{"from":"2018-02-19T19:14:08.456Z","to":null,"include_lower":true,"include_upper":true},"_cache":false}},{"has_parent":{"query":{"filtered":{"query":{"match_all":{}},"filter":{"bool":{"must":{"or":{"filters":[{"term":{"users":"admin"}},{"term":{"groups":"Anyone"}},{"term":{"groups":"sonar-users"}},{"term":{"groups":"sonar-administrators"}}]}},"_cache":true}}}},"parent_type":"authorization"}}]}}}},"sort":[{"project":{"order":"asc","missing":"_first"}},{"filePath":{"order":"asc","missing":"_first"}},{"line":{"order":"asc","missing":"_first"}},{"severityValue":{"order":"desc","missing":"_first"}},{"key":{"order":"asc","missing":"_first"}}],"aggregations":{"types":{"global":{},"aggregations":{"types_filter":{"filter":{"bool":{"must":[{"query":{"match_all":{}}},{"terms":{"modulePath":["AWHS0koMv8wI-iczVKXw"]}},{"missing":{"field":"resolution"}},{"range":{"issueCreatedAt":{"from":"2018-02-19T19:14:08.456Z","to":null,"include_lower":true,"include_upper":true},"_cache":false}},{"has_parent":{"query":{"filtered":{"query":{"match_all":{}},"filter":{"bool":{"must":{"or":{"filters":[{"term":{"users":"admin"}},{"term":{"groups":"Anyone"}},{"term":{"groups":"sonar-users"}},{"term":{"groups":"sonar-administrators"}}]}},"_cache":true}}}},"parent_type":"authorization"}}]}},"aggregations":{"types":{"terms":{"field":"type","size":10,"min_doc_count":1,"order":{"_count":"desc"}}},"types_selected":{"terms":{"field":"type","include":"VULNERABILITY"}}}}}},"resolutions":{"global":{},"aggregations":{"resolutions_filter":{"filter":{"bool":{"must":[{"query":{"match_all":{}}},{"terms":{"modulePath":["AWHS0koMv8wI-iczVKXw"]}},{"terms":{"type":["VULNERABILITY"]}},{"range":{"issueCreatedAt":{"from":"2018-02-19T19:14:08.456Z","to":null,"include_lower":true,"include_upper":true},"_cache":false}},{"has_parent":{"query":{"filtered":{"query":{"match_all":{}},"filter":{"bool":{"must":{"or":{"filters":[{"term":{"users":"admin"}},{"term":{"groups":"Anyone"}},{"term":{"groups":"sonar-users"}},{"term":{"groups":"sonar-administrators"}}]}},"_cache":true}}}},"parent_type":"authorization"}}]}},"aggregations":{"resolutions":{"terms":{"field":"resolution","size":15,"min_doc_count":1,"order":{"_count":"desc"}}},"resolutions_missing":{"missing":{"field":"resolution"}}}}}},"createdAt":{"date_histogram":{"field":"issueCreatedAt","interval":"1d","min_doc_count":0,"pre_zone":"GMT","offset":"-3600s","format":"yyyy-MM-dd'T'HH:mm:ssZ","extended_bounds":{"min":1519067648456,"max":1519914131788}}}}}' on indices '[issues]' on types '[issue]'
at org.sonar.server.es.request.ProxySearchRequestBuilder.get(ProxySearchRequestBuilder.java:47) ~[sonar-server-5.6.7.jar:na]
at org.sonar.server.es.request.ProxySearchRequestBuilder.get(ProxySearchRequestBuilder.java:35) ~[sonar-server-5.6.7.jar:na]
at org.sonar.server.issue.index.IssueIndex.search(IssueIndex.java:235) ~[sonar-server-5.6.7.jar:na]
at org.sonar.server.issue.ws.SearchAction.doHandle(SearchAction.java:301) ~[sonar-server-5.6.7.jar:na]
at org.sonar.server.issue.ws.SearchAction.handle(SearchAction.java:288) ~[sonar-server-5.6.7.jar:na]
at org.sonar.server.ws.WebServiceEngine.execute(WebServiceEngine.java:107) ~[sonar-server-5.6.7.jar:na]
at sun.reflect.GeneratedMethodAccessor228.invoke(Unknown Source) ~[na:na]
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[na:1.8.0_91]
at java.lang.reflect.Method.invoke(Method.java:498) ~[na:1.8.0_91]
at org.jruby.javasupport.JavaMethod.invokeDirectWithExceptionHandling(JavaMethod.java:425) [jruby-complete-1.7.9.jar:na]
at org.jruby.javasupport.JavaMethod.invokeDirect(JavaMethod.java:292) [jruby-complete-1.7.9.jar:na]
at org.jruby.java.invokers.InstanceMethodInvoker.call(InstanceMethodInvoker.java:44) [jruby-complete-1.7.9.jar:na]
是的，它确实在崩溃前请求了一段时间，并拒绝在 UI 上显示任何问题。请帮助，所有准备升级到 SQ 5.6.7 以便稍后我们可以升级到 SQ 6.7.1 但现在我们仍然是 运行 SQ 4.5.4 在生产中（它能够处理如此大量的数据)
How large is our DB? Our database has close to 27 million LOC, with
5.6 million closed/open issues on Database.
Proper Solution
- 升级到 SonarQube 6.1 LTS
- 如果您是 Oracle DB，请确保分析了 tables。即，继续索引 up-to-date
- 如果您没有升级空间并且索引是 up-to-date 但您遇到缓慢，请为数据库清理器设置最少周数，甚至比 SonarQube 和 运行 分析推荐的还要少.
当 SonarQube 删除旧快照时，所有相关的旧问题都将被正确删除，只有当新分析是 运行 时，才会在组件上执行 db cleaner。如果您有幽灵项目（不再在 SCM 中）创建空项目并使用相同的密钥清除数据库。
Stupid Hack
长期以来，我尝试按照问题 SONAR-8067 中的建议更改 SonarQube 的代码（在 Ann 的评论中链接）。但我没能做到，描述太高级了，我几乎没有时间疯狂地逆向工程，所以我选择了暴力破解。
特别提及：我相信我们已经对已关闭的问题制定了相当严厉的政策，但他们仍然顽固不化。
多亏了扫描缩小版的 JS，我们的数据库每天都在呈指数级增长，现在它有 680 万行问题table，无奈我执行了以下命令。
delete from issues where status = 'CLOSED' or status = 'RESOLVED'
这个 sql 语句用了 14 分钟，从 table 中清除了 450 万行。后来从 SonarQube 主目录中删除 data、temp 目录并重新启动 SonarQube，现在 ES 正在呼吸。 但故事还没有结束。现在仪表板是 unstable，680 万行又回到了 table！ （我什至不知道这是怎么发生的，我怀疑是 SonarQube 中的一些流氓代码）。
问题仍然可以从问题、措施和代码仪表板中浏览。如果您有自定义仪表板添加问题小部件，请像我们一样，
版主注意事项：有时候，愚蠢的 hack 可能就足够了。
我可以进行分析，sonar.search.javaAdditionalOpts=-Dbootstrap.system_call_filter=false
我还提供了大量的内存，
sonar.search.javaOpts=-Xmx2G -Xms1G -Xss1m -Djava.net.preferIPv4Stack=true \
-XX:+UseParNewGC -XX:+UseConcMarkSweepGC -XX:CMSInitiatingOccupancyFraction=75 \
-XX:+UseCMSInitiatingOccupancyOnly -XX:+HeapDumpOnOutOfMemoryError
系统 64 位，16 GM RAM 和 2 核；运行 RHEL 6.7
并且 ES 出现故障并显示以下日志。
2018.03.01 14:32:34 WARN es[o.e.indices.breaker] [sonar-1519909671773] [FIELDDATA] New used memory 1285367970 [1.1gb] from field [key] would be larger than configured breaker: 1278030643 [1.1gb], breaking
2018.03.01 15:09:43 WARN es[o.e.indices.breaker] [sonar-1519909671773] [FIELDDATA] New used memory 1285374084 [1.1gb] from field [key] would be larger than configured breaker: 1278030643 [1.1gb], breaking
2018.03.01 15:10:04 WARN es[o.e.indices.breaker] [sonar-1519909671773] [FIELDDATA] New used memory 1285374084 [1.1gb] from field [key] would be larger than configured breaker: 1278030643 [1.1gb], breaking
2018.03.01 15:20:15 WARN es[o.e.indices.breaker] [sonar-1519909671773] [FIELDDATA] New used memory 1285379720 [1.1gb] from field [key] would be larger than configured breaker: 1278030643 [1.1gb], breaking
2018.03.01 15:20:47 WARN es[o.e.indices.breaker] [sonar-1519909671773] [FIELDDATA] New used memory 1285379720 [1.1gb] from field [key] would be larger than configured breaker: 1278030643 [1.1gb], breaking
2018.03.01 15:21:30 WARN es[o.e.indices.breaker] [sonar-1519909671773] [FIELDDATA] New used memory 1285379720 [1.1gb] from field [key] would be larger than configured breaker: 1278030643 [1.1gb], breaking
2018.03.01 15:21:39 WARN es[o.e.indices.breaker] [sonar-1519909671773] [FIELDDATA] New used memory 1287374978 [1.1gb] from field [key] would be larger than configured breaker: 1278030643 [1.1gb], breaking
2018.03.01 15:22:02 WARN es[o.e.indices.breaker] [sonar-1519909671773] [FIELDDATA] New used memory 1287374978 [1.1gb] from field [key] would be larger than configured breaker: 1278030643 [1.1gb], breaking
2018.03.01 15:22:12 WARN es[o.e.indices.breaker] [sonar-1519909671773] [FIELDDATA] New used memory 1279785543 [1.1gb] from field [issueCreatedAt] would be larger than configured breaker: 1278030643 [1.1gb], breaking
2018.03.01 15:22:12 WARN es[o.e.indices.breaker] [sonar-1519909671773] [FIELDDATA] New used memory 1279798446 [1.1gb] from field [issueCreatedAt] would be larger than configured breaker: 1278030643 [1.1gb], breaking
2018.03.01 15:22:12 WARN es[o.e.indices.breaker] [sonar-1519909671773] [FIELDDATA] New used memory 1279787446 [1.1gb] from field [issueCreatedAt] would be larger than configured breaker: 1278030643 [1.1gb], breaking
2018.03.01 15:22:12 WARN es[o.e.indices.breaker] [sonar-1519909671773] [FIELDDATA] New used memory 1279771440 [1.1gb] from field [issueCreatedAt] would be larger than configured breaker: 1278030643 [1.1gb], breaking
2018.03.01 15:22:12 WARN es[o.e.indices.breaker] [sonar-1519909671773] [FIELDDATA] New used memory 1279800560 [1.1gb] from field [issueCreatedAt] would be larger than configured breaker: 1278030643 [1.1gb], breaking
2018.03.01 15:22:13 ERROR web[o.s.s.w.WebServiceEngine] Fail to process request http://172.17.33.86:9091/sonar/api/issues/search?p=1&ps=50&s=FILE_LINE&asc=true&additionalFields=_all&facets=types%2Cresolutions%2CcreatedAt&resolved=false&types=VULNERABILITY&sinceLeakPeriod=true&componentUuids=AWHS0koMv8wI-iczVKXw
java.lang.IllegalStateException: Fail to execute ES search request '{"from":0,"size":50,"query":{"filtered":{"query":{"match_all":{}},"filter":{"bool":{"must":[{"terms":{"modulePath":["AWHS0koMv8wI-iczVKXw"]}},{"missing":{"field":"resolution"}},{"terms":{"type":["VULNERABILITY"]}},{"range":{"issueCreatedAt":{"from":"2018-02-19T19:14:08.456Z","to":null,"include_lower":true,"include_upper":true},"_cache":false}},{"has_parent":{"query":{"filtered":{"query":{"match_all":{}},"filter":{"bool":{"must":{"or":{"filters":[{"term":{"users":"admin"}},{"term":{"groups":"Anyone"}},{"term":{"groups":"sonar-users"}},{"term":{"groups":"sonar-administrators"}}]}},"_cache":true}}}},"parent_type":"authorization"}}]}}}},"sort":[{"project":{"order":"asc","missing":"_first"}},{"filePath":{"order":"asc","missing":"_first"}},{"line":{"order":"asc","missing":"_first"}},{"severityValue":{"order":"desc","missing":"_first"}},{"key":{"order":"asc","missing":"_first"}}],"aggregations":{"types":{"global":{},"aggregations":{"types_filter":{"filter":{"bool":{"must":[{"query":{"match_all":{}}},{"terms":{"modulePath":["AWHS0koMv8wI-iczVKXw"]}},{"missing":{"field":"resolution"}},{"range":{"issueCreatedAt":{"from":"2018-02-19T19:14:08.456Z","to":null,"include_lower":true,"include_upper":true},"_cache":false}},{"has_parent":{"query":{"filtered":{"query":{"match_all":{}},"filter":{"bool":{"must":{"or":{"filters":[{"term":{"users":"admin"}},{"term":{"groups":"Anyone"}},{"term":{"groups":"sonar-users"}},{"term":{"groups":"sonar-administrators"}}]}},"_cache":true}}}},"parent_type":"authorization"}}]}},"aggregations":{"types":{"terms":{"field":"type","size":10,"min_doc_count":1,"order":{"_count":"desc"}}},"types_selected":{"terms":{"field":"type","include":"VULNERABILITY"}}}}}},"resolutions":{"global":{},"aggregations":{"resolutions_filter":{"filter":{"bool":{"must":[{"query":{"match_all":{}}},{"terms":{"modulePath":["AWHS0koMv8wI-iczVKXw"]}},{"terms":{"type":["VULNERABILITY"]}},{"range":{"issueCreatedAt":{"from":"2018-02-19T19:14:08.456Z","to":null,"include_lower":true,"include_upper":true},"_cache":false}},{"has_parent":{"query":{"filtered":{"query":{"match_all":{}},"filter":{"bool":{"must":{"or":{"filters":[{"term":{"users":"admin"}},{"term":{"groups":"Anyone"}},{"term":{"groups":"sonar-users"}},{"term":{"groups":"sonar-administrators"}}]}},"_cache":true}}}},"parent_type":"authorization"}}]}},"aggregations":{"resolutions":{"terms":{"field":"resolution","size":15,"min_doc_count":1,"order":{"_count":"desc"}}},"resolutions_missing":{"missing":{"field":"resolution"}}}}}},"createdAt":{"date_histogram":{"field":"issueCreatedAt","interval":"1d","min_doc_count":0,"pre_zone":"GMT","offset":"-3600s","format":"yyyy-MM-dd'T'HH:mm:ssZ","extended_bounds":{"min":1519067648456,"max":1519914131788}}}}}' on indices '[issues]' on types '[issue]'
at org.sonar.server.es.request.ProxySearchRequestBuilder.get(ProxySearchRequestBuilder.java:47) ~[sonar-server-5.6.7.jar:na]
at org.sonar.server.es.request.ProxySearchRequestBuilder.get(ProxySearchRequestBuilder.java:35) ~[sonar-server-5.6.7.jar:na]
at org.sonar.server.issue.index.IssueIndex.search(IssueIndex.java:235) ~[sonar-server-5.6.7.jar:na]
at org.sonar.server.issue.ws.SearchAction.doHandle(SearchAction.java:301) ~[sonar-server-5.6.7.jar:na]
at org.sonar.server.issue.ws.SearchAction.handle(SearchAction.java:288) ~[sonar-server-5.6.7.jar:na]
at org.sonar.server.ws.WebServiceEngine.execute(WebServiceEngine.java:107) ~[sonar-server-5.6.7.jar:na]
at sun.reflect.GeneratedMethodAccessor228.invoke(Unknown Source) ~[na:na]
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[na:1.8.0_91]
at java.lang.reflect.Method.invoke(Method.java:498) ~[na:1.8.0_91]
at org.jruby.javasupport.JavaMethod.invokeDirectWithExceptionHandling(JavaMethod.java:425) [jruby-complete-1.7.9.jar:na]
at org.jruby.javasupport.JavaMethod.invokeDirect(JavaMethod.java:292) [jruby-complete-1.7.9.jar:na]
at org.jruby.java.invokers.InstanceMethodInvoker.call(InstanceMethodInvoker.java:44) [jruby-complete-1.7.9.jar:na]
是的，它确实在崩溃前请求了一段时间，并拒绝在 UI 上显示任何问题。请帮助，所有准备升级到 SQ 5.6.7 以便稍后我们可以升级到 SQ 6.7.1 但现在我们仍然是 运行 SQ 4.5.4 在生产中（它能够处理如此大量的数据)
How large is our DB? Our database has close to 27 million LOC, with 5.6 million closed/open issues on Database.
Proper Solution
- 升级到 SonarQube 6.1 LTS
- 如果您是 Oracle DB，请确保分析了 tables。即，继续索引 up-to-date
- 如果您没有升级空间并且索引是 up-to-date 但您遇到缓慢，请为数据库清理器设置最少周数，甚至比 SonarQube 和 运行 分析推荐的还要少.
当 SonarQube 删除旧快照时，所有相关的旧问题都将被正确删除，只有当新分析是 运行 时，才会在组件上执行 db cleaner。如果您有幽灵项目（不再在 SCM 中）创建空项目并使用相同的密钥清除数据库。
Stupid Hack
长期以来，我尝试按照问题 SONAR-8067 中的建议更改 SonarQube 的代码（在 Ann 的评论中链接）。但我没能做到，描述太高级了，我几乎没有时间疯狂地逆向工程，所以我选择了暴力破解。
特别提及：我相信我们已经对已关闭的问题制定了相当严厉的政策，但他们仍然顽固不化。
多亏了扫描缩小版的 JS，我们的数据库每天都在呈指数级增长，现在它有 680 万行问题table，无奈我执行了以下命令。
delete from issues where status = 'CLOSED' or status = 'RESOLVED'
这个 sql 语句用了 14 分钟，从 table 中清除了 450 万行。后来从 SonarQube 主目录中删除 data、temp 目录并重新启动 SonarQube，现在 ES 正在呼吸。 但故事还没有结束。现在仪表板是 unstable，680 万行又回到了 table！ （我什至不知道这是怎么发生的，我怀疑是 SonarQube 中的一些流氓代码）。
问题仍然可以从问题、措施和代码仪表板中浏览。如果您有自定义仪表板添加问题小部件，请像我们一样，
版主注意事项：有时候，愚蠢的 hack 可能就足够了。