Query Failed [Failed to execute main query] OutOfMemoryError: Java heap space
Query Failed [Failed to execute main query] OutOfMemoryError: Java heap space
嗨，我有弹性数据 80 M 数据有 52 Go，我用这个查询在文本中搜索
$json_doc = '{
"query": {
"query_string": {
"query": "*' . $sujet . '*",
"lenient": true
}
},
"size": 10000000,
"from": 0,
"fields": [
"association_key","text"
]
}';
当我搜索一个词时，它看起来小于 20K，我有我的数据，但当他超过 20K 时，我有这个错误
[2015-11-14 20:03:04,663][DEBUG][action.search.type ] [Pixx] [myindex3][2], node[x9gOi0WuQviXeE0eM6MsAg], [P], s[STARTED]: Failed to execute [org.elasticsearch.action.search.SearchRequest@3989d22b] lastShard [true]
org.elasticsearch.search.query.QueryPhaseExecutionException: [myindex3][2]: query[filtered(_all:*startup _all:maroc*)->cache(_type:tweecoms3)],from[0],size[10000000]: Query Failed [Failed to execute main query]
at org.elasticsearch.search.query.QueryPhase.execute(QueryPhase.java:163)
at org.elasticsearch.search.SearchService.loadOrExecuteQueryPhase(SearchService.java:286)
at org.elasticsearch.search.SearchService.executeQueryPhase(SearchService.java:297)
at org.elasticsearch.search.action.SearchServiceTransportAction.call(SearchServiceTransportAction.java:231)
at org.elasticsearch.search.action.SearchServiceTransportAction.call(SearchServiceTransportAction.java:228)
at org.elasticsearch.search.action.SearchServiceTransportAction.run(SearchServiceTransportAction.java:559)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.OutOfMemoryError: Java heap space
at org.apache.lucene.util.PriorityQueue.<init>(PriorityQueue.java:56)
at org.apache.lucene.search.HitQueue.<init>(HitQueue.java:64)
at org.apache.lucene.search.TopScoreDocCollector.<init>(TopScoreDocCollector.java:275)
at org.apache.lucene.search.TopScoreDocCollector.<init>(TopScoreDocCollector.java:37)
at org.apache.lucene.search.TopScoreDocCollector$OutOfOrderTopScoreDocCollector.<init>(TopScoreDocCollector.java:135)
at org.apache.lucene.search.TopScoreDocCollector$OutOfOrderTopScoreDocCollector.<init>(TopScoreDocCollector.java:133)
at org.apache.lucene.search.TopScoreDocCollector.create(TopScoreDocCollector.java:262)
at org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:490)
at org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:448)
at org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:281)
at org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:269)
at org.elasticsearch.search.query.QueryPhase.execute(QueryPhase.java:157)
... 8 more
[2015-11-14 20:03:11,764][DEBUG][action.search.type ] [Pixx] [myindex3][0], node[x9gOi0WuQviXeE0eM6MsAg], [P], s[STARTED]: Failed to execute [org.elasticsearch.action.search.SearchRequest@3989d22b] lastShard [true]
org.elasticsearch.search.query.QueryPhaseExecutionException: [myindex3][0]: query[filtered(_all:*startup _all:maroc*)->cache(_type:tweecoms3)],from[0],size[10000000]: Query Failed [Failed to execute main query]
at org.elasticsearch.search.query.QueryPhase.execute(QueryPhase.java:163)
at org.elasticsearch.search.SearchService.loadOrExecuteQueryPhase(SearchService.java:286)
at org.elasticsearch.search.SearchService.executeQueryPhase(SearchService.java:297)
at org.elasticsearch.search.action.SearchServiceTransportAction.call(SearchServiceTransportAction.java:231)
at org.elasticsearch.search.action.SearchServiceTransportAction.call(SearchServiceTransportAction.java:228)
at org.elasticsearch.search.action.SearchServiceTransportAction.run(SearchServiceTransportAction.java:559)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.OutOfMemoryError: Java heap space
at org.apache.lucene.util.FixedBitSet.<init>(FixedBitSet.java:187)
at org.elasticsearch.common.lucene.docset.DocIdSets.toCacheable(DocIdSets.java:81)
at org.elasticsearch.index.cache.filter.weighted.WeightedFilterCache$FilterCacheFilterWrapper.getDocIdSet(WeightedFilterCache.java:177)
at org.elasticsearch.common.lucene.search.ApplyAcceptedDocsFilter.getDocIdSet(ApplyAcceptedDocsFilter.java:46)
at org.apache.lucene.search.FilteredQuery.bulkScorer(FilteredQuery.java:144)
at org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:618)
at org.elasticsearch.search.internal.ContextIndexSearcher.search(ContextIndexSearcher.java:191)
at org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:491)
at org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:448)
at org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:281)
at org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:269)
at org.elasticsearch.search.query.QueryPhase.execute(QueryPhase.java:157)
... 8 more
我如何解决这个问题我有专用服务器仅包含 Intel Xeon W3520 c 4/8 t + 2.66 GHz RAM 32 GB ECC 在 Debian 下
似乎您没有足够的 space 用于 Java 的堆。
由于您似乎拥有更多硬件，因此您可以尝试增加它：
Increase heap size in Java
更新：ES_HEAP_SIZE 不再是记录的选项
Elastic Search 不会在未配置的情况下使用所有内存。
您需要配置和调整 Elastic Search 和 Debian 的内存设置。
您需要配置其中一项：
- /etc/elasticsearch/jvm.options：包含 JVM 选项的文件（文件位于 config/jvm.options（在 TAR 或 ZIP 发行版中而不是 Debian 软件包）
- ES_JAVA_OPTS：一个环境变量，带有额外的 JVM 设置，包括内存和 GC 调整（如果需要）（几乎是 jvm.options 中的内容，但在一行中以空格分隔)
以下是 Elastic 截至 2020 年 2 月的最新文档的相关摘录：
Here are examples of how to set the heap size via the jvm.options
file:
-Xms2g ①
-Xmx2g ②
- ① Set the minimum heap size to 2g.
- ② Set the maximum heap size to 2g.
It is also possible to set the heap size via an environment variable.
This can be done by commenting out the Xms and Xmx settings in the
jvm.options file and setting these values via ES_JAVA_OPTS:
ES_JAVA_OPTS="-Xms2g -Xmx2g" ./bin/elasticsearch ①
ES_JAVA_OPTS="-Xms4000m -Xmx4000m" ./bin/elasticsearch ②
- ① Set
the minimum and maximum heap size to 2 GB.
- ② Set the minimum and maximum heap size to 4000 MB.
在此处查看更多信息：
嗨，我有弹性数据 80 M 数据有 52 Go，我用这个查询在文本中搜索
$json_doc = '{
"query": {
"query_string": {
"query": "*' . $sujet . '*",
"lenient": true
}
},
"size": 10000000,
"from": 0,
"fields": [
"association_key","text"
]
}';
当我搜索一个词时，它看起来小于 20K，我有我的数据，但当他超过 20K 时，我有这个错误
[2015-11-14 20:03:04,663][DEBUG][action.search.type ] [Pixx] [myindex3][2], node[x9gOi0WuQviXeE0eM6MsAg], [P], s[STARTED]: Failed to execute [org.elasticsearch.action.search.SearchRequest@3989d22b] lastShard [true]
org.elasticsearch.search.query.QueryPhaseExecutionException: [myindex3][2]: query[filtered(_all:*startup _all:maroc*)->cache(_type:tweecoms3)],from[0],size[10000000]: Query Failed [Failed to execute main query]
at org.elasticsearch.search.query.QueryPhase.execute(QueryPhase.java:163)
at org.elasticsearch.search.SearchService.loadOrExecuteQueryPhase(SearchService.java:286)
at org.elasticsearch.search.SearchService.executeQueryPhase(SearchService.java:297)
at org.elasticsearch.search.action.SearchServiceTransportAction.call(SearchServiceTransportAction.java:231)
at org.elasticsearch.search.action.SearchServiceTransportAction.call(SearchServiceTransportAction.java:228)
at org.elasticsearch.search.action.SearchServiceTransportAction.run(SearchServiceTransportAction.java:559)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.OutOfMemoryError: Java heap space
at org.apache.lucene.util.PriorityQueue.<init>(PriorityQueue.java:56)
at org.apache.lucene.search.HitQueue.<init>(HitQueue.java:64)
at org.apache.lucene.search.TopScoreDocCollector.<init>(TopScoreDocCollector.java:275)
at org.apache.lucene.search.TopScoreDocCollector.<init>(TopScoreDocCollector.java:37)
at org.apache.lucene.search.TopScoreDocCollector$OutOfOrderTopScoreDocCollector.<init>(TopScoreDocCollector.java:135)
at org.apache.lucene.search.TopScoreDocCollector$OutOfOrderTopScoreDocCollector.<init>(TopScoreDocCollector.java:133)
at org.apache.lucene.search.TopScoreDocCollector.create(TopScoreDocCollector.java:262)
at org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:490)
at org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:448)
at org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:281)
at org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:269)
at org.elasticsearch.search.query.QueryPhase.execute(QueryPhase.java:157)
... 8 more
[2015-11-14 20:03:11,764][DEBUG][action.search.type ] [Pixx] [myindex3][0], node[x9gOi0WuQviXeE0eM6MsAg], [P], s[STARTED]: Failed to execute [org.elasticsearch.action.search.SearchRequest@3989d22b] lastShard [true]
org.elasticsearch.search.query.QueryPhaseExecutionException: [myindex3][0]: query[filtered(_all:*startup _all:maroc*)->cache(_type:tweecoms3)],from[0],size[10000000]: Query Failed [Failed to execute main query]
at org.elasticsearch.search.query.QueryPhase.execute(QueryPhase.java:163)
at org.elasticsearch.search.SearchService.loadOrExecuteQueryPhase(SearchService.java:286)
at org.elasticsearch.search.SearchService.executeQueryPhase(SearchService.java:297)
at org.elasticsearch.search.action.SearchServiceTransportAction.call(SearchServiceTransportAction.java:231)
at org.elasticsearch.search.action.SearchServiceTransportAction.call(SearchServiceTransportAction.java:228)
at org.elasticsearch.search.action.SearchServiceTransportAction.run(SearchServiceTransportAction.java:559)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.OutOfMemoryError: Java heap space
at org.apache.lucene.util.FixedBitSet.<init>(FixedBitSet.java:187)
at org.elasticsearch.common.lucene.docset.DocIdSets.toCacheable(DocIdSets.java:81)
at org.elasticsearch.index.cache.filter.weighted.WeightedFilterCache$FilterCacheFilterWrapper.getDocIdSet(WeightedFilterCache.java:177)
at org.elasticsearch.common.lucene.search.ApplyAcceptedDocsFilter.getDocIdSet(ApplyAcceptedDocsFilter.java:46)
at org.apache.lucene.search.FilteredQuery.bulkScorer(FilteredQuery.java:144)
at org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:618)
at org.elasticsearch.search.internal.ContextIndexSearcher.search(ContextIndexSearcher.java:191)
at org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:491)
at org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:448)
at org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:281)
at org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:269)
at org.elasticsearch.search.query.QueryPhase.execute(QueryPhase.java:157)
... 8 more
我如何解决这个问题我有专用服务器仅包含 Intel Xeon W3520 c 4/8 t + 2.66 GHz RAM 32 GB ECC 在 Debian 下
似乎您没有足够的 space 用于 Java 的堆。 由于您似乎拥有更多硬件，因此您可以尝试增加它： Increase heap size in Java
更新：ES_HEAP_SIZE 不再是记录的选项
Elastic Search 不会在未配置的情况下使用所有内存。
您需要配置和调整 Elastic Search 和 Debian 的内存设置。
您需要配置其中一项：
- /etc/elasticsearch/jvm.options：包含 JVM 选项的文件（文件位于 config/jvm.options（在 TAR 或 ZIP 发行版中而不是 Debian 软件包）
- ES_JAVA_OPTS：一个环境变量，带有额外的 JVM 设置，包括内存和 GC 调整（如果需要）（几乎是 jvm.options 中的内容，但在一行中以空格分隔)
以下是 Elastic 截至 2020 年 2 月的最新文档的相关摘录：
Here are examples of how to set the heap size via the jvm.options file:
-Xms2g ① -Xmx2g ②
- ① Set the minimum heap size to 2g.
- ② Set the maximum heap size to 2g.
It is also possible to set the heap size via an environment variable. This can be done by commenting out the Xms and Xmx settings in the jvm.options file and setting these values via ES_JAVA_OPTS:
ES_JAVA_OPTS="-Xms2g -Xmx2g" ./bin/elasticsearch ① ES_JAVA_OPTS="-Xms4000m -Xmx4000m" ./bin/elasticsearch ②
- ① Set the minimum and maximum heap size to 2 GB.
- ② Set the minimum and maximum heap size to 4000 MB.
在此处查看更多信息：