所有简单(循环)路径查询 returns 数据过多 -> 堆溢出
All simple (cyclic) paths query returns too much data -> heap overflow
我是 Neo4j 的新手,使用带有 Javascript 驱动程序的 Neo4j 桌面版。我的图有大约 30.000 个节点和 40.000 个边。最终的方法是获取图中的所有简单循环,但我在尝试执行以下密码查询时遇到堆溢出:
let res = await session.run('MATCH p=(n)-[*2..4]-(n) RETURN nodes(p)')
这当然是因为查询尝试 return 找到的所有简单循环,这对我的堆来说太过分了,不幸的是,增加 node.js 的内存不是一个选项。
那么有什么方法可以绕过这个问题并从图中获取所有简单循环吗?
这是我得到的堆溢出错误:
/Users/paulus/.nvm/versions/node/v15.10.0/bin/node /Users/paulus/routeplanner/RouteFinder.js
<--- Last few GCs --->
[39630:0x104e00000] 227813 ms: Scavenge (reduce) 2033.5 (2077.9) -> 2032.1 (2072.1) MB, 113.5 / 0.0 ms (average mu = 0.577, current mu = 0.321) allocation failure
[39630:0x104e00000] 228171 ms: Scavenge (reduce) 2036.5 (2072.6) -> 2035.5 (2075.6) MB, 28.6 / 0.0 ms (average mu = 0.577, current mu = 0.321) allocation failure
[39630:0x104e00000] 228558 ms: Scavenge (reduce) 2039.1 (2075.6) -> 2038.5 (2080.4) MB, 28.2 / 0.0 ms (average mu = 0.577, current mu = 0.321) allocation failure
<--- JS stacktrace --->
<--- Last few GCs --->
[39630:0x104e00000] 227813 ms: Scavenge (reduce) 2033.5 (2077.9) -> 2032.1 (2072.1) MB, 113.5 / 0.0 ms (average mu = 0.577, current mu = 0.321) allocation failure
[39630:0x104e00000] 228171 ms: Scavenge (reduce) 2036.5 (2072.6) -> 2035.5 (2075.6) MB, 28.6 / 0.0 ms (average mu = 0.577, current mu = 0.321) allocation failure
[39630:0x104e00000] 228558 ms: Scavenge (reduce) 2039.1 (2075.6) -> 2038.5 (2080.4) MB, 28.2 / 0.0 ms (average mu = 0.577, current mu = 0.321) allocation failure
<--- JS stacktrace --->
FATAL ERROR: MarkCompactCollector: young object promotion failed Allocation failed - JavaScript heap out of memory
FATAL ERROR: MarkCompactCollector: young object promotion failed Allocation failed - JavaScript heap out of memory
<--- Last few GCs --->
[39630:0x104e00000] 227813 ms: Scavenge (reduce) 2033.5 (2077.9) -> 2032.1 (2072.1) MB, 113.5 / 0.0 ms (average mu = 0.577, current mu = 0.321) allocation failure
[39630:0x104e00000] 228171 ms: Scavenge (reduce) 2036.5 (2072.6) -> 2035.5 (2075.6) MB, 28.6 / 0.0 ms (average mu = 0.577, current mu = 0.321) allocation failure
[39630:0x104e00000] 228558 ms: Scavenge (reduce) 2039.1 (2075.6) -> 2038.5 (2080.4) MB, 28.2 / 0.0 ms (average mu = 0.577, current mu = 0.321) allocation failure
<--- JS stacktrace --->
FATAL ERROR: MarkCompactCollector: young object promotion failed Allocation failed - JavaScript heap out of memory
Process finished with exit code 139 (interrupted by signal 11: SIGSEGV)
如果不了解图表的数据模型,很难给出准确的解决方案,但使用 skip/limit 获取更小批量的数据可能会有所帮助。
这将获得前 100 个模式:
MATCH p=(n)-[*2..4]-(n) RETURN nodes(p) SKIP 0 LIMIT 100
接下来的 100 个:
MATCH p=(n)-[*2..4]-(n) RETURN nodes(p) SKIP 100 LIMIT 100
我是 Neo4j 的新手,使用带有 Javascript 驱动程序的 Neo4j 桌面版。我的图有大约 30.000 个节点和 40.000 个边。最终的方法是获取图中的所有简单循环,但我在尝试执行以下密码查询时遇到堆溢出:
let res = await session.run('MATCH p=(n)-[*2..4]-(n) RETURN nodes(p)')
这当然是因为查询尝试 return 找到的所有简单循环,这对我的堆来说太过分了,不幸的是,增加 node.js 的内存不是一个选项。
那么有什么方法可以绕过这个问题并从图中获取所有简单循环吗?
这是我得到的堆溢出错误:
/Users/paulus/.nvm/versions/node/v15.10.0/bin/node /Users/paulus/routeplanner/RouteFinder.js
<--- Last few GCs --->
[39630:0x104e00000] 227813 ms: Scavenge (reduce) 2033.5 (2077.9) -> 2032.1 (2072.1) MB, 113.5 / 0.0 ms (average mu = 0.577, current mu = 0.321) allocation failure
[39630:0x104e00000] 228171 ms: Scavenge (reduce) 2036.5 (2072.6) -> 2035.5 (2075.6) MB, 28.6 / 0.0 ms (average mu = 0.577, current mu = 0.321) allocation failure
[39630:0x104e00000] 228558 ms: Scavenge (reduce) 2039.1 (2075.6) -> 2038.5 (2080.4) MB, 28.2 / 0.0 ms (average mu = 0.577, current mu = 0.321) allocation failure
<--- JS stacktrace --->
<--- Last few GCs --->
[39630:0x104e00000] 227813 ms: Scavenge (reduce) 2033.5 (2077.9) -> 2032.1 (2072.1) MB, 113.5 / 0.0 ms (average mu = 0.577, current mu = 0.321) allocation failure
[39630:0x104e00000] 228171 ms: Scavenge (reduce) 2036.5 (2072.6) -> 2035.5 (2075.6) MB, 28.6 / 0.0 ms (average mu = 0.577, current mu = 0.321) allocation failure
[39630:0x104e00000] 228558 ms: Scavenge (reduce) 2039.1 (2075.6) -> 2038.5 (2080.4) MB, 28.2 / 0.0 ms (average mu = 0.577, current mu = 0.321) allocation failure
<--- JS stacktrace --->
FATAL ERROR: MarkCompactCollector: young object promotion failed Allocation failed - JavaScript heap out of memory
FATAL ERROR: MarkCompactCollector: young object promotion failed Allocation failed - JavaScript heap out of memory
<--- Last few GCs --->
[39630:0x104e00000] 227813 ms: Scavenge (reduce) 2033.5 (2077.9) -> 2032.1 (2072.1) MB, 113.5 / 0.0 ms (average mu = 0.577, current mu = 0.321) allocation failure
[39630:0x104e00000] 228171 ms: Scavenge (reduce) 2036.5 (2072.6) -> 2035.5 (2075.6) MB, 28.6 / 0.0 ms (average mu = 0.577, current mu = 0.321) allocation failure
[39630:0x104e00000] 228558 ms: Scavenge (reduce) 2039.1 (2075.6) -> 2038.5 (2080.4) MB, 28.2 / 0.0 ms (average mu = 0.577, current mu = 0.321) allocation failure
<--- JS stacktrace --->
FATAL ERROR: MarkCompactCollector: young object promotion failed Allocation failed - JavaScript heap out of memory
Process finished with exit code 139 (interrupted by signal 11: SIGSEGV)
如果不了解图表的数据模型,很难给出准确的解决方案,但使用 skip/limit 获取更小批量的数据可能会有所帮助。
这将获得前 100 个模式:
MATCH p=(n)-[*2..4]-(n) RETURN nodes(p) SKIP 0 LIMIT 100
接下来的 100 个:
MATCH p=(n)-[*2..4]-(n) RETURN nodes(p) SKIP 100 LIMIT 100