使用 Apache Ignite 和 SQL 过滤时使用率高 CPU

Question

我正面临使用 SQL 的 Apache Ignite 内存缓存的高 CPU 消耗相关问题。 Where 子句包含 3 列与“=”匹配并且所有三列都被索引。它导致 Intel i7 上的所有 8 个内核都达到最大值。我附上了与导致 CPU 使用率高的线程相关的堆栈跟踪。在此堆栈跟踪中，CPU 的 64% 用于 TableFilter.next()，其余用于 Select.isConditionMet() - 此 table 中的行数约为 6000并且需要根据条件过滤并返回1行。 SqlFieldsQuery 就是为此目的而构造的。这种查找在应用程序中发生了数千次。这是 Ignite 的错误用法 SQL 还是有什么设置不正确？

尝试了以下选项：

keepBinary() 同时获取缓存
为所有列编制索引 - 所查找的三列中有 2 列具有高基数
尝试过惰性模式
尝试更改 cache.query(cacheQuery).getAll();到 cache.query(cacheQuery).iterator() 和一个 next()

以上更改并没有多大帮助。请帮忙，我是不是遗漏了什么？

添加复合索引后（从 H2 控制台，我遇到了以下异常）

javax.cache.CacheException: Failed to execute map query on remote node [nodeId=08fe1345-5b85-4a74-bb63-67cccf67b137, errMsg=Failed to parse SQL query: SELECT
__Z0.NORMALIZEDVALUE __C0_0
FROM "DM".DM __Z0
WHERE (__Z0.DN = 'countryOfTaxationMap') AND ((__Z0.VAL = 'KOR') AND ((__Z0.DS = 'trds') AND ((__Z0.RD = 'countryCodes') AND ((__Z0.STATUS = 'A')))))]
        at org.apache.ignite.internal.processors.query.h2.twostep.GridReduceQueryExecutor.fail(GridReduceQueryExecutor.java:290) ~[ignite-indexing-2.7.6.jar:2.7.6]
        at org.apache.ignite.internal.processors.query.h2.twostep.GridReduceQueryExecutor.onFail(GridReduceQueryExecutor.java:280) ~[ignite-indexing-2.7.6.jar:2.7.6]
        at org.apache.ignite.internal.processors.query.h2.twostep.GridReduceQueryExecutor.onMessage(GridReduceQueryExecutor.java:259) ~[ignite-indexing-2.7.6.jar:2.7.6]
        at org.apache.ignite.internal.processors.query.h2.twostep.GridMapQueryExecutor.sendError(GridMapQueryExecutor.java:1198) ~[ignite-indexing-2.7.6.jar:2.7.6]
        at org.apache.ignite.internal.processors.query.h2.twostep.GridMapQueryExecutor.onQueryRequest0(GridMapQueryExecutor.java:1051) ~[ignite-indexing-2.7.6.jar:2.7.6]
        at org.apache.ignite.internal.processors.query.h2.twostep.GridMapQueryExecutor.onQueryRequest(GridMapQueryExecutor.java:705) ~[ignite-indexing-2.7.6.jar:2.7.6]
        at org.apache.ignite.internal.processors.query.h2.twostep.GridMapQueryExecutor.onMessage(GridMapQueryExecutor.java:240) ~[ignite-indexing-2.7.6.jar:2.7.6]
        at org.apache.ignite.internal.processors.query.h2.twostep.GridReduceQueryExecutor.applyx(GridReduceQueryExecutor.java:170) ~[ignite-indexing-2.7.6.jar:2.7.6]
        at org.apache.ignite.internal.processors.query.h2.twostep.GridReduceQueryExecutor.applyx(GridReduceQueryExecutor.java:168) ~[ignite-indexing-2.7.6.jar:2.7.6]
        at org.apache.ignite.internal.util.lang.IgniteInClosure2X.apply(IgniteInClosure2X.java:38) ~[ignite-core-2.7.6.jar:2.7.6]
        at org.apache.ignite.internal.processors.query.h2.IgniteH2Indexing.send(IgniteH2Indexing.java:3405) ~[ignite-indexing-2.7.6.jar:2.7.6]
        at org.apache.ignite.internal.processors.query.h2.twostep.GridReduceQueryExecutor.send(GridReduceQueryExecutor.java:1642) ~[ignite-indexing-2.7.6.jar:2.7.6]
        at org.apache.ignite.internal.processors.query.h2.twostep.GridReduceQueryExecutor.query(GridReduceQueryExecutor.java:876) ~[ignite-indexing-2.7.6.jar:2.7.6]
        at org.apache.ignite.internal.processors.query.h2.IgniteH2Indexing.iterator(IgniteH2Indexing.java:1809) ~[ignite-indexing-2.7.6.jar:2.7.6]
        at org.apache.ignite.internal.processors.cache.QueryCursorImpl.iterator(QueryCursorImpl.java:95) ~[ignite-core-2.7.6.jar:2.7.6]
        at org.apache.ignite.internal.processors.cache.QueryCursorImpl.getAll(QueryCursorImpl.java:114) ~[ignite-core-2.7.6.jar:2.7.6]

Answer 1

您的 WHERE 中有三个条件，但仅在其中一个条件上使用了索引。考虑创建复合索引（请注意，Ignite 每次 table 每个查询只能使用一个索引）。

使用 Apache Ignite 和 SQL 过滤时使用率高 CPU

High CPU usage when filtering using Apache Ignite and SQL

java

sql

performance

cpu-usage

ignite