hazelcast 启动缓慢

Slow startup of hazelcast

我有一个 spring-boot 应用程序,我想使用 hazelcast 作为具有 spring-boot 缓存的缓存提供程序。

我在 hazelcast.yaml 文件中有以下配置:

hazelcast:
  cluster-name: message-handler-cluster
  network:
    join:
      auto-detection:
        enabled: false
      multicast:
        enabled: false
      tcp-ip:
        enabled: true
        member-list:
          - 127.0.0.1
  integrity-checker:
    enabled: false

当我使用 hazelcast-spring 5.1.1 maven 依赖项时,hazelcast 开始非常缓慢。这是启动日志:

2022-04-19 09:59:30 WARN (main) [] [StandardLoggerFactory$StandardLogger] [] [] Hazelcast is starting in a Java modular environment (Java 9 and newer) but without proper access to required Java packages. Use additional Java arguments to provide Hazelcast access to Java internal API. The internal API access is used to get the best performance results. Arguments to be used:
 --add-modules java.se --add-exports java.base/jdk.internal.ref=ALL-UNNAMED --add-opens java.base/java.lang=ALL-UNNAMED --add-opens java.base/sun.nio.ch=ALL-UNNAMED --add-opens java.management/sun.management=ALL-UNNAMED --add-opens jdk.management/com.sun.management.internal=ALL-UNNAMED
2022-04-19 09:59:30 INFO (main) [] [StandardLoggerFactory$StandardLogger] [] [] [LOCAL] [message-handler-cluster] [5.1.1] Interfaces is disabled, trying to pick one address from TCP-IP config addresses: [127.0.0.1]
2022-04-19 09:59:30 INFO (main) [] [StandardLoggerFactory$StandardLogger] [] [] [127.0.0.1]:5701 [message-handler-cluster] [5.1.1] 
    +       +  o    o     o     o---o o----o o      o---o     o     o----o o--o--o
    + +   + +  |    |    / \       /  |      |     /         / \    |         |   
    + + + + +  o----o   o   o     o   o----o |    o         o   o   o----o    |   
    + +   + +  |    |  /     \   /    |      |     \       /     \       |    |   
    +       +  o    o o       o o---o o----o o----o o---o o       o o----o    o   
2022-04-19 09:59:30 INFO (main) [] [StandardLoggerFactory$StandardLogger] [] [] [127.0.0.1]:5701 [message-handler-cluster] [5.1.1] Copyright (c) 2008-2022, Hazelcast, Inc. All Rights Reserved.
2022-04-19 09:59:30 INFO (main) [] [StandardLoggerFactory$StandardLogger] [] [] [127.0.0.1]:5701 [message-handler-cluster] [5.1.1] Hazelcast Platform 5.1.1 (20220317 - 5b5fa10) starting at [127.0.0.1]:5701
2022-04-19 09:59:30 INFO (main) [] [StandardLoggerFactory$StandardLogger] [] [] [127.0.0.1]:5701 [message-handler-cluster] [5.1.1] Cluster name: message-handler-cluster
2022-04-19 09:59:30 INFO (main) [] [StandardLoggerFactory$StandardLogger] [] [] [127.0.0.1]:5701 [message-handler-cluster] [5.1.1] Integrity Checker is disabled. Fail-fast on corrupted executables will not be performed.
To enable integrity checker do one of the following: 
  - Change member config using Java API: config.setIntegrityCheckerEnabled(true);
  - Change XML/YAML configuration property: Set hazelcast.integrity-checker.enabled to true
  - Add system property: -Dhz.integritychecker.enabled=true (for Hazelcast embedded, works only when loading config via Config.load)
  - Add environment variable: HZ_INTEGRITYCHECKER_ENABLED=true (recommended when running container image. For Hazelcast embedded, works only when loading config via Config.load)
2022-04-19 09:59:30 INFO (main) [] [StandardLoggerFactory$StandardLogger] [] [] [127.0.0.1]:5701 [message-handler-cluster] [5.1.1] The Jet engine is disabled.
To enable the Jet engine on the members, do one of the following:
  - Change member config using Java API: config.getJetConfig().setEnabled(true)
  - Change XML/YAML configuration property: Set hazelcast.jet.enabled to true
  - Add system property: -Dhz.jet.enabled=true (for Hazelcast embedded, works only when loading config via Config.load)
  - Add environment variable: HZ_JET_ENABLED=true (recommended when running container image. For Hazelcast embedded, works only when loading config via Config.load)
2022-04-19 10:00:22 INFO (main) [] [StandardLoggerFactory$StandardLogger] [] [] [127.0.0.1]:5701 [message-handler-cluster] [5.1.1] Enable DEBUG/FINE log level for log category com.hazelcast.system.security  or use -Dhazelcast.security.recommendations system property to see security recommendations and the status of current config.
2022-04-19 10:00:23 INFO (main) [] [StandardLoggerFactory$StandardLogger] [] [] [127.0.0.1]:5701 [message-handler-cluster] [5.1.1] Using TCP/IP discovery
2022-04-19 10:00:23 WARN (main) [] [StandardLoggerFactory$StandardLogger] [] [] [127.0.0.1]:5701 [message-handler-cluster] [5.1.1] CP Subsystem is not enabled. CP data structures will operate in UNSAFE mode! Please note that UNSAFE mode will not provide strong consistency guarantees.
2022-04-19 10:00:23 INFO (main) [] [StandardLoggerFactory$StandardLogger] [] [] [127.0.0.1]:5701 [message-handler-cluster] [5.1.1] Diagnostics disabled. To enable add -Dhazelcast.diagnostics.enabled=true to the JVM arguments.
2022-04-19 10:00:23 INFO (main) [] [StandardLoggerFactory$StandardLogger] [] [] [127.0.0.1]:5701 [message-handler-cluster] [5.1.1] [127.0.0.1]:5701 is STARTING
WARNING: An illegal reflective access operation has occurred
WARNING: Illegal reflective access by com.hazelcast.internal.networking.nio.SelectorOptimizer (file:/C:/Users/ZC15PL/.m2/repository/com/hazelcast/hazelcast/5.1.1/hazelcast-5.1.1.jar) to field sun.nio.ch.SelectorImpl.selectedKeys
WARNING: Please consider reporting this to the maintainers of com.hazelcast.internal.networking.nio.SelectorOptimizer
WARNING: Use --illegal-access=warn to enable warnings of further illegal reflective access operations
WARNING: All illegal access operations will be denied in a future release
2022-04-19 10:00:25 INFO (hz.upbeat_engelbart.cached.thread-2) [] [StandardLoggerFactory$StandardLogger] [] [] [127.0.0.1]:5701 [message-handler-cluster] [5.1.1] [127.0.0.1]:5703 is added to the blacklist.
2022-04-19 10:00:25 INFO (hz.upbeat_engelbart.cached.thread-1) [] [StandardLoggerFactory$StandardLogger] [] [] [127.0.0.1]:5701 [message-handler-cluster] [5.1.1] [127.0.0.1]:5702 is added to the blacklist.
2022-04-19 10:00:26 INFO (main) [] [StandardLoggerFactory$StandardLogger] [] [] [127.0.0.1]:5701 [message-handler-cluster] [5.1.1] 

Members {size:1, ver:1} [
    Member [127.0.0.1]:5701 - 01db03fc-2dc4-45a6-813c-80b843d1e1b4 this
]

2022-04-19 10:00:26 INFO (main) [] [StandardLoggerFactory$StandardLogger] [] [] [127.0.0.1]:5701 [message-handler-cluster] [5.1.1] [127.0.0.1]:5701 is STARTED

启动时间约为 30-40 秒。

当我使用 hazelcast-spring 4.2.4 maven 依赖项时,hazelcast 启动非常快。这是启动日志:

2022-04-19 10:04:41 WARN (main) [] [StandardLoggerFactory$StandardLogger] [] [] Hazelcast is starting in a Java modular environment (Java 9 and newer) but without proper access to required Java packages. Use additional Java arguments to provide Hazelcast access to Java internal API. The internal API access is used to get the best performance results. Arguments to be used:
 --add-modules java.se --add-exports java.base/jdk.internal.ref=ALL-UNNAMED --add-opens java.base/java.lang=ALL-UNNAMED --add-opens java.base/java.nio=ALL-UNNAMED --add-opens java.base/sun.nio.ch=ALL-UNNAMED --add-opens java.management/sun.management=ALL-UNNAMED --add-opens jdk.management/com.sun.management.internal=ALL-UNNAMED
2022-04-19 10:04:41 INFO (main) [] [StandardLoggerFactory$StandardLogger] [] [] [LOCAL] [message-handler-cluster] [4.2.4] Interfaces is disabled, trying to pick one address from TCP-IP config addresses: [127.0.0.1]
2022-04-19 10:04:41 INFO (main) [] [StandardLoggerFactory$StandardLogger] [] [] [127.0.0.1]:5701 [message-handler-cluster] [4.2.4] Hazelcast 4.2.4 (20211220 - 25f0049) starting at [127.0.0.1]:5701
2022-04-19 10:04:42 INFO (main) [] [StandardLoggerFactory$StandardLogger] [] [] [127.0.0.1]:5701 [message-handler-cluster] [4.2.4] Using TCP/IP discovery
2022-04-19 10:04:42 WARN (main) [] [StandardLoggerFactory$StandardLogger] [] [] [127.0.0.1]:5701 [message-handler-cluster] [4.2.4] CP Subsystem is not enabled. CP data structures will operate in UNSAFE mode! Please note that UNSAFE mode will not provide strong consistency guarantees.
2022-04-19 10:04:43 INFO (main) [] [StandardLoggerFactory$StandardLogger] [] [] [127.0.0.1]:5701 [message-handler-cluster] [4.2.4] Diagnostics disabled. To enable add -Dhazelcast.diagnostics.enabled=true to the JVM arguments.
2022-04-19 10:04:43 INFO (main) [] [StandardLoggerFactory$StandardLogger] [] [] [127.0.0.1]:5701 [message-handler-cluster] [4.2.4] [127.0.0.1]:5701 is STARTING
WARNING: An illegal reflective access operation has occurred
WARNING: Illegal reflective access by com.hazelcast.internal.networking.nio.SelectorOptimizer (file:/C:/Users/ZC15PL/.m2/repository/com/hazelcast/hazelcast/4.2.4/hazelcast-4.2.4.jar) to field sun.nio.ch.SelectorImpl.selectedKeys
WARNING: Please consider reporting this to the maintainers of com.hazelcast.internal.networking.nio.SelectorOptimizer
WARNING: Use --illegal-access=warn to enable warnings of further illegal reflective access operations
WARNING: All illegal access operations will be denied in a future release
2022-04-19 10:04:45 INFO (hz.vibrant_ganguly.cached.thread-3) [] [StandardLoggerFactory$StandardLogger] [] [] [127.0.0.1]:5701 [message-handler-cluster] [4.2.4] [127.0.0.1]:5703 is added to the blacklist.
2022-04-19 10:04:45 INFO (hz.vibrant_ganguly.cached.thread-2) [] [StandardLoggerFactory$StandardLogger] [] [] [127.0.0.1]:5701 [message-handler-cluster] [4.2.4] [127.0.0.1]:5702 is added to the blacklist.
2022-04-19 10:04:46 INFO (main) [] [StandardLoggerFactory$StandardLogger] [] [] [127.0.0.1]:5701 [message-handler-cluster] [4.2.4] 

Members {size:1, ver:1} [
    Member [127.0.0.1]:5701 - c810dae2-5697-4d3d-9c60-0a7ed89f66d7 this
]

2022-04-19 10:04:46 INFO (main) [] [StandardLoggerFactory$StandardLogger] [] [] [127.0.0.1]:5701 [message-handler-cluster] [4.2.4] [127.0.0.1]:5701 is STARTED

启动时间为 3-4 秒。

我应该如何配置 hazelcast 5.1.1 以实现上述目标,启动时间非常快,或者我在配置中缺少什么?

更新: 5.1.1 的更详细 (DEBUG) 日志(演示 github 项目):

2022-04-19 16:44:42.425 DEBUG 14620 --- [           main] c.h.s.i.o.impl.BackpressureRegulator     : [127.0.0.1]:5701 [demo-cluster] [5.1.1] Backpressure is disabled
2022-04-19 16:44:42.494 DEBUG 14620 --- [           main] h.s.i.o.i.InboundResponseHandlerSupplier : [127.0.0.1]:5701 [demo-cluster] [5.1.1] Running with 2 response threads
2022-04-19 16:45:35.617 DEBUG 14620 --- [           main] c.h.i.server.tcp.LocalAddressRegistry    : [127.0.0.1]:5701 [demo-cluster] [5.1.1] LinkedAddresses{primaryAddress=[127.0.0.1]:5701, allLinkedAddresses=[[fe80:0:0:0:5582:4f40:ed57:9ab0%eth3]:5701, [fe80:0:0:0:e154:4b5e:dd8b:23aa%wlan4]:5701, [172.31.64.1]:5701, [fe80:0:0:0:f161:66b7:46bd:3d9c%eth26]:5701, [172.25.112.1]:5701, [fe80:0:0:0:e0b7:e6fc:65d7:29c9]:5701, [fe80:0:0:0:f161:66b7:46bd:3d9c]:5701, [fe80:0:0:0:e154:4b5e:dd8b:23aa]:5701, [fe80:0:0:0:150:ba24:fd1c:f8d6]:5701, [192.168.96.1]:5701, [fe80:0:0:0:f1d8:d5be:d257:80f4]:5701, [fe80:0:0:0:7d64:92ab:6a06:9db2%eth8]:5701, [fe80:0:0:0:b45f:145f:a925:afd%eth15]:5701, [fe80:0:0:0:81c6:d272:c19d:e398%eth51]:5701, [fe80:0:0:0:5918:ecac:57d2:bbc1%net0]:5701, [fe80:0:0:0:4cd7:74e4:1e3c:e686]:5701, [127.0.0.1]:5701, [fe80:0:0:0:4cd7:74e4:1e3c:e686%eth23]:5701, [fe80:0:0:0:e0b7:e6fc:65d7:29c9%eth2]:5701, [fe80:0:0:0:81c6:d272:c19d:e398]:5701, [fe80:0:0:0:1c97:db6c:48a5:dee5]:5701, [fe80:0:0:0:1c97:db6c:48a5:dee5%eth10]:5701, [192.168.160.1]:5701, [fe80:0:0:0:5582:4f40:ed57:9ab0]:5701, [fe80:0:0:0:f1d8:d5be:d257:80f4%eth31]:5701, [fe80:0:0:0:150:ba24:fd1c:f8d6%eth7]:5701, [fe80:0:0:0:20ca:afcd:2c09:c89a]:5701, [fe80:0:0:0:f145:d1f6:dc9b:a5a8]:5701, [fe80:0:0:0:b45f:145f:a925:afd]:5701, [fe80:0:0:0:e927:f676:20d4:509d%wlan3]:5701, [172.16.128.213]:5701, [fe80:0:0:0:20ca:afcd:2c09:c89a%eth21]:5701, [fe80:0:0:0:f145:d1f6:dc9b:a5a8%wlan0]:5701, [10.83.179.202]:5701, [fe80:0:0:0:e927:f676:20d4:509d]:5701, [192.168.224.1]:5701, [172.31.112.1]:5701, [0:0:0:0:0:0:0:1]:5701, [fe80:0:0:0:7d64:92ab:6a06:9db2]:5701, [fe80:0:0:0:5918:ecac:57d2:bbc1]:5701, [172.27.16.1]:5701]} are registered for the local member with local uuid=e291a410-c08f-498d-9cc1-445bf3999ace
2022-04-19 16:45:35.885 DEBUG 14620 --- [           main] com.hazelcast.system.security            : [127.0.0.1]:5701 [demo-cluster] [5.1.1] 

干杯, Zsolt

我是在 5.1 中添加此更改的人。之前我没想到会花这么长时间。在 5.1 中,我们开始在注册表中注册服务器套接字的所有地址,以便 Hazelcast 成员知道这些地址是它自己的地址。参见:https://github.com/hazelcast/hazelcast/blob/bbcb69ae732a0717cbc5a28339c94dfd49a73493/hazelcast/src/main/java/com/hazelcast/internal/server/tcp/LocalAddressRegistry.java#L246-L261。这导致如果服务器将套接字绑定到任何本地地址,它会迭代所有网络接口并将这些地址记录在注册表中作为成员自己的地址,这在您的环境中似乎花费了太长时间。您能否尝试使用 -Dhazelcast.socket.bind.any=false 属性 来避免为成员的服务器套接字绑定任何接口,否则它会绑定 0.0.0.0 并且我们遍历所有接口。