Hazelcast错误节点加入失败原因:加入节点的版本3.12.7与集群版本3.9不兼容

Hazelcast error Reason of failure for node join: Joining node's version 3.12.7 is not compatible with cluster version 3.9

我在 Spring Boot 微服务 JHipster 项目中遇到了这个问题,这个问题是在没有更改任何代码的情况下开始的。微服务器部署在 Kubernetes Microsoft Azure 环境中。该问题与 Hazelcast 问题有关。

堆栈跟踪:

2021-08-05T15:36:00.921185708Z 2021-08-05 15:36:00.920 ERROR 7 --- [ration.thread-0] com.hazelcast.security                   : [10.32.0.18]:5701 [dev] [3.12.7] Node could not join cluster. Before join check failed node is going to shutdown now!
2021-08-05T15:36:00.92269528Z 2021-08-05 15:36:00.922 ERROR 7 --- [ration.thread-0] com.hazelcast.security                   : [10.32.0.18]:5701 [dev] [3.12.7] Reason of failure for node join: Joining node's version 3.12.7 is not compatible with cluster version 3.9 (Rolling Member Upgrades are only supported in Hazelcast Enterprise)
2021-08-05T15:36:00.923792359Z 2021-08-05 15:36:00.923  WARN 7 --- [ration.thread-0] com.hazelcast.instance.Node              : [10.32.0.18]:5701 [dev] [3.12.7] Terminating forcefully...
2021-08-05T15:36:00.924224151Z 2021-08-05 15:36:00.924  INFO 7 --- [ration.thread-0] com.hazelcast.instance.Node              : [10.32.0.18]:5701 [dev] [3.12.7] Shutting down connection manager...
2021-08-05T15:36:00.929472551Z 2021-08-05 15:36:00.929  INFO 7 --- [ration.thread-0] com.hazelcast.nio.tcp.TcpIpConnection    : [10.32.0.18]:5701 [dev] [3.12.7] Connection[id=1, /10.32.0.18:49367->/10.40.0.24:5701, qualifier=null, endpoint=[10.40.0.24]:5701, alive=false, type=MEMBER] closed. Reason: EndpointManager is stopping
2021-08-05T15:36:00.937643697Z 2021-08-05 15:36:00.937  INFO 7 --- [ration.thread-0] com.hazelcast.instance.Node              : [10.32.0.18]:5701 [dev] [3.12.7] Shutting down node engine...
2021-08-05T15:36:01.002501471Z 2021-08-05 15:36:01.002  INFO 7 --- [ration.thread-0] com.hazelcast.instance.NodeExtension     : [10.32.0.18]:5701 [dev] [3.12.7] Destroying node NodeExtension.
2021-08-05T15:36:01.002715567Z 2021-08-05 15:36:01.002  INFO 7 --- [ration.thread-0] com.hazelcast.instance.Node              : [10.32.0.18]:5701 [dev] [3.12.7] Hazelcast Shutdown is completed in 79 ms.
2021-08-05T15:36:04.930340712Z 2021-08-05 15:36:04.926 ERROR 7 --- [           main] com.hazelcast.instance.Node              : [10.32.0.18]:5701 [dev] [3.12.7] Could not join cluster. Shutting down now!
2021-08-05T15:36:04.930373712Z 2021-08-05 15:36:04.927  INFO 7 --- [           main] com.hazelcast.core.LifecycleService      : [10.32.0.18]:5701 [dev] [3.12.7] [10.32.0.18]:5701 is SHUTTING_DOWN
2021-08-05T15:36:04.932312175Z 2021-08-05 15:36:04.931  INFO 7 --- [           main] com.hazelcast.instance.Node              : [10.32.0.18]:5701 [dev] [3.12.7] Node is already shutting down... Waiting for shutdown process to complete...
2021-08-05T15:36:04.932877065Z 2021-08-05 15:36:04.932  INFO 7 --- [           main] com.hazelcast.core.LifecycleService      : [10.32.0.18]:5701 [dev] [3.12.7] [10.32.0.18]:5701 is SHUTDOWN
2021-08-05T15:36:05.038848261Z 2021-08-05 15:36:05.038  WARN 7 --- [           main] com.hazelcast.util.PhoneHome             : [10.32.0.18]:5701 [dev] [3.12.7] Could not schedule phone home task! Most probably Hazelcast failed to start.
2021-08-05T15:36:05.04047403Z 2021-08-05 15:36:05.039  INFO 7 --- [           main] com.hazelcast.instance.Node              : [10.32.0.18]:5701 [dev] [3.12.7] Node is already shutting down... Waiting for shutdown process to complete...
2021-08-05T15:36:05.042475292Z 2021-08-05 15:36:05.042  WARN 7 --- [           main] ConfigServletWebServerApplicationContext : Exception encountered during context initialization - cancelling refresh attempt: org.springframework.context.ApplicationContextException: Unable to start web server; nested exception is java.lang.RuntimeException: org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'servletEndpointRegistrar' defined in class path resource [org/springframework/boot/actuate/autoconfigure/endpoint/web/ServletEndpointManagementContextConfiguration$WebMvcServletEndpointManagementContextConfiguration.class]: Bean instantiation via factory method failed; nested exception is org.springframework.beans.BeanInstantiationException: Failed to instantiate [org.springframework.boot.actuate.endpoint.web.ServletEndpointRegistrar]: Factory method 'servletEndpointRegistrar' threw exception; nested exception is org.springframework.beans.factory.UnsatisfiedDependencyException: Error creating bean with name 'healthEndpoint' defined in class path resource [org/springframework/boot/actuate/autoconfigure/health/HealthEndpointConfiguration.class]: Unsatisfied dependency expressed through method 'healthEndpoint' parameter 0; nested exception is org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'healthContributorRegistry' defined in class path resource [org/springframework/boot/actuate/autoconfigure/health/HealthEndpointConfiguration.class]: Bean instantiation via factory method failed; nested exception is org.springframework.beans.BeanInstantiationException: Failed to instantiate [org.springframework.boot.actuate.health.HealthContributorRegistry]: Factory method 'healthContributorRegistry' threw exception; nested exception is org.springframework.beans.factory.UnsatisfiedDependencyException: Error creating bean with name 'hazelcastHealthContributor' defined in class path resource [org/springframework/boot/actuate/autoconfigure/hazelcast/HazelcastHealthContributorAutoConfiguration.class]: Unsatisfied dependency expressed through method 'hazelcastHealthContributor' parameter 0; nested exception is org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'hazelcastInstance' defined in class path resource [.../.../.../.../config/CacheConfiguration.class]: Bean instantiation via factory method failed; nested exception is org.springframework.beans.BeanInstantiationException: Failed to instantiate [com.hazelcast.core.HazelcastInstance]: Factory method 'hazelcastInstance' threw exception; nested exception is java.lang.IllegalStateException: Node failed to start!
2021-08-05T15:36:05.042724688Z 2021-08-05 15:36:05.042  INFO 7 --- [           main] x.x.x.x.config.CacheConfiguration      : Closing Cache Manager
2021-08-05T15:36:05.254458785Z 2021-08-05 15:36:05.253 ERROR 7 --- [           main] o.s.boot.SpringApplication               : Application run failed
2021-08-05T15:36:05.254488484Z 
2021-08-05T15:36:05.254494684Z org.springframework.context.ApplicationContextException: Unable to start web server; nested exception is java.lang.RuntimeException: org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'servletEndpointRegistrar' defined in class path resource [org/springframework/boot/actuate/autoconfigure/endpoint/web/ServletEndpointManagementContextConfiguration$WebMvcServletEndpointManagementContextConfiguration.class]: Bean instantiation via factory method failed; nested exception is org.springframework.beans.BeanInstantiationException: Failed to instantiate [org.springframework.boot.actuate.endpoint.web.ServletEndpointRegistrar]: Factory method 'servletEndpointRegistrar' threw exception; nested exception is org.springframework.beans.factory.UnsatisfiedDependencyException: Error creating bean with name 'healthEndpoint' defined in class path resource [org/springframework/boot/actuate/autoconfigure/health/HealthEndpointConfiguration.class]: Unsatisfied dependency expressed through method 'healthEndpoint' parameter 0; nested exception is org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'healthContributorRegistry' defined in class path resource [org/springframework/boot/actuate/autoconfigure/health/HealthEndpointConfiguration.class]: Bean instantiation via factory method failed; nested exception is org.springframework.beans.BeanInstantiationException: Failed to instantiate [org.springframework.boot.actuate.health.HealthContributorRegistry]: Factory method 'healthContributorRegistry' threw exception; nested exception is org.springframework.beans.factory.UnsatisfiedDependencyException: Error creating bean with name 'hazelcastHealthContributor' defined in class path resource [org/springframework/boot/actuate/autoconfigure/hazelcast/HazelcastHealthContributorAutoConfiguration.class]: Unsatisfied dependency expressed through method 'hazelcastHealthContributor' parameter 0; nested exception is org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'hazelcastInstance' defined in class path resource [.../.../.../.../config/CacheConfiguration.class]: Bean instantiation via factory method failed; nested exception is org.springframework.beans.BeanInstantiationException: Failed to instantiate [com.hazelcast.core.HazelcastInstance]: Factory method 'hazelcastInstance' threw exception; nested exception is java.lang.IllegalStateException: Node failed to start!
2021-08-05T15:36:05.254519084Z  at org.springframework.boot.web.servlet.context.ServletWebServerApplicationContext.onRefresh(ServletWebServerApplicationContext.java:156)
2021-08-05T15:36:05.254522883Z  at org.springframework.context.support.AbstractApplicationContext.refresh(AbstractApplicationContext.java:544)
2021-08-05T15:36:05.254526583Z  at org.springframework.boot.web.servlet.context.ServletWebServerApplicationContext.refresh(ServletWebServerApplicationContext.java:141)
2021-08-05T15:36:05.254530283Z  at org.springframework.boot.SpringApplication.refresh(SpringApplication.java:747)
2021-08-05T15:36:05.254534083Z  at org.springframework.boot.SpringApplication.refreshContext(SpringApplication.java:397)
2021-08-05T15:36:05.254537583Z  at org.springframework.boot.SpringApplication.run(SpringApplication.java:315)

缓存配置class

import io.github.jhipster.config.JHipsterProperties;

import com.hazelcast.config.*;
import com.hazelcast.core.HazelcastInstance;
import com.hazelcast.core.Hazelcast;

import org.slf4j.Logger;
import org.slf4j.LoggerFactory;

import org.springframework.boot.autoconfigure.web.ServerProperties;

import org.springframework.cache.CacheManager;

import org.springframework.boot.info.BuildProperties;
import org.springframework.boot.info.GitProperties;
import org.springframework.cache.interceptor.KeyGenerator;
import org.springframework.beans.factory.annotation.Autowired;
import io.github.jhipster.config.cache.PrefixedKeyGenerator;
import org.springframework.cache.annotation.EnableCaching;
import org.springframework.cloud.client.ServiceInstance;
import org.springframework.cloud.client.discovery.DiscoveryClient;
import org.springframework.cloud.client.serviceregistry.Registration;
import org.springframework.context.annotation.*;
import org.springframework.core.env.Environment;
import org.springframework.core.env.Profiles;

import javax.annotation.PreDestroy;

@Configuration
@EnableCaching
public class CacheConfiguration {
    private GitProperties gitProperties;
    private BuildProperties buildProperties;

    private final Logger log = LoggerFactory.getLogger(CacheConfiguration.class);

    private final Environment env;

    private final ServerProperties serverProperties;

    private final DiscoveryClient discoveryClient;

    private Registration registration;

    public CacheConfiguration(Environment env, ServerProperties serverProperties, DiscoveryClient discoveryClient) {
        this.env = env;
        this.serverProperties = serverProperties;
        this.discoveryClient = discoveryClient;
    }

    @Autowired(required = false)
    public void setRegistration(Registration registration) {
        this.registration = registration;
    }

    @PreDestroy
    public void destroy() {
        log.info("Closing Cache Manager");
        Hazelcast.shutdownAll();
    }

    @Bean
    public CacheManager cacheManager(HazelcastInstance hazelcastInstance) {
        log.debug("Starting HazelcastCacheManager");
        return new com.hazelcast.spring.cache.HazelcastCacheManager(hazelcastInstance);
    }

    @Bean
    public HazelcastInstance hazelcastInstance(JHipsterProperties jHipsterProperties) {
        log.debug("Configuring Hazelcast");
        HazelcastInstance hazelCastInstance = Hazelcast.getHazelcastInstanceByName("bpm");
        if (hazelCastInstance != null) {
            log.debug("Hazelcast already initialized");
            return hazelCastInstance;
        }
        Config config = new Config();
        config.setInstanceName("bpm");
        config.getNetworkConfig().getJoin().getMulticastConfig().setEnabled(false);
        if (this.registration == null) {
            log.warn("No discovery service is set up, Hazelcast cannot create a cluster.");
        } else {
            // The serviceId is by default the application's name,
            // see the "spring.application.name" standard Spring property
            String serviceId = registration.getServiceId();
            log.debug("Configuring Hazelcast clustering for instanceId: {}", serviceId);
            // In development, everything goes through 127.0.0.1, with a different port
            if (env.acceptsProfiles(Profiles.of(JHipsterConstants.SPRING_PROFILE_DEVELOPMENT))) {
                log.debug("Application is running with the \"dev\" profile, Hazelcast " +
                          "cluster will only work with localhost instances");

                System.setProperty("hazelcast.local.localAddress", "127.0.0.1");
                config.getNetworkConfig().setPort(serverProperties.getPort() + 5701);
                config.getNetworkConfig().getJoin().getTcpIpConfig().setEnabled(true);
                for (ServiceInstance instance : discoveryClient.getInstances(serviceId)) {
                    String clusterMember = "127.0.0.1:" + (instance.getPort() + 5701);
                    log.debug("Adding Hazelcast (dev) cluster member {}", clusterMember);
                    config.getNetworkConfig().getJoin().getTcpIpConfig().addMember(clusterMember);
                }
            } else { // Production configuration, one host per instance all using port 5701
                config.getNetworkConfig().setPort(5701);
                config.getNetworkConfig().getJoin().getTcpIpConfig().setEnabled(true);
                for (ServiceInstance instance : discoveryClient.getInstances(serviceId)) {
                    String clusterMember = instance.getHost() + ":5701";
                    log.debug("Adding Hazelcast (prod) cluster member {}", clusterMember);
                    config.getNetworkConfig().getJoin().getTcpIpConfig().addMember(clusterMember);
                }
            }
        }
        config.getMapConfigs().put("default", initializeDefaultMapConfig(jHipsterProperties));

        // Full reference is available at: https://docs.hazelcast.org/docs/management-center/3.9/manual/html/Deploying_and_Starting.html
        config.setManagementCenterConfig(initializeDefaultManagementCenterConfig(jHipsterProperties));
        config.getMapConfigs().put("xxx.xxx.xxx.xxx.domain.*", initializeDomainMapConfig(jHipsterProperties));
        return Hazelcast.newHazelcastInstance(config);
    }

    private ManagementCenterConfig initializeDefaultManagementCenterConfig(JHipsterProperties jHipsterProperties) {
        ManagementCenterConfig managementCenterConfig = new ManagementCenterConfig();
        managementCenterConfig.setEnabled(jHipsterProperties.getCache().getHazelcast().getManagementCenter().isEnabled());
        managementCenterConfig.setUrl(jHipsterProperties.getCache().getHazelcast().getManagementCenter().getUrl());
        managementCenterConfig.setUpdateInterval(jHipsterProperties.getCache().getHazelcast().getManagementCenter().getUpdateInterval());
        return managementCenterConfig;
    }

    private MapConfig initializeDefaultMapConfig(JHipsterProperties jHipsterProperties) {
        MapConfig mapConfig = new MapConfig();

        /*
        Number of backups. If 1 is set as the backup-count for example,
        then all entries of the map will be copied to another JVM for
        fail-safety. Valid numbers are 0 (no backup), 1, 2, 3.
        */
        mapConfig.setBackupCount(jHipsterProperties.getCache().getHazelcast().getBackupCount());

        /*
        Valid values are:
        NONE (no eviction),
        LRU (Least Recently Used),
        LFU (Least Frequently Used).
        NONE is the default.
        */
        mapConfig.setEvictionPolicy(EvictionPolicy.LRU);

        /*
        Maximum size of the map. When max size is reached,
        map is evicted based on the policy defined.
        Any integer between 0 and Integer.MAX_VALUE. 0 means
        Integer.MAX_VALUE. Default is 0.
        */
        mapConfig.setMaxSizeConfig(new MaxSizeConfig(0, MaxSizeConfig.MaxSizePolicy.USED_HEAP_SIZE));

        return mapConfig;
    }

    private MapConfig initializeDomainMapConfig(JHipsterProperties jHipsterProperties) {
        MapConfig mapConfig = new MapConfig();
        mapConfig.setTimeToLiveSeconds(jHipsterProperties.getCache().getHazelcast().getTimeToLiveSeconds());
        return mapConfig;
    }

    @Autowired(required = false)
    public void setGitProperties(GitProperties gitProperties) {
        this.gitProperties = gitProperties;
    }

    @Autowired(required = false)
    public void setBuildProperties(BuildProperties buildProperties) {
        this.buildProperties = buildProperties;
    }

    @Bean
    public KeyGenerator keyGenerator() {
        return new PrefixedKeyGenerator(this.gitProperties, this.buildProperties);
    }
}

pom.xml 依赖版本

<maven.version>3.3.9</maven.version>
<java.version>1.8</java.version>

Hazelcast 依赖项:

<dependency>
    <groupId>com.hazelcast</groupId>
    <artifactId>hazelcast</artifactId>
</dependency>
<dependency>
    <groupId>com.hazelcast</groupId>
    <artifactId>hazelcast-hibernate53</artifactId>
</dependency>
<dependency>
    <groupId>com.hazelcast</groupId>
    <artifactId>hazelcast-spring</artifactId>
</dependency>

请帮忙。

具有(默认)集群名称 dev 的两个进程已找到彼此并试图聚集在一起。版本不兼容,因此失败。

您可以关闭两个进程的发现,但如果您不控制另一个,这可能不可行。

相反,您可以使用 config.getGroupConfig().setName('...') 更改集群的名称以防止加入尝试。