在 ignite 部署示例之后,无法 运行 在 azure kubernetes 上使用 ignitevisorcmd

Can't run ignitevisorcmd on azure kubernetes following ignite deploying example

我已按照 ignite 2.9 文档中的 Azure Kubernetes 部署说明进行操作 https://ignite.apache.org/docs/latest/installation/kubernetes/azure-deployment。但是我不能使用 ignitevisorcmd.sh

如果我 运行 它我不能 select 使用打开命令的配置文件,因为它从 url https://raw.githubusercontent.com/apache/ignite/master/modules/kubernetes/config/example-kube-persistence-and-wal.xml 加载配置。所以我将配置文件放在 ignite 集群上的某个位置 (/opt/ignite/apache-ignite/custom-config/cm-ignite.xml)。我连接到其中一个节点,并在其上 运行 ingitevisorcmd。我 select 打开命令的配置文件但有错误消息:

visor> open
Local configuration files:
+==========================================================================================+
| #  |                                 Configuration File                                  |
+==========================================================================================+
| 0  | config/default-config.xml                                                           |
| 1  | benchmarks/config/ignite-base-config.xml                                            |
| 2  | benchmarks/config/ignite-localhost-config.xml                                       |
| 3  | benchmarks/config/ignite-multicast-config.xml                                       |
| 4  | benchmarks/config/ignite-remote-config.xml                                          |
| 5  | benchmarks/sources/config/ignite-base-config.xml                                    |
| 6  | benchmarks/sources/config/ignite-localhost-config.xml                               |
| 7  | benchmarks/sources/config/ignite-multicast-config.xml                               |
| 8  | benchmarks/sources/config/ignite-remote-config.xml                                  |
| 9  | (?) config/router/default-router.xml                                                |
| 10 | custom-config/..2020_11_26_08_55_28.816549898/cm-ignite.xml                         |
| 11 | custom-config/..data/cm-ignite.xml                                                  |
| 12 | custom-config/cm-ignite.xml                                                         |
| 13 | examples/config/encryption/example-encrypted-store.xml                              |
| 14 | examples/config/example-cache.xml                                                   |
| 15 | examples/config/example-data-regions.xml                                            |
| 16 | examples/config/example-default.xml                                                 |
| 17 | (?) examples/config/example-ignite-ml.xml                                           |
| 18 | (?) examples/config/example-ignite.xml                                              |
| 19 | examples/config/persistentstore/example-persistent-store.xml                        |
| 20 | examples/config/redis/example-redis.xml                                             |
| 21 | examples/config/spark/example-shared-rdd.xml                                        |
| 22 | (?) examples/config/spring/example-spring-data.xml                                  |
| 23 | examples/src/main/java/org/apache/ignite/examples/misc/springbean/spring-bean.xml   |
| 24 | platforms/cpp/examples/compute-example/config/compute-example.xml                   |
| 25 | platforms/cpp/examples/continuous-query-example/config/continuous-query-example.xml |
| 26 | platforms/cpp/examples/odbc-example/config/example-odbc.xml                         |
| 27 | platforms/cpp/examples/put-get-example/config/example-cache.xml                     |
| 28 | platforms/cpp/examples/query-example/config/query-example.xml                       |
+------------------------------------------------------------------------------------------+

Choose configuration file number ('c' to cancel) [0]: 12

+-----------------------------------------------------------------------------+
| Using configuration | /opt/ignite/apache-ignite/custom-config/cm-ignite.xml |
+-----------------------------------------------------------------------------+

Nov 26, 2020 9:08:35 AM org.springframework.beans.factory.xml.XmlBeanDefinitionReader loadBeanDefinitions
INFO: Loading XML bean definitions from URL [file:/opt/ignite/apache-ignite/custom-config/cm-ignite.xml]
Nov 26, 2020 9:08:35 AM org.springframework.context.support.AbstractApplicationContext prepareRefresh
INFO: Refreshing org.springframework.context.support.GenericApplicationContext@333291e3: startup date [Thu Nov 26 09:08:35 GMT 2020]; root of context hierarchy
Nov 26, 2020 9:08:36 AM org.springframework.context.support.AbstractApplicationContext refresh
WARNING: Exception encountered during context initialization - cancelling refresh attempt: org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'org.apache.ignite.configuration.IgniteConfiguration#0' defined in URL [file:/opt/ignite/apache-ignite/custom-config/cm-ignite.xml]: Cannot create inner bean 'org.apache.ignite.spi.discovery.tcp.TcpDiscoverySpi#2de23121' of type [org.apache.ignite.spi.discovery.tcp.TcpDiscoverySpi] while setting bean property 'discoverySpi'; nested exception is org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'org.apache.ignite.spi.discovery.tcp.TcpDiscoverySpi#2de23121' defined in URL [file:/opt/ignite/apache-ignite/custom-config/cm-ignite.xml]: Cannot create inner bean 'org.apache.ignite.spi.discovery.tcp.ipfinder.kubernetes.TcpDiscoveryKubernetesIpFinder#22fcf7ab' of type [org.apache.ignite.spi.discovery.tcp.ipfinder.kubernetes.TcpDiscoveryKubernetesIpFinder] while setting bean property 'ipFinder'; nested exception is org.springframework.beans.factory.CannotLoadBeanClassException: Cannot find class [org.apache.ignite.spi.discovery.tcp.ipfinder.kubernetes.TcpDiscoveryKubernetesIpFinder] for bean with name 'org.apache.ignite.spi.discovery.tcp.ipfinder.kubernetes.TcpDiscoveryKubernetesIpFinder#22fcf7ab' defined in URL [file:/opt/ignite/apache-ignite/custom-config/cm-ignite.xml]; nested exception is java.lang.ClassNotFoundException: org.apache.ignite.spi.discovery.tcp.ipfinder.kubernetes.TcpDiscoveryKubernetesIpFinder
class org.apache.ignite.IgniteCheckedException: Failed to instantiate Spring XML application context (make sure all classes used in Spring configuration are present at CLASSPATH) [springUrl=file:/opt/ignite/apache-ignite/custom-config/cm-ignite.xml]
    at org.apache.ignite.internal.util.spring.IgniteSpringHelperImpl.applicationContext(IgniteSpringHelperImpl.java:387)
    at org.apache.ignite.internal.util.spring.IgniteSpringHelperImpl.loadConfigurations(IgniteSpringHelperImpl.java:104)
    at org.apache.ignite.internal.util.spring.IgniteSpringHelperImpl.loadConfigurations(IgniteSpringHelperImpl.java:98)
    at org.apache.ignite.visor.commands.open.VisorOpenCommand.configuration(VisorOpenCommand.scala:153)
    at org.apache.ignite.visor.commands.open.VisorOpenCommand.open(VisorOpenCommand.scala:212)
    at org.apache.ignite.visor.commands.open.VisorOpenCommand.open(VisorOpenCommand.scala:94)
    at org.apache.ignite.visor.commands.open.VisorOpenCommand$$anonfun.apply$mcV$sp(VisorOpenCommand.scala:305)
    at org.apache.ignite.visor.commands.VisorConsole.mainLoop(VisorConsole.scala:237)
    at org.apache.ignite.visor.commands.VisorConsole$.delayedEndpoint$org$apache$ignite$visor$commands$VisorConsole(VisorConsole.scala:354)
    at org.apache.ignite.visor.commands.VisorConsole$delayedInit$body.apply(VisorConsole.scala:343)
    at scala.Function0$class.apply$mcV$sp(Function0.scala:34)
    at scala.runtime.AbstractFunction0.apply$mcV$sp(AbstractFunction0.scala:12)
    at scala.App$$anonfun$main.apply(App.scala:76)
    at scala.App$$anonfun$main.apply(App.scala:76)
    at scala.collection.immutable.List.foreach(List.scala:392)
    at scala.collection.generic.TraversableForwarder$class.foreach(TraversableForwarder.scala:35)
    at scala.App$class.main(App.scala:76)
    at org.apache.ignite.visor.commands.VisorConsole$.main(VisorConsole.scala:343)
    at org.apache.ignite.visor.commands.VisorConsole.main(VisorConsole.scala)
Caused by: org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'org.apache.ignite.configuration.IgniteConfiguration#0' defined in URL [file:/opt/ignite/apache-ignite/custom-config/cm-ignite.xml]: Cannot create inner bean 'org.apache.ignite.spi.discovery.tcp.TcpDiscoverySpi#2de23121' of type [org.apache.ignite.spi.discovery.tcp.TcpDiscoverySpi] while setting bean property 'discoverySpi'; nested exception is org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'org.apache.ignite.spi.discovery.tcp.TcpDiscoverySpi#2de23121' defined in URL [file:/opt/ignite/apache-ignite/custom-config/cm-ignite.xml]: Cannot create inner bean 'org.apache.ignite.spi.discovery.tcp.ipfinder.kubernetes.TcpDiscoveryKubernetesIpFinder#22fcf7ab' of type [org.apache.ignite.spi.discovery.tcp.ipfinder.kubernetes.TcpDiscoveryKubernetesIpFinder] while setting bean property 'ipFinder'; nested exception is org.springframework.beans.factory.CannotLoadBeanClassException: Cannot find class [org.apache.ignite.spi.discovery.tcp.ipfinder.kubernetes.TcpDiscoveryKubernetesIpFinder] for bean with name 'org.apache.ignite.spi.discovery.tcp.ipfinder.kubernetes.TcpDiscoveryKubernetesIpFinder#22fcf7ab' defined in URL [file:/opt/ignite/apache-ignite/custom-config/cm-ignite.xml]; nested exception is java.lang.ClassNotFoundException: org.apache.ignite.spi.discovery.tcp.ipfinder.kubernetes.TcpDiscoveryKubernetesIpFinder
    at org.springframework.beans.factory.support.BeanDefinitionValueResolver.resolveInnerBean(BeanDefinitionValueResolver.java:313)
    at org.springframework.beans.factory.support.BeanDefinitionValueResolver.resolveValueIfNecessary(BeanDefinitionValueResolver.java:122)
    at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.applyPropertyValues(AbstractAutowireCapableBeanFactory.java:1522)
    at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.populateBean(AbstractAutowireCapableBeanFactory.java:1269)
    at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.doCreateBean(AbstractAutowireCapableBeanFactory.java:551)
    at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBean(AbstractAutowireCapableBeanFactory.java:481)
    at org.springframework.beans.factory.support.AbstractBeanFactory.getObject(AbstractBeanFactory.java:312)
    at org.springframework.beans.factory.support.DefaultSingletonBeanRegistry.getSingleton(DefaultSingletonBeanRegistry.java:230)
    at org.springframework.beans.factory.support.AbstractBeanFactory.doGetBean(AbstractBeanFactory.java:308)
    at org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:197)
    at org.springframework.beans.factory.support.DefaultListableBeanFactory.preInstantiateSingletons(DefaultListableBeanFactory.java:757)
    at org.springframework.context.support.AbstractApplicationContext.finishBeanFactoryInitialization(AbstractApplicationContext.java:867)
    at org.springframework.context.support.AbstractApplicationContext.refresh(AbstractApplicationContext.java:542)
    at org.apache.ignite.internal.util.spring.IgniteSpringHelperImpl.applicationContext(IgniteSpringHelperImpl.java:381)
    ... 18 more
Caused by: org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'org.apache.ignite.spi.discovery.tcp.TcpDiscoverySpi#2de23121' defined in URL [file:/opt/ignite/apache-ignite/custom-config/cm-ignite.xml]: Cannot create inner bean 'org.apache.ignite.spi.discovery.tcp.ipfinder.kubernetes.TcpDiscoveryKubernetesIpFinder#22fcf7ab' of type [org.apache.ignite.spi.discovery.tcp.ipfinder.kubernetes.TcpDiscoveryKubernetesIpFinder] while setting bean property 'ipFinder'; nested exception is org.springframework.beans.factory.CannotLoadBeanClassException: Cannot find class [org.apache.ignite.spi.discovery.tcp.ipfinder.kubernetes.TcpDiscoveryKubernetesIpFinder] for bean with name 'org.apache.ignite.spi.discovery.tcp.ipfinder.kubernetes.TcpDiscoveryKubernetesIpFinder#22fcf7ab' defined in URL [file:/opt/ignite/apache-ignite/custom-config/cm-ignite.xml]; nested exception is java.lang.ClassNotFoundException: org.apache.ignite.spi.discovery.tcp.ipfinder.kubernetes.TcpDiscoveryKubernetesIpFinder
    at org.springframework.beans.factory.support.BeanDefinitionValueResolver.resolveInnerBean(BeanDefinitionValueResolver.java:313)
    at org.springframework.beans.factory.support.BeanDefinitionValueResolver.resolveValueIfNecessary(BeanDefinitionValueResolver.java:122)
    at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.applyPropertyValues(AbstractAutowireCapableBeanFactory.java:1522)
    at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.populateBean(AbstractAutowireCapableBeanFactory.java:1269)
    at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.doCreateBean(AbstractAutowireCapableBeanFactory.java:551)
    at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBean(AbstractAutowireCapableBeanFactory.java:481)
    at org.springframework.beans.factory.support.BeanDefinitionValueResolver.resolveInnerBean(BeanDefinitionValueResolver.java:299)
    ... 31 more
Caused by: org.springframework.beans.factory.CannotLoadBeanClassException: Cannot find class [org.apache.ignite.spi.discovery.tcp.ipfinder.kubernetes.TcpDiscoveryKubernetesIpFinder] for bean with name 'org.apache.ignite.spi.discovery.tcp.ipfinder.kubernetes.TcpDiscoveryKubernetesIpFinder#22fcf7ab' defined in URL [file:/opt/ignite/apache-ignite/custom-config/cm-ignite.xml]; nested exception is java.lang.ClassNotFoundException: org.apache.ignite.spi.discovery.tcp.ipfinder.kubernetes.TcpDiscoveryKubernetesIpFinder
    at org.springframework.beans.factory.support.AbstractBeanFactory.resolveBeanClass(AbstractBeanFactory.java:1397)
    at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBean(AbstractAutowireCapableBeanFactory.java:454)
    at org.springframework.beans.factory.support.BeanDefinitionValueResolver.resolveInnerBean(BeanDefinitionValueResolver.java:299)
    ... 37 more
Caused by: java.lang.ClassNotFoundException: org.apache.ignite.spi.discovery.tcp.ipfinder.kubernetes.TcpDiscoveryKubernetesIpFinder
    at java.net.URLClassLoader.findClass(URLClassLoader.java:382)
    at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
    at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:349)
    at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
    at org.springframework.util.ClassUtils.forName(ClassUtils.java:251)
    at org.springframework.beans.factory.support.AbstractBeanDefinition.resolveBeanClass(AbstractBeanDefinition.java:408)
    at org.springframework.beans.factory.support.AbstractBeanFactory.doResolveBeanClass(AbstractBeanFactory.java:1444)
    at org.springframework.beans.factory.support.AbstractBeanFactory.resolveBeanClass(AbstractBeanFactory.java:1389)
    ... 39 more
[WARN ] Invalid command name: 'open'
[WARN ] Type 'help' to print commands list.

如何在我的案例中使用 ignitevisorcmd

不可能 运行 ignitevisorcmd 来自 Kubernetes 中与 运行 点燃服务节点相同的 pod,因为 TcpDiscoveryKubernetesIpFinder 不能与不同的发现端口一起使用(来自 javadocs:确保所有 Ignite pods 占用相似的发现端口,否则它们将无法使用此 IP 查找器相互发现。)。发现端口已被 ignite 服务器使用,因此 visor 只能 运行 作为一个单独的 pod。

还应该安装库 ignite-kubernetes 以便使用 ignite 遮阳板:

kubectl -n ignite exec -it ignite-0 -- bash
cp -R /opt/ignite/apache-ignite/libs/optional/ignite-kubernetes/* /opt/ignite/apache-ignite/libs
/opt/ignite/apache-ignite/bin/ignitevisorcmd.sh

还要考虑 ignitevisorcmd 是一个胖客户端,不能 运行 在通过负载均衡器部署 ignite 服务器节点的 Kubernetes 集群之外。 (目前无法通过 NAT、负载平衡器和防火墙工作)。

注意:我已经设法从胖客户端连接到部署在 Kubernetes 内部的 ignite,该客户端可以通过 ip 直接访问 ignite 节点。我在配置中使用了 TcpDiscoveryVmIpFinder,并通过暴露的 ignite 服务域名使用 nslookup 确定服务器节点 ip。