NiFi 不会从 Zookeeper Kerberos 配置开始

NiFi Won't Start with Zookeeper Kerberos Config

我目前在让 NiFi 和 Zookeeper 使用 Kerberos 进行身份验证时遇到一些问题。任何帮助将不胜感激。

当我尝试使用 Kerberos 配置启动 NiFI 时,它只是在启动过程中关闭。

我正在使用外部 Zookeeper 集群(不是嵌入式集群)。我已根据 NiFi Admin Guide:

将以下内容添加到我的配置文件中

$NIFI_HOME/conf/bootstrap.conf:

java.arg.15=-Djava.security.auth.login.config=/opt/nifi/conf/zookeeper-jaas.conf

$NIFI_HOME/conf/nifi.属性:

nifi.zookeeper.auth.type=sasl
nifi.zookeeper.kerberos.removeHostFromPrincipal=true
nifi.zookeeper.kerberos.removeRealmFromPrincipal=true

$NIFI_HOME/conf/zookeeper-jaas.conf:

Client {
    com.sun.security.auth.module.Krb5LoginModule required
    useKeyTab=true
    keyTab="/opt/nifi/conf/nifi.keytab"
    storeKey=true
    useTicketCache=false
    principal="HTTP/nifi-2.nifi.nifi4.svc.cluster.local@MYDOMAIN.NET";
    };

我也在初始化密钥表使用:

kinit -kt /opt/nifi/conf/nifi.keytab HTTP/nifi-2.nifi.nifi4.svc.cluster.local@MYDOMAIN.NET

当我启动 NiFi 时,在它关闭之前我得到以下堆栈跟踪:

2020-04-21 18:41:43,736 WARN [main] org.eclipse.jetty.webapp.WebAppContext Failed startup of context o.e.j.w.WebAppContext@504497fa{nifi-api,/nifi-api,file:///opt/nifi/work/jetty/nifi-web-api-1.10.0.war/webapp/,UNAVAILABLE}{./work/nar/framework/nifi-framework-nar-1.10.0.nar-unpacked/NAR-INF/bundled-dependencies/nifi-web-api-1.10.0.war}
org.apache.nifi.web.NiFiCoreException: Unable to start Flow Controller.
    at org.apache.nifi.web.contextlistener.ApplicationStartupContextListener.contextInitialized(ApplicationStartupContextListener.java:88)
    at org.eclipse.jetty.server.handler.ContextHandler.callContextInitialized(ContextHandler.java:959)
    at org.eclipse.jetty.servlet.ServletContextHandler.callContextInitialized(ServletContextHandler.java:553)
    at org.eclipse.jetty.server.handler.ContextHandler.startContext(ContextHandler.java:924)
    at org.eclipse.jetty.servlet.ServletContextHandler.startContext(ServletContextHandler.java:365)
    at org.eclipse.jetty.webapp.WebAppContext.startWebapp(WebAppContext.java:1497)
    at org.eclipse.jetty.webapp.WebAppContext.startContext(WebAppContext.java:1459)
    at org.eclipse.jetty.server.handler.ContextHandler.doStart(ContextHandler.java:854)
    at org.eclipse.jetty.servlet.ServletContextHandler.doStart(ServletContextHandler.java:278)
    at org.eclipse.jetty.webapp.WebAppContext.doStart(WebAppContext.java:545)
    at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:68)
    at org.eclipse.jetty.util.component.ContainerLifeCycle.start(ContainerLifeCycle.java:167)
    at org.eclipse.jetty.util.component.ContainerLifeCycle.doStart(ContainerLifeCycle.java:119)
    at org.eclipse.jetty.server.handler.AbstractHandler.doStart(AbstractHandler.java:113)
    at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:68)
    at org.eclipse.jetty.util.component.ContainerLifeCycle.start(ContainerLifeCycle.java:167)
    at org.eclipse.jetty.util.component.ContainerLifeCycle.doStart(ContainerLifeCycle.java:110)
    at org.eclipse.jetty.server.handler.AbstractHandler.doStart(AbstractHandler.java:113)
    at org.eclipse.jetty.server.handler.gzip.GzipHandler.doStart(GzipHandler.java:406)
    at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:68)
    at org.eclipse.jetty.util.component.ContainerLifeCycle.start(ContainerLifeCycle.java:167)
    at org.eclipse.jetty.util.component.ContainerLifeCycle.doStart(ContainerLifeCycle.java:119)
    at org.eclipse.jetty.server.handler.AbstractHandler.doStart(AbstractHandler.java:113)
    at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:68)
    at org.eclipse.jetty.util.component.ContainerLifeCycle.start(ContainerLifeCycle.java:167)
    at org.eclipse.jetty.util.component.ContainerLifeCycle.doStart(ContainerLifeCycle.java:119)
    at org.eclipse.jetty.server.handler.AbstractHandler.doStart(AbstractHandler.java:113)
    at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:68)
    at org.eclipse.jetty.util.component.ContainerLifeCycle.start(ContainerLifeCycle.java:167)
    at org.eclipse.jetty.server.Server.start(Server.java:418)
    at org.eclipse.jetty.util.component.ContainerLifeCycle.doStart(ContainerLifeCycle.java:110)
    at org.eclipse.jetty.server.handler.AbstractHandler.doStart(AbstractHandler.java:113)
    at org.eclipse.jetty.server.Server.doStart(Server.java:382)
    at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:68)
    at org.apache.nifi.web.server.JettyServer.start(JettyServer.java:952)
    at org.apache.nifi.NiFi.<init>(NiFi.java:158)
    at org.apache.nifi.NiFi.<init>(NiFi.java:72)
    at org.apache.nifi.NiFi.main(NiFi.java:301)
Caused by: org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'flowService': FactoryBean threw exception on object creation; nested exception is org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'flowController': FactoryBean threw exception on object creation; nested exception is java.lang.StringIndexOutOfBoundsException: begin 0, end -1, length 87
    at org.springframework.beans.factory.support.FactoryBeanRegistrySupport.doGetObjectFromFactoryBean(FactoryBeanRegistrySupport.java:185)
    at org.springframework.beans.factory.support.FactoryBeanRegistrySupport.getObjectFromFactoryBean(FactoryBeanRegistrySupport.java:103)
    at org.springframework.beans.factory.support.AbstractBeanFactory.getObjectForBeanInstance(AbstractBeanFactory.java:1640)
    at org.springframework.beans.factory.support.AbstractBeanFactory.doGetBean(AbstractBeanFactory.java:323)
    at org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:202)
    at org.springframework.context.support.AbstractApplicationContext.getBean(AbstractApplicationContext.java:1086)
    at org.apache.nifi.web.contextlistener.ApplicationStartupContextListener.contextInitialized(ApplicationStartupContextListener.java:55)
    ... 37 common frames omitted
Caused by: org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'flowController': FactoryBean threw exception on object creation; nested exception is java.lang.StringIndexOutOfBoundsException: begin 0, end -1, length 87
    at org.springframework.beans.factory.support.FactoryBeanRegistrySupport.doGetObjectFromFactoryBean(FactoryBeanRegistrySupport.java:185)
    at org.springframework.beans.factory.support.FactoryBeanRegistrySupport.getObjectFromFactoryBean(FactoryBeanRegistrySupport.java:103)
    at org.springframework.beans.factory.support.AbstractBeanFactory.getObjectForBeanInstance(AbstractBeanFactory.java:1640)
    at org.springframework.beans.factory.support.AbstractBeanFactory.doGetBean(AbstractBeanFactory.java:323)
    at org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:202)
    at org.springframework.context.support.AbstractApplicationContext.getBean(AbstractApplicationContext.java:1086)
    at org.apache.nifi.spring.StandardFlowServiceFactoryBean.getObject(StandardFlowServiceFactoryBean.java:48)
    at org.springframework.beans.factory.support.FactoryBeanRegistrySupport.doGetObjectFromFactoryBean(FactoryBeanRegistrySupport.java:178)
    ... 43 common frames omitted
Caused by: java.lang.StringIndexOutOfBoundsException: begin 0, end -1, length 87
    at java.base/java.lang.String.checkBoundsBeginEnd(String.java:3319)
    at java.base/java.lang.String.substring(String.java:1874)
    at org.apache.nifi.controller.leader.election.CuratorACLProviderFactory$SaslACLProvider.<init>(CuratorACLProviderFactory.java:49)
    at org.apache.nifi.controller.leader.election.CuratorACLProviderFactory$SaslACLProvider.<init>(CuratorACLProviderFactory.java:40)
    at org.apache.nifi.controller.leader.election.CuratorACLProviderFactory.create(CuratorACLProviderFactory.java:37)
    at org.apache.nifi.controller.leader.election.CuratorLeaderElectionManager.createClient(CuratorLeaderElectionManager.java:389)
    at org.apache.nifi.controller.leader.election.CuratorLeaderElectionManager.determineLeaderExternal(CuratorLeaderElectionManager.java:343)
    at org.apache.nifi.controller.leader.election.CuratorLeaderElectionManager.getLeader(CuratorLeaderElectionManager.java:240)
    at org.apache.nifi.controller.FlowController.<init>(FlowController.java:680)
    at org.apache.nifi.controller.FlowController.createClusteredInstance(FlowController.java:413)
    at org.apache.nifi.spring.FlowControllerFactoryBean.getObject(FlowControllerFactoryBean.java:65)
    at org.springframework.beans.factory.support.FactoryBeanRegistrySupport.doGetObjectFromFactoryBean(FactoryBeanRegistrySupport.java:178)
    ... 50 common frames omitted
2020-04-21 18:41:43,983 INFO [main] o.e.j.a.AnnotationConfiguration Scanning elapsed time=97ms
2020-04-21 18:41:43,984 INFO [main] o.e.j.s.h.C._nifi_content_viewer No Spring WebApplicationInitializer types detected on classpath
2020-04-21 18:41:44,010 INFO [main] o.e.jetty.server.handler.ContextHandler Started o.e.j.w.WebAppContext@5618fc1f{nifi-content-viewer,/nifi-content-viewer,file:///opt/nifi/work/jetty/nifi-web-content-viewer-1.10.0.war/webapp/,AVAILABLE}{./work/nar/framework/nifi-framework-nar-1.10.0.nar-unpacked/NAR-INF/bundled-dependencies/nifi-web-content-viewer-1.10.0.war}
2020-04-21 18:41:44,044 INFO [main] o.e.j.a.AnnotationConfiguration Scanning elapsed time=22ms
2020-04-21 18:41:44,046 WARN [main] o.e.j.webapp.StandardDescriptorProcessor Duplicate mapping from / to default
2020-04-21 18:41:44,046 INFO [main] o.e.j.s.h.ContextHandler._nifi_docs No Spring WebApplicationInitializer types detected on classpath
2020-04-21 18:41:44,071 INFO [main] o.e.jetty.server.handler.ContextHandler Started o.e.j.w.WebAppContext@60b1ff3b{nifi-docs,/nifi-docs,file:///opt/nifi/work/jetty/nifi-web-docs-1.10.0.war/webapp/,AVAILABLE}{./work/nar/framework/nifi-framework-nar-1.10.0.nar-unpacked/NAR-INF/bundled-dependencies/nifi-web-docs-1.10.0.war}
2020-04-21 18:41:44,091 INFO [main] o.e.j.a.AnnotationConfiguration Scanning elapsed time=11ms
2020-04-21 18:41:44,093 INFO [main] o.e.j.server.handler.ContextHandler._ No Spring WebApplicationInitializer types detected on classpath
2020-04-21 18:41:44,116 INFO [main] o.e.jetty.server.handler.ContextHandler Started o.e.j.w.WebAppContext@374c3975{nifi-error,/,file:///opt/nifi/work/jetty/nifi-web-error-1.10.0.war/webapp/,AVAILABLE}{./work/nar/framework/nifi-framework-nar-1.10.0.nar-unpacked/NAR-INF/bundled-dependencies/nifi-web-error-1.10.0.war}
2020-04-21 18:41:44,137 INFO [main] o.e.jetty.util.ssl.SslContextFactory x509=X509@5fb8db6c(nifi-2,h=[nifi-2-fix, nifi-2, test.gdn.network, test-api.gdn.network, nifi.gdn.network],w=[]) for SslContextFactory@4556a9a7[provider=null,keyStore=file:///opt/nifi/conf/nifi-2.p12,trustStore=file:///opt/nifi/conf/truststore.jks]
2020-04-21 18:41:44,149 INFO [main] o.eclipse.jetty.server.AbstractConnector Started ServerConnector@4d81e83a{SSL,[ssl, http/1.1]}{nifi-2:8443}
2020-04-21 18:41:44,149 INFO [main] org.eclipse.jetty.server.Server Started @38713ms
2020-04-21 18:41:44,150 WARN [main] org.apache.nifi.web.server.JettyServer Failed to start web server... shutting down.
org.apache.nifi.web.NiFiCoreException: Unable to start Flow Controller.
    at org.apache.nifi.web.contextlistener.ApplicationStartupContextListener.contextInitialized(ApplicationStartupContextListener.java:88)
    at org.eclipse.jetty.server.handler.ContextHandler.callContextInitialized(ContextHandler.java:959)
    at org.eclipse.jetty.servlet.ServletContextHandler.callContextInitialized(ServletContextHandler.java:553)
    at org.eclipse.jetty.server.handler.ContextHandler.startContext(ContextHandler.java:924)
    at org.eclipse.jetty.servlet.ServletContextHandler.startContext(ServletContextHandler.java:365)
    at org.eclipse.jetty.webapp.WebAppContext.startWebapp(WebAppContext.java:1497)
    at org.eclipse.jetty.webapp.WebAppContext.startContext(WebAppContext.java:1459)
    at org.eclipse.jetty.server.handler.ContextHandler.doStart(ContextHandler.java:854)
    at org.eclipse.jetty.servlet.ServletContextHandler.doStart(ServletContextHandler.java:278)
    at org.eclipse.jetty.webapp.WebAppContext.doStart(WebAppContext.java:545)
    at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:68)
    at org.eclipse.jetty.util.component.ContainerLifeCycle.start(ContainerLifeCycle.java:167)
    at org.eclipse.jetty.util.component.ContainerLifeCycle.doStart(ContainerLifeCycle.java:119)
    at org.eclipse.jetty.server.handler.AbstractHandler.doStart(AbstractHandler.java:113)
    at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:68)
    at org.eclipse.jetty.util.component.ContainerLifeCycle.start(ContainerLifeCycle.java:167)
    at org.eclipse.jetty.util.component.ContainerLifeCycle.doStart(ContainerLifeCycle.java:110)
    at org.eclipse.jetty.server.handler.AbstractHandler.doStart(AbstractHandler.java:113)
    at org.eclipse.jetty.server.handler.gzip.GzipHandler.doStart(GzipHandler.java:406)
    at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:68)
    at org.eclipse.jetty.util.component.ContainerLifeCycle.start(ContainerLifeCycle.java:167)
    at org.eclipse.jetty.util.component.ContainerLifeCycle.doStart(ContainerLifeCycle.java:119)
    at org.eclipse.jetty.server.handler.AbstractHandler.doStart(AbstractHandler.java:113)
    at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:68)
    at org.eclipse.jetty.util.component.ContainerLifeCycle.start(ContainerLifeCycle.java:167)
    at org.eclipse.jetty.util.component.ContainerLifeCycle.doStart(ContainerLifeCycle.java:119)
    at org.eclipse.jetty.server.handler.AbstractHandler.doStart(AbstractHandler.java:113)
    at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:68)
    at org.eclipse.jetty.util.component.ContainerLifeCycle.start(ContainerLifeCycle.java:167)
    at org.eclipse.jetty.server.Server.start(Server.java:418)
    at org.eclipse.jetty.util.component.ContainerLifeCycle.doStart(ContainerLifeCycle.java:110)
    at org.eclipse.jetty.server.handler.AbstractHandler.doStart(AbstractHandler.java:113)
    at org.eclipse.jetty.server.Server.doStart(Server.java:382)
    at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:68)
    at org.apache.nifi.web.server.JettyServer.start(JettyServer.java:952)
    at org.apache.nifi.NiFi.<init>(NiFi.java:158)
    at org.apache.nifi.NiFi.<init>(NiFi.java:72)
    at org.apache.nifi.NiFi.main(NiFi.java:301)
Caused by: org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'flowService': FactoryBean threw exception on object creation; nested exception is org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'flowController': FactoryBean threw exception on object creation; nested exception is java.lang.StringIndexOutOfBoundsException: begin 0, end -1, length 87
    at org.springframework.beans.factory.support.FactoryBeanRegistrySupport.doGetObjectFromFactoryBean(FactoryBeanRegistrySupport.java:185)
    at org.springframework.beans.factory.support.FactoryBeanRegistrySupport.getObjectFromFactoryBean(FactoryBeanRegistrySupport.java:103)
    at org.springframework.beans.factory.support.AbstractBeanFactory.getObjectForBeanInstance(AbstractBeanFactory.java:1640)
    at org.springframework.beans.factory.support.AbstractBeanFactory.doGetBean(AbstractBeanFactory.java:323)
    at org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:202)
    at org.springframework.context.support.AbstractApplicationContext.getBean(AbstractApplicationContext.java:1086)
    at org.apache.nifi.web.contextlistener.ApplicationStartupContextListener.contextInitialized(ApplicationStartupContextListener.java:55)
    ... 37 common frames omitted
Caused by: org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'flowController': FactoryBean threw exception on object creation; nested exception is java.lang.StringIndexOutOfBoundsException: begin 0, end -1, length 87
    at org.springframework.beans.factory.support.FactoryBeanRegistrySupport.doGetObjectFromFactoryBean(FactoryBeanRegistrySupport.java:185)
    at org.springframework.beans.factory.support.FactoryBeanRegistrySupport.getObjectFromFactoryBean(FactoryBeanRegistrySupport.java:103)
    at org.springframework.beans.factory.support.AbstractBeanFactory.getObjectForBeanInstance(AbstractBeanFactory.java:1640)
    at org.springframework.beans.factory.support.AbstractBeanFactory.doGetBean(AbstractBeanFactory.java:323)
    at org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:202)
    at org.springframework.context.support.AbstractApplicationContext.getBean(AbstractApplicationContext.java:1086)
    at org.apache.nifi.spring.StandardFlowServiceFactoryBean.getObject(StandardFlowServiceFactoryBean.java:48)
    at org.springframework.beans.factory.support.FactoryBeanRegistrySupport.doGetObjectFromFactoryBean(FactoryBeanRegistrySupport.java:178)
    ... 43 common frames omitted
Caused by: java.lang.StringIndexOutOfBoundsException: begin 0, end -1, length 87
    at java.base/java.lang.String.checkBoundsBeginEnd(String.java:3319)
    at java.base/java.lang.String.substring(String.java:1874)
    at org.apache.nifi.controller.leader.election.CuratorACLProviderFactory$SaslACLProvider.<init>(CuratorACLProviderFactory.java:49)
    at org.apache.nifi.controller.leader.election.CuratorACLProviderFactory$SaslACLProvider.<init>(CuratorACLProviderFactory.java:40)
    at org.apache.nifi.controller.leader.election.CuratorACLProviderFactory.create(CuratorACLProviderFactory.java:37)
    at org.apache.nifi.controller.leader.election.CuratorLeaderElectionManager.createClient(CuratorLeaderElectionManager.java:389)
    at org.apache.nifi.controller.leader.election.CuratorLeaderElectionManager.determineLeaderExternal(CuratorLeaderElectionManager.java:343)
    at org.apache.nifi.controller.leader.election.CuratorLeaderElectionManager.getLeader(CuratorLeaderElectionManager.java:240)
    at org.apache.nifi.controller.FlowController.<init>(FlowController.java:680)
    at org.apache.nifi.controller.FlowController.createClusteredInstance(FlowController.java:413)
    at org.apache.nifi.spring.FlowControllerFactoryBean.getObject(FlowControllerFactoryBean.java:65)
    at org.springframework.beans.factory.support.FactoryBeanRegistrySupport.doGetObjectFromFactoryBean(FactoryBeanRegistrySupport.java:178)
    ... 50 common frames omitted
2020-04-21 18:41:44,150 INFO [Thread-0] org.apache.nifi.NiFi Initiating shutdown of Jetty web server...

如果我删除 Kerberos 配置,那么 NiFi 将 运行 就好了(尽管没有 Kerberised Zookeeper)。我已经尝试 运行 使用全新的 Zookeeper 来解决这个问题,以防与之前的非 kerberised 节点发生某种冲突,但没有成功。

对 NiFi/Zookeeper 有更多了解的人能帮助我理解这个错误吗?

非常感谢,

哈利

您仍然需要在 nifi.properties 中设置 kerberos 服务主体:

nifi.kerberos.service.principal

管理员指南中关于 ZK 客户端的文档应该更新,但它假设 NiFi 本身已经 "kerberized" 可以设置服务主体。