Hadoop 集群 kerberized 时无法访问 HDFS

Can't access HDFS when Hadoop cluster kerberized

我成功地对一个测试 Hortonworks 集群进行了 kerberos 化。 Ambari 为服务创建了密钥表，它们都已启动。名称节点有 HA。备用名称节点启动速度快，活动名称节点需要更长的时间。 Namenode UI 显示一切正确。可以使用kerberos登录。名称节点是 nn1.zim.com 和 nn2.zim.com

此配置可能有什么问题？以 hdfs 身份登录，使用 kinit -kt 加载密钥表。在列表 HDFS 尝试中我收到此错误：

[root@nn1 hdfs]# hdfs dfs -ls / 18/12/02 16:18:22 WARN ipc.Client: Exception encountered while connecting to the server : javax.security.sasl.SaslException: GSS initiate failed [Caused by GSSE xception: No valid credentials provided (Mechanism level: Failed to find any Ker beros tgt)] 18/12/02 16:18:22 INFO retry.RetryInvocationHandler: java.io.IOException: Failed on local exception: java.io.IOException: javax.security.sasl.SaslException: GSS initiate failed [Caused by GSSException: No valid credentials provided (Mechani sm level: Failed to find any Kerberos tgt)]; Host Details : local host is: "nn1. zim.com/192.168.50.10"; destination host is: "nn2.zim.com":8020; , while invokin g ClientNamenodeProtocolTranslatorPB.getFileInfo over nn2.zim.com/192.168.50.11: 8020 after 1 failover attempts. Trying to failover after sleeping for 1123ms.

主机的 Kerberos 主体是：

nn1.zim.com/192.168.50.10@ZIM.COM nn1.zim.com@ZIM.COM nn2.zim.com/192.168.50.11@ZIM.COM nn2.zim.com@ZIM.COM host/nn1.zim.com@ZIM.COM host/nn2.zim.com@ZIM.COM

krb5.cfg:

[logging] default = FILE:/var/log/krb5libs.log
kdc = FILE:/var/log/krb5kdc.log
admin_server = FILE:/var/log/kadmind.log

[libdefaults] dns_lookup_realm = false
ticket_lifetime = 24h renew_lifetime = 7d
forwardable = true
rdns = false default_realm = ZIM.COM
default_ccache_name = KEYRING:persistent:%{uid}

[realms] ZIM.COM = {
kdc = kb.zim.com
admin_server = kb.zim.com
}

[domain_realm]
.zim.com = ZIM.COM
zim.com = ZIM.COM

解决方案：必须为每个主机创建两个 kerberos 主体：FQDN 和短。我只创建了 FQDN (nn1.zim.com) - 这就是问题的原因。创建第二个主体 (nn1) 后，一切都开始工作了。

当您使用 Active Directory 时，两种类型的主体都将在创建 AD 计算机对象时自动创建。

Hadoop 集群 kerberized 时无法访问 HDFS

Can't access HDFS when Hadoop cluster kerberized

hadoop

hdfs

kerberos

ambari