共享 NFS 主目录上的无密码 ssh 不起作用 (centos 7)
Passwordless ssh on shared NFS home directory does not work (centos 7)
我有一个包含 7 个节点的集群（所有 Centos 7 OS）。主节点为maercher5，其余为从节点。我需要在主节点上设置无密码 ssh 到从节点到 运行 MPI 程序。
主目录由 NFS 从主节点共享到所有从节点。
我按照这个 tutorial 进行了从主节点到从节点的无密码 ssh。
我在所有机器上都有相同的 UID 和 GID。
由于所有节点上只有 1 个 ssh 文件夹共享。 ssh 文件夹的权限是：
$ ls -al $HOME/.ssh
total 28
drwx------. 2 sarah sarah 76 Apr 16 21:17 .
drwx------. 17 sarah sarah 4096 Apr 17 13:51 ..
-rw-------. 1 sarah sarah 11895 Apr 16 21:17 authorized_keys
-rw-------. 1 sarah sarah 1679 Apr 3 00:55 id_rsa
-rw-r--r--. 1 sarah sarah 411 Apr 10 14:24 id_rsa.pub
-rw-------. 1 sarah sarah 2265 Apr 10 13:58 known_hosts
节点之间可以ping通。 Marcher5为主节点。
[sarah@marcher5]$ cat /etc/hosts
127.0.0.1 localhost localhost.localdomain localhost4 localhost4.localdomain4
::1 localhost localhost.localdomain localhost6 localhost6.localdomain6
192.168.250.15 marcher5.cs.txstate.edu marcher5
192.168.250.17 marcher7.cs.txstate.edu marcher7
192.168.250.18 marcher8.cs.txstate.edu marcher8
192.168.250.19 marcher9.cs.txstate.edu marcher9
192.168.250.20 marcher10.cs.txstate.edu marcher10
192.168.250.21 marcher11.cs.txstate.edu marcher11
192.168.250.22 marcher12.cs.txstate.edu marcher12
在所有slave节点上，NFS Mount如下：
[sarah@marcher11 ~]$ cat /etc/fstab
/dev/mapper/centos-root / xfs defaults 1 1
UUID=79c2716b-9099-4731-82cc-094ca26eb837 /boot xfs defaults 1 2
#/dev/mapper/centos-home /home xfs defaults 1 2
/dev/mapper/centos-swap swap swap defaults 0 0
marcher5:/home/sge_users /home/sge_users nfs soft,intr,bg,nosuid,timeo=20,retrans=10,async,wsize=8192,rsize=8192 0 0
[sarah@marcher11 ~]$ mount |grep home
/dev/mapper/centos-home on /home type xfs (rw,relatime,seclabel,attr2,inode64,noquota)
marcher5:/home/sge_users on /home/sge_users type nfs4 (rw,nosuid,relatime,vers=4.0,rsize=8192,wsize=8192,namlen=255,soft,proto=tcp,port=0,timeo=20,retrans=10,sec=sys,clientaddr=192.168.250.21,local_lock=none,addr=192.168.250.15)
问题是无密码 ssh 不起作用。
[sarah@marcher5 mpi2007]$ ssh -v marcher11
OpenSSH_6.6.1, OpenSSL 1.0.1e-fips 11 Feb 2013
debug1: Reading configuration data /etc/ssh/ssh_config
debug1: /etc/ssh/ssh_config line 56: Applying options for *
debug1: Connecting to marcher11 [192.168.250.21] port 22.
debug1: Connection established.
debug1: identity file /home/sge_users/sarah/.ssh/id_rsa type 1
debug1: identity file /home/sge_users/sarah/.ssh/id_rsa-cert type -1
debug1: identity file /home/sge_users/sarah/.ssh/id_dsa type -1
debug1: identity file /home/sge_users/sarah/.ssh/id_dsa-cert type -1
debug1: identity file /home/sge_users/sarah/.ssh/id_ecdsa type -1
debug1: identity file /home/sge_users/sarah/.ssh/id_ecdsa-cert type -1 [29/1894]
debug1: identity file /home/sge_users/sarah/.ssh/id_ed25519 type -1
debug1: identity file /home/sge_users/sarah/.ssh/id_ed25519-cert type -1
debug1: Enabling compatibility mode for protocol 2.0
debug1: Local version string SSH-2.0-OpenSSH_6.6.1
debug1: Remote protocol version 2.0, remote software version OpenSSH_6.6.1
debug1: match: OpenSSH_6.6.1 pat OpenSSH_6.6.1* compat 0x04000000
debug1: SSH2_MSG_KEXINIT sent
debug1: SSH2_MSG_KEXINIT received
debug1: kex: server->client aes128-ctr hmac-md5-etm@openssh.com none
debug1: kex: client->server aes128-ctr hmac-md5-etm@openssh.com none
debug1: kex: curve25519-sha256@libssh.org need=16 dh_need=16
debug1: kex: curve25519-sha256@libssh.org need=16 dh_need=16
debug1: sending SSH2_MSG_KEX_ECDH_INIT
debug1: expecting SSH2_MSG_KEX_ECDH_REPLY
debug1: Server host key: ECDSA 80:81:97:62:dd:9b:fc:e2:76:bc:13:ce:30:07:79:49
debug1: Host 'marcher11' is known and matches the ECDSA host key.
debug1: Found key in /home/sge_users/sarah/.ssh/known_hosts:5
debug1: ssh_ecdsa_verify: signature correct
debug1: SSH2_MSG_NEWKEYS sent
debug1: expecting SSH2_MSG_NEWKEYS
debug1: SSH2_MSG_NEWKEYS received
debug1: Roaming not allowed by server
debug1: SSH2_MSG_SERVICE_REQUEST sent
debug1: SSH2_MSG_SERVICE_ACCEPT received
debug1: Authentications that can continue: publickey,gssapi-keyex,gssapi-with-mic,password
debug1: Next authentication method: gssapi-keyex
debug1: No valid Key exchange context
debug1: Next authentication method: gssapi-with-mic
debug1: Unspecified GSS failure. Minor code may provide more information
debug1: Unspecified GSS failure. Minor code may provide more information
No Kerberos credentials available
debug1: Unspecified GSS failure. Minor code may provide more information
No Kerberos credentials available
debug1: Unspecified GSS failure. Minor code may provide more information
debug1: Unspecified GSS failure. Minor code may provide more information
No Kerberos credentials available
debug1: Next authentication method: publickey
debug1: Offering RSA public key: /home/sge_users/sarah/.ssh/id_rsa
debug1: Authentications that can continue: publickey,gssapi-keyex,gssapi-with-mic,password
debug1: Trying private key: /home/sge_users/sarah/.ssh/id_dsa
debug1: Trying private key: /home/sge_users/sarah/.ssh/id_ecdsa
debug1: Trying private key: /home/sge_users/sarah/.ssh/id_ed25519
debug1: Next authentication method: password
sarah@marcher11's password:
debug1: Authentication succeeded (password).
Authenticated to marcher11 ([192.168.250.21]:22).
debug1: channel 0: new [client-session]
debug1: Requesting no-more-sessions@openssh.com
debug1: Entering interactive session.
debug1: Sending environment.
debug1: Sending env LC_ALL = C
debug1: Sending env LANG = en_US.UTF-8
我已经被这个问题关了一个多月了，如果有任何帮助，我们将不胜感激。我尝试从 root@master 到 root@slave 执行此操作并且它有效。
我正在检查主节点上的 /var/log/messages，但那里什么也没有。但是当我在从属节点上检查它时，我发现了错误。
在/var/log/messages中：
Apr 17 23:32:00 marcher9 python: SELinux is preventing /usr/sbin/sshd from read access on the file authorized_keys.
***** Plugin catchall_boolean (89.3 confidence) suggests ******************
If you want to allow use to nfs home dirs
Then you must tell SELinux about this by enabling the 'use_nfs_home_dirs' boolean.
You can read 'None' man page for more details.
Do
setsebool -P use_nfs_home_dirs 1
***** Plugin catchall (11.6 confidence) suggests **************************
If you believe that sshd should be allowed read access on the authorized_keys file by default.
Then you should report this as a bug.
You can generate a local policy module to allow this access.
Do
allow this access for now by executing:
# grep sshd /var/log/audit/audit.log | audit2allow -M mypol
# semodule -i mypol.pp
原来我只需要 运行 'setsebool -P use_nfs_home_dirs 1' 作为 root。然后一切都像一个魅力。谢谢@user_ABCD
我有一个包含 7 个节点的集群（所有 Centos 7 OS）。主节点为maercher5，其余为从节点。我需要在主节点上设置无密码 ssh 到从节点到 运行 MPI 程序。 主目录由 NFS 从主节点共享到所有从节点。 我按照这个 tutorial 进行了从主节点到从节点的无密码 ssh。 我在所有机器上都有相同的 UID 和 GID。 由于所有节点上只有 1 个 ssh 文件夹共享。 ssh 文件夹的权限是：
$ ls -al $HOME/.ssh
total 28
drwx------. 2 sarah sarah 76 Apr 16 21:17 .
drwx------. 17 sarah sarah 4096 Apr 17 13:51 ..
-rw-------. 1 sarah sarah 11895 Apr 16 21:17 authorized_keys
-rw-------. 1 sarah sarah 1679 Apr 3 00:55 id_rsa
-rw-r--r--. 1 sarah sarah 411 Apr 10 14:24 id_rsa.pub
-rw-------. 1 sarah sarah 2265 Apr 10 13:58 known_hosts
节点之间可以ping通。 Marcher5为主节点。
[sarah@marcher5]$ cat /etc/hosts
127.0.0.1 localhost localhost.localdomain localhost4 localhost4.localdomain4
::1 localhost localhost.localdomain localhost6 localhost6.localdomain6
192.168.250.15 marcher5.cs.txstate.edu marcher5
192.168.250.17 marcher7.cs.txstate.edu marcher7
192.168.250.18 marcher8.cs.txstate.edu marcher8
192.168.250.19 marcher9.cs.txstate.edu marcher9
192.168.250.20 marcher10.cs.txstate.edu marcher10
192.168.250.21 marcher11.cs.txstate.edu marcher11
192.168.250.22 marcher12.cs.txstate.edu marcher12
在所有slave节点上，NFS Mount如下：
[sarah@marcher11 ~]$ cat /etc/fstab
/dev/mapper/centos-root / xfs defaults 1 1
UUID=79c2716b-9099-4731-82cc-094ca26eb837 /boot xfs defaults 1 2
#/dev/mapper/centos-home /home xfs defaults 1 2
/dev/mapper/centos-swap swap swap defaults 0 0
marcher5:/home/sge_users /home/sge_users nfs soft,intr,bg,nosuid,timeo=20,retrans=10,async,wsize=8192,rsize=8192 0 0
[sarah@marcher11 ~]$ mount |grep home
/dev/mapper/centos-home on /home type xfs (rw,relatime,seclabel,attr2,inode64,noquota)
marcher5:/home/sge_users on /home/sge_users type nfs4 (rw,nosuid,relatime,vers=4.0,rsize=8192,wsize=8192,namlen=255,soft,proto=tcp,port=0,timeo=20,retrans=10,sec=sys,clientaddr=192.168.250.21,local_lock=none,addr=192.168.250.15)
问题是无密码 ssh 不起作用。
[sarah@marcher5 mpi2007]$ ssh -v marcher11
OpenSSH_6.6.1, OpenSSL 1.0.1e-fips 11 Feb 2013
debug1: Reading configuration data /etc/ssh/ssh_config
debug1: /etc/ssh/ssh_config line 56: Applying options for *
debug1: Connecting to marcher11 [192.168.250.21] port 22.
debug1: Connection established.
debug1: identity file /home/sge_users/sarah/.ssh/id_rsa type 1
debug1: identity file /home/sge_users/sarah/.ssh/id_rsa-cert type -1
debug1: identity file /home/sge_users/sarah/.ssh/id_dsa type -1
debug1: identity file /home/sge_users/sarah/.ssh/id_dsa-cert type -1
debug1: identity file /home/sge_users/sarah/.ssh/id_ecdsa type -1
debug1: identity file /home/sge_users/sarah/.ssh/id_ecdsa-cert type -1 [29/1894]
debug1: identity file /home/sge_users/sarah/.ssh/id_ed25519 type -1
debug1: identity file /home/sge_users/sarah/.ssh/id_ed25519-cert type -1
debug1: Enabling compatibility mode for protocol 2.0
debug1: Local version string SSH-2.0-OpenSSH_6.6.1
debug1: Remote protocol version 2.0, remote software version OpenSSH_6.6.1
debug1: match: OpenSSH_6.6.1 pat OpenSSH_6.6.1* compat 0x04000000
debug1: SSH2_MSG_KEXINIT sent
debug1: SSH2_MSG_KEXINIT received
debug1: kex: server->client aes128-ctr hmac-md5-etm@openssh.com none
debug1: kex: client->server aes128-ctr hmac-md5-etm@openssh.com none
debug1: kex: curve25519-sha256@libssh.org need=16 dh_need=16
debug1: kex: curve25519-sha256@libssh.org need=16 dh_need=16
debug1: sending SSH2_MSG_KEX_ECDH_INIT
debug1: expecting SSH2_MSG_KEX_ECDH_REPLY
debug1: Server host key: ECDSA 80:81:97:62:dd:9b:fc:e2:76:bc:13:ce:30:07:79:49
debug1: Host 'marcher11' is known and matches the ECDSA host key.
debug1: Found key in /home/sge_users/sarah/.ssh/known_hosts:5
debug1: ssh_ecdsa_verify: signature correct
debug1: SSH2_MSG_NEWKEYS sent
debug1: expecting SSH2_MSG_NEWKEYS
debug1: SSH2_MSG_NEWKEYS received
debug1: Roaming not allowed by server
debug1: SSH2_MSG_SERVICE_REQUEST sent
debug1: SSH2_MSG_SERVICE_ACCEPT received
debug1: Authentications that can continue: publickey,gssapi-keyex,gssapi-with-mic,password
debug1: Next authentication method: gssapi-keyex
debug1: No valid Key exchange context
debug1: Next authentication method: gssapi-with-mic
debug1: Unspecified GSS failure. Minor code may provide more information
debug1: Unspecified GSS failure. Minor code may provide more information
No Kerberos credentials available
debug1: Unspecified GSS failure. Minor code may provide more information
No Kerberos credentials available
debug1: Unspecified GSS failure. Minor code may provide more information
debug1: Unspecified GSS failure. Minor code may provide more information
No Kerberos credentials available
debug1: Next authentication method: publickey
debug1: Offering RSA public key: /home/sge_users/sarah/.ssh/id_rsa
debug1: Authentications that can continue: publickey,gssapi-keyex,gssapi-with-mic,password
debug1: Trying private key: /home/sge_users/sarah/.ssh/id_dsa
debug1: Trying private key: /home/sge_users/sarah/.ssh/id_ecdsa
debug1: Trying private key: /home/sge_users/sarah/.ssh/id_ed25519
debug1: Next authentication method: password
sarah@marcher11's password:
debug1: Authentication succeeded (password).
Authenticated to marcher11 ([192.168.250.21]:22).
debug1: channel 0: new [client-session]
debug1: Requesting no-more-sessions@openssh.com
debug1: Entering interactive session.
debug1: Sending environment.
debug1: Sending env LC_ALL = C
debug1: Sending env LANG = en_US.UTF-8
我已经被这个问题关了一个多月了，如果有任何帮助，我们将不胜感激。我尝试从 root@master 到 root@slave 执行此操作并且它有效。
我正在检查主节点上的 /var/log/messages，但那里什么也没有。但是当我在从属节点上检查它时，我发现了错误。
在/var/log/messages中：
Apr 17 23:32:00 marcher9 python: SELinux is preventing /usr/sbin/sshd from read access on the file authorized_keys.
***** Plugin catchall_boolean (89.3 confidence) suggests ******************
If you want to allow use to nfs home dirs
Then you must tell SELinux about this by enabling the 'use_nfs_home_dirs' boolean.
You can read 'None' man page for more details.
Do
setsebool -P use_nfs_home_dirs 1
***** Plugin catchall (11.6 confidence) suggests **************************
If you believe that sshd should be allowed read access on the authorized_keys file by default.
Then you should report this as a bug.
You can generate a local policy module to allow this access.
Do
allow this access for now by executing:
# grep sshd /var/log/audit/audit.log | audit2allow -M mypol
# semodule -i mypol.pp
原来我只需要 运行 'setsebool -P use_nfs_home_dirs 1' 作为 root。然后一切都像一个魅力。谢谢@user_ABCD