从 EC2 实例安装 s3 存储桶时如何使 S3FS 使用 IMDS v2

Question

我们无法在需要 IMDS v2 的 AWS EC2 实例上安装 S3 存储桶（尽管 IMDS v1 工作正常！）。我们使用的是 s3fs-fuse v1.90，它应该支持 IMDS v2（在 Ubuntu 18.04 上）。

这就是我们尝试装载 S3 存储桶的方式：

sudo s3fs <bucket> <mount-point> -o iam_role=auto,url=https://s3.eu-central-1.amazonaws.com,endpoint=eu-central-1,allow_other,uid=1000,gid=1000,mp_umask=007,use_cache=/tmp/s3foldercache,dbglevel=debug -f

但是，我们收到此错误消息：

2021-09-02T12:40:53.157Z [INF]       curl.cpp:LoadIAMRoleFromMetaData(2919): Get IAM Role name
2021-09-02T12:40:53.157Z [DBG] curl_handlerpool.cpp:GetHandler(81): Get handler from pool: rest = 31
2021-09-02T12:40:53.158Z [DBG] curl.cpp:RequestPerform(2283): connecting to URL http://169.254.169.254/latest/meta-data/iam/security-credentials/
2021-09-02T12:40:53.159Z [ERR] curl.cpp:RequestPerform(2394): HTTP response code 401, returning EIO. Body Text: <?xml version="1.0" encoding="iso-8859-1"?>
<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN"
        "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
<html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en" lang="en">
 <head>
  <title>401 - Unauthorized</title>
 </head>
 <body>
  <h1>401 - Unauthorized</h1>
 </body>
</html>

2021-09-02T12:40:53.160Z [CRT] s3fs.cpp:s3fs_init(3395): could not load IAM role name from meta data.
2021-09-02T12:40:53.160Z [ERR] s3fs.cpp:s3fs_exit_fuseloop(3372): Exiting FUSE event loop due to errors

同样，当我们在 EC2 实例上允许 IMDS v1 时，mount 命令工作正常。

我们如何确保 s3fs 将使用 IMDS v2？

编辑：非常感谢@Noah 为 s3fs 贡献了一个修复程序。我已经提取了最新的提交，但现在看到了一条新的错误消息。显然 IMDSv2 令牌检索失败，无论我是否应用 iam_role=auto 或 iam_role=some_profile（删除 -f 也不会改变任何东西）：

2021-09-06T05:43:58.857Z [INF] s3fs.cpp:s3fs_init(3382): init v1.90(commit:9d1552a) with OpenSSL
2021-09-06T05:43:58.857Z [INF]       curl.cpp:LoadIAMRoleFromMetaData(3181): Get IAM Role name
2021-09-06T05:43:58.857Z [DBG] curl_handlerpool.cpp:GetHandler(81): Get handler from pool: rest = 31
2021-09-06T05:43:58.857Z [DBG] curl.cpp:RequestPerform(2509): connecting to URL http://169.254.169.254/latest/api/token
2021-09-06T05:43:58.889Z [ERR] curl.cpp:RequestPerform(2622): HTTP response code 417, returning EIO. Body Text: <?xml version="1.0" encoding="iso-8859-1"?>
<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN"
         "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
<html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en" lang="en">
 <head>
  <title>417 - Expectation Failed</title>
 </head>
 <body>
  <h1>417 - Expectation Failed</h1>
 </body>
</html>
2021-09-06T05:43:58.890Z [ERR] curl.cpp:LoadIAMRoleFromMetaData(3212): AWS IMDSv2 token retrieval failed: -5
2021-09-06T05:43:58.890Z [DBG] curl.cpp:RequestPerform(2509): connecting to URL http://169.254.169.254/latest/meta-data/iam/security-credentials/
2021-09-06T05:43:58.892Z [ERR] curl.cpp:RequestPerform(2622): HTTP response code 401, returning EIO. Body Text: <?xml version="1.0" encoding="iso-8859-1"?>
<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN"
        "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
<html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en" lang="en">
 <head>
  <title>401 - Unauthorized</title>
 </head>
 <body>
  <h1>401 - Unauthorized</h1>
 </body>
</html>
2021-09-06T05:43:58.892Z [CRT] s3fs.cpp:s3fs_init(3395): could not load IAM role name from meta data.
2021-09-06T05:43:58.892Z [ERR] s3fs.cpp:s3fs_exit_fuseloop(3372): Exiting FUSE event loop due to errors

Answer 1

如果您明确指定 IAM 角色是否有效？

当需要令牌时，检索 IAM 角色信息（iam_role=auto 需要）似乎可能存在问题。

此外，在测试这个时，我在使用 -f 选项时遇到问题，所以也尝试删除它。

以下日志显示使用以下命令安装存储桶时正确的令牌检索。通过挂载点的存储桶访问按预期工作。

$ sudo s3fs <REDACTED> /mnt -o iam_role=s3fs-testing,allow_other,uid=1000,gid=1000,mp_umask=007,use_cache=/tmp/s3foldercache,dbglevel=debug

Sep 02 16:06:46 ip-10-0-0-77 s3fs[600]: init v1.89(commit:unknown) with GnuTLS(gcrypt)
Sep 02 16:06:46 ip-10-0-0-77 s3fs[600]: check services.
Sep 02 16:06:46 ip-10-0-0-77 s3fs[600]: IAM Access Token refreshing...
Sep 02 16:06:46 ip-10-0-0-77 s3fs[600]:       [IAM role=s3fs-testing]
Sep 02 16:06:46 ip-10-0-0-77 s3fs[600]: curl_handlerpool.cpp:GetHandler(81): Get handler from pool: rest = 31
Sep 02 16:06:46 ip-10-0-0-77 s3fs[600]: curl.cpp:RequestPerform(2254): connecting to URL http://169.254.169.254/latest/api/token
Sep 02 16:06:46 ip-10-0-0-77 s3fs[600]:       HTTP response code 200
Sep 02 16:06:46 ip-10-0-0-77 s3fs[600]:       Setting AWS IMDSv2 API token to <REDACTED>
Sep 02 16:06:46 ip-10-0-0-77 s3fs[600]: curl.cpp:RequestPerform(2254): connecting to URL http://169.254.169.254/latest/meta-data/iam/security-credentials/s3fs-testing
Sep 02 16:06:46 ip-10-0-0-77 s3fs[600]:       HTTP response code 200
Sep 02 16:06:46 ip-10-0-0-77 s3fs[600]:       IAM credential response = "{
                                          "Code" : "Success",
                                          "LastUpdated" : "2021-09-02T15:36:20Z",
                                          "Type" : "AWS-HMAC",
                                          "AccessKeyId" : "REDACTED",
                                          "SecretAccessKey" : "REDACTED",
                                          "Token" : "REDACTED",
                                          "Expiration" : "2021-09-02T22:10:48Z"
                                        }"
Sep 02 16:06:46 ip-10-0-0-77 s3fs[600]: IAM Access Token refreshed

从 EC2 实例安装 s3 存储桶时如何使 S3FS 使用 IMDS v2

How to make S3FS use IMDS v2 when mounting s3 buckets from EC2 instance

ubuntu

amazon-s3

amazon-ec2

s3fs