如何从 AWS 中的 kubernetes sa 访问 EFS？
How to access EFS from kubernetes sa in AWS?
我根据以下文档在 AWS 中部署了一个 EFS 并在 EKS 上部署了一个测试容器：Amazon EFS CSI driver。
EFS CSI 控制器 pods 在 kube-system:
kube-system efs-csi-controller-5bb76d96d8-b7qhk 3/3 Running 0 26s
kube-system efs-csi-controller-5bb76d96d8-hcgvc 3/3 Running 0 26s
从文档部署示例应用程序后，当确认 efs-csi-controller sa pod 日志时，它们似乎运行不正常。
连播机 1：
$ kubectl logs efs-csi-controller-5bb76d96d8-b7qhk \
> -n kube-system \
> -c csi-provisioner \
> --tail 10
W1030 08:15:59.073406 1 feature_gate.go:235] Setting GA feature gate Topology=true. It will be removed in a future release.
I1030 08:15:59.073485 1 feature_gate.go:243] feature gates: &{map[Topology:true]}
I1030 08:15:59.073500 1 csi-provisioner.go:132] Version: v2.1.1-0-g353098c90
I1030 08:15:59.073520 1 csi-provisioner.go:155] Building kube configs for running in cluster...
I1030 08:15:59.087072 1 connection.go:153] Connecting to unix:///var/lib/csi/sockets/pluginproxy/csi.sock
I1030 08:15:59.087512 1 common.go:111] Probing CSI driver for readiness
I1030 08:15:59.090672 1 csi-provisioner.go:202] Detected CSI driver efs.csi.aws.com
I1030 08:15:59.091694 1 csi-provisioner.go:244] CSI driver does not support PUBLISH_UNPUBLISH_VOLUME, not watching VolumeAttachments
I1030 08:15:59.091997 1 controller.go:756] Using saving PVs to API server in background
I1030 08:15:59.092834 1 leaderelection.go:243] attempting to acquire leader lease kube-system/efs-csi-aws-com...
连播机 2：
$ kubectl logs efs-csi-controller-5bb76d96d8-hcgvc \
> -n kube-system \
> -c csi-provisioner \
> --tail 10
I1030 08:16:32.628759 1 controller.go:1099] Final error received, removing PVC 111111a-d6fb-440a-9bb1-132901jfas from claims in progress
W1030 08:16:32.628783 1 controller.go:958] Retrying syncing claim "111111a-d6fb-440a-9bb1-132901jfas", failure 5
E1030 08:16:32.628798 1 controller.go:981] error syncing claim "111111a-d6fb-440a-9bb1-132901jfas": failed to provision volume with StorageClass "efs-sc": rpc error: code = Unauthenticated desc = Access Denied. Please ensure you have the right AWS permissions: Access denied
I1030 08:16:32.628845 1 event.go:282] Event(v1.ObjectReference{Kind:"PersistentVolumeClaim", Namespace:"default", Name:"efs-claim", UID:"111111a-d6fb-440a-9bb1-132901jfas", APIVersion:"v1", ResourceVersion:"1724705", FieldPath:""}): type: 'Warning' reason: 'ProvisioningFailed' failed to provision volume with StorageClass "efs-sc": rpc error: code = Unauthenticated desc = Access Denied. Please ensure you have the right AWS permissions: Access denied
I1030 08:17:04.628997 1 controller.go:1332] provision "default/efs-claim" class "efs-sc": started
I1030 08:17:04.629193 1 event.go:282] Event(v1.ObjectReference{Kind:"PersistentVolumeClaim", Namespace:"default", Name:"efs-claim", UID:"111111a-d6fb-440a-9bb1-132901jfas", APIVersion:"v1", ResourceVersion:"1724705", FieldPath:""}): type: 'Normal' reason: 'Provisioning' External provisioner is provisioning volume for claim "default/efs-claim"
I1030 08:17:04.687957 1 controller.go:1099] Final error received, removing PVC 111111a-d6fb-440a-9bb1-132901jfas from claims in progress
W1030 08:17:04.687977 1 controller.go:958] Retrying syncing claim "111111a-d6fb-440a-9bb1-132901jfas", failure 6
E1030 08:17:04.688001 1 controller.go:981] error syncing claim "111111a-d6fb-440a-9bb1-132901jfas": failed to provision volume with StorageClass "efs-sc": rpc error: code = Unauthenticated desc = Access Denied. Please ensure you have the right AWS permissions: Access denied
I1030 08:17:04.688044 1 event.go:282] Event(v1.ObjectReference{Kind:"PersistentVolumeClaim", Namespace:"default", Name:"efs-claim", UID:"111111a-d6fb-440a-9bb1-132901jfas", APIVersion:"v1", ResourceVersion:"1724705", FieldPath:""}): type: 'Warning' reason: 'ProvisioningFailed' failed to provision volume with StorageClass "efs-sc": rpc error: code = Unauthenticated desc = Access Denied. Please ensure you have the right AWS permissions: Access denied
从事件中可以看出：
$ kubectl get events
27m Warning FailedScheduling pod/efs-app skip schedule deleting pod: default/efs-app
7m38s Warning FailedScheduling pod/efs-app 0/2 nodes are available: 2 pod has unbound immediate PersistentVolumeClaims.
7m24s Warning FailedScheduling pod/efs-app 0/2 nodes are available: 2 persistentvolumeclaim "efs-claim" is being deleted.
7m24s Warning FailedScheduling pod/efs-app skip schedule deleting pod: default/efs-app
17s Warning FailedScheduling pod/efs-app 0/2 nodes are available: 2 pod has unbound immediate PersistentVolumeClaims.
27m Normal ExternalProvisioning persistentvolumeclaim/efs-claim waiting for a volume to be created, either by external provisioner "efs.csi.aws.com" or manually created by system administrator
10m Normal ExternalProvisioning persistentvolumeclaim/efs-claim waiting for a volume to be created, either by external provisioner "efs.csi.aws.com" or manually created by system administrator
11m Normal Provisioning persistentvolumeclaim/efs-claim External provisioner is provisioning volume for claim "default/efs-claim"
11m Warning ProvisioningFailed persistentvolumeclaim/efs-claim failed to provision volume with StorageClass "efs-sc": rpc error: code = Unauthenticated desc = Access Denied. Please ensure you have the right AWS permissions: Access denied
7m47s Normal Provisioning persistentvolumeclaim/efs-claim External provisioner is provisioning volume for claim "default/efs-claim"
7m47s Warning ProvisioningFailed persistentvolumeclaim/efs-claim failed to provision volume with StorageClass "efs-sc": rpc error: code = Unauthenticated desc = Access Denied. Please ensure you have the right AWS permissions: Access denied
74s Normal ExternalProvisioning persistentvolumeclaim/efs-claim waiting for a volume to be created, either by external provisioner "efs.csi.aws.com" or manually created by system administrator
2m56s Normal Provisioning persistentvolumeclaim/efs-claim External provisioner is provisioning volume for claim "default/efs-claim"
2m56s Warning ProvisioningFailed persistentvolumeclaim/efs-claim failed to provision volume with StorageClass "efs-sc": rpc error: code = Unauthenticated desc = Access Denied. Please ensure you have the right AWS permissions: Access denied
ServiceAccount 创建者：
---
apiVersion: v1
kind: ServiceAccount
metadata:
name: efs-csi-controller-sa
namespace: kube-system
labels:
app.kubernetes.io/name: aws-efs-csi-driver
annotations:
eks.amazonaws.com/role-arn: arn:aws:iam::111111111111:role/AmazonEKS_EFS_CSI_Driver_Policy
AmazonEKS_EFS_CSI_Driver_Policy 是 here 的 json。
示例代码
storageclass.yaml
kind: StorageClass
apiVersion: storage.k8s.io/v1
metadata:
name: efs-sc
provisioner: efs.csi.aws.com
parameters:
provisioningMode: efs-ap
fileSystemId: fs-92107410
directoryPerms: "700"
gidRangeStart: "1000" # optional
gidRangeEnd: "2000" # optional
basePath: "/dynamic_provisioning" # optional
pod.yaml
---
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
name: efs-claim
spec:
accessModes:
- ReadWriteMany
storageClassName: efs-sc
resources:
requests:
storage: 5Gi
---
apiVersion: v1
kind: Pod
metadata:
name: efs-app
spec:
containers:
- name: app
image: centos
command: ["/bin/sh"]
args: ["-c", "while true; do echo $(date -u) >> /data/out; sleep 5; done"]
volumeMounts:
- name: persistent-storage
mountPath: /data
volumes:
- name: persistent-storage
persistentVolumeClaim:
claimName: efs-claim
已发布社区 Wiki 答案以获得更好的可见性。随意扩展它。
基于@面田评论：
The reason was the efs driver image is using the different region from mine. I changed to the right one and it works.
您可以在 this documentation 中的适当区域找到设置 Amazon EFS CSI 驱动程序的步骤。
我根据以下文档在 AWS 中部署了一个 EFS 并在 EKS 上部署了一个测试容器：Amazon EFS CSI driver。
EFS CSI 控制器 pods 在 kube-system:
kube-system efs-csi-controller-5bb76d96d8-b7qhk 3/3 Running 0 26s
kube-system efs-csi-controller-5bb76d96d8-hcgvc 3/3 Running 0 26s
从文档部署示例应用程序后，当确认 efs-csi-controller sa pod 日志时，它们似乎运行不正常。
连播机 1：
$ kubectl logs efs-csi-controller-5bb76d96d8-b7qhk \
> -n kube-system \
> -c csi-provisioner \
> --tail 10
W1030 08:15:59.073406 1 feature_gate.go:235] Setting GA feature gate Topology=true. It will be removed in a future release.
I1030 08:15:59.073485 1 feature_gate.go:243] feature gates: &{map[Topology:true]}
I1030 08:15:59.073500 1 csi-provisioner.go:132] Version: v2.1.1-0-g353098c90
I1030 08:15:59.073520 1 csi-provisioner.go:155] Building kube configs for running in cluster...
I1030 08:15:59.087072 1 connection.go:153] Connecting to unix:///var/lib/csi/sockets/pluginproxy/csi.sock
I1030 08:15:59.087512 1 common.go:111] Probing CSI driver for readiness
I1030 08:15:59.090672 1 csi-provisioner.go:202] Detected CSI driver efs.csi.aws.com
I1030 08:15:59.091694 1 csi-provisioner.go:244] CSI driver does not support PUBLISH_UNPUBLISH_VOLUME, not watching VolumeAttachments
I1030 08:15:59.091997 1 controller.go:756] Using saving PVs to API server in background
I1030 08:15:59.092834 1 leaderelection.go:243] attempting to acquire leader lease kube-system/efs-csi-aws-com...
连播机 2：
$ kubectl logs efs-csi-controller-5bb76d96d8-hcgvc \
> -n kube-system \
> -c csi-provisioner \
> --tail 10
I1030 08:16:32.628759 1 controller.go:1099] Final error received, removing PVC 111111a-d6fb-440a-9bb1-132901jfas from claims in progress
W1030 08:16:32.628783 1 controller.go:958] Retrying syncing claim "111111a-d6fb-440a-9bb1-132901jfas", failure 5
E1030 08:16:32.628798 1 controller.go:981] error syncing claim "111111a-d6fb-440a-9bb1-132901jfas": failed to provision volume with StorageClass "efs-sc": rpc error: code = Unauthenticated desc = Access Denied. Please ensure you have the right AWS permissions: Access denied
I1030 08:16:32.628845 1 event.go:282] Event(v1.ObjectReference{Kind:"PersistentVolumeClaim", Namespace:"default", Name:"efs-claim", UID:"111111a-d6fb-440a-9bb1-132901jfas", APIVersion:"v1", ResourceVersion:"1724705", FieldPath:""}): type: 'Warning' reason: 'ProvisioningFailed' failed to provision volume with StorageClass "efs-sc": rpc error: code = Unauthenticated desc = Access Denied. Please ensure you have the right AWS permissions: Access denied
I1030 08:17:04.628997 1 controller.go:1332] provision "default/efs-claim" class "efs-sc": started
I1030 08:17:04.629193 1 event.go:282] Event(v1.ObjectReference{Kind:"PersistentVolumeClaim", Namespace:"default", Name:"efs-claim", UID:"111111a-d6fb-440a-9bb1-132901jfas", APIVersion:"v1", ResourceVersion:"1724705", FieldPath:""}): type: 'Normal' reason: 'Provisioning' External provisioner is provisioning volume for claim "default/efs-claim"
I1030 08:17:04.687957 1 controller.go:1099] Final error received, removing PVC 111111a-d6fb-440a-9bb1-132901jfas from claims in progress
W1030 08:17:04.687977 1 controller.go:958] Retrying syncing claim "111111a-d6fb-440a-9bb1-132901jfas", failure 6
E1030 08:17:04.688001 1 controller.go:981] error syncing claim "111111a-d6fb-440a-9bb1-132901jfas": failed to provision volume with StorageClass "efs-sc": rpc error: code = Unauthenticated desc = Access Denied. Please ensure you have the right AWS permissions: Access denied
I1030 08:17:04.688044 1 event.go:282] Event(v1.ObjectReference{Kind:"PersistentVolumeClaim", Namespace:"default", Name:"efs-claim", UID:"111111a-d6fb-440a-9bb1-132901jfas", APIVersion:"v1", ResourceVersion:"1724705", FieldPath:""}): type: 'Warning' reason: 'ProvisioningFailed' failed to provision volume with StorageClass "efs-sc": rpc error: code = Unauthenticated desc = Access Denied. Please ensure you have the right AWS permissions: Access denied
从事件中可以看出：
$ kubectl get events
27m Warning FailedScheduling pod/efs-app skip schedule deleting pod: default/efs-app
7m38s Warning FailedScheduling pod/efs-app 0/2 nodes are available: 2 pod has unbound immediate PersistentVolumeClaims.
7m24s Warning FailedScheduling pod/efs-app 0/2 nodes are available: 2 persistentvolumeclaim "efs-claim" is being deleted.
7m24s Warning FailedScheduling pod/efs-app skip schedule deleting pod: default/efs-app
17s Warning FailedScheduling pod/efs-app 0/2 nodes are available: 2 pod has unbound immediate PersistentVolumeClaims.
27m Normal ExternalProvisioning persistentvolumeclaim/efs-claim waiting for a volume to be created, either by external provisioner "efs.csi.aws.com" or manually created by system administrator
10m Normal ExternalProvisioning persistentvolumeclaim/efs-claim waiting for a volume to be created, either by external provisioner "efs.csi.aws.com" or manually created by system administrator
11m Normal Provisioning persistentvolumeclaim/efs-claim External provisioner is provisioning volume for claim "default/efs-claim"
11m Warning ProvisioningFailed persistentvolumeclaim/efs-claim failed to provision volume with StorageClass "efs-sc": rpc error: code = Unauthenticated desc = Access Denied. Please ensure you have the right AWS permissions: Access denied
7m47s Normal Provisioning persistentvolumeclaim/efs-claim External provisioner is provisioning volume for claim "default/efs-claim"
7m47s Warning ProvisioningFailed persistentvolumeclaim/efs-claim failed to provision volume with StorageClass "efs-sc": rpc error: code = Unauthenticated desc = Access Denied. Please ensure you have the right AWS permissions: Access denied
74s Normal ExternalProvisioning persistentvolumeclaim/efs-claim waiting for a volume to be created, either by external provisioner "efs.csi.aws.com" or manually created by system administrator
2m56s Normal Provisioning persistentvolumeclaim/efs-claim External provisioner is provisioning volume for claim "default/efs-claim"
2m56s Warning ProvisioningFailed persistentvolumeclaim/efs-claim failed to provision volume with StorageClass "efs-sc": rpc error: code = Unauthenticated desc = Access Denied. Please ensure you have the right AWS permissions: Access denied
ServiceAccount 创建者：
---
apiVersion: v1
kind: ServiceAccount
metadata:
name: efs-csi-controller-sa
namespace: kube-system
labels:
app.kubernetes.io/name: aws-efs-csi-driver
annotations:
eks.amazonaws.com/role-arn: arn:aws:iam::111111111111:role/AmazonEKS_EFS_CSI_Driver_Policy
AmazonEKS_EFS_CSI_Driver_Policy 是 here 的 json。
示例代码
storageclass.yaml
kind: StorageClass
apiVersion: storage.k8s.io/v1
metadata:
name: efs-sc
provisioner: efs.csi.aws.com
parameters:
provisioningMode: efs-ap
fileSystemId: fs-92107410
directoryPerms: "700"
gidRangeStart: "1000" # optional
gidRangeEnd: "2000" # optional
basePath: "/dynamic_provisioning" # optional
pod.yaml
---
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
name: efs-claim
spec:
accessModes:
- ReadWriteMany
storageClassName: efs-sc
resources:
requests:
storage: 5Gi
---
apiVersion: v1
kind: Pod
metadata:
name: efs-app
spec:
containers:
- name: app
image: centos
command: ["/bin/sh"]
args: ["-c", "while true; do echo $(date -u) >> /data/out; sleep 5; done"]
volumeMounts:
- name: persistent-storage
mountPath: /data
volumes:
- name: persistent-storage
persistentVolumeClaim:
claimName: efs-claim
已发布社区 Wiki 答案以获得更好的可见性。随意扩展它。
基于@面田评论：
The reason was the efs driver image is using the different region from mine. I changed to the right one and it works.
您可以在 this documentation 中的适当区域找到设置 Amazon EFS CSI 驱动程序的步骤。