缺少 AWS 卷，但 kubernetes 中有 "bound" PVC

Question

查看下面的输出。让我感到困惑的是，状态是绑定的，但 AWS 中不存在该卷。我正在使用 Kubernetes 1.17

我还检查过没有 POD 在使用这个 PVC（另外使用 https://github.com/yashbhutwala/kubectl-df-pv 来描述下面的命令）

任何想法 - 这怎么可能发生？如果卷是通过 AWS CLI（或 GUI web UI）手动删除的——这是否意味着 Kubernetes 没有正确处理这种情况？

 k get pvc -n metrics   
NAME                                   STATUS   VOLUME                                     CAPACITY   ACCESS MODES   STORAGECLASS   AGE
grafana-persistent-storage             Bound    pvc-1395291c-d89b-11e9-8a64-0a4976158cfe   1Gi        RWO            gp2            398d

➜ k describe pv pvc-1395291c-d89b-11e9-8a64-0a4976158cfe           
Name:              pvc-1395291c-d89b-11e9-8a64-0a4976158cfe
Labels:            failure-domain.beta.kubernetes.io/region=eu-central-1
                   failure-domain.beta.kubernetes.io/zone=eu-central-1c
Annotations:       kubernetes.io/createdby: aws-ebs-dynamic-provisioner
                   pv.kubernetes.io/bound-by-controller: yes
                   pv.kubernetes.io/provisioned-by: kubernetes.io/aws-ebs
Finalizers:        [kubernetes.io/pv-protection]
StorageClass:      gp2
Status:            Bound
Claim:             metrics/grafana-persistent-storage
Reclaim Policy:    Delete
Access Modes:      RWO
VolumeMode:        Filesystem
Capacity:          1Gi
Node Affinity:     
  Required Terms:  
    Term 0:        failure-domain.beta.kubernetes.io/zone in [eu-central-1c]
                   failure-domain.beta.kubernetes.io/region in [eu-central-1]
Message:           
Source:
    Type:       AWSElasticBlockStore (a Persistent Disk resource in AWS)
    VolumeID:   aws://eu-central-1c/vol-0b92b7db07b87b3e8
    FSType:     ext4
    Partition:  0
    ReadOnly:   false
Events:         <none>

➜ aws ec2 describe-volumes --volume-ids vol-0b92b7db07b87b3e8

An error occurred (InvalidVolume.NotFound) when calling the DescribeVolumes operation: The volume 'vol-0b92b7db07b87b3e8' does not exist.

➜ env | grep AWS              
AWS_ACCESS_KEY_ID=xxx
AWS_SECRET_ACCESS_KEY=yyy
AWS_DEFAULT_REGION=eu-central-1
AWS_DEFAULT_OUTPUT=table

➜ kubectl version  
'Tipz:' k version
Client Version: version.Info{Major:"1", Minor:"17", GitVersion:"v1.17.3", GitCommit:"06ad960bfd03b39c8310aaf92d1e7c12ce618213", GitTreeState:"clean", BuildDate:"2020-02-11T18:14:22Z", GoVersion:"go1.13.6", Compiler:"gc", Platform:"linux/amd64"}
Server Version: version.Info{Major:"1", Minor:"17+", GitVersion:"v1.17.9-eks-4c6976", GitCommit:"4c6976793196d70bc5cd29d56ce5440c9473648e", GitTreeState:"clean", BuildDate:"2020-07-17T18:46:04Z", GoVersion:"go1.13.9", Compiler:"gc", Platform:"linux/amd64"}

Answer 1

你应该使用 Amazon EBS CSI driver instead of in-tree Amazon EBS storage provisioner。通过创建存储 class

使用动态配置

kind: StorageClass
apiVersion: storage.k8s.io/v1
metadata:
  name: ebs-sc
provisioner: ebs.csi.aws.com
volumeBindingMode: WaitForFirstConsumer

Answer 2

很有可能从 AWS 控制台删除卷，而它们被 K8s 持久卷引用！但是，仅当 没有 Pod 挂载它们时。

如果 Pod 正在安装有问题的 PV 的 PVC，则无法从 AWS 控制台删除不可能，因为存储正在使用（附加）。

因此，换句话说，删除存储上仅存在 PVC 和 PV，不会导致 K8s 使这些资源失效。

缺少 AWS 卷，但 kubernetes 中有 "bound" PVC

Absent AWS volume but "bound" PVC in kubernetes

kubernetes

kubernetes-pvc

amazon-eks