从 VolumeSnapshot 恢复的 Kubernetes PVC 为空

Question

我正在尝试使用在 kubernetes 中从 1.17 升级到 beta 的 VolumeSnapshot 备份 mechanism。

这是我的场景：

创建 nginx 部署及其使用的 PVC

apiVersion: apps/v1
kind: Deployment
metadata:
  name: nginx-deployment
spec:
  selector:
    matchLabels:
      app: nginx
  replicas: 1
  template:
    metadata:
      labels:
        app: nginx
    spec:
      containers:
      - name: nginx
        image: nginx:1.14.2
        ports:
        - containerPort: 80
        volumeMounts:
        - name: my-pvc
          mountPath: /root/test
      volumes:
        - name: my-pvc
          persistentVolumeClaim:
            claimName: nginx-pvc

apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  finalizers: null
  labels:
    name: nginx-pvc
  name: nginx-pvc
  namespace: default
spec:
  accessModes:
  - ReadWriteOnce
  resources:
    requests:
      storage: 8Gi
  storageClassName: premium-rwo

执行进入运行 nginx容器，cd进入PVC挂载路径并创建一些文件

▶ k exec -it nginx-deployment-84765795c-7hz5n bash
root@nginx-deployment-84765795c-7hz5n:/# cd /root/test
root@nginx-deployment-84765795c-7hz5n:~/test# touch {1..10}.txt
root@nginx-deployment-84765795c-7hz5n:~/test# ls
1.txt  10.txt  2.txt  3.txt  4.txt  5.txt  6.txt  7.txt  8.txt  9.txt  lost+found
root@nginx-deployment-84765795c-7hz5n:~/test#

使用 nginx-pvc

作为源创建以下 VolumeSnapshot

apiVersion: snapshot.storage.k8s.io/v1beta1
kind: VolumeSnapshot
metadata:
  namespace: default
  name: nginx-volume-snapshot
spec:
  volumeSnapshotClassName: pd-retain-vsc
  source:
    persistentVolumeClaimName: nginx-pvc

使用的VolumeSnapshotClass是下面的

apiVersion: snapshot.storage.k8s.io/v1beta1
deletionPolicy: Retain
driver: pd.csi.storage.gke.io
kind: VolumeSnapshotClass
metadata:
  creationTimestamp: "2020-09-25T09:10:16Z"
  generation: 1
  name: pd-retain-vsc

等到它变成readyToUse: true

apiVersion: v1
items:
- apiVersion: snapshot.storage.k8s.io/v1beta1
  kind: VolumeSnapshot
  metadata:
    creationTimestamp: "2020-11-04T09:38:00Z"
    finalizers:
    - snapshot.storage.kubernetes.io/volumesnapshot-as-source-protection
    generation: 1
    name: nginx-volume-snapshot
    namespace: default
    resourceVersion: "34170857"
    selfLink: /apis/snapshot.storage.k8s.io/v1beta1/namespaces/default/volumesnapshots/nginx-volume-snapshot
    uid: ce1991f8-a44c-456f-8b2a-2e12f8df28fc
  spec:
    source:
      persistentVolumeClaimName: nginx-pvc
    volumeSnapshotClassName: pd-retain-vsc
  status:
    boundVolumeSnapshotContentName: snapcontent-ce1991f8-a44c-456f-8b2a-2e12f8df28fc
    creationTime: "2020-11-04T09:38:02Z"
    readyToUse: true
    restoreSize: 8Gi
kind: List
metadata:
  resourceVersion: ""
  selfLink: ""

删除 nginx 部署和初始 PVC

▶ k delete pvc,deploy --all
persistentvolumeclaim "nginx-pvc" deleted
deployment.apps "nginx-deployment" deleted

创建一个新的 PVC，使用之前创建的 VolumeSnapshot 作为它的 dataSource

apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  finalizers: null
  labels:
    name: nginx-pvc-restored
  name: nginx-pvc-restored
  namespace: default
spec:
  accessModes:
  - ReadWriteOnce
  resources:
    requests:
      storage: 8Gi
  dataSource:
    name: nginx-volume-snapshot
    kind: VolumeSnapshot
    apiGroup: snapshot.storage.k8s.io

▶ k create -f nginx-pvc-restored.yaml
persistentvolumeclaim/nginx-pvc-restored created

▶ k get pvc
NAME                 STATUS   VOLUME                                     CAPACITY   ACCESS MODES   STORAGECLASS   AGE
nginx-pvc-restored   Bound    pvc-56d0a898-9f65-464f-8abf-90fa0a58a048   8Gi        RWO            standard       39s

将新的（恢复的）PVC 的名称设置为 nginx 部署

apiVersion: apps/v1
kind: Deployment
metadata:
  name: nginx-deployment
spec:
  selector:
    matchLabels:
      app: nginx
  replicas: 1
  template:
    metadata:
      labels:
        app: nginx
    spec:
      containers:
      - name: nginx
        image: nginx:1.14.2
        ports:
        - containerPort: 80
        volumeMounts:
        - name: my-pvc
          mountPath: /root/test
      volumes:
        - name: my-pvc
          persistentVolumeClaim:
            claimName: nginx-pvc-restored

并再次创建 Deployment

▶ k create -f nginx-deployment-restored.yaml
deployment.apps/nginx-deployment created

cd进入PVC挂载目录。它应该包含以前创建的文件，但它是空的

▶ k exec -it nginx-deployment-67c7584d4b-l7qrq bash
root@nginx-deployment-67c7584d4b-l7qrq:/# cd /root/test
root@nginx-deployment-67c7584d4b-l7qrq:~/test# ls
lost+found
root@nginx-deployment-67c7584d4b-l7qrq:~/test#

▶ k version
Client Version: version.Info{Major:"1", Minor:"17", GitVersion:"v1.17.12", GitCommit:"5ec472285121eb6c451e515bc0a7201413872fa3", GitTreeState:"clean", BuildDate:"2020-09-16T13:39:51Z", GoVersion:"go1.13.15", Compiler:"gc", Platform:"darwin/amd64"}
Server Version: version.Info{Major:"1", Minor:"17+", GitVersion:"v1.17.12-gke.1504", GitCommit:"17061f5bd4ee34f72c9281d49f94b4f3ac31ac25", GitTreeState:"clean", BuildDate:"2020-10-19T17:00:22Z", GoVersion:"go1.13.15b4", Compiler:"gc", Platform:"linux/amd64"}

Answer 1

这是为了更清楚地说明当前问题而发布的社区 Wiki 答案。随意扩展它。

如@pkaramol 所述，这是一个 on-going 在以下线程下注册的问题：

Creating an intree PVC with datasource should fail #96225

What happened: In clusters that have intree drivers as the default storageclass, if you try to create a PVC with snapshot data source and forget to put the csi storageclass in it, then an empty PVC will be provisioned using the default storageclass.

What you expected to happen: PVC creation should not proceed and instead have an event with an incompatible error, similar to how we check proper csi driver in the csi provisioner.

在撰写此回答时，此问题尚未解决。

从 VolumeSnapshot 恢复的 Kubernetes PVC 为空

Kubernetes PVC restored from VolumeSnapshot is empty

volume

kubernetes

persistent-volume-claims