在 Kubernetes pod 中安装 vSphere 存储不起作用

Question

我正在尝试在 pod 中安装 vSphere 卷，但我不断收到：

vsphere_volume_util.go:123] Cloud provider not initialized properly

/etc/kubernetes/environment/vsphere.conf

[Global]
    user="xxxxxx"
    password="xxxxxx"
    server="xxxxxx"
    port="443"
    insecure-flag="1"
    datacenter="Frankfurt"
    datastore="dfrclupoc01-001"
    #working-dir="dockvols"
[Disk]
    scsicontrollertype=pvscsi

在 "vmWare vSphere Web Client" 我看到：

<mltdfrd01.xx.com>
  <Frankfurt>
    <dfrclupoc01-001>

在那个商店下面我有一个文件夹 "dockvols"，其中有一个子目录“11111111-1111-1111-1111-111111111111”。

apiVersion: v1
kind: PersistentVolume
metadata:
  name: pv0001
spec:
  capacity:
    storage: 10Gi
  accessModes:
    - ReadWriteOnce
  persistentVolumeReclaimPolicy: Retain
  vsphereVolume:
    volumePath: "[Frankfurt/dfrclupoc01-001] dockvols/11111111-1111-1111-1111-111111111111/MyVolume.vmdk"
    fsType: ext4

kind: PersistentVolumeClaim
apiVersion: v1
metadata:
  name: pvcmilo1
spec:
  accessModes:
    - ReadWriteOnce
  resources:
    requests:
      storage: 10Gi

apiVersion: v1
kind: Pod
metadata:
  name: pod0001
spec:
  containers:
  - image: busybox
    name: pod0001
    volumeMounts:
    - mountPath: /data
      name: pod-volume
  volumes:
  - name: pod-volume
    persistentVolumeClaim:
      claimName: pvcmilo1

我尝试了不同的卷路径，但我认为问题出在过程的早期。

从我创建 pod 开始的节点日志：

I0602 05:43:20.781563   84854 operation_executor.go:917] MountVolume.SetUp succeeded for volume "kubernetes.io/secret/224c6b51-24fc-11e7-adcd-005056890fa6-default-token-j6vgj" (spec.Name: "default-                 token-j6vgj") pod "224c6b51-24fc-11e7-adcd-005056890fa6" (UID: "224c6b51-24fc-11e7-adcd-005056890fa6").
I0602 05:43:24.279729   84854 kubelet.go:1781] SyncLoop (ADD, "api"): "pod0001_default(ebe97189-4777-11e7-8979-005056890fa6)"
E0602 05:43:24.378657   84854 vsphere_volume_util.go:123] Cloud provider not initialized properly
I0602 05:43:24.382952   84854 reconciler.go:230] VerifyControllerAttachedVolume operation started for volume "kubernetes.io/vsphere-volume/ebe97189-4777-11e7-8979-005056890fa6-pv0001" (spec.Name: "                 pv0001") pod "ebe97189-4777-11e7-8979-005056890fa6" (UID: "ebe97189-4777-11e7-8979-005056890fa6")
I0602 05:43:24.382985   84854 reconciler.go:230] VerifyControllerAttachedVolume operation started for volume "kubernetes.io/secret/ebe97189-4777-11e7-8979-005056890fa6-default-token-zsrfn" (spec.Na                 me: "default-token-zsrfn") pod "ebe97189-4777-11e7-8979-005056890fa6" (UID: "ebe97189-4777-11e7-8979-005056890fa6")
I0602 05:43:24.483237   84854 reconciler.go:306] MountVolume operation started for volume "kubernetes.io/secret/ebe97189-4777-11e7-8979-005056890fa6-default-token-zsrfn" (spec.Name: "default-token-                 zsrfn") to pod "ebe97189-4777-11e7-8979-005056890fa6" (UID: "ebe97189-4777-11e7-8979-005056890fa6").
E0602 05:43:24.483265   84854 vsphere_volume_util.go:123] Cloud provider not initialized properly
I0602 05:43:24.483296   84854 reconciler.go:306] MountVolume operation started for volume "kubernetes.io/vsphere-volume/ebe97189-4777-11e7-8979-005056890fa6-pv0001" (spec.Name: "pv0001") to pod "eb                 e97189-4777-11e7-8979-005056890fa6" (UID: "ebe97189-4777-11e7-8979-005056890fa6").
E0602 05:43:24.492507   84854 mount_linux.go:119] Mount failed: exit status 32
Mounting command: mount
Mounting arguments: /var/lib/kubelet/plugins/kubernetes.io/vsphere-volume/mounts/[dfrclupoc01-001] dockvols/11111111-1111-1111-1111-111111111111/MyVolume.vmdk /var/lib/kubelet/pods/ebe97189-4777-11                 e7-8979-005056890fa6/volumes/kubernetes.io~vsphere-volume/pv0001  [bind]
Output: mount: special device /var/lib/kubelet/plugins/kubernetes.io/vsphere-volume/mounts/[dfrclupoc01-001] dockvols/11111111-1111-1111-1111-111111111111/MyVolume.vmdk does not exist

Kubernete 版本：1.5.2

感谢您的帮助，米洛

Answer 1

好像漏掉了很多细节:

不仅 kubelet 需要 cloudconfig，还需要 api-server 和 controller-manager
wwn 磁盘 by-id 缺失。我必须通过将 disk.EnableUUID 设置为 TRUE
删除工作目录条目。似乎让 kubelet 崩溃了...
一些我忘了的细节

详情见https://vanderzee.org/linux/article-170620-144221。

在 Kubernetes pod 中安装 vSphere 存储不起作用

mounting vSphere store in Kubernetes pod not working

vsphere

kubernetes