Digital Ocean 管理的 Kubernetes 卷处于挂起状态

Question

它不是数字海洋特有的，如果能验证这是否是预期的行为会非常好。

我正在尝试使用来自 ElasticSearch itself

的 helm chart 在 DO 管理的 Kubernetes 集群上设置 ElasticSearch 集群

他们说我需要在 volumeClaimTemplate 中指定 storageClassName 才能使用托管 kubernetes 服务提供的卷。对于 DO，根据他们的 docs，它是 do-block-storages。似乎也没有必要定义 PVC，helm chart 应该自己定义。

这是我正在使用的配置

# Specify node pool
nodeSelector:
    doks.digitalocean.com/node-pool: elasticsearch

# Shrink default JVM heap.
esJavaOpts: "-Xmx128m -Xms128m"

# Allocate smaller chunks of memory per pod.
resources:
  requests:
    cpu: "100m"
    memory: "512M"
  limits:
    cpu: "1000m"
    memory: "512M"

# Specify Digital Ocean storage
# Request smaller persistent volumes.
volumeClaimTemplate:
  accessModes: [ "ReadWriteOnce" ]
  storageClassName: do-block-storage
  resources:
    requests:
      storage: 10Gi
extraInitContainers: |
  - name: create
    image: busybox:1.28
    command: ['mkdir', '/usr/share/elasticsearch/data/nodes/']
    volumeMounts:
    - mountPath: /usr/share/elasticsearch/data
      name: elasticsearch-master
  - name: file-permissions
    image: busybox:1.28
    command: ['chown', '-R', '1000:1000', '/usr/share/elasticsearch/']
    volumeMounts:
    - mountPath: /usr/share/elasticsearch/data
      name: elasticsearch-master

Helm chart 我正在用 terraform 设置，但不管怎样，你会怎么做：

resource "helm_release" "elasticsearch" {
  name      = "elasticsearch"
  chart     = "elastic/elasticsearch"
  namespace = "elasticsearch"

  values = [
    file("charts/elasticsearch.yaml")
  ]
}

这是我在检查 pod 日志时得到的信息：

51s         Normal    Provisioning           persistentvolumeclaim/elasticsearch-master-elasticsearch-master-2   External provisioner is provisioning volume for claim "elasticsearch/elasticsearch-master-elasticsearch-master-2"
2m28s       Normal    ExternalProvisioning   persistentvolumeclaim/elasticsearch-master-elasticsearch-master-2   waiting for a volume to be created, either by external provisioner "dobs.csi.digitalocean.com" or manually created by system administrator

我很确定问题出在音量上。它应该由 kubernetes 自动提供。描述持久存储给出了这个：

holms@debian ~/D/c/s/b/t/s/post-infra> kubectl describe pvc elasticsearch-master-elasticsearch-master-0 --namespace elasticsearch
Name:          elasticsearch-master-elasticsearch-master-0
Namespace:     elasticsearch
StorageClass:  do-block-storage
Status:        Pending
Volume:        
Labels:        app=elasticsearch-master
Annotations:   volume.beta.kubernetes.io/storage-provisioner: dobs.csi.digitalocean.com
Finalizers:    [kubernetes.io/pvc-protection]
Capacity:      
Access Modes:  
VolumeMode:    Filesystem
Mounted By:    elasticsearch-master-0
Events:
  Type    Reason                Age                    From                                                                              Message
  ----    ------                ----                   ----                                                                              -------
  Normal  Provisioning          4m57s (x176 over 14h)  dobs.csi.digitalocean.com_master-setupad-eu_04e43747-fafb-11e9-b7dd-e6fd8fbff586  External provisioner is provisioning volume for claim "elasticsearch/elasticsearch-master-elasticsearch-master-0"
  Normal  ExternalProvisioning  93s (x441 over 111m)   persistentvolume-controller                                                       waiting for a volume to be created, either by external provisioner "dobs.csi.digitalocean.com" or manually created by system administrator

我已经 google 一切，似乎一切都是正确的，DO 方面的音量应该没有问题，但它挂在挂起状态。这是预期的行为还是我应该要求 DO 支持人员检查他们那边发生了什么？

Answer 1

是的，这是预期的行为。此图表可能与 Digital Ocean Kubernetes 服务不兼容。

Digital Ocean 文档在已知问题部分包含以下信息：

Support for resizing DigitalOcean Block Storage Volumes in Kubernetes has not yet been implemented.

In the DigitalOcean Control Panel, cluster resources (worker nodes, load balancers, and block storage volumes) are listed outside of the Kubernetes page. If you rename or otherwise modify these resources in the control panel, you may render them unusable to the cluster or cause the reconciler to provision replacement resources. To avoid this, manage your cluster resources exclusively with kubectl or from the control panel’s Kubernetes page.

在charts/stable/elasticsearch中提到了具体要求：

Prerequisites Details

Kubernetes 1.10+

PV dynamic provisioning support on the underlying infrastructure

您可以向 Digital Ocean 支持寻求帮助或尝试在没有 helm chart 的情况下部署 ElasticSearch。

github 甚至提到：

Automated testing of this chart is currently only run against GKE (Google Kubernetes Engine).

更新：

我的 kubeadm ha 集群也存在同样的问题。

但是我设法通过为我的 storageclass.

手动创建 PersistentVolumes 使其工作

我的存储类定义：storageclass.yaml:

kind: StorageClass
apiVersion: storage.k8s.io/v1
metadata:
  name: ssd
provisioner: kubernetes.io/no-provisioner
volumeBindingMode: WaitForFirstConsumer
parameters:
  type: pd-ssd

$ kubectl apply -f storageclass.yaml

$ kubectl get sc
NAME   PROVISIONER   AGE
ssd    local         50m

我的 PersistentVolume 定义：pv.yaml:

apiVersion: v1
kind: PersistentVolume
metadata:
  name: task-pv-volume
  labels:
    type: local
spec:
  storageClassName: ssd
  capacity:
    storage: 30Gi
  accessModes:
    - ReadWriteOnce
  hostPath:
    path: "/mnt/data"
  nodeAffinity:
    required:
      nodeSelectorTerms:
      - matchExpressions:
        - key: kubernetes.io/hostname
          operator: In
          values:
          - <name of the node>

kubectl apply -f pv.yaml

之后我运行掌舵图：

helm install stable/elasticsearch --name my-release --set data.persistence.storageClass=ssd,data.storage=30Gi --set data.persistence.storageClass=ssd,master.storage=30Gi

PVC终于绑定了

$ kubectl get pvc -A
NAMESPACE   NAME                                     STATUS    VOLUME            CAPACITY   ACCESS MODES   STORAGECLASS   AGE
default     data-my-release-elasticsearch-data-0     Bound     task-pv-volume2   30Gi       RWO            ssd            17m
default     data-my-release-elasticsearch-master-0   Pending                                                              17m

请注意，我仅手动满足单个 pvc 和 ElasticSearch 手动卷配置可能非常低效。

我建议联系 DO 支持以获取自动卷配置解决方案。

Answer 2

多么奇怪的情况，在我将 10Gi 更改为 10G 之后它开始工作了。也许它必须对存储做一些事情 class 它是自己的，但它开始工作了。

Digital Ocean 管理的 Kubernetes 卷处于挂起状态

Digital Ocean managed Kubernetes volume in pending state

elasticsearch

kubernetes

digital-ocean

volumes

Prerequisites Details