PersistemVolumeClaim 元数据无法正确解码

Question

环境

Kubectl 版本

Client Version: version.Info{Major:"1", Minor:"18", GitVersion:"v1.18.4", GitCommit:"c96aede7b5205121079932896c4ad89bb93260af", GitTreeState:"clean", BuildDate:"2020-06-18T02:59:13Z", GoVersion:"go1.14.3", Compiler:"gc", Platform:"darwin/amd64"}
Server Version: version.Info{Major:"1", Minor:"20+", GitVersion:"v1.20.4-80+89e0897d2cb807", GitCommit:"89e0897d2cb8073fbb8f700258573f1478d4826a", GitTreeState:"clean", BuildDate:"2021-11-22T03:53:35Z", GoVersion:"go1.15.8", Compiler:"gc", Platform:"linux/amd64"}

Kubernetes版本（Kind集群）

kind: Cluster
apiVersion: kind.x-k8s.io/v1alpha4
nodes:
  - role: control-plane
    image: kindest/node:v1.20.7@sha256:cbeaf907fc78ac97ce7b625e4bf0de16e3ea725daf6b04f930bd14c67c671ff9
  - role: worker
    image: kindest/node:v1.20.7@sha256:cbeaf907fc78ac97ce7b625e4bf0de16e3ea725daf6b04f930bd14c67c671ff9
  - role: worker
    image: kindest/node:v1.20.7@sha256:cbeaf907fc78ac97ce7b625e4bf0de16e3ea725daf6b04f930bd14c67c671ff9

Kubebuilder 版本

Version: main.version{KubeBuilderVersion:"3.1.0", KubernetesVendor:"1.19.2", GitCommit:"92e0349ca7334a0a8e5e499da4fb077eb524e94a", BuildDate:"2021-05-27T17:54:28Z", GoOs:"darwin", GoArch:"amd64"}

Os

Macos Big Sur 11.6

我使用 kubebuilder 来定义我自己的 CRD 如下，它包含 VolumeClaimTemplates 类型为 []coreV1.PersistentVolumeClaim

的字段

package v1alpha1
import (
    apps "k8s.io/api/apps/v1"
    coreV1 "k8s.io/api/core/v1"
    metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"
    "k8s.io/apimachinery/pkg/util/intstr"
)

type DatabaseSetSpec struct {
        ...
        // +optional
    VolumeClaimTemplates []coreV1.PersistentVolumeClaim `json:"volumeClaimTemplates,omitempty" protobuf:"bytes,4,rep,name=volumeClaimTemplates"`
}

// +kubebuilder:object:root=true
// +kubebuilder:subresource:status
// +kubebuilder:resource:shortName=ami-dbs

// DatabaseSet is the Schema for the databasesets API
type DatabaseSet struct {
    metav1.TypeMeta   `json:",inline"`
    metav1.ObjectMeta `json:"metadata,omitempty"`

    Spec   DatabaseSetSpec   `json:"spec,omitempty"`
    Status DatabaseSetStatus `json:"status,omitempty"`
}

// DatabaseSetStatus defines the observed state of DatabaseSet
type DatabaseSetStatus struct {
    ...
}

//+kubebuilder:object:root=true

// DatabaseSetList contains a list of DatabaseSet
type DatabaseSetList struct {
    metav1.TypeMeta `json:",inline"`
    metav1.ListMeta `json:"metadata,omitempty"`
    Items           []DatabaseSet `json:"items"`
}

func init() {
    SchemeBuilder.Register(&DatabaseSet{}, &DatabaseSetList{})
}

但是当我像下面这样应用CR时，我发现metadata字段是空的

apiVersion: apps.analyticdb.aliyun.com/v1alpha1
kind: DatabaseSet
metadata:
  name: databaseset-sample
spec: 
  ...
  volumeClaimTemplates:
    - metadata:
        name: pvc-test
      spec:
        accessModes: [ "ReadWriteOnce" ]
        storageClassName: "manual"
        resources:
          requests:
            storage: 3Gi

这里是从k8s ectd得到的yaml，发现volumeClaimTemplates的metadata是空的

apiVersion: apps.analyticdb.aliyun.com/v1alpha1
kind: DatabaseSet
metadata:
  creationTimestamp: "2021-12-24T09:46:22Z"
  generation: 1
  name: databaseset-sample
  namespace: default
  resourceVersion: "98727469"
  uid: e64107f2-7a4b-473b-9275-39ab5e2e88dc
spec:
  ...
  volumeClaimTemplates:
  - metadata: {}
    spec:
      accessModes:
      - ReadWriteOnce
      resources:
        requests:
          storage: 3Gi
      storageClassName: manual

有谁知道为什么吗？

当我用下面的注释标记 volumeclaimtemplate 字段时，metada 可以正确解码

// +kubebuilder:pruning:PreserveUnknownFields
// +kubebuilder:validation:Schemaless

Answer 1

请参阅 kubebuilder issue#2460，使用 controller-gen 选项 crd:generateEmbeddedObjectMeta=true 将起作用

Answer 2

我已经发布了社区维基答案来总结主题：

有人问了同样的问题 on the github。解决方案就像提到的OP：

have found the solution, use the controller-gen crd option crd:generateEmbeddedObjectMeta=true will work

在 github 上被提及：

I found this option through controller-gen -h, and there is no mention of this option in the official kubebuilder controller-gen CLI documention .

是的，没有提到（这个工具的官方文档有点旧 - 2019 年 8 月）但是看看 this problem and this answer:

hi @numbnut I haven't tested yet with v1.21.2 but I'll take a look.

Using controller-tools@0.2.4 was needed before to be able to run the operator with k8s >1.18 but that has been addressed in this fork at least up until v1.20.x`

The newer0.6.1 introduces generateEmbeddedObjectMeta which is required specifically to add the metadata for the PersistentVolumeClaims without it claims will not be deleted when the cluster is deleted;

@clouddra check if your env is not using a previous version of controller-tools

您还可以从 source code.

中找到合适的行

PersistemVolumeClaim 元数据无法正确解码

PersistemVolumeClaim Metadata can not be decoded correctly

go

kubernetes

kubebuilder