如何更新 K8S Pod 的 DNS 配置

Question

我有一个项目想用 Operator 更新 Pod 的 DNS 配置，

get dns message
get matched pod
modify:
    pod.Spec.DNSConfig = CRD_SPEC
    pod.Spec.DNSPolicy = corev1.DNSNone
client.Update(ctx,&pod)

但是我在执行的时候出现了如下错误：

 ERROR   controller-runtime.manager.controller.dnsinjection      Reconciler error        {"reconciler group": "xxxx", "reconciler kind": "xxxxx", "name": "dnsinjection", "namespace": "default", "error": "Pod \"busybox\" is invalid: spec: Forbidden: pod updates may not change fields other than `spec.containers[*].image`, `spec.initContainers[*].image`, `spec.activeDeadlineSeconds` or `spec.tolerations` (only additions to existing tolerations)\n  core.PodSpec{\n  \t... // 21 identical fields\n  \tPriority:         &0,\n  \tPreemptionPolicy: nil,\n  \tDNSConfig: &core.PodDNSConfig{\n  \t\tNameservers: []string{\n  \t\t\t\"1.2.3.4\",\n- \t\t\t\"0.0.0.0\",\n  \t\t},\n  \t\tSearches: []string{\"ns1.svc.cluster-domain.example\", \"my.dns.search.suffix\"},\n  \t\tOptions:  []core.PodDNSConfigOption{{Name: \"ndots\", Value: &\"2\"}, {Name: \"edns0\"}},\n  \t},\n  \tReadinessGates:   nil,\n  \tRuntimeClassName: nil,\n  \t... // 3 identical fields\n  }\n"}

DNSConfig和DNSPoicy字段在源码中没有声明不能更新，那为什么更新失败了？

我在 kubect edit pod busybox 和 kubectl apply -f modifyed_pod.yml(add DNSConfig) 命令中遇到了同样的错误。

如果你能告诉我如何解决它，我将不胜感激。

Answer 1

就像消息说您无法更新 pod 的 DNS 配置：Forbidden: pod updates may not change fields other than spec.containers[*].image, spec.initContainers[*].image。

如果要将 DNS 配置注入所有 pods，则需要在创建 pod 之前添加配置。研究 MutatingAdmissionWebhook 作为解决此问题的方法。

Answer 2

正如消息所说，Kubernetes does not support updating most pod's fields directly:

Kubernetes doesn't prevent you from managing Pods directly. It is possible to update some fields of a running Pod, in place. However, Pod update operations like patch, and replace have some limitations:

Most of the metadata about a Pod is immutable. For example, you cannot change the namespace, name, uid, or creationTimestamp fields; the generation field is unique. It only accepts updates that increment the field's current value.

If the metadata.deletionTimestamp is set, no new entry can be added to the metadata.finalizers list.

Pod updates may not change fields other than spec.containers[*].image, spec.initContainers[*].image, spec.activeDeadlineSeconds or spec.tolerations. For spec.tolerations, you can only add new entries.

这是为什么？

Pods 在 Kubernetes 中 are designed as relatively ephemeral, disposable entities:

You'll rarely create individual Pods directly in Kubernetes—even singleton Pods. This is because Pods are designed as relatively ephemeral, disposable entities. When a Pod gets created (directly by you, or indirectly by a controller), the new Pod is scheduled to run on a Node in your cluster. The Pod remains on that node until the Pod finishes execution, the Pod object is deleted, the Pod is evicted for lack of resources, or the node fails.

Kubernetes assumes that for managing pods and doing any updates you should use a workload resources 而不是直接创建 pods：

Pods are generally not created directly and are created using workload resources. See Working with Pods for more information on how Pods are used with workload resources. Here are some examples of workload resources that manage one or more Pods:

Deployment

StatefulSet

DaemonSet

您可以轻松地更新工作负载资源定义中的大多数字段，它会正常工作。请记住，它不会编辑任何现有的 pods - it will delete the currently running pods with old configuration and start the new ones - Kubernetes will make sure that this process goes smoothly:

Modifying the pod template or switching to a new pod template has no direct effect on the Pods that already exist. If you change the pod template for a workload resource, that resource needs to create replacement Pods that use the updated template.

For example, the StatefulSet controller ensures that the running Pods match the current pod template for each StatefulSet object. If you edit the StatefulSet to change its pod template, the StatefulSet starts to create new Pods based on the updated template. Eventually, all of the old Pods are replaced with new Pods, and the update is complete.

Each workload resource implements its own rules for handling changes to the Pod template. If you want to read more about StatefulSet specifically, read Update strategy in the StatefulSet Basics tutorial.

因此，根据以上所有信息，我建议切换到工作负载资源，例如 deployment:

A Deployment provides declarative updates for Pods and ReplicaSets.

例如 - 现在我的 pod 定义如下：

apiVersion: v1
kind: Pod
metadata:
  name: busybox
  namespace: default
spec:
  containers:
  - image: busybox:1.28
    command:
      - sleep
      - "9999999"
    imagePullPolicy: IfNotPresent
    name: busybox
  restartPolicy: Always
  hostNetwork: true
  dnsPolicy: ClusterFirstWithHostNet

当我尝试运行 kubectl edit pod busybox 命令更改 dnsPolicy 时，我遇到了与您相同的错误。但是，如果我更改为具有相同 pod 定义的部署：

apiVersion: apps/v1
kind: Deployment
metadata:
  name: busybox-deployment
  labels:
    app: busybox
spec:
  replicas: 1
  selector:
    matchLabels:
      app: busybox
  template:
    metadata:
      labels:
        app: busybox
    spec:
      containers:
      - image: busybox:1.28
        command:
        - sleep
        - "9999999"
        imagePullPolicy: IfNotPresent
        name: busybox
      restartPolicy: Always
      hostNetwork: true
      dnsPolicy: ClusterFirstWithHostNet

然后如果我运行 kubectl edit deploy busybox-deployment 并更改 dnsPolicy 字段，我将获得一个具有新配置的新 pod（旧 pod 将被自动删除）。

请记住，如果您想坚持使用直接 pod 定义，您始终可以删除 pod 并在尝试时应用新的、修改后的 yaml (kubectl delete pod {your-pod-name} && kubectl apply -f {modified.yaml})。它会正常工作。

同时检查：

如何更新 K8S Pod 的 DNS 配置

How to update DNS configuration of K8S Pod

kubernetes

kubernetes-operator

kubernetes-dns