k8s 从 pod 规范中删除 nodeAffinity

Question

我们的系统运行在 VPC 原生网络中的 GKE 上。我们最近从 v1.9 升级到 v1.21，当我们转移配置时，我注意到 kube-dns 部署中的 spec.template.spec.affinity.nodeAffinity 被删除并被忽略。我尝试使用“kubectl apply -f kube-dns-deployment.yaml”手动添加它“=12=]

我得到“部署。apps/kube-dns 配置”，但几秒钟后，kube-dns 恢复到没有这种关联的配置。

这是yaml中的相关代码：

apiVersion: apps/v1
kind: Deployment
metadata:
  labels:
    addonmanager.kubernetes.io/mode: Reconcile
    k8s-app: kube-dns
    kubernetes.io/cluster-service: "true"
  name: kube-dns
  namespace: kube-system
spec:
  progressDeadlineSeconds: 600
  replicas: 2
  revisionHistoryLimit: 10
  selector:
    matchLabels:
      k8s-app: kube-dns
  strategy:
    rollingUpdate:
      maxSurge: 10%
      maxUnavailable: 0
    type: RollingUpdate
  template:
    metadata:
      annotations:
        components.gke.io/component-name: kubedns
        prometheus.io/port: "10054"
        prometheus.io/scrape: "true"
        scheduler.alpha.kubernetes.io/critical-pod: ""
        seccomp.security.alpha.kubernetes.io/pod: runtime/default
      creationTimestamp: null
      labels:
        k8s-app: kube-dns
    spec:
      affinity:
        nodeAffinity:
          preferredDuringSchedulingIgnoredDuringExecution:
          - preference:
              matchExpressions:
              - key: cloud.google.com/gke-nodepool
                operator: In
                values:
                - pool-1
            weight: 20
          - preference:
              matchExpressions:
              - key: cloud.google.com/gke-nodepool
                operator: In
                values:
                - pool-3
                - training-pool
            weight: 1
          requiredDuringSchedulingIgnoredDuringExecution:
            nodeSelectorTerms:
            - matchExpressions:
              - key: cloud.google.com/gke-nodepool
                operator: In
                values:
                - pool-1
                - pool-3
                - training-pool
        podAntiAffinity:
          preferredDuringSchedulingIgnoredDuringExecution:
          - podAffinityTerm:
              labelSelector:
                matchExpressions:
                - key: k8s-app
                  operator: In
                  values:
                  - kube-dns
              topologyKey: kubernetes.io/hostname
            weight: 100
          requiredDuringSchedulingIgnoredDuringExecution:
          - labelSelector:
              matchExpressions:
              - key: k8s-app
                operator: In
                values:
                - kube-dns
            topologyKey: cloud.google.com/hostname
      containers:
      ....
      dnsPolicy: Default
      nodeSelector:
        kubernetes.io/os: linux

这是我在运行 $ kubectl get deployment kube-dns -n kube-system -o yaml:

时得到的结果

apiVersion: apps/v1
kind: Deployment
metadata:
  annotations:
    ....
  labels:
    addonmanager.kubernetes.io/mode: Reconcile
    k8s-app: kube-dns
    kubernetes.io/cluster-service: "true"
  name: kube-dns
  namespace: kube-system
  resourceVersion: "16650828"
  uid: ....
spec:
  progressDeadlineSeconds: 600
  replicas: 2
  revisionHistoryLimit: 10
  selector:
    matchLabels:
      k8s-app: kube-dns
  strategy:
    rollingUpdate:
      maxSurge: 10%
      maxUnavailable: 0
    type: RollingUpdate
  template:
    metadata:
      annotations:
        components.gke.io/component-name: kubedns
        prometheus.io/port: "10054"
        prometheus.io/scrape: "true"
        scheduler.alpha.kubernetes.io/critical-pod: ""
        seccomp.security.alpha.kubernetes.io/pod: runtime/default
      creationTimestamp: null
      labels:
        k8s-app: kube-dns
    spec:
      affinity:
        podAntiAffinity:
          preferredDuringSchedulingIgnoredDuringExecution:
          - podAffinityTerm:
              labelSelector:
                matchExpressions:
                - key: k8s-app
                  operator: In
                  values:
                  - kube-dns
              topologyKey: kubernetes.io/hostname
            weight: 100
      containers:
      ...
      dnsPolicy: Default
      nodeSelector:
        kubernetes.io/os: linux
      priorityClassName: system-cluster-critical
      restartPolicy: Always
      schedulerName: default-scheduler
      securityContext:
        fsGroup: 65534
        supplementalGroups:
        - 65534
      serviceAccount: kube-dns
      serviceAccountName: kube-dns
      terminationGracePeriodSeconds: 30
      tolerations:
      - key: CriticalAddonsOnly
        operator: Exists
      - key: components.gke.io/gke-managed-components
        operator: Exists
      volumes:
      - configMap:
          defaultMode: 420
          name: kube-dns
          optional: true
        name: kube-dns-config
status:
  ...

如您所见，GKE 只是删除了 NodeAffinity 部分，以及 podAffinity 的一部分。

Answer 1

kube-dns 是 GKE 中的一种服务发现机制，也是集群使用的默认 DNS 提供商。它由 Google 管理，这就是更改未保留的原因，很可能那部分代码已在新版本中删除。

如果您需要应用自定义配置，可以按照指南进行操作 Setting up a custom kube-dns Deployment。

k8s 从 pod 规范中删除 nodeAffinity

k8s deletes nodeAffinity from pod specs

kubernetes

kube-dns

gke-networking