如何更改 Kubernetes 工作节点的内部 IP？

Question

我正在尝试使用 Kelsey Hightower 的 Learn Kubernetes the Hard Way 指南从头开始部署 K8s 集群。就我而言，我使用的是 Vagrant 和 VirtualBox。

我的每个 Master 和 Worker 在 eth0（10.0.2.x 范围）中都有一个 DHCP 网络，用于从 Internet 提取位，以及一个 eth1 静态范围（10.10.10.x/24）用于内部 k8s 通信。

 [vagrant@master-1 ~]$ kubectl get nodes -o wide
NAME       STATUS   ROLES    AGE   VERSION   INTERNAL-IP   EXTERNAL-IP   OS-IMAGE                KERNEL-VERSION              CONTAINER-RUNTIME
worker-1   Ready    <none>   32s   v1.12.0   10.0.2.15     <none>        CentOS Linux 7 (Core)   3.10.0-957.1.3.el7.x86_64   containerd://1.2.0-rc.0
worker-2   Ready    <none>   2s    v1.12.0   10.0.2.15     <none>        CentOS Linux 7 (Core)   3.10.0-957.1.3.el7.x86_64   containerd://1.2.0-rc.0

我最初没有标志 -node-ip="10.10.10.x 和 -address="10.10.10.x" 设置。

添加后 - 我确实删除了节点并重新启动 kubelet 服务，希望能再次注册节点，但它似乎不想更新。

== 以下是 kubelet 配置示例：

/var/lib/kubelet/kubelet-config.yaml
kind: KubeletConfiguration
apiVersion: kubelet.config.k8s.io/v1beta1
authentication:
  anonymous:
    enabled: false
  webhook:
    enabled: true
  x509:
    clientCAFile: "/var/lib/kubernetes/ca.pem"
authorization:
  mode: Webhook
clusterDomain: "cluster.local"
clusterDNS:
  - "10.32.0.10"
podCIDR: "${POD_CIDR}"
resolvConf: "/run/systemd/resolve/resolv.conf"
runtimeRequestTimeout: "15m"
tlsCertFile: "/var/lib/kubelet/${HOSTNAME}.pem"
tlsPrivateKeyFile: "/var/lib/kubelet/${HOSTNAME}-key.pem"
EOF


/etc/systemd/system/kubelet.service
[Unit]
Description=Kubernetes Kubelet
Documentation=https://github.com/kubernetes/kubernetes
After=containerd.service
Requires=containerd.service

[Service]
ExecStart=/usr/local/bin/kubelet \
  --config=/var/lib/kubelet/kubelet-config.yaml \
  --container-runtime=remote \
  --container-runtime-endpoint=unix:///var/run/containerd/containerd.sock \
  --image-pull-progress-deadline=2m \
  --kubeconfig=/var/lib/kubelet/kubeconfig \
  --network-plugin=cni \
  --node-ip="$NODE_IP"
  --address="$NODE_IP"
  --register-node=true \
  --v=2

和 kube-api 服务器：

[Service]
ExecStart=/usr/local/bin/kube-apiserver \
  --advertise-address=${INTERNAL_IP} \
  --allow-privileged=true \
  --apiserver-count=3 \
  --audit-log-maxage=30 \
  --audit-log-maxbackup=3 \
  --audit-log-maxsize=100 \
  --audit-log-path=/var/log/audit.log \
  --authorization-mode=Node,RBAC \
  --bind-address=0.0.0.0 \
  --client-ca-file=/var/lib/kubernetes/ca.pem \
  --enable-admission-plugins=Initializers,NamespaceLifecycle,NodeRestriction,LimitRanger,ServiceAccount,DefaultStorageClass,ResourceQuota \
  --enable-swagger-ui=true \
  --etcd-cafile=/var/lib/kubernetes/ca.pem \
  --etcd-certfile=/var/lib/kubernetes/kubernetes.pem \
  --etcd-keyfile=/var/lib/kubernetes/kubernetes-key.pem \
  --etcd-servers=https://10.10.10.11:2379,https://10.10.10.12:2379 \
  --event-ttl=1h \
  --experimental-encryption-provider-config=/var/lib/kubernetes/encryption-config.yaml \
  --kubelet-certificate-authority=/var/lib/kubernetes/ca.pem \
  --kubelet-client-certificate=/var/lib/kubernetes/kubernetes.pem \
  --kubelet-client-key=/var/lib/kubernetes/kubernetes-key.pem \
  --kubelet-https=true \
  --runtime-config=api/all \
  --service-account-key-file=/var/lib/kubernetes/service-account.pem \
  --service-cluster-ip-range=10.32.0.0/24 \
  --service-node-port-range=30000-32767 \
  --tls-cert-file=/var/lib/kubernetes/kubernetes.pem \
  --tls-private-key-file=/var/lib/kubernetes/kubernetes-key.pem \
  --v=2

同样在 vagrant 中，我相信 eth0 是 NAT 设备，因为我看到 10.0.2.15 分配给所有虚拟机的 ip (master/slaves)

[vagrant@worker-1 ~]$ ip a s
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
    inet6 ::1/128 scope host
       valid_lft forever preferred_lft forever
2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP group default qlen 1000
    link/ether 52:54:00:75:dc:3d brd ff:ff:ff:ff:ff:ff
    inet 10.0.2.15/24 brd 10.0.2.255 scope global noprefixroute dynamic eth0
       valid_lft 84633sec preferred_lft 84633sec
    inet6 fe80::5054:ff:fe75:dc3d/64 scope link
       valid_lft forever preferred_lft forever
3: eth1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP group default qlen 1000
    link/ether 08:00:27:24:a4:c2 brd ff:ff:ff:ff:ff:ff
    inet 192.168.0.206/24 brd 192.168.0.255 scope global noprefixroute dynamic eth1
       valid_lft 3600sec preferred_lft 3600sec
4: eth2: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP group default qlen 1000
    link/ether 08:00:27:76:22:4a brd ff:ff:ff:ff:ff:ff
    inet 10.10.10.21/24 brd 10.10.10.255 scope global noprefixroute eth2
       valid_lft forever preferred_lft forever
    inet6 fe80::a00:27ff:fe76:224a/64 scope link
       valid_lft forever preferred_lft forever
[vagrant@worker-1 ~]$

我想问的是如何更新内部 IP 和外部 IP post 更改 kubelet 配置

Answer 1

我编辑了 /etc/systemd/system/kubelet.service.d/10-kubeadm.conf - 将 --node-ip flag 添加到 KUBELET_CONFIG_ARGS 并重新启动了 kubelet：

systemctl daemon-reload
systemctl restart kubelet

并且kubectl get nodes -o wide立即报告了新的IP地址。我在 master 上做的时候花了一点时间 - 但最终发生了。

Answer 2

我所做的与Michael的回答类似，只是我没有编辑/etc/systemd/system/kubelet.service.d/10-kubeadm.conf。

As suggested here 说明了（我强调了一些点）

The file that can contain user-specified flag overrides with KUBELET_EXTRA_ARGS is sourced from /etc/default/kubelet (for DEBs), or /etc/sysconfig/kubelet (for RPMs). KUBELET_EXTRA_ARGS is last in the flag chain and has the highest priority in the event of conflicting settings

因此，我没有更改本应自动更改的文件（可能是由于升级等原因），而是将 /etc/sysconfig/kubelet 更改为 KUBELET_EXTRA_ARGS='--node-ip 10.0.0.3'（可以使用的选项见kubelet --help).

然后我按照其他答案的说明做了：

systemctl daemon-reload
systemctl restart kubelet
kubectl get nodes -o wide

并且内部 IP 已相应更改。

如何更改 Kubernetes 工作节点的内部 IP？

How to change the internal IP of Kubernetes worker nodes?

kubernetes

kubelet