Ufw 防火墙阻止 kubernetes（使用 calico）
Ufw firewall blocks kubernetes (with calico)
我正在尝试在我的服务器 (Debian 10) 上安装 kubernetes 集群。在我的服务器上，我使用 ufw 作为防火墙。
在创建集群之前，我在 ufw 上允许了这些端口：
179/tcp, 4789/udp, 5473/tcp, 443 /tcp, 6443/tcp, 2379/tcp, 4149/tcp, 10250/tcp, 10255/tcp, 10256/tcp, 9099/tcp, 6443/tcp
正如 calico 文档所建议的那样 (https://docs.projectcalico.org/getting-started/kubernetes/requirements) and this git repo on kubernetes security too (https://github.com/freach/kubernetes-security-best-practice)。
但是当我想创建集群时，calico/node pod 无法启动，因为 Felix 没有运行（我在 ufw 上允许 9099/tcp）：
Liveness probe failed: calico/node is not ready: Felix is not live: Get http://localhost:9099/liveness: dial tcp [::1]:9099: connect: connection refused
如果我禁用 ufw，集群会创建并且没有错误。
所以我想知道我应该如何配置 ufw 才能让 kubernetes 工作。
如果有人能帮助我，那就太好了，谢谢！
编辑：我的 ufw 状态
To Action From
6443/tcp ALLOW Anywhere
9099 ALLOW Anywhere
179/tcp ALLOW Anywhere
4789/udp ALLOW Anywhere
5473/tcp ALLOW Anywhere
2379/tcp ALLOW Anywhere
8181 ALLOW Anywhere
8080 ALLOW Anywhere
###### (v6) LIMIT Anywhere (v6) # allow ssh connections in
Postfix (v6) ALLOW Anywhere (v6)
KUBE (v6) ALLOW Anywhere (v6)
6443 (v6) ALLOW Anywhere (v6)
6783/udp (v6) ALLOW Anywhere (v6)
6784/udp (v6) ALLOW Anywhere (v6)
6783/tcp (v6) ALLOW Anywhere (v6)
443/tcp (v6) ALLOW Anywhere (v6)
80/tcp (v6) ALLOW Anywhere (v6)
4149/tcp (v6) ALLOW Anywhere (v6)
10250/tcp (v6) ALLOW Anywhere (v6)
10255/tcp (v6) ALLOW Anywhere (v6)
10256/tcp (v6) ALLOW Anywhere (v6)
9099/tcp (v6) ALLOW Anywhere (v6)
6443/tcp (v6) ALLOW Anywhere (v6)
9099 (v6) ALLOW Anywhere (v6)
179/tcp (v6) ALLOW Anywhere (v6)
4789/udp (v6) ALLOW Anywhere (v6)
5473/tcp (v6) ALLOW Anywhere (v6)
2379/tcp (v6) ALLOW Anywhere (v6)
8181 (v6) ALLOW Anywhere (v6)
8080 (v6) ALLOW Anywhere (v6)
53 ALLOW OUT Anywhere # allow DNS calls out
123 ALLOW OUT Anywhere # allow NTP out
80/tcp ALLOW OUT Anywhere # allow HTTP traffic out
443/tcp ALLOW OUT Anywhere # allow HTTPS traffic out
21/tcp ALLOW OUT Anywhere # allow FTP traffic out
43/tcp ALLOW OUT Anywhere # allow whois
SMTPTLS ALLOW OUT Anywhere # open TLS port 465 for use with SMPT to send e-mails
10.32.0.0/12 ALLOW OUT Anywhere on weave
53 (v6) ALLOW OUT Anywhere (v6) # allow DNS calls out
123 (v6) ALLOW OUT Anywhere (v6) # allow NTP out
80/tcp (v6) ALLOW OUT Anywhere (v6) # allow HTTP traffic out
443/tcp (v6) ALLOW OUT Anywhere (v6) # allow HTTPS traffic out
21/tcp (v6) ALLOW OUT Anywhere (v6) # allow FTP traffic out
43/tcp (v6) ALLOW OUT Anywhere (v6) # allow whois
SMTPTLS (v6) ALLOW OUT Anywhere (v6) # open TLS port 465 for use with SMPT to send e-mails
抱歉，我的 ufw 规则有点乱，我尝试了太多东西让 kubernetes 工作。
I'm trying to install a kubernetes cluster on my server (Debian 10). On my server I used ufw as firewall. Before creating the cluster I allowed these ports on ufw:
179/tcp, 4789/udp, 5473/tcp, 443 /tcp, 6443/tcp, 2379/tcp, 4149/tcp, 10250/tcp, 10255/tcp, 10256/tcp, 9099/tcp, 6443/tcp
注意： 所有可执行命令都以 $
开头
- 按照这个初始说明，我在 Debian 10 上安装了 ufw 并启用了您提到的相同端口：
$ sudo apt update && sudo apt-upgrade -y
$ sudo apt install ufw -y
$ sudo ufw allow ssh
Rule added
Rule added (v6)
$ sudo ufw enable
Command may disrupt existing ssh connections. Proceed with operation (y|n)? y
Firewall is active and enabled on system startup
$ sudo ufw allow 179/tcp
$ sudo ufw allow 4789/tcp
$ sudo ufw allow 5473/tcp
$ sudo ufw allow 443/tcp
$ sudo ufw allow 6443/tcp
$ sudo ufw allow 2379/tcp
$ sudo ufw allow 4149/tcp
$ sudo ufw allow 10250/tcp
$ sudo ufw allow 10255/tcp
$ sudo ufw allow 10256/tcp
$ sudo ufw allow 9099/tcp
$ sudo ufw status
Status: active
To Action From
-- ------ ----
22/tcp ALLOW Anywhere
179/tcp ALLOW Anywhere
4789/tcp ALLOW Anywhere
5473/tcp ALLOW Anywhere
443/tcp ALLOW Anywhere
6443/tcp ALLOW Anywhere
2379/tcp ALLOW Anywhere
4149/tcp ALLOW Anywhere
10250/tcp ALLOW Anywhere
10255/tcp ALLOW Anywhere
10256/tcp ALLOW Anywhere
22/tcp (v6) ALLOW Anywhere (v6)
179/tcp (v6) ALLOW Anywhere (v6)
4789/tcp (v6) ALLOW Anywhere (v6)
5473/tcp (v6) ALLOW Anywhere (v6)
443/tcp (v6) ALLOW Anywhere (v6)
6443/tcp (v6) ALLOW Anywhere (v6)
2379/tcp (v6) ALLOW Anywhere (v6)
4149/tcp (v6) ALLOW Anywhere (v6)
10250/tcp (v6) ALLOW Anywhere (v6)
10255/tcp (v6) ALLOW Anywhere (v6)
10256/tcp (v6) ALLOW Anywhere (v6)
- 现在我将安装 Docker:
$ sudo apt-get update
$ sudo apt-get install -y apt-transport-https ca-certificates curl gnupg2 software-properties-common=
- 正在添加 Docker 存储库：
$ curl -fsSL https://download.docker.com/linux/debian/gpg | sudo apt-key add -
$ sudo apt-key fingerprint 0EBFCD88
$ sudo add-apt-repository "deb [arch=amd64] https://download.docker.com/linux/debian buster stable"
- 更新源列表并安装 Docker-ce:
$ sudo apt-get update
$ sudo apt-get -y install docker-ce
注意： 在生产系统上建议安装 docker 的固定版本：
$ apt-cache madison docker-ce
$ sudo apt-get install docker-ce=<VERSION>
- 正在安装 Kube Tools - kubeadm、kubectl、kubelet：
$ curl -s https://packages.cloud.google.com/apt/doc/apt-key.gpg | sudo apt-key add -
- 配置 Kubernetes 存储库（复制 3 行并立即粘贴）：
$ cat <<EOF | sudo tee /etc/apt/sources.list.d/kubernetes.list
deb https://apt.kubernetes.io/ kubernetes-xenial main
EOF
- 正在安装软件包：
$ sudo apt-get update
$ sudo apt-get install -y kubelet kubeadm kubectl
- 安装后将这些包标记为不自动更新：
$ sudo apt-mark hold kubelet kubeadm kubectl
$ sudo kubeadm init --pod-network-cidr=192.168.0.0/16
- 为非 root 用户启用 kubectl：
$ mkdir -p $HOME/.kube
$ sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
$ sudo chown $(id -u):$(id -g) $HOME/.kube/config
$ kubectl apply -f https://docs.projectcalico.org/manifests/calico.yaml
configmap/calico-config created
customresourcedefinition.apiextensions.k8s.io/bgpconfigurations.crd.projectcalico.org created
customresourcedefinition.apiextensions.k8s.io/bgppeers.crd.projectcalico.org created
customresourcedefinition.apiextensions.k8s.io/blockaffinities.crd.projectcalico.org created
customresourcedefinition.apiextensions.k8s.io/clusterinformations.crd.projectcalico.org created
customresourcedefinition.apiextensions.k8s.io/felixconfigurations.crd.projectcalico.org created
customresourcedefinition.apiextensions.k8s.io/globalnetworkpolicies.crd.projectcalico.org created
customresourcedefinition.apiextensions.k8s.io/globalnetworksets.crd.projectcalico.org created
customresourcedefinition.apiextensions.k8s.io/hostendpoints.crd.projectcalico.org created
customresourcedefinition.apiextensions.k8s.io/ipamblocks.crd.projectcalico.org created
customresourcedefinition.apiextensions.k8s.io/ipamconfigs.crd.projectcalico.org created
customresourcedefinition.apiextensions.k8s.io/ipamhandles.crd.projectcalico.org created
customresourcedefinition.apiextensions.k8s.io/ippools.crd.projectcalico.org created
customresourcedefinition.apiextensions.k8s.io/networkpolicies.crd.projectcalico.org created
customresourcedefinition.apiextensions.k8s.io/networksets.crd.projectcalico.org created
clusterrole.rbac.authorization.k8s.io/calico-kube-controllers created
clusterrolebinding.rbac.authorization.k8s.io/calico-kube-controllers created
clusterrole.rbac.authorization.k8s.io/calico-node created
clusterrolebinding.rbac.authorization.k8s.io/calico-node created
daemonset.apps/calico-node created
serviceaccount/calico-node created
deployment.apps/calico-kube-controllers created
serviceaccount/calico-kube-controllers created
- 查看状态：
$ kubectl get pods -n kube-system
NAME READY STATUS RESTARTS AGE
calico-kube-controllers-555fc8cc5c-wnnvq 1/1 Running 0 26m
calico-node-sngt8 1/1 Running 0 26m
coredns-66bff467f8-2qqlv 1/1 Running 0 55m
coredns-66bff467f8-vptpr 1/1 Running 0 55m
etcd-kubeadm-ufw-debian10 1/1 Running 0 55m
kube-apiserver-kubeadm-ufw-debian10 1/1 Running 0 55m
kube-controller-manager-kubeadm-ufw-debian10 1/1 Running 0 55m
kube-proxy-nx8cz 1/1 Running 0 55m
kube-scheduler-kubeadm-ufw-debian10 1/1 Running 0 55m
考虑因素：
Sorry my ufw rules are a bit messy, I tried too many things to get kubernetes working.
- 为了使某事成功而尝试很多事情是正常的，但有时它本身最终会成为问题。
- 我将一步一步地发布给您，以便将它部署在与您相同的环境中，以便您可以再次按照它来获得相同的结果。
- 我的 felix 探测器没有出现任何错误，只有当我尝试（有意）部署 kubernetes 而没有在 ufw 上创建规则时出现错误。
如果没有解决，下一步：
- 现在，如果按照本教程进行操作后您仍然遇到类似问题，请使用以下信息更新问题：
kubectl describe <pod_name> -n kube-systemkubectl get pod <pod_name> -n kube-systemkubectl logs <pod_name> -n kube-system- 始终建议从 Linux 的全新安装开始，如果您是 运行 VM，请删除该 VM 并创建一个新的。
- 如果您 运行 在裸机上，请考虑 运行 服务器上还有什么，也许有其他软件干扰了网络通信。
如果您按照这些故障排除步骤发现任何问题，请在评论中告诉我。
我正在尝试在我的服务器 (Debian 10) 上安装 kubernetes 集群。在我的服务器上，我使用 ufw 作为防火墙。 在创建集群之前，我在 ufw 上允许了这些端口：
179/tcp, 4789/udp, 5473/tcp, 443 /tcp, 6443/tcp, 2379/tcp, 4149/tcp, 10250/tcp, 10255/tcp, 10256/tcp, 9099/tcp, 6443/tcp
正如 calico 文档所建议的那样 (https://docs.projectcalico.org/getting-started/kubernetes/requirements) and this git repo on kubernetes security too (https://github.com/freach/kubernetes-security-best-practice)。
但是当我想创建集群时，calico/node pod 无法启动，因为 Felix 没有运行（我在 ufw 上允许 9099/tcp）：
Liveness probe failed: calico/node is not ready: Felix is not live: Get http://localhost:9099/liveness: dial tcp [::1]:9099: connect: connection refused
如果我禁用 ufw，集群会创建并且没有错误。
所以我想知道我应该如何配置 ufw 才能让 kubernetes 工作。 如果有人能帮助我，那就太好了，谢谢！
编辑：我的 ufw 状态
To Action From
6443/tcp ALLOW Anywhere
9099 ALLOW Anywhere
179/tcp ALLOW Anywhere
4789/udp ALLOW Anywhere
5473/tcp ALLOW Anywhere
2379/tcp ALLOW Anywhere
8181 ALLOW Anywhere
8080 ALLOW Anywhere
###### (v6) LIMIT Anywhere (v6) # allow ssh connections in
Postfix (v6) ALLOW Anywhere (v6)
KUBE (v6) ALLOW Anywhere (v6)
6443 (v6) ALLOW Anywhere (v6)
6783/udp (v6) ALLOW Anywhere (v6)
6784/udp (v6) ALLOW Anywhere (v6)
6783/tcp (v6) ALLOW Anywhere (v6)
443/tcp (v6) ALLOW Anywhere (v6)
80/tcp (v6) ALLOW Anywhere (v6)
4149/tcp (v6) ALLOW Anywhere (v6)
10250/tcp (v6) ALLOW Anywhere (v6)
10255/tcp (v6) ALLOW Anywhere (v6)
10256/tcp (v6) ALLOW Anywhere (v6)
9099/tcp (v6) ALLOW Anywhere (v6)
6443/tcp (v6) ALLOW Anywhere (v6)
9099 (v6) ALLOW Anywhere (v6)
179/tcp (v6) ALLOW Anywhere (v6)
4789/udp (v6) ALLOW Anywhere (v6)
5473/tcp (v6) ALLOW Anywhere (v6)
2379/tcp (v6) ALLOW Anywhere (v6)
8181 (v6) ALLOW Anywhere (v6)
8080 (v6) ALLOW Anywhere (v6)
53 ALLOW OUT Anywhere # allow DNS calls out
123 ALLOW OUT Anywhere # allow NTP out
80/tcp ALLOW OUT Anywhere # allow HTTP traffic out
443/tcp ALLOW OUT Anywhere # allow HTTPS traffic out
21/tcp ALLOW OUT Anywhere # allow FTP traffic out
43/tcp ALLOW OUT Anywhere # allow whois
SMTPTLS ALLOW OUT Anywhere # open TLS port 465 for use with SMPT to send e-mails
10.32.0.0/12 ALLOW OUT Anywhere on weave
53 (v6) ALLOW OUT Anywhere (v6) # allow DNS calls out
123 (v6) ALLOW OUT Anywhere (v6) # allow NTP out
80/tcp (v6) ALLOW OUT Anywhere (v6) # allow HTTP traffic out
443/tcp (v6) ALLOW OUT Anywhere (v6) # allow HTTPS traffic out
21/tcp (v6) ALLOW OUT Anywhere (v6) # allow FTP traffic out
43/tcp (v6) ALLOW OUT Anywhere (v6) # allow whois
SMTPTLS (v6) ALLOW OUT Anywhere (v6) # open TLS port 465 for use with SMPT to send e-mails
抱歉，我的 ufw 规则有点乱，我尝试了太多东西让 kubernetes 工作。
I'm trying to install a kubernetes cluster on my server (Debian 10). On my server I used ufw as firewall. Before creating the cluster I allowed these ports on ufw: 179/tcp, 4789/udp, 5473/tcp, 443 /tcp, 6443/tcp, 2379/tcp, 4149/tcp, 10250/tcp, 10255/tcp, 10256/tcp, 9099/tcp, 6443/tcp
注意： 所有可执行命令都以 $
- 按照这个初始说明，我在 Debian 10 上安装了 ufw 并启用了您提到的相同端口：
$ sudo apt update && sudo apt-upgrade -y
$ sudo apt install ufw -y
$ sudo ufw allow ssh
Rule added
Rule added (v6)
$ sudo ufw enable
Command may disrupt existing ssh connections. Proceed with operation (y|n)? y
Firewall is active and enabled on system startup
$ sudo ufw allow 179/tcp
$ sudo ufw allow 4789/tcp
$ sudo ufw allow 5473/tcp
$ sudo ufw allow 443/tcp
$ sudo ufw allow 6443/tcp
$ sudo ufw allow 2379/tcp
$ sudo ufw allow 4149/tcp
$ sudo ufw allow 10250/tcp
$ sudo ufw allow 10255/tcp
$ sudo ufw allow 10256/tcp
$ sudo ufw allow 9099/tcp
$ sudo ufw status
Status: active
To Action From
-- ------ ----
22/tcp ALLOW Anywhere
179/tcp ALLOW Anywhere
4789/tcp ALLOW Anywhere
5473/tcp ALLOW Anywhere
443/tcp ALLOW Anywhere
6443/tcp ALLOW Anywhere
2379/tcp ALLOW Anywhere
4149/tcp ALLOW Anywhere
10250/tcp ALLOW Anywhere
10255/tcp ALLOW Anywhere
10256/tcp ALLOW Anywhere
22/tcp (v6) ALLOW Anywhere (v6)
179/tcp (v6) ALLOW Anywhere (v6)
4789/tcp (v6) ALLOW Anywhere (v6)
5473/tcp (v6) ALLOW Anywhere (v6)
443/tcp (v6) ALLOW Anywhere (v6)
6443/tcp (v6) ALLOW Anywhere (v6)
2379/tcp (v6) ALLOW Anywhere (v6)
4149/tcp (v6) ALLOW Anywhere (v6)
10250/tcp (v6) ALLOW Anywhere (v6)
10255/tcp (v6) ALLOW Anywhere (v6)
10256/tcp (v6) ALLOW Anywhere (v6)
- 现在我将安装 Docker:
$ sudo apt-get update
$ sudo apt-get install -y apt-transport-https ca-certificates curl gnupg2 software-properties-common=
- 正在添加 Docker 存储库：
$ curl -fsSL https://download.docker.com/linux/debian/gpg | sudo apt-key add -
$ sudo apt-key fingerprint 0EBFCD88
$ sudo add-apt-repository "deb [arch=amd64] https://download.docker.com/linux/debian buster stable"
- 更新源列表并安装 Docker-ce:
$ sudo apt-get update
$ sudo apt-get -y install docker-ce
注意： 在生产系统上建议安装 docker 的固定版本：
$ apt-cache madison docker-ce
$ sudo apt-get install docker-ce=<VERSION>
- 正在安装 Kube Tools - kubeadm、kubectl、kubelet：
$ curl -s https://packages.cloud.google.com/apt/doc/apt-key.gpg | sudo apt-key add -
- 配置 Kubernetes 存储库（复制 3 行并立即粘贴）：
$ cat <<EOF | sudo tee /etc/apt/sources.list.d/kubernetes.list
deb https://apt.kubernetes.io/ kubernetes-xenial main
EOF
- 正在安装软件包：
$ sudo apt-get update
$ sudo apt-get install -y kubelet kubeadm kubectl
- 安装后将这些包标记为不自动更新：
$ sudo apt-mark hold kubelet kubeadm kubectl
$ sudo kubeadm init --pod-network-cidr=192.168.0.0/16
- 为非 root 用户启用 kubectl：
$ mkdir -p $HOME/.kube
$ sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
$ sudo chown $(id -u):$(id -g) $HOME/.kube/config
$ kubectl apply -f https://docs.projectcalico.org/manifests/calico.yaml
configmap/calico-config created
customresourcedefinition.apiextensions.k8s.io/bgpconfigurations.crd.projectcalico.org created
customresourcedefinition.apiextensions.k8s.io/bgppeers.crd.projectcalico.org created
customresourcedefinition.apiextensions.k8s.io/blockaffinities.crd.projectcalico.org created
customresourcedefinition.apiextensions.k8s.io/clusterinformations.crd.projectcalico.org created
customresourcedefinition.apiextensions.k8s.io/felixconfigurations.crd.projectcalico.org created
customresourcedefinition.apiextensions.k8s.io/globalnetworkpolicies.crd.projectcalico.org created
customresourcedefinition.apiextensions.k8s.io/globalnetworksets.crd.projectcalico.org created
customresourcedefinition.apiextensions.k8s.io/hostendpoints.crd.projectcalico.org created
customresourcedefinition.apiextensions.k8s.io/ipamblocks.crd.projectcalico.org created
customresourcedefinition.apiextensions.k8s.io/ipamconfigs.crd.projectcalico.org created
customresourcedefinition.apiextensions.k8s.io/ipamhandles.crd.projectcalico.org created
customresourcedefinition.apiextensions.k8s.io/ippools.crd.projectcalico.org created
customresourcedefinition.apiextensions.k8s.io/networkpolicies.crd.projectcalico.org created
customresourcedefinition.apiextensions.k8s.io/networksets.crd.projectcalico.org created
clusterrole.rbac.authorization.k8s.io/calico-kube-controllers created
clusterrolebinding.rbac.authorization.k8s.io/calico-kube-controllers created
clusterrole.rbac.authorization.k8s.io/calico-node created
clusterrolebinding.rbac.authorization.k8s.io/calico-node created
daemonset.apps/calico-node created
serviceaccount/calico-node created
deployment.apps/calico-kube-controllers created
serviceaccount/calico-kube-controllers created
- 查看状态：
$ kubectl get pods -n kube-system
NAME READY STATUS RESTARTS AGE
calico-kube-controllers-555fc8cc5c-wnnvq 1/1 Running 0 26m
calico-node-sngt8 1/1 Running 0 26m
coredns-66bff467f8-2qqlv 1/1 Running 0 55m
coredns-66bff467f8-vptpr 1/1 Running 0 55m
etcd-kubeadm-ufw-debian10 1/1 Running 0 55m
kube-apiserver-kubeadm-ufw-debian10 1/1 Running 0 55m
kube-controller-manager-kubeadm-ufw-debian10 1/1 Running 0 55m
kube-proxy-nx8cz 1/1 Running 0 55m
kube-scheduler-kubeadm-ufw-debian10 1/1 Running 0 55m
考虑因素：
Sorry my ufw rules are a bit messy, I tried too many things to get kubernetes working.
- 为了使某事成功而尝试很多事情是正常的，但有时它本身最终会成为问题。
- 我将一步一步地发布给您，以便将它部署在与您相同的环境中，以便您可以再次按照它来获得相同的结果。
- 我的 felix 探测器没有出现任何错误，只有当我尝试（有意）部署 kubernetes 而没有在 ufw 上创建规则时出现错误。
如果没有解决，下一步：
- 现在，如果按照本教程进行操作后您仍然遇到类似问题，请使用以下信息更新问题：
kubectl describe <pod_name> -n kube-systemkubectl get pod <pod_name> -n kube-systemkubectl logs <pod_name> -n kube-system- 始终建议从 Linux 的全新安装开始，如果您是 运行 VM，请删除该 VM 并创建一个新的。
- 如果您 运行 在裸机上，请考虑 运行 服务器上还有什么，也许有其他软件干扰了网络通信。
如果您按照这些故障排除步骤发现任何问题，请在评论中告诉我。