Kubernetes API 服务器无法使用 Calico CNI 解析 ESK 上的 webhook

Question

我正在尝试在 AWS EKS 中部署应用程序。我按照 the official Calico documentation. I have also installed the AWS load balancer controller by following the docs here.

使用 Calico CNI 创建了一个 EKS 集群

这是我的集群、部署和入口配置文件。

cluster.yaml

apiVersion: eksctl.io/v1alpha5
kind: ClusterConfig

metadata:
  name: clustername
  region: us-east-2

nodeGroups:
  - name: ng1
    instanceType: t3.medium
    desiredCapacity: 1
    volumeSize: 30
    maxPodsPerNode: 250
    ami: auto
    ssh:
      publicKeyName: keyname

deployment.yaml

apiVersion: apps/v1
kind: Deployment
metadata:
  namespace: my_namspace
  name: deployment-2048
spec:
  selector:
    matchLabels:
      app.kubernetes.io/name: app-2048
  replicas: 1
  template:
    metadata:
      labels:
        app.kubernetes.io/name: app-2048
    spec:
      containers:
      - image: alexwhen/docker-2048
        imagePullPolicy: Always
        name: app-2048
        ports:
        - containerPort: 80
---
apiVersion: v1
kind: Service
metadata:
  namespace: my_namspace
  name: service-2048
spec:
  ports:
    - port: 80
      targetPort: 80
      protocol: TCP
  type: NodePort
  selector:
    app.kubernetes.io/name: app-2048

ingress.yaml

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: my_namspace-ingress
  namespace: my_namspace
  annotations:
    kubernetes.io/ingress.class: alb
    alb.ingress.kubernetes.io/scheme: internet-facing
spec:
  rules:
    - host: domain.io
      http:
        paths:
        - path: /*
          pathType: ImplementationSpecific
          backend:
            service:
              name: service-2048
              port:
                number: 80

kubectl get pods --namespace kube-system -o wide

NAME                                            READY   STATUS    RESTARTS   AGE    IP              NODE                                          NOMINATED NODE   READINESS GATES
aws-load-balancer-controller-568d85bd58-6jpk5   1/1     Running   0          74m    172.16.22.4     ip-192-168-32-46.us-east-2.compute.internal   <none>           <none>
aws-load-balancer-controller-568d85bd58-ph44m   1/1     Running   0          74m    172.16.22.5     ip-192-168-32-46.us-east-2.compute.internal   <none>           <none>
calico-kube-controllers-6fd7b9848d-8lw4t        1/1     Running   0          91m    172.16.22.3     ip-192-168-32-46.us-east-2.compute.internal   <none>           <none>
calico-node-xdw2h                               1/1     Running   0          87m    192.168.32.46   ip-192-168-32-46.us-east-2.compute.internal   <none>           <none>
coredns-f47955f89-5qwh4                         1/1     Running   0          110m   172.16.22.2     ip-192-168-32-46.us-east-2.compute.internal   <none>           <none>
coredns-f47955f89-qfpbl                         1/1     Running   0          111m   172.16.22.1     ip-192-168-32-46.us-east-2.compute.internal   <none>           <none>
kube-proxy-bnw6v                                1/1     Running   0          87m    192.168.32.46   ip-192-168-32-46.us-east-2.compute.internal   <none>           <none>

如您所见，一切运行顺利。问题是当我尝试使用 kubectl apply -f ingress.yaml

应用我的入口时

Error from server (InternalError): error when creating "ingress-alb.yaml": Internal error occurred: failed calling webhook "vingress.elbv2.k8s.aws": Post "https://aws-load-balancer-webhook-service.kube-system.svc:443/validate-networking-v1-ingress?timeout=10s": Address is not allowed

我了解到 from here 这是 EKS 上 calico 的常见问题，并且还尝试按照提供的解决方案在部署文件和负载均衡器中使用 hostNetwork: true控制器。

helm upgrade aws-load-balancer-controller eks/aws-load-balancer-controller \
  -n kube-system \
  --set clusterName=clustername \
  --set serviceAccount.create=false \
  --set serviceAccount.name=aws-load-balancer-controller \
  --set hostNetwork=true

但反应是一样的。不知何故，对其他人有用的解决方案对我不起作用。也许我漏掉了什么，我真的很想找出来。

Answer 1

我建议您不要将 Calico CNI 方法与 Amazon EKS 服务一起使用。

去年，我花时间研究了这个解决方案，并联系了 AWS 专家团队来解决一个用这个方法的用例。过了一会儿，Calico 官方网站上有一个大大的NOTE，如下所示。我相信 EKS 控制平面（以及 GKE 控制平面）都无法验证您 pods 的 NATed IP 地址。两家云厂商管理的控制面IP范围都在VPC内。

参考：https://projectcalico.docs.tigera.io/getting-started/kubernetes/managed-public-cloud/eks

Kubernetes API 服务器无法使用 Calico CNI 解析 ESK 上的 webhook

Kubernetes API server failed to resolve webhook on ESK with Calico CNI

amazon-web-services

kubernetes

amazon-eks

calico