无法从 Windows GKE pod 访问互联网

Question

我使用

创建了一个带 Windows 池的 Kubernetes 集群

gcloud beta container clusters create test-cluster --enable-ip-alias  --num-nodes=1  --release-channel=rapid
gcloud container node-pools create rpp2-pool  --cluster=test-cluster --image-type=WINDOWS_LTSC --enable-autoupgrade --machine-type=n1-standard-2
gcloud container clusters get-credentials test-cluster

并在其中部署了一个基于mcr.microsoft.com/windows/servercore:ltsc2019的容器。

当我 kubectl exec -it win-webserver powershell 进入 pod 时，我无法 ping google.com。但是，我可以从 pod 所在的 VM 实例 ping google.com。google.com。

当我 kubectl run 基于 linux (busybox) ping 的图像从那里工作时。

我已经应用了这些说明，但它仍然不起作用： https://cloud.google.com/compute/docs/containers/#mtu_failures

编辑：我可以通过它们的 pod 名称和服务名称访问集群中的其他 pods（但只有当它们部署到同一节点时，请查看下面的 EDIT2）。我也可以 ping 邻居 linux 虚拟机（默认池）。无法 ping Windows 虚拟机的默认网关 - 10.132.0.1 - 不确定是否应该是这种情况。

在 Windows VM 实例上：

ipconfig

Windows IP Configuration
Ethernet adapter vEthernet (Ethernet):

   Connection-specific DNS Suffix  . : europe-west1-b.c.rpp2-261008.internal
   Link-local IPv6 Address . . . . . : fe80::2d21:4bd7:fd85:2533%14
   IPv4 Address. . . . . . . . . . . : 10.132.0.7
   Subnet Mask . . . . . . . . . . . : 255.255.240.0
   Default Gateway . . . . . . . . . : 10.132.0.1

Ethernet adapter vEthernet (cbr0):

   Connection-specific DNS Suffix  . :
   Link-local IPv6 Address . . . . . : fe80::1111:61b8:97de:83f8%21
   IPv4 Address. . . . . . . . . . . : 10.44.1.2
   Subnet Mask . . . . . . . . . . . : 255.255.255.0
   Default Gateway . . . . . . . . . :

Ethernet adapter vEthernet (nat):

   Connection-specific DNS Suffix  . :
   Link-local IPv6 Address . . . . . : fe80::5c57:5e77:1a8a:eccc%9
   IPv4 Address. . . . . . . . . . . : 172.21.48.1
   Subnet Mask . . . . . . . . . . . : 255.255.240.0
   Default Gateway . . . . . . . . . :

我使用的 pod yaml 文件 kubectl apply:

apiVersion: v1
kind: Service
metadata:
  name: win-webserver
  labels:
    app: win-webserver
spec:
  ports:
    # the port that this service should serve on
    - port: 80
      targetPort: 80
  selector:
    app: win-webserver
  type: NodePort
---
apiVersion: apps/v1
kind: Deployment
metadata:
  labels:
    app: win-webserver
  name: win-webserver
spec:
  replicas: 1
  selector:
    matchLabels:
      app: win-webserver
  template:
    metadata:
      labels:
        app: win-webserver
      name: win-webserver
    spec:
     containers:
      - name: windowswebserver
        image: mcr.microsoft.com/windows/servercore:ltsc2019
        command:
        - powershell.exe
        - -command
        - "<#code used from https://gist.github.com/wagnerandrade/5424431#> ; $$listener = New-Object System.Net.HttpListener ; $$listener.Prefixes.Add('http://*:80/') ; $$listener.Start() ; $$callerCounts = @{} ; Write-Host('Listening at http://*:80/') ; while ($$listener.IsListening) { ;$$context = $$listener.GetContext() ;$$requestUrl = $$context.Request.Url ;$$clientIP = $$context.Request.RemoteEndPoint.Address ;$$response = $$context.Response ;Write-Host '' ;Write-Host('> {0}' -f $$requestUrl) ;  ;$$count = 1 ;$$k=$$callerCounts.Get_Item($$clientIP) ;if ($$k -ne $$null) { $$count += $$k } ;$$callerCounts.Set_Item($$clientIP, $$count) ;$$ip=(Get-NetAdapter | Get-NetIpAddress); $$header='<html><body><H1>Windows Container Web Server</H1>' ;$$callerCountsString='' ;$$callerCounts.Keys | % { $$callerCountsString+='<p>IP {0} callerCount {1} ' -f $$ip[1].IPAddress,$$callerCounts.Item($$_) } ;$$footer='</body></html>' ;$$content='{0}{1}{2}' -f $$header,$$callerCountsString,$$footer ;Write-Output $$content ;$$buffer = [System.Text.Encoding]::UTF8.GetBytes($$content) ;$$response.ContentLength64 = $$buffer.Length ;$$response.OutputStream.Write($$buffer, 0, $$buffer.Length) ;$$response.Close() ;$$responseStatus = $$response.StatusCode ;Write-Host('< {0}' -f $$responseStatus)  } ; "
     nodeSelector:
      beta.kubernetes.io/os: windows

编辑2：我注意到我可以通过名称 ping 集群中的其他 pods，但前提是它们已部署到同一节点：

NAME                             READY   STATUS    RESTARTS   AGE     IP
win-webserver-75bc4c4c6f-5w9q5   1/1     Running   0          8m34s   10.52.2.4
win-webserver-75bc4c4c6f-d5wlv   1/1     Running   0          22h     10.52.1.4
win-webserver-75bc4c4c6f-pjz57   1/1     Running   0          8m34s   10.52.2.5

从第一个 pod 我只能通过它的名字 ping 第三个而不是第二个，但是我可以正常 ping IP 10.52.1.4。

Answer 1

在这里找到答案： https://kubernetes.io/docs/setup/production-environment/windows/intro-windows-in-kubernetes/#troubleshooting

3. My Windows Pods do not have network connectivity

If you are using virtual machines, ensure that MAC spoofing is enabled on all the VM network adapter(s).

4. My Windows Pods cannot ping external resources

Windows Pods do not have outbound rules programmed for the ICMP protocol today. However, TCP/UDP is supported. When trying to demonstrate connectivity to resources outside of the cluster, please substitute ping <IP> with corresponding curl <IP> commands.

If you are still facing problems, most likely your network configuration in cni.conf deserves some extra attention. You can always edit this static file. The configuration update will apply to any newly created Kubernetes resources.

One of the Kubernetes networking requirements (see Kubernetes model) is for cluster communication to occur without NAT internally. To honor this requirement, there is an ExceptionList for all the communication where we do not want outbound NAT to occur. However, this also means that you need to exclude the external IP you are trying to query from the ExceptionList. Only then will the traffic originating from your Windows pods be SNAT’ed correctly to receive a response from the outside world. In this regard, your ExceptionList in cni.conf should look as follows:

"ExceptionList": [
                "10.244.0.0/16",  # Cluster subnet
                "10.96.0.0/12",   # Service subnet
                "10.127.130.0/24" # Management (host) subnet
            ]

wget google.com -UseBasicParsing returns 200 行。

无法从 Windows GKE pod 访问互联网

Can't access internet from within Windows GKE pod

kubernetes

google-cloud-platform

google-kubernetes-engine

docker-for-windows

windows-server-2019