在入口处公开 Google 端点 API

Question

我在 google 端点上有一个 API，后端部署在 GKE 上，我想通过入口公开它，以便我可以在其上使用 IAP。
我正在使用 ESP2。
我首先将我的服务部署为 LoadBalancer，它正在运行。

事情是我的入口说： “所有后端服务都处于不健康状态”
我知道健康检查没有通过，但我不明白为什么...

服务和对应的 pod 没有显示错误，但是在我的 pod 事件中我可以看到： “就绪探测失败：获取 http://10.32.1.27:8000/swagger：拨打 tcp 10.32.1.27:8000：连接：连接被拒绝“

我的 Pod 和服务配置如下所示：

apiVersion: v1
kind: Service
metadata:
 name: devfleet-django-endpoint-service
 namespace: my-ns
spec:
  # NodePort is mandatory for Ingress to perform load balancer
  type: NodePort
  ports:
  - port: 443
    protocol: TCP
    targetPort: 9000
    name: https
  selector:
    app: devfleet-django-endpoint
---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: devfleet-django-endpoint
  namespace: my-ns
spec:
  replicas: 2
  selector:
    matchLabels:
      app: devfleet-django-endpoint
   template:
    metadata:
      labels:
        app: devfleet-django-endpoint
    spec:
      containers:
      - name: esp
        image: gcr.io/endpoints-release/endpoints-runtime:2
        args: [
          "--listener_port=9000",
          "--backend", "127.0.0.1:8080",
          "--service=my-custom-domain.io",
          "--rollout_strategy=managed",
          "-z", "healthz",
          "--ssl_server_cert_path", "/etc/esp/ssl"
        ]
        volumeMounts:
        - mountPath: /etc/esp/ssl
          name: esp-ssl
          readOnly: true
        - mountPath: /etc/nginx/custom
          name: nginx-config
          readOnly: true        
        ports:
          - containerPort: 9000
      - name: devfleet-django-endpoint
        image: my-img
        ports:
        - containerPort: 8000
        imagePullPolicy: Always
        env:
         some_env_data
        readinessProbe:
          httpGet:
            path: /swagger
            port: 8000
          initialDelaySeconds: 10
          periodSeconds: 10
          timeoutSeconds: 10
          failureThreshold: 3
        livenessProbe:
          httpGet:
           path: /swagger
            port: 8000
          initialDelaySeconds: 15
          periodSeconds: 60
          timeoutSeconds: 10
          failureThreshold: 3
        volumeMounts:
          - name: devfleet-storage
            mountPath: /secrets/cloudstorage
            readOnly: true
        resources:
          requests:
            cpu: 30m
            memory: 90Mi
          limits:
            cpu: 200m
            memory: 400Mi            
      - image: b.gcr.io/cloudsql-docker/gce-proxy:1.13
        name: cloudsql-proxy
        command: ["/cloud_sql_proxy", "--dir=/app",
                  "-instances=production-213911:europe-west1:dev-postgresql=tcp:5432",
                  "-credential_file=/secrets/cloudsql/credentials.json"]
        volumeMounts:
           - name: prodsql-oauth-credentials
            mountPath: /secrets/cloudsql
            readOnly: true
         resources:
          requests:
            cpu: 10m
            memory: 10Mi
          limits:
            cpu: 20m
            memory: 50Mi          
      volumes:
        - name: prodsql-oauth-credentials
          secret:
            secretName: secret-name
        - name: devfleet-storage
           secret:
            secretName: secret-name
        - name: app
           emptyDir:
        - name: esp-ssl
          secret:
            secretName: secret-name
        - name: nginx-config
          configMap:
            name: nginx-config
          
      nodeSelector:
        cloud.google.com/gke-nodepool: default-pool

我的入口配置是：

apiVersion: extensions/v1beta1
kind: Ingress
metadata:
  name: devapi
  namespace: my-ns
  annotations:
    kubernetes.io/ingress.global-static-ip-name: ingress-devapi
    ingress.gcp.kubernetes.io/pre-shared-cert: "my-cert"
    kubernetes.io/ingress.class: gce
    ingress.kubernetes.io/enable-cors: "true"
    kubernetes.io/ingress.allow-http: "false"
spec:
  backend:
    serviceName: devfleet-django-endpoint-service
    servicePort: 443

知道我做错了什么吗？
谢谢

Answer 1

因此，为了将您的 API 部署在 google 端点上暴露在同一个 pod 中的 esp 容器和您的应用程序容器中，您将需要创建具有特定健康检查的后端配置.

这是我所做的：

添加了后端配置：

apiVersion: cloud.google.com/v1  
kind: BackendConfig  
metadata:  
   name: esp-config  
  namespace: my-namespace  
spec:  
  healthCheck:  
    checkIntervalSec: 80  
    port: the-node-port-on-which-your-service-is-exposed  
    type: HTTPS  
    requestPath: /healthz

apiVersion: cloud.google.com/v1
kind: BackendConfig
metadata:
  name: devapi-config
  namespace: my-namespace
spec:
  healthCheck:
    checkIntervalSec: 80
    port: the-node-port-on-which-your-service-is-exposed
    requestPath: /

更新我的服务：

apiVersion: v1
kind: Service
metadata:
  name: name
  namespace: my-namespace
  annotations:
    cloud.google.com/backend-config: '{"ports": {
    "443":"esp-config",
    "8035":"devapi-config"
    }}'

spec:
  ports:
  - port: Exposed port
    targetPort: target-port-you-set-in-deployment
    protocol: TCP
    name: name
  - port: Exposed port
    targetPort: target-port-you-set-in-deployment
    protocol: TCP
    name: another-name
  selector:
    app: name
  type: NodePort

更新我的入口

apiVersion: extensions/v1beta1
kind: Ingress
metadata:
  name: name
  namespace: my-namespace
  annotations:
    kubernetes.io/ingress.global-static-ip-name: static-ip-name
    ingress.gcp.kubernetes.io/pre-shared-cert: my-cert
    kubernetes.io/ingress.class: gce
    ingress.kubernetes.io/enable-cors: "true"

spec:
  backend:
    serviceName: my-service-name
    servicePort: The-exposed-port-of-my-application (not esp)

在入口处公开 Google 端点 API

Expose Google Endpoint API on ingress

google-cloud-endpoints

kubernetes-health-check

kubernetes-ingress