Kubernetes nginx ingress 定期给出 404

Question

我已经在 AWS 中部署了 kops k8s，一切都在同一个命名空间中。

nginx 入口控制器将流量路由到 https 后端（wordpress 应用程序）。

我可以访问该网站，但不幸的是，每 10~ 次调用只有 1 次获得 http 200。所有其他 9 次获得 404 nginx 未找到。试图到处搜索但没有运气:(

我的配置： DNS -> AWS NLB -> 2 个节点

ingress.yaml

apiVersion: extensions/v1beta1
kind: Ingress
metadata:
  name: ingress-nginx
  namespace: example-ns
  labels:
    app.kubernetes.io/name: ingress-nginx
    app.kubernetes.io/part-of: ingress-nginx
  annotations:
    kubernetes.io/ingress.class: nginx
    nginx.ingress.kubernetes.io/backend-protocol: "HTTPS"
    nginx.ingress.kubernetes.io/ssl-passthrough: "True"
    nginx.org/ssl-services: test-service
    nginx.ingress.kubernetes.io/affinity: "cookie"
spec:
  rules:
  - host: "test.example.com"
    http:
      paths:
      - path: /
        backend:
          serviceName: test-service
          servicePort: 8443

nginx-service.yaml:

kind: Service
apiVersion: v1
metadata:
  name: ingress-nginx
  namespace: example-ns
  labels:
    app.kubernetes.io/name: ingress-nginx
    app.kubernetes.io/part-of: ingress-nginx
  annotations:
    service.beta.kubernetes.io/aws-load-balancer-type: nlb
    service.beta.kubernetes.io/aws-load-balancer-connection-idle-timeout: "60"
spec:
  externalTrafficPolicy: Local
  type: LoadBalancer
  selector:
    app.kubernetes.io/name: ingress-nginx
    app.kubernetes.io/part-of: ingress-nginx
  ports:
    - name: http
      port: 80
      targetPort: http
    - name: https
      port: 443
      targetPort: https

nginx-daemonset.yaml:

kind: DaemonSet
apiVersion: extensions/v1beta1
metadata:
  name: nginx-ingress-controller
  namespace: example-ns
  labels:
    app.kubernetes.io/name: ingress-nginx
    app.kubernetes.io/part-of: ingress-nginx
spec:
  selector:
    matchLabels:
      app.kubernetes.io/name: ingress-nginx
      app.kubernetes.io/part-of: ingress-nginx
  template:
    metadata:
      labels:
        app.kubernetes.io/name: ingress-nginx
        app.kubernetes.io/part-of: ingress-nginx
      annotations:
        prometheus.io/port: "10254"
        prometheus.io/scrape: "true"
    spec:
      serviceAccountName: nginx-ingress-serviceaccount
      imagePullSecrets:
      - name: private-repo
      containers:
        - name: nginx-ingress-controller
          image: private_repo/private_image
          args:
            - /nginx-ingress-controller
            - --configmap=$(POD_NAMESPACE)/nginx-configuration
            - --tcp-services-configmap=$(POD_NAMESPACE)/tcp-services
            - --udp-services-configmap=$(POD_NAMESPACE)/udp-services
            - --publish-service=$(POD_NAMESPACE)/ingress-nginx
            - --annotations-prefix=nginx.ingress.kubernetes.io
            - --default-ssl-certificate=$(POD_NAMESPACE)/tls-cert
          securityContext:
            allowPrivilegeEscalation: true
            capabilities:
              drop:
                - ALL
              add:
                - NET_BIND_SERVICE
            runAsUser: 33
          resources:
            limits:
              cpu: 500m
              memory: 300Mi
            requests:
              cpu: 400m
              memory: 200Mi
          env:
            - name: POD_NAME
              valueFrom:
                fieldRef:
                  fieldPath: metadata.name
            - name: POD_NAMESPACE
              valueFrom:
                fieldRef:
                  fieldPath: metadata.namespace
          ports:
            - name: https
              containerPort: 443
          livenessProbe:
            failureThreshold: 3
            httpGet:
              path: /healthz
              port: 10254
              scheme: HTTP
            initialDelaySeconds: 10
            periodSeconds: 10
            successThreshold: 1
            timeoutSeconds: 10
          readinessProbe:
            failureThreshold: 3
            httpGet:
              path: /healthz
              port: 10254
              scheme: HTTP
            periodSeconds: 10
            successThreshold: 1
            timeoutSeconds: 10

wordpress.yaml:

apiVersion: apps/v1
kind: Deployment

metadata:
  name: test-example
  namespace: example-ns
  labels:
    app.kubernetes.io/name: ingress-nginx
    app.kubernetes.io/part-of: ingress-nginx
spec:
  replicas: 1
  selector:
    matchLabels:
      app.kubernetes.io/name: ingress-nginx
      app.kubernetes.io/part-of: ingress-nginx
  strategy:
    type: RollingUpdate
  template:
    metadata:
      labels:
        app.kubernetes.io/name: ingress-nginx
        app.kubernetes.io/part-of: ingress-nginx
    spec:
      restartPolicy: Always
      volumes:
      - name: volume
        persistentVolumeClaim:
           claimName: volume-claim
      imagePullSecrets:
      - name: private-repo

      containers:
      - name: test-example-httpd
        image: private_repo/private_image
        imagePullPolicy: Always
        ports:
        - containerPort: 8443
          name: https

      - name: test-example-php-fpm
        image: private_repo/private_image
        imagePullPolicy: Always
        securityContext:
          runAsUser: 82
        securityContext:
          allowPrivilegeEscalation: false

---
apiVersion: v1
kind: Service
metadata:
  name: test-service
  namespace: example-ns
  labels:
    app.kubernetes.io/name: ingress-nginx
    app.kubernetes.io/part-of: ingress-nginx
spec:
  ports:
  - name: https-web
    targetPort: 8443
    port: 8443
  selector:
    app.kubernetes.io/name: ingress-nginx
    app.kubernetes.io/part-of: ingress-nginx

---更新---

kubectl get  endpoints,services -n example-ns
NAME                           ENDPOINTS                                            AGE
endpoints/ingress-nginx        100.101.0.1:8443,100.100.0.4:443,100.101.0.2:443     1d
endpoints/test-service   100.100.0.1:8443,100.101.0.1:8443,100.101.0.2:8443   4h

NAME                         TYPE           CLUSTER-IP       EXTERNAL-IP                                                                     PORT(S)                      AGE
service/ingress-nginx        LoadBalancer   SOME-IP     sometext.elb.us-west-3.amazonaws.com   80:31541/TCP,443:31017/TCP   1d
service/test-service   ClusterIP      SOME-IP   <none>                                                                          8443/TCP                     4h

谢谢！

Answer 1

显然将注释 nginx.ingress.kubernetes.io/ssl-passthrough 从 "True" 更改为 "False" 解决了这个问题。

可能必须在 NGINX 而不是 apache 中对 ssl 终止做一些事情。

Kubernetes nginx ingress 定期给出 404

Kubernetes nginx ingress periodically gives 404

amazon-web-services

kubernetes

aws-elb

nginx-ingress