如何使用从 Kubernetes 入口到 pods 的 SSL 请求

Question

我正在使用 gitlab kubernetes 集成进行 kubernetes 应用程序部署。我运行遇到一个问题，即在将 pods（容器）置于 ssl 之后，浏览器响应：

    Bad Request
    Your browser sent a request that this server could not understand.
    Reason: You're speaking plain HTTP to an SSL-enabled server port.
    Instead use the HTTPS scheme to access this URL, please.
    Apache/2.4.38 (Debian) Server at docker.vm Port 80

我正在使用 https://***********.eu/ 访问浏览器 url 并且不知道为什么它在 kubernetes 中从 https 重定向到 http去 pods.

的路

我的Ingress.yaml

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  annotations:
    field.cattle.io/publicEndpoints: '[{"addresses":["******"],"port":443,"protocol":"HTTPS","serviceName":"******","ingressName":"******","hostname":"******","path":"/","allNodes":true},{"addresses":["******"],"port":443,"protocol":"HTTPS","serviceName":"******","ingressName":"******","hostname":"******","path":"/","allNodes":true}]'
    kubernetes.io/ingress.class: nginx
    kubernetes.io/tls-acme: "true"
  creationTimestamp: "2021-05-21T12:54:44Z"
  generation: 1
  labels:
    app: development
    chart: auto-deploy-app-1.0.7
    heritage: Tiller
    release: development
  managedFields:
  - apiVersion: extensions/v1beta1
    fieldsType: FieldsV1
    fieldsV1:
      f:metadata:
        f:annotations:
          .: {}
          f:kubernetes.io/ingress.class: {}
          f:kubernetes.io/tls-acme: {}
        f:labels:
          .: {}
          f:app: {}
          f:chart: {}
          f:heritage: {}
          f:release: {}
      f:spec:
        f:rules: {}
        f:tls: {}
    manager: Go-http-client
    operation: Update
    time: "2021-05-21T12:54:44Z"
  - apiVersion: networking.k8s.io/v1beta1
    fieldsType: FieldsV1
    fieldsV1:
      f:status:
        f:loadBalancer:
          f:ingress: {}
    manager: nginx-ingress-controller
    operation: Update
    time: "2021-05-21T12:55:25Z"
  - apiVersion: extensions/v1beta1
    fieldsType: FieldsV1
    fieldsV1:
      f:metadata:
        f:annotations:
          f:field.cattle.io/publicEndpoints: {}
    manager: rancher
    operation: Update
    time: "2021-05-21T12:55:25Z"
  name: development-auto-deploy
  namespace: ******
  resourceVersion: "******"
  selfLink: /apis/networking.k8s.io/v1/namespaces/******
  uid: ******
spec:
  rules:
  - host: ******
    http:
      paths:
      - backend:
          service:
            name: development-auto-deploy
            port:
              number: 443
        path: /
        pathType: ImplementationSpecific
  - host: ******
    http:
      paths:
      - backend:
          service:
            name: development-auto-deploy
            port:
              number: 443
        path: /
        pathType: ImplementationSpecific
  tls:
  - hosts:
    - ******
    - ******
    secretName: development-auto-deploy-tls
</pre>

My Service.yaml

<pre>
apiVersion: v1
kind: Service
metadata:
  creationTimestamp: "2021-05-21T12:54:44Z"
  labels:
    app: development
    chart: auto-deploy-app-1.0.7
    heritage: Tiller
    release: development
  managedFields:
  - apiVersion: v1
    fieldsType: FieldsV1
    fieldsV1:
      f:metadata:
        f:labels:
          .: {}
          f:app: {}
          f:chart: {}
          f:heritage: {}
          f:release: {}
      f:spec:
        f:ports:
          .: {}
          k:{"port":443,"protocol":"TCP"}:
            .: {}
            f:name: {}
            f:port: {}
            f:protocol: {}
            f:targetPort: {}
        f:selector:
          .: {}
          f:app: {}
          f:tier: {}
        f:sessionAffinity: {}
        f:type: {}
    manager: Go-http-client
    operation: Update
    time: "2021-05-21T12:54:44Z"
  name: development-auto-deploy
  namespace: ******
  resourceVersion: "******"
  selfLink: /api/v1/namespaces/******
  uid: ******
spec:
  clusterIP: ******
  ports:
  - name: web
    port: 443
    protocol: TCP
    targetPort: 443
  selector:
    app: development
    tier: web
  sessionAffinity: None
  type: ClusterIP
status:
  loadBalancer: {}
>

和 deployment.yaml 用于 pod 部署：

apiVersion: apps/v1
kind: Deployment
metadata:
  annotations:
    app.gitlab.com/app: *******
    app.gitlab.com/env: development
    deployment.kubernetes.io/revision: "1"
    field.cattle.io/publicEndpoints: '[{"addresses":["*******"],"port":443,"protocol":"HTTPS","serviceName":"*******","ingressName":"*******","hostname":"*******","path":"/","allNodes":true},{"addresses":["*******"],"port":443,"protocol":"HTTPS","serviceName":"*******","ingressName":"*******","hostname":"*******","path":"/","allNodes":true}]'
  creationTimestamp: "2021-05-21T12:54:44Z"
  generation: 2
  labels:
    app: development
    chart: auto-deploy-app-1.0.7
    heritage: Tiller
    release: development
    tier: web
    track: stable
  managedFields:
  - apiVersion: apps/v1
    fieldsType: FieldsV1
    fieldsV1:
      f:metadata:
        f:annotations:
          .: {}
          f:app.gitlab.com/app: {}
          f:app.gitlab.com/env: {}
        f:labels:
          .: {}
          f:app: {}
          f:chart: {}
          f:heritage: {}
          f:release: {}
          f:tier: {}
          f:track: {}
      f:spec:
        f:progressDeadlineSeconds: {}
        f:replicas: {}
        f:revisionHistoryLimit: {}
        f:selector:
          f:matchLabels:
            .: {}
            f:app: {}
            f:release: {}
            f:tier: {}
            f:track: {}
        f:strategy:
          f:rollingUpdate:
            .: {}
            f:maxSurge: {}
            f:maxUnavailable: {}
          f:type: {}
        f:template:
          f:metadata:
            f:annotations:
              .: {}
              f:app.gitlab.com/app: {}
              f:app.gitlab.com/env: {}
              f:checksum/application-secrets: {}
            f:labels:
              .: {}
              f:app: {}
              f:release: {}
              f:tier: {}
              f:track: {}
          f:spec:
            f:containers:
              k:{"name":"auto-deploy-app"}:
                .: {}
                f:env:
                  .: {}
                  k:{"name":"DATABASE_URL"}:
                    .: {}
                    f:name: {}
                    f:value: {}
                  k:{"name":"GITLAB_ENVIRONMENT_NAME"}:
                    .: {}
                    f:name: {}
                    f:value: {}
                  k:{"name":"GITLAB_ENVIRONMENT_URL"}:
                    .: {}
                    f:name: {}
                    f:value: {}
                f:envFrom: {}
                f:image: {}
                f:imagePullPolicy: {}
                f:livenessProbe:
                  .: {}
                  f:failureThreshold: {}
                  f:httpGet:
                    .: {}
                    f:path: {}
                    f:port: {}
                    f:scheme: {}
                  f:initialDelaySeconds: {}
                  f:periodSeconds: {}
                  f:successThreshold: {}
                  f:timeoutSeconds: {}
                f:name: {}
                f:ports:
                  .: {}
                  k:{"containerPort":443,"protocol":"TCP"}:
                    .: {}
                    f:containerPort: {}
                    f:name: {}
                    f:protocol: {}
                f:readinessProbe:
                  .: {}
                  f:failureThreshold: {}
                  f:httpGet:
                    .: {}
                    f:path: {}
                    f:port: {}
                    f:scheme: {}
                  f:initialDelaySeconds: {}
                  f:periodSeconds: {}
                  f:successThreshold: {}
                  f:timeoutSeconds: {}
                f:resources: {}
                f:terminationMessagePath: {}
                f:terminationMessagePolicy: {}
            f:dnsPolicy: {}
            f:imagePullSecrets:
              .: {}
              k:{"name":"*******"}:
                .: {}
                f:name: {}
            f:restartPolicy: {}
            f:schedulerName: {}
            f:securityContext: {}
            f:terminationGracePeriodSeconds: {}
    manager: Go-http-client
    operation: Update
    time: "2021-05-21T12:54:44Z"
  - apiVersion: apps/v1
    fieldsType: FieldsV1
    fieldsV1:
      f:metadata:
        f:annotations:
          f:deployment.kubernetes.io/revision: {}
      f:status:
        f:availableReplicas: {}
        f:conditions:
          .: {}
          k:{"type":"Available"}:
            .: {}
            f:lastTransitionTime: {}
            f:lastUpdateTime: {}
            f:message: {}
            f:reason: {}
            f:status: {}
            f:type: {}
          k:{"type":"Progressing"}:
            .: {}
            f:lastTransitionTime: {}
            f:lastUpdateTime: {}
            f:message: {}
            f:reason: {}
            f:status: {}
            f:type: {}
        f:observedGeneration: {}
        f:readyReplicas: {}
        f:replicas: {}
        f:updatedReplicas: {}
    manager: kube-controller-manager
    operation: Update
    time: "2021-05-21T12:54:55Z"
  - apiVersion: apps/v1
    fieldsType: FieldsV1
    fieldsV1:
      f:metadata:
        f:annotations:
          f:field.cattle.io/publicEndpoints: {}
    manager: rancher
    operation: Update
    time: "2021-05-21T12:55:25Z"
  name: development
  namespace: *******
  resourceVersion: "*******"
  selfLink: /apis/apps/v1/namespaces/*******
  uid: *******
spec:
  progressDeadlineSeconds: 600
  replicas: 1
  revisionHistoryLimit: 10
  selector:
    matchLabels:
      app: development
      release: development
      tier: web
      track: stable
  strategy:
    rollingUpdate:
      maxSurge: 25%
      maxUnavailable: 25%
    type: RollingUpdate
  template:
    metadata:
      annotations:
        app.gitlab.com/app: *******
        app.gitlab.com/env: development
        checksum/application-secrets: *******
      creationTimestamp: null
      labels:
        app: development
        release: development
        tier: web
        track: stable
    spec:
      containers:
      - env:
        - name: DATABASE_URL
          value: ' '
        - name: GITLAB_ENVIRONMENT_NAME
          value: development
        - name: GITLAB_ENVIRONMENT_URL
          value: *******
        envFrom:
        - secretRef:
            name: development-secret
        image: *******
        imagePullPolicy: IfNotPresent
        livenessProbe:
          failureThreshold: 3
          httpGet:
            path: /
            port: 443
            scheme: HTTPS
          initialDelaySeconds: 15
          periodSeconds: 10
          successThreshold: 1
          timeoutSeconds: 15
        name: auto-deploy-app
        ports:
        - containerPort: 443
          name: web
          protocol: TCP
        readinessProbe:
          failureThreshold: 3
          httpGet:
            path: /
            port: 443
            scheme: HTTPS
          initialDelaySeconds: 5
          periodSeconds: 10
          successThreshold: 1
          timeoutSeconds: 3
        resources: {}
        terminationMessagePath: /dev/termination-log
        terminationMessagePolicy: File
      dnsPolicy: ClusterFirst
      imagePullSecrets:
      - name: *******
      restartPolicy: Always
      schedulerName: default-scheduler
      securityContext: {}
      terminationGracePeriodSeconds: 30
status:
  availableReplicas: 1
  conditions:
  - lastTransitionTime: "2021-05-21T12:54:55Z"
    lastUpdateTime: "2021-05-21T12:54:55Z"
    message: Deployment has minimum availability.
    reason: MinimumReplicasAvailable
    status: "True"
    type: Available
  - lastTransitionTime: "2021-05-21T12:54:44Z"
    lastUpdateTime: "2021-05-21T12:54:55Z"
    message: ReplicaSet "*******" has successfully progressed.
    reason: NewReplicaSetAvailable
    status: "True"
    type: Progressing
  observedGeneration: 2
  readyReplicas: 1
  replicas: 1
  updatedReplicas: 1

SSL 在某处被删除，kubernetes ingress 使用 http:// 而不是 https:// 调用 pods，但我不知道如何修复它。

所以问题是：如何从 kubernetes ingress 中删除 SSL 终止？

Answer 1

如果您希望 SSL 终止发生在服务器而不是 ingress/LoadBalancer，您可以使用一种叫做 SSL Passthrough 的东西。负载均衡器将不会在入口处终止 SSL 请求，但您的服务器应该能够终止这些 SSL 请求。根据您的入口 class

在您的 ingress.yaml 文件中使用这些配置

   annotations:
    ingress.kubernetes.io/ssl-passthrough: "true"
    nginx.ingress.kubernetes.io/ssl-passthrough: "true"

您可以在 nginx 中使用另一种注解。 backend-protocol 注释可以指示 NGINX 应如何与后端服务通信。

nginx.ingress.kubernetes.io/backend-protocol: "HTTPS"

NGINX 默认使用 HTTP.

在此处阅读更多相关信息 https://kubernetes.github.io/ingress-nginx/user-guide/nginx-configuration/annotations/#backend-protocol

如何使用从 Kubernetes 入口到 pods 的 SSL 请求

How to use SSL requests from Kubernetes ingress to pods

kubernetes

kubernetes-helm

kubernetes-ingress