Traefik 作为 Ingress 控制器:使用 Letsencrypt 进行 https 时为 404

Traefik as Ingress controller: 404 when using Letsencrypt for https

几天前,我使用 Traefik 作为 Ingress 控制器创建了一个 Kubernetes 集群。之后,我为子域 traefik.mydomain.de 启用了 Traefik web ui。现在我正在尝试使用 Letsencrypt

这是我的完整配置traefik.yml:

---
kind: ClusterRole
apiVersion: rbac.authorization.k8s.io/v1beta1
metadata:
  name: traefik-ingress-controller
rules:
  - apiGroups:
      - ""
    resources:
      - services
      - endpoints
      - secrets
    verbs:
      - get
      - list
      - watch
  - apiGroups:
      - extensions
    resources:
      - ingresses
    verbs:
      - get
      - list
      - watch
---
kind: ClusterRoleBinding
apiVersion: rbac.authorization.k8s.io/v1beta1
metadata:
  name: traefik-ingress-controller
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: ClusterRole
  name: traefik-ingress-controller
subjects:
- kind: ServiceAccount
  name: traefik-ingress-controller
  namespace: kube-system
---
apiVersion: v1
kind: ServiceAccount
metadata:
  name: traefik-ingress-controller
  namespace: kube-system
---
apiVersion: v1
kind: ConfigMap
metadata:
  name: traefik-config
  namespace: kube-system
data:
  traefik.toml: |
    [entryPoints]
      [entryPoints.http]
      address = ":80"
        [entryPoints.http.redirect]
        entryPoint = "https"
      [entryPoints.https]
      address = ":443"
        [entryPoints.https.tls]

    [acme]
    email = "admin@mydomain.de"
    storage = "/acme/acme.json"
    onHostRule = true
    caServer = "https://acme-staging-v02.api.letsencrypt.org/directory"
    entryPoint = "https"
      [acme.httpChallenge]
      entryPoint = "http"

    [[acme.domains]]
    main = "mydomain.de"
    sans = ["traefik.mydomain.de"]
---
kind: DaemonSet
apiVersion: extensions/v1beta1
metadata:
  name: traefik-ingress-controller
  namespace: kube-system
  labels:
    k8s-app: traefik-ingress-lb
spec:
  template:
    metadata:
      labels:
        k8s-app: traefik-ingress-lb
        name: traefik-ingress-lb
    spec:
      serviceAccountName: traefik-ingress-controller
      terminationGracePeriodSeconds: 60
      hostNetwork: true
      volumes:
        - name: config
          configMap:
            name: traefik-config
        - name: acme
          hostPath:
            path: /srv/configs/acme.json
            type: File
      containers:
      - image: traefik
        name: traefik-ingress-lb
        volumeMounts:
            - mountPath: "/config"
              name: "config"
            - mountPath: "/acme/acme.json"
              name: "acme"
        ports:
        - name: http
          containerPort: 80
          hostPort: 80
        - name: https
          containerPort: 443
          hostPort: 443
        securityContext:
          capabilities:
            drop:
            - ALL
            add:
            - NET_BIND_SERVICE
        args:
        - --configfile=/config/traefik.toml
        - --api
        - --kubernetes
        - --logLevel=DEBUG
---
kind: Service
apiVersion: v1
metadata:
  name: traefik-ingress-service
  namespace: kube-system
spec:
  selector:
    k8s-app: traefik-ingress-lb
  ports:
  - protocol: TCP
    port: 80
    name: http
  - protocol: TCP
    port: 443
    name: https
  type: NodePort
---
apiVersion: v1
kind: Service
metadata:
  name: traefik-web-ui
  namespace: kube-system
spec:
  selector:
    k8s-app: traefik-ingress-lb
  ports:
  - protocol: TCP
    port: 8080
    name: webui
---
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
  name: traefik-web-ui
  namespace: kube-system
  annotations:
    kubernetes.io/ingress.class: traefik
spec:
  rules:
  - host: traefik.mydomain.de
    http:
      paths:
      - path: /
        backend:
          serviceName: traefik-web-ui
          servicePort: 8080

结果:

调试输出:

time="2018-05-31T10:54:58Z" level=info msg="Using TOML configuration file /config/traefik.toml"
time="2018-05-31T10:54:58Z" level=info msg="Traefik version v1.6.2 built on 2018-05-22_03:19:06PM"
time="2018-05-31T10:54:58Z" level=info msg="\nStats collection is disabled.\nHelp us improve Traefik by turning this feature on :)\nMore details on: https://docs.traefik.io/basics/#collected-data\n"
time="2018-05-31T10:54:58Z" level=debug msg="Global configuration loaded {\"LifeCycle\":{\"RequestAcceptGraceTimeout\":0,\"GraceTimeOut\":10000000000},\"GraceTimeOut\":0,\"Debug\":false,\"CheckNewVersion\":true,\"SendAnonymousUsage\":false,\"AccessLogsFile\":\"\",\"AccessLog\":null,\"TraefikLogsFile\":\"\",\"TraefikLog\":null,\"Tracing\":null,\"LogLevel\":\"DEBUG\",\"EntryPoints\":{\"http\":{\"Address\":\":80\",\"TLS\":null,\"Redirect\":{\"entryPoint\":\"https\"},\"Auth\":null,\"WhitelistSourceRange\":null,\"WhiteList\":null,\"Compress\":false,\"ProxyProtocol\":null,\"ForwardedHeaders\":{\"Insecure\":true,\"TrustedIPs\":null}},\"https\":{\"Address\":\":443\",\"TLS\":{\"MinVersion\":\"\",\"CipherSuites\":null,\"Certificates\":null,\"ClientCAFiles\":null,\"ClientCA\":{\"Files\":null,\"Optional\":false}},\"Redirect\":null,\"Auth\":null,\"WhitelistSourceRange\":null,\"WhiteList\":null,\"Compress\":false,\"ProxyProtocol\":null,\"ForwardedHeaders\":{\"Insecure\":true,\"TrustedIPs\":null}},\"traefik\":{\"Address\":\":8080\",\"TLS\":null,\"Redirect\":null,\"Auth\":null,\"WhitelistSourceRange\":null,\"WhiteList\":null,\"Compress\":false,\"ProxyProtocol\":null,\"ForwardedHeaders\":{\"Insecure\":true,\"TrustedIPs\":null}}},\"Cluster\":null,\"Constraints\":[],\"ACME\":null,\"DefaultEntryPoints\":[\"http\"],\"ProvidersThrottleDuration\":2000000000,\"MaxIdleConnsPerHost\":200,\"IdleTimeout\":0,\"InsecureSkipVerify\":false,\"RootCAs\":null,\"Retry\":null,\"HealthCheck\":{\"Interval\":30000000000},\"RespondingTimeouts\":null,\"ForwardingTimeouts\":null,\"AllowMinWeightZero\":false,\"Web\":null,\"Docker\":null,\"File\":null,\"Marathon\":null,\"Consul\":null,\"ConsulCatalog\":null,\"Etcd\":null,\"Zookeeper\":null,\"Boltdb\":null,\"Kubernetes\":{\"Watch\":true,\"Filename\":\"\",\"Constraints\":[],\"Trace\":false,\"TemplateVersion\":0,\"DebugLogGeneratedTemplate\":false,\"Endpoint\":\"\",\"Token\":\"\",\"CertAuthFilePath\":\"\",\"DisablePassHostHeaders\":false,\"EnablePassTLSCert\":false,\"Namespaces\":null,\"LabelSelector\":\"\",\"IngressClass\":\"\"},\"Mesos\":null,\"Eureka\":null,\"ECS\":null,\"Rancher\":null,\"DynamoDB\":null,\"ServiceFabric\":null,\"Rest\":null,\"API\":{\"EntryPoint\":\"traefik\",\"Dashboard\":true,\"Debug\":false,\"CurrentConfigurations\":null,\"Statistics\":null},\"Metrics\":null,\"Ping\":null}"
time="2018-05-31T10:54:58Z" level=info msg="Preparing server https &{Address::443 TLS:0xc42057e900 Redirect:<nil> Auth:<nil> WhitelistSourceRange:[] WhiteList:<nil> Compress:false ProxyProtocol:<nil> ForwardedHeaders:0xc420020480} with readTimeout=0s writeTimeout=0s idleTimeout=3m0s"
time="2018-05-31T10:54:59Z" level=info msg="Preparing server http &{Address::80 TLS:<nil> Redirect:0xc420092a80 Auth:<nil> WhitelistSourceRange:[] WhiteList:<nil> Compress:false ProxyProtocol:<nil> ForwardedHeaders:0xc4200204a0} with readTimeout=0s writeTimeout=0s idleTimeout=3m0s"
time="2018-05-31T10:54:59Z" level=info msg="Preparing server traefik &{Address::8080 TLS:<nil> Redirect:<nil> Auth:<nil> WhitelistSourceRange:[] WhiteList:<nil> Compress:false ProxyProtocol:<nil> ForwardedHeaders:0xc4200204c0} with readTimeout=0s writeTimeout=0s idleTimeout=3m0s"
time="2018-05-31T10:54:59Z" level=info msg="Starting provider configuration.providerAggregator {}"
time="2018-05-31T10:54:59Z" level=info msg="Starting server on :443"
time="2018-05-31T10:54:59Z" level=info msg="Starting server on :80"
time="2018-05-31T10:54:59Z" level=info msg="Starting server on :8080"
time="2018-05-31T10:54:59Z" level=info msg="Starting provider *kubernetes.Provider {\"Watch\":true,\"Filename\":\"\",\"Constraints\":[],\"Trace\":false,\"TemplateVersion\":0,\"DebugLogGeneratedTemplate\":false,\"Endpoint\":\"\",\"Token\":\"\",\"CertAuthFilePath\":\"\",\"DisablePassHostHeaders\":false,\"EnablePassTLSCert\":false,\"Namespaces\":null,\"LabelSelector\":\"\",\"IngressClass\":\"\"}"
time="2018-05-31T10:54:59Z" level=info msg="Starting provider *acme.Provider {\"Email\":\"admin@mydomain.de\",\"ACMELogging\":false,\"CAServer\":\"https://acme-staging-v02.api.letsencrypt.org/directory\",\"Storage\":\"/acme/acme.json\",\"EntryPoint\":\"https\",\"OnHostRule\":true,\"OnDemand\":false,\"DNSChallenge\":null,\"HTTPChallenge\":{\"EntryPoint\":\"http\"},\"Domains\":[{\"Main\":\"mydomain.de\",\"SANs\":[\"traefik.mydomain.de\"]}],\"Store\":{}}"
time="2018-05-31T10:54:59Z" level=debug msg="Using Ingress label selector: \"\""
time="2018-05-31T10:54:59Z" level=info msg="ingress label selector is: \"\""
time="2018-05-31T10:54:59Z" level=info msg="Creating in-cluster Provider client"
time="2018-05-31T10:54:59Z" level=info msg="Testing certificate renew..."
time="2018-05-31T10:54:59Z" level=debug msg="Configuration received from provider ACME: {\"tls\":[{\"EntryPoints\":[\"https\"],\"Certificate\":{\"CertFile\":\"-----BEGIN CERTIFICATE-----<<< cert here >>>-----END CERTIFICATE-----\n\n-----BEGIN CERTIFICATE-----<<< another cert here >>>-----END CERTIFICATE-----\n\",\"KeyFile\":\"-----BEGIN RSA PRIVATE<<< rsa data here >>>-----END RSA PRIVATE KEY-----\n\"}}]}"
time="2018-05-31T10:54:59Z" level=debug msg="Looking for provided certificate(s) to validate [\"mydomain.de\" \"traefik.mydomain.de\"]..."
time="2018-05-31T10:54:59Z" level=debug msg="No ACME certificate to generate for domains [\"mydomain.de\" \"traefik.mydomain.de\"]."
time="2018-05-31T10:54:59Z" level=debug msg="Add certificate for domains mydomain.de,traefik.mydomain.de"
time="2018-05-31T10:54:59Z" level=info msg="Server configuration reloaded on :8080"
time="2018-05-31T10:54:59Z" level=info msg="Server configuration reloaded on :443"
time="2018-05-31T10:54:59Z" level=info msg="Server configuration reloaded on :80"
time="2018-05-31T10:54:59Z" level=debug msg="Received Kubernetes event kind *v1.Service"
time="2018-05-31T10:54:59Z" level=debug msg="Configuration received from provider kubernetes: {\"backends\":{\"traefik.mydomain.de/\":{\"servers\":{\"traefik-ingress-controller-lqkjn\":{\"url\":\"https://11.22.33.44:8080\",\"weight\":1}},\"loadBalancer\":{\"method\":\"wrr\"}}},\"frontends\":{\"traefik.mydomain.de/\":{\"entryPoints\":[\"http\"],\"backend\":\"traefik.mydomain.de/\",\"routes\":{\"/\":{\"rule\":\"PathPrefix:/\"},\"traefik.mydomain.de\":{\"rule\":\"Host:traefik.mydomain.de\"}},\"passHostHeader\":true,\"priority\":0,\"basicAuth\":[]}}}"
time="2018-05-31T10:54:59Z" level=debug msg="Creating frontend traefik.mydomain.de/"
time="2018-05-31T10:54:59Z" level=debug msg="Wiring frontend traefik.mydomain.de/ to entryPoint http"
time="2018-05-31T10:54:59Z" level=debug msg="Creating route traefik.mydomain.de Host:traefik.mydomain.de"
time="2018-05-31T10:54:59Z" level=debug msg="Creating route / PathPrefix:/"
time="2018-05-31T10:54:59Z" level=debug msg="Creating entry point redirect http -> https"
time="2018-05-31T10:54:59Z" level=debug msg="Creating backend traefik.mydomain.de/"
time="2018-05-31T10:54:59Z" level=debug msg="Creating load-balancer wrr"
time="2018-05-31T10:54:59Z" level=debug msg="Creating server traefik-ingress-controller-lqkjn at https://11.22.33.44:8080 with weight 1"
time="2018-05-31T10:54:59Z" level=debug msg="Add certificate for domains mydomain.de,traefik.mydomain.de"
time="2018-05-31T10:54:59Z" level=info msg="Server configuration reloaded on :443"
time="2018-05-31T10:54:59Z" level=info msg="Server configuration reloaded on :80"
time="2018-05-31T10:54:59Z" level=info msg="Server configuration reloaded on :8080"
time="2018-05-31T10:54:59Z" level=debug msg="Try to challenge certificate for domain [traefik.mydomain.de] founded in Host rule"
time="2018-05-31T10:54:59Z" level=debug msg="No domain parsed in rule \"PathPrefix:/\""
time="2018-05-31T10:54:59Z" level=debug msg="Looking for provided certificate(s) to validate [\"traefik.mydomain.de\"]..."
time="2018-05-31T10:54:59Z" level=debug msg="No ACME certificate to generate for domains [\"traefik.mydomain.de\"]."
time="2018-05-31T10:54:59Z" level=debug msg="Received Kubernetes event kind *v1.Secret"
time="2018-05-31T10:54:59Z" level=debug msg="Skipping Kubernetes event kind *v1.Secret"
time="2018-05-31T10:54:59Z" level=debug msg="Received Kubernetes event kind *v1.Secret"
<<< many more skipped events >>>
time="2018-05-31T10:55:16Z" level=debug msg="Skipping Kubernetes event kind *v1.Endpoints"
time="2018-05-31T10:55:16Z" level=debug msg="Received Kubernetes event kind *v1.Endpoints"
time="2018-05-31T10:55:16Z" level=debug msg="Skipping Kubernetes event kind *v1.Endpoints"
<<< many more skipped events >>>

不幸的是,我缺乏 required 调试技能来进一步分析。我检查了我配置的配置文件是否可用且可读。我检查了 acme.json 是否正在使用 - 它包含有关已颁发证书的信息。

注意:我尽我所能使它保持可读性和尽可能短(不遗漏重要信息),但很可能无法将其保持在应有的最小限度。请原谅我 - 当您对某个主题还不坚定时,提问会更难。

经过大量研究:我的配置中(至少)有 2 个错误。

  1. 失踪defaultEntryPoints。似乎 Traefik Web Ui 没有为前端配置端点(至少它自己没有)。结果,在我的配置中无法访问网络 ui。添加默认入口点(在没有特定前端入口点配置时使用)可以解决该问题。

解决方案:在 traefik.toml 定义中添加这一行 ConfigMap 命名为 traefik-config(查看完整配置 下面):

defaultEntryPoints = ["http", "https"]
  1. 错误的端口配置。端口配置未从暴露的服务端口 80 正确链接到 Traefik web ui 使用的内部服务端口 8080。这可以通过更新名为 traefik-web-uiService 的配置并添加 targetPort: 8080 来更改(请参阅下面的完整配置)。

进行这些更改后,我的设置将按预期运行。

完整配置仅供参考:

---
kind: ClusterRole
apiVersion: rbac.authorization.k8s.io/v1beta1
metadata:
name: traefik-ingress-controller
rules:
- apiGroups:
    - ""
    resources:
    - services
    - endpoints
    - secrets
    verbs:
    - get
    - list
    - watch
- apiGroups:
    - extensions
    resources:
    - ingresses
    verbs:
    - get
    - list
    - watch
---
kind: ClusterRoleBinding
apiVersion: rbac.authorization.k8s.io/v1beta1
metadata:
name: traefik-ingress-controller
roleRef:
apiGroup: rbac.authorization.k8s.io
kind: ClusterRole
name: traefik-ingress-controller
subjects:
- kind: ServiceAccount
name: traefik-ingress-controller
namespace: kube-system
---
apiVersion: v1
kind: ServiceAccount
metadata:
name: traefik-ingress-controller
namespace: kube-system
---
apiVersion: v1
kind: ConfigMap
metadata:
name: traefik-config
namespace: kube-system
data:
traefik.toml: |
    defaultEntryPoints = ["http", "https"]

    [entryPoints]
    [entryPoints.http]
    address = ":80"
        [entryPoints.http.redirect]
        entryPoint = "https"
    [entryPoints.https]
    address = ":443"
        [entryPoints.https.tls]

    [acme]
    email = "admin@mydomain.de"
    storage = "/acme/acme.json"
    onHostRule = true
    caServer = "https://acme-staging-v02.api.letsencrypt.org/directory"
    entryPoint = "https"
    [acme.httpChallenge]
    entryPoint = "http"

    [[acme.domains]]
    main = "mydomain.de"
    sans = ["traefik.mydomain.de"]
---
kind: DaemonSet
apiVersion: extensions/v1beta1
metadata:
name: traefik-ingress-controller
namespace: kube-system
labels:
    k8s-app: traefik-ingress-lb
spec:
template:
    metadata:
    labels:
        k8s-app: traefik-ingress-lb
        name: traefik-ingress-lb
    spec:
    serviceAccountName: traefik-ingress-controller
    terminationGracePeriodSeconds: 60
    hostNetwork: true
    volumes:
        - name: config
        configMap:
            name: traefik-config
        - name: acme
        hostPath:
            path: /srv/configs/acme.json
            type: File
    containers:
    - image: traefik
        name: traefik-ingress-lb
        volumeMounts:
            - mountPath: "/config"
            name: "config"
            - mountPath: "/acme/acme.json"
            name: "acme"
        ports:
        - name: http
        containerPort: 80
        hostPort: 80
        - name: https
        containerPort: 443
        hostPort: 443
        securityContext:
        capabilities:
            drop:
            - ALL
            add:
            - NET_BIND_SERVICE
        args:
        - --configfile=/config/traefik.toml
        - --api
        - --kubernetes
        - --logLevel=DEBUG
---
kind: Service
apiVersion: v1
metadata:
name: traefik-ingress-service
namespace: kube-system
spec:
selector:
    k8s-app: traefik-ingress-lb
ports:
- protocol: TCP
    port: 80
    name: http
- protocol: TCP
    port: 443
    name: https
type: NodePort
---
apiVersion: v1
kind: Service
metadata:
name: traefik-web-ui
namespace: kube-system
spec:
selector:
    k8s-app: traefik-ingress-lb
ports:
- protocol: TCP
    port: 80
    targetPort: 8080
    name: webui
---
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
name: traefik-web-ui
namespace: kube-system
annotations:
    kubernetes.io/ingress.class: traefik
spec:
rules:
- host: traefik.mydomain.de
    http:
    paths:
    - path: /
        backend:
        serviceName: traefik-web-ui
        servicePort: 80