在 k8s 上使用 elasticsearch 的 Zipkin 无法访问

Question

我正在尝试在 k8s 中部署 zipkin。我正在使用 elasticsearch（版本 6.8.8）作为存储。部署工作正常，服务器启动。但是，我只能通过端口转发访问服务器。

$ kubectl -n ns-zipkin port-forward zipkin-bdcf7f78b-shd9p 8888:9411

之后我可以访问http://localhost:8888/zipkin/

可能是什么原因？服务的部署已经没有得到我期望的端点（见下面的输出）。

deployment.yaml

apiVersion: apps/v1
kind: Deployment
metadata:
  name: zipkin
  labels:
    app: zipkin
spec:
  replicas: 1
  selector:
    matchLabels:
      app: zipkin
  template:
    metadata:
      name: zipkin
      labels:
        app: zipkin
    spec:
      containers:
        - name: zipkin
          image: openzipkin/zipkin:2.21
          imagePullPolicy: Always
          ports:
            - containerPort: 9411
              protocol: TCP
          env:
          - name: STORAGE_TYPE
            value: elasticsearch
          - name: ES_HOSTS
            value: https://my-es-host:9243
          - name: ES_USERNAME
            value: myUser
          - name: ES_PASSWORD
            value: myPassword
          - name: ES_HTTP_LOGGING
            value: HEADERS
          readinessProbe:
            httpGet:
              path: /api/v2/services
              port: 9411
            initialDelaySeconds: 5
            timeoutSeconds: 3

$ kubectl -n ns-zipkin describe deployment zipkin

Name:                   zipkin
Namespace:              ns-zipkin
CreationTimestamp:      Fri, 04 Sep 2020 09:16:38 +0200
Labels:                 app=zipkin
Annotations:            deployment.kubernetes.io/revision: 1
Selector:               app=zipkin
Replicas:               1 desired | 1 updated | 1 total | 1 available | 0 unavailable
StrategyType:           RollingUpdate
MinReadySeconds:        0
RollingUpdateStrategy:  25% max unavailable, 25% max surge
Pod Template:
  Labels:  app=zipkin
  Containers:
   zipkin:
    Image:      openzipkin/zipkin:2.21
    Port:       9411/TCP
    Host Port:  0/TCP
    Readiness:  http-get http://:9411/api/v2/services delay=5s timeout=3s period=10s #success=1 #failure=3
    Environment:
      STORAGE_TYPE:     elasticsearch
      ES_HOSTS:         https://my-es-host:9243
      ES_USERNAME:      myUser
      ES_PASSWORD:      myPassword
      ES_HTTP_LOGGING:  HEADERS
    Mounts:             <none>
  Volumes:              <none>
Conditions:
  Type           Status  Reason
  ----           ------  ------
  Available      True    MinimumReplicasAvailable
  Progressing    True    NewReplicaSetAvailable
OldReplicaSets:  <none>
NewReplicaSet:   zipkin-6788f4d555 (1/1 replicas created)
Events:
  Type    Reason             Age   From                   Message
  ----    ------             ----  ----                   -------
  Normal  ScalingReplicaSet  43m   deployment-controller  Scaled up replica set zipkin-6788f4d555 to 1

service.yaml

apiVersion: v1
kind: Service
metadata:
  name: zipkin
  labels:
    app.kubernetes.io/name: zipkin
    app.kubernetes.io/instance: zipkin
    app: zipkin
spec:
  type: ClusterIP
  ports:
    - port: 9411
      targetPort: 9411
      protocol: TCP
      name: http
  selector:
    app.kubernetes.io/name: zipkin
    app.kubernetes.io/instance: zipkin
    app: zipkin

$ kubectl -n ns-zipkin describe service zipkin

Name:              zipkin
Namespace:         ns-zipkin
Labels:            app=zipkin
                   app.kubernetes.io/instance=zipkin
                   app.kubernetes.io/name=zipkin
Annotations:       Selector:  app.kubernetes.io/instance=zipkin,app.kubernetes.io/name=zipkin,app=zipkin
Type:              ClusterIP
IP:                CONCRETE-IP-ADDRESS
Port:              http  9411/TCP
TargetPort:        9411/TCP
Endpoints:         <none>
Session Affinity:  None
Events:            <none>

ingress.yaml

apiVersion: extensions/v1beta1
kind: Ingress
metadata:
  name: zipkin
  labels:
    app.kubernetes.io/name: zipkin
    app.kubernetes.io/instance: zipkin
    app.kubernetes.io/managed-by: zipkin
  annotations:
    kubernetes.io/ingress.class: nginx
    kubernetes.io/tls-acme: "true"
    nginx.ingress.kubernetes.io/rewrite-target: /
    nginx.ingress.kubernetes.io/proxy-read-timeout: "3600"
    nginx.ingress.kubernetes.io/proxy-body-size: "0"
    nginx.ingress.kubernetes.io/cors-allow-methods: "PUT, GET, POST, OPTIONS, DELETE"
    nginx.ingress.kubernetes.io/cors-allow-origin: "*"
    nginx.ingress.kubernetes.io/enable-cors: "true"
spec:
  tls:
    - hosts:
        - ns-zipkin.my-host
      secretName: .my-host
  rules:
    - host: ns-zipkin.my-host
      http:
        paths:
          - path: /zipkin/?(.*)
            backend:
              serviceName: zipkin
              servicePort: 80

$ kubectl -n ns-zipkin describe ingress

Name:             zipkin
Namespace:        ns-zipkin
Address:          
Default backend:  default-http-backend:80 (<error: endpoints "default-http-backend" not found>)
TLS:
  .my-host terminates ns-zipkin.my-host
Rules:
  Host                                   Path  Backends
  ----                                   ----  --------
  ns-zipkin.my-host  
                                         /zipkin/?(.*)   zipkin:80 (<none>)
Annotations:                             kubernetes.io/ingress.class: nginx
                                         kubernetes.io/tls-acme: true
                                         nginx.ingress.kubernetes.io/cors-allow-methods: PUT, GET, POST, OPTIONS, DELETE
                                         nginx.ingress.kubernetes.io/cors-allow-origin: *
                                         nginx.ingress.kubernetes.io/enable-cors: true
                                         nginx.ingress.kubernetes.io/proxy-body-size: 0
                                         nginx.ingress.kubernetes.io/proxy-read-timeout: 3600
                                         nginx.ingress.kubernetes.io/rewrite-target: /
Events:
  Type     Reason     Age   From                      Message
  ----     ------     ----  ----                      -------
  Normal   CREATE     6s    nginx-ingress-controller  Ingress ns-zipkin/zipkin
  Warning  BadConfig  6s    cert-manager              Could not determine issuer for ingress due to bad annotations: failed to determine issuer name to be used for ingress resource
  Normal   CREATE     6s    nginx-ingress-controller  Ingress ns-zipkin/zipkin
  Normal   CREATE     6s    nginx-ingress-controller  Ingress ns-zipkin/zipkin
  Normal   CREATE     6s    nginx-ingress-controller  Ingress ns-zipkin/zipkin
  Normal   CREATE     6s    nginx-ingress-controller  Ingress ns-zipkin/zipkin
  Normal   CREATE     6s    nginx-ingress-controller  Ingress ns-zipkin/zipkin
  Normal   CREATE     6s    nginx-ingress-controller  Ingress ns-zipkin/zipkin
  Normal   CREATE     6s    nginx-ingress-controller  Ingress ns-zipkin/zipkin
  Normal   CREATE     6s    nginx-ingress-controller  Ingress ns-zipkin/zipkin
  Normal   CREATE     6s    nginx-ingress-controller  Ingress ns-zipkin/zipkin

Answer 1

您的服务需要 pod 上的以下标签：

selector:
  app.kubernetes.io/name: zipkin
  app.kubernetes.io/instance: zipkin
  app: zipkin

虽然看起来你在 zipkin 上只有一个标签 pods:

labels:
    app: zipkin

标签选择器使用逻辑与 (&&)，这意味着指定的所有标签都必须在 pod 上才能与之匹配。

Answer 2

以下有效。抱歉，我无法提供所有详细信息，因为我不认识他们:(也许其他人可以。

deployment.yaml

apiVersion: apps/v1
kind: Deployment
metadata:
  name: zipkin
  labels:
    app.kubernetes.io/name: zipkin
    app.kubernetes.io/instance: zipkin
    app: zipkin
spec:
  replicas: 1
  selector:
    matchLabels:
      app: zipkin
  template:
    metadata:
      name: zipkin
      labels:
        app: zipkin
    spec:
      containers:
        - name: zipkin
          image: openzipkin/zipkin:2.21
          imagePullPolicy: Always
          ports:
            - containerPort: 9411
              protocol: TCP
          env:
          - name: STORAGE_TYPE
            value: elasticsearch
          - name: ES_HOSTS
            value: https://my-es-host:9243
          - name: ES_USERNAME
            value: myUser
          - name: ES_PASSWORD
            value: myPassword
          - name: ES_HTTP_LOGGING
            value: HEADERS
          readinessProbe:
            httpGet:
              path: /api/v2/services
              port: 9411
            initialDelaySeconds: 5
            timeoutSeconds: 3

service.yaml

apiVersion: v1
kind: Service
metadata:
  name: zipkin
  labels:
    app.kubernetes.io/name: zipkin
    app.kubernetes.io/instance: zipkin
    app: zipkin
spec:
  type: ClusterIP
  ports:
    - port: 9411
      targetPort: 9411
      protocol: TCP
      name: http  <-- DELETED
  selector:
    app.kubernetes.io/name: zipkin    <-- DELETED
    app.kubernetes.io/instance: zipkin    <-- DELETED
    app: zipkin

ingress.yaml

apiVersion: extensions/v1beta1
kind: Ingress
metadata:
  name: zipkin
  labels:
    app.kubernetes.io/name: zipkin
    app.kubernetes.io/instance: zipkin
    app.kubernetes.io/managed-by: zipkin
  annotations:
    kubernetes.io/ingress.class: nginx
    kubernetes.io/tls-acme: "true"
    nginx.ingress.kubernetes.io/rewrite-target: /    <-- DELETED
    nginx.ingress.kubernetes.io/proxy-read-timeout: "3600"
    nginx.ingress.kubernetes.io/proxy-body-size: "0"
    nginx.ingress.kubernetes.io/cors-allow-methods: "PUT, GET, POST, OPTIONS, DELETE"
    nginx.ingress.kubernetes.io/cors-allow-origin: "*"
    nginx.ingress.kubernetes.io/enable-cors: "true"
spec:
  tls:
    - hosts:
        - ns-zipkin.my-host
      secretName: .my-host
  rules:
    - host: ns-zipkin.my-host
      http:
        paths:
          - path: /  <-- CHANGED
            backend:
              serviceName: zipkin
              servicePort: 9411   <-- CHANGED

在 k8s 上使用 elasticsearch 的 Zipkin 无法访问

Zipkin with elasticsearch on k8s no access

zipkin

kubernetes