对 kubernetes 上的 ravendb 部署进行故障排除

Question

我在 google 云上的 3 节点集群上部署了 ravendb。但是它不能从浏览器访问。这是我遵循的过程和配置。你能帮我解决服务和部署问题吗？当我运行 get pods 和 get svc 命令时 pods 和服务运行ning 但无法从浏览器访问数据库。

程序如下：

I suggest you first run the setup wizard on your local dev machine and get the Let's Encrypt certificate. Just use 127.0.0.1:8080 as the IP, it's not important at the moment.
(Even better will be to get your own domain + certificate for production use)

You need to convert both the pfx file and the license.json file to base64, In c# for example:
Convert.ToBase64String(File.ReadAllBytes(@"C:\work\certs\cluster.server.certificate.iftah.pfx"))

Convert.ToBase64String(File.ReadAllBytes(@"C:\work\license.json"))

1. Create a GKE standard cluster with 3 nodes, no special settings. Let's call it raven-cluster
2. Install gcloud and kubectl (follow the getting started guide: https://cloud.google.com/kubernetes-engine/docs/quickstart)
run:
3. > gcloud container clusters get-credentials raven-cluster
4. > kubectl create clusterrolebinding my-cluster-admin-binding --clusterrole cluster-admin --user $(gcloud config get-value account)

Now you're ready to deploy. 
Edit the cluster.yaml file to include the base64 certificate (name: raven-ssl)
Edit the license.secret.yaml file to include the base64 license (name: ravendb-license)

4a) kubectl label node role=ingress-controller --all
5. kubectl create -f license.secret.yaml
6. kubectl create -f haproxy.yaml
7. kubectl create -f cluster.yaml
9. kubectl get pod
8. kubectl get svc

ravendb 规范 YAML：

apiVersion: v1
items:
- apiVersion: v1
  data:
    raven-0: "{\r\n  \"Setup.Mode\": \"None\",\r\n  \"DataDir\": \"/data/RavenData\",\r\n
      \ \"Security.Certificate.Path\": \"/ssl/ssl\",\r\n  \"ServerUrl\": \"https://0.0.0.0\",\r\n
      \ \"ServerUrl.Tcp\": \"tcp://0.0.0.0:38888\",\r\n  \"PublicServerUrl\": \"https://a.tej-test001.ravendb.community\",\r\n
      \ \"PublicServerUrl.Tcp\": \"tcp://tcp-a.tej-test001.ravendb.community:443\",\r\n
      \ \"License.Path\": \"/license/license.json\",\r\n  \"License.Eula.Accepted\":
      \"true\",\r\n  \"License.CanActivate\": \"false\",\r\n  \"License.CanForceUpdate\":
      \"false\",\r\n  \"Server.AllowedDestinations\": \"Azure\",\r\n}"
    raven-1: "{\r\n  \"Setup.Mode\": \"None\",\r\n  \"DataDir\": \"/data/RavenData\",\r\n
      \ \"Security.Certificate.Path\": \"/ssl/ssl\",\r\n  \"ServerUrl\": \"https://0.0.0.0\",\r\n
      \ \"ServerUrl.Tcp\": \"tcp://0.0.0.0:38888\",\r\n  \"PublicServerUrl\": \"https://b.tej-test001.ravendb.community\",\r\n
      \ \"PublicServerUrl.Tcp\": \"tcp://tcp-b.tej-test001.ravendb.community:443\",\r\n
      \ \"License.Path\": \"/license/license.json\",\r\n  \"License.Eula.Accepted\":
      \"true\",\r\n  \"License.CanActivate\": \"false\",\r\n  \"License.CanForceUpdate\":
      \"false\",\r\n  \"Server.AllowedDestinations\": \"Azure\",\r\n}"
    raven-2: "{\r\n  \"Setup.Mode\": \"None\",\r\n  \"DataDir\": \"/data/RavenData\",\r\n
      \ \"Security.Certificate.Path\": \"/ssl/ssl\",\r\n  \"ServerUrl\": \"https://0.0.0.0\",\r\n
      \ \"ServerUrl.Tcp\": \"tcp://0.0.0.0:38888\",\r\n  \"PublicServerUrl\": \"https://c.tej-test001.ravendb.community\",\r\n
      \ \"PublicServerUrl.Tcp\": \"tcp://tcp-c.tej-test001.ravendb.community:443\",\r\n
      \ \"License.Path\": \"/license/license.json\",\r\n  \"License.Eula.Accepted\":
      \"true\",\r\n  \"License.CanActivate\": \"false\",\r\n  \"License.CanForceUpdate\":
      \"false\",\r\n  \"Server.AllowedDestinations\": \"Azure\",\r\n}"
  kind: ConfigMap
  metadata:
    labels:
      app: ravendb
      cluster: ee632d20-0a5f-40e4-a84a-5294da32d6d5
    name: raven-settings
    namespace: default
- apiVersion: apps/v1
  kind: StatefulSet
  metadata:
    labels:
      app: ravendb
      cluster: ee632d20-0a5f-40e4-a84a-5294da32d6d5
    name: raven
    namespace: default
  spec:
    podManagementPolicy: OrderedReady
    replicas: 3
    revisionHistoryLimit: 10
    selector:
      matchLabels:
        app: ravendb
        cluster: ee632d20-0a5f-40e4-a84a-5294da32d6d5
    serviceName: raven
    template:
      metadata:
        labels:
          app: ravendb
          cluster: ee632d20-0a5f-40e4-a84a-5294da32d6d5
      spec:
        affinity:
          podAntiAffinity:
            requiredDuringSchedulingIgnoredDuringExecution:
            - labelSelector:
                matchExpressions:
                - key: cluster
                  operator: In
                  values:
                  - ee632d20-0a5f-40e4-a84a-5294da32d6d5
              topologyKey: kubernetes.io/hostname
        containers:
        - command:
          - /bin/sh
          - -c
          - /opt/RavenDB/Server/Raven.Server --config-path /config/$HOSTNAME
          image: ravendb/ravendb:latest
          imagePullPolicy: Always
          name: ravendb
          ports:
          - containerPort: 443
            name: http-api
            protocol: TCP
          - containerPort: 38888
            name: tcp-server
            protocol: TCP
          - containerPort: 161
            name: snmp
            protocol: TCP
          resources:
            limits:
              cpu: 256m
              memory: 1900Mi
            requests:
              cpu: 256m
              memory: 1900Mi
          terminationMessagePath: /dev/termination-log
          terminationMessagePolicy: File
          volumeMounts:
          - mountPath: /data
            name: data
          - mountPath: /ssl
            name: ssl
          - mountPath: /license
            name: license
          - mountPath: /config
            name: config
        dnsPolicy: ClusterFirst
        restartPolicy: Always
        schedulerName: default-scheduler
        securityContext: {}
        terminationGracePeriodSeconds: 120
        volumes:
        - name: ssl
          secret:
            defaultMode: 420
            secretName: raven-ssl
        - configMap:
            defaultMode: 420
            name: raven-settings
          name: config
        - name: license
          secret:
            defaultMode: 420
            secretName: ravendb-license
    updateStrategy:
      rollingUpdate:
        partition: 0
      type: RollingUpdate
    volumeClaimTemplates:
    - metadata:
        labels:
          app: ravendb
          cluster: ee632d20-0a5f-40e4-a84a-5294da32d6d5
        name: data
      spec:
        accessModes:
        - ReadWriteOnce
        resources:
          requests:
            storage: 10Gi
- apiVersion: extensions/v1beta1
  kind: Ingress  
  metadata:
    annotations:
      ingress.kubernetes.io/ssl-passthrough: "true"
      kubernetes.io/ingress.class: "haproxy"
    labels:
      app: ravendb
      cluster: ee632d20-0a5f-40e4-a84a-5294da32d6d5
    name: raven
    namespace: default
  spec:
    rules:
    - host: a.tej-test001.ravendb.community
      http:
        paths:
        - backend:
            serviceName: raven-0
            servicePort: 443
          path: /
    - host: tcp-a.tej-test001.ravendb.community
      http:
        paths:
        - backend:
            serviceName: raven-0
            servicePort: 38888
          path: /
    - host: b.tej-test001.ravendb.community
      http:
        paths:
        - backend:
            serviceName: raven-1
            servicePort: 443
          path: /
    - host: tcp-b.tej-test001.ravendb.community
      http:
        paths:
        - backend:
            serviceName: raven-1
            servicePort: 38888
          path: /
    - host: c.tej-test001.ravendb.community
      http:
        paths:
        - backend:
            serviceName: raven-2
            servicePort: 443
          path: /
    - host: tcp-c.tej-test001.ravendb.community
      http:
        paths:
        - backend:
            serviceName: raven-2
            servicePort: 38888
          path: /
- apiVersion: v1
  data:
    ssl: dfdjfdkljfdkjdkjd;kfjdkfjdklfj
  kind: Secret
  metadata:
    labels:
      app: ravendb
      cluster: ee632d20-0a5f-40e4-a84a-5294da32d6d5
    name: raven-ssl
    namespace: default
  type: Opaque
- apiVersion: v1
  kind: Service
  metadata:
    labels:
      app: ravendb
      cluster: ee632d20-0a5f-40e4-a84a-5294da32d6d5
    name: raven
    namespace: default
  spec:
    clusterIP: None
    ports:
    - name: http-api
      port: 443
      protocol: TCP
      targetPort: 443
    - name: tcp-server
      port: 38888
      protocol: TCP
      targetPort: 38888
    - name: snmp
      port: 161
      protocol: TCP
      targetPort: 161
    selector:
      app: ravendb
      cluster: ee632d20-0a5f-40e4-a84a-5294da32d6d5
    sessionAffinity: None
    type: ClusterIP
  status:
    loadBalancer: {}
- apiVersion: v1
  kind: Service
  metadata:
    labels:
      app: ravendb
      cluster: ee632d20-0a5f-40e4-a84a-5294da32d6d5
      node: "0"
    name: raven-0
    namespace: default
  spec:
    ports:
    - name: http-api
      port: 443
      protocol: TCP
      targetPort: 443
    - name: tcp-server
      port: 38888
      protocol: TCP
      targetPort: 38888
    - name: snmp
      port: 161
      protocol: TCP
      targetPort: 161
    selector:
      app: ravendb
      cluster: ee632d20-0a5f-40e4-a84a-5294da32d6d5
      statefulset.kubernetes.io/pod-name: raven-0
    sessionAffinity: None
    type: ClusterIP
  status:
    loadBalancer: {}
- apiVersion: v1
  kind: Service
  metadata:
    labels:
      app: ravendb
      cluster: ee632d20-0a5f-40e4-a84a-5294da32d6d5
      node: "1"
    name: raven-1
    namespace: default
  spec:
    ports:
    - name: http-api
      port: 443
      protocol: TCP
      targetPort: 443
    - name: tcp-server
      port: 38888
      protocol: TCP
      targetPort: 38888
    - name: snmp
      port: 161
      protocol: TCP
      targetPort: 161
    selector:
      app: ravendb
      cluster: ee632d20-0a5f-40e4-a84a-5294da32d6d5
      statefulset.kubernetes.io/pod-name: raven-1
    sessionAffinity: None
    type: ClusterIP
  status:
    loadBalancer: {}
- apiVersion: v1
  kind: Service
  metadata:
    labels:
      app: ravendb
      cluster: ee632d20-0a5f-40e4-a84a-5294da32d6d5
      node: "2"
    name: raven-2
    namespace: default
  spec:
    ports:
    - name: http-api
      port: 443
      protocol: TCP
      targetPort: 443
    - name: tcp-server
      port: 38888
      protocol: TCP
      targetPort: 38888
    - name: snmp
      port: 161
      protocol: TCP
      targetPort: 161
    selector:
      app: ravendb
      cluster: ee632d20-0a5f-40e4-a84a-5294da32d6d5
      statefulset.kubernetes.io/pod-name: raven-2
    sessionAffinity: None
    type: ClusterIP
  status:
    loadBalancer: {}
kind: List

haproxy 规范 yaml:

---
apiVersion: v1
kind: ServiceAccount
metadata:
  name: ingress-controller
  namespace: default
---
apiVersion: rbac.authorization.k8s.io/v1beta1
kind: ClusterRole
metadata:
  name: ingress-controller
rules:
  - apiGroups:
      - ""
    resources:
      - configmaps
      - endpoints
      - nodes
      - pods
      - secrets
    verbs:
      - list
      - watch
  - apiGroups:
      - ""
    resources:
      - nodes
    verbs:
      - get
  - apiGroups:
      - ""
    resources:
      - services
    verbs:
      - get
      - list
      - watch
  - apiGroups:
      - "extensions"
    resources:
      - ingresses
    verbs:
      - get
      - list
      - watch
  - apiGroups:
      - ""
    resources:
      - events
    verbs:
      - create
      - patch
  - apiGroups:
      - "extensions"
    resources:
      - ingresses/status
    verbs:
      - update
---
apiVersion: rbac.authorization.k8s.io/v1beta1
kind: Role
metadata:
  name: ingress-controller
  namespace: default
rules:
  - apiGroups:
      - ""
    resources:
      - configmaps
      - pods
      - secrets
      - namespaces
    verbs:
      - get
  - apiGroups:
      - ""
    resources:
      - configmaps
    verbs:
      - get
      - update
  - apiGroups:
      - ""
    resources:
      - configmaps
    verbs:
      - create
  - apiGroups:
      - ""
    resources:
      - endpoints
    verbs:
      - get
      - create
      - update
---
apiVersion: rbac.authorization.k8s.io/v1beta1
kind: ClusterRoleBinding
metadata:
  name: ingress-controller
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: ClusterRole
  name: ingress-controller
subjects:
  - kind: ServiceAccount
    name: ingress-controller
    namespace: default
  - apiGroup: rbac.authorization.k8s.io
    kind: User
    name: ingress-controller
---
apiVersion: rbac.authorization.k8s.io/v1beta1
kind: RoleBinding
metadata:
  name: ingress-controller
  namespace: default
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: Role
  name: ingress-controller
subjects:
  - kind: ServiceAccount
    name: ingress-controller
    namespace: default
  - apiGroup: rbac.authorization.k8s.io
    kind: User
    name: ingress-controller
---
apiVersion: extensions/v1beta1
kind: Deployment
metadata:
  labels:
    run: ingress-default-backend
  name: ingress-default-backend
  namespace: default
spec:
  selector:
    matchLabels:
      run: ingress-default-backend
  template:
    metadata:
      labels:
        run: ingress-default-backend
    spec:
      containers:
      - name: ingress-default-backend
        image: gcr.io/google_containers/defaultbackend:1.0
        ports:
        - containerPort: 8080
        resources:
          limits:
            cpu: 10m
            memory: 20Mi
---
apiVersion: v1
kind: Service
metadata:
  name: ingress-default-backend
  namespace: default
spec:
  ports:
  - port: 8080
  selector:
    run: ingress-default-backend
---
apiVersion: v1
data:
  backend-server-slots-increment: "4"
  dynamic-scaling: "true"
kind: ConfigMap
metadata:
  name: haproxy-ingress
  namespace: default
---
apiVersion: apps/v1beta2
kind: Deployment
metadata:
  labels:
    run: haproxy-ingress
  name: haproxy-ingress
spec:
  selector:
    matchLabels:
      run: haproxy-ingress
  template:
    metadata:
      labels:
        run: haproxy-ingress
    spec:
      serviceAccountName: ingress-controller
      containers:
      - name: haproxy-ingress
        image: quay.io/jcmoraisjr/haproxy-ingress
        args:
        - --default-backend-service=$(POD_NAMESPACE)/ingress-default-backend
        - --configmap=$(POD_NAMESPACE)/haproxy-ingress
        - --reload-strategy=reusesocket
        ports:
        - name: https
          containerPort: 443
        - name: stat
          containerPort: 1936
        livenessProbe:
          httpGet:
            path: /healthz
            port: 10253
        env:
        - name: POD_NAME
          valueFrom:
            fieldRef:
              fieldPath: metadata.name
        - name: POD_NAMESPACE
          valueFrom:
            fieldRef:
              fieldPath: metadata.namespace
---
apiVersion: v1
kind: Service
metadata:
  labels:
    run: haproxy-ingress
  name: haproxy-ingress
  namespace: default
spec:
  type: LoadBalancer
  ports:
  - name: https
    port: 443
  - name: stat
    port: 1936
  selector:
    run: haproxy-ingress

Answer 1

您的说明不完整。您没有更新 DNS 记录。

kubectl 获取 pod
kubectl 获取 svc
记下 haproxy-ingress 服务的 EXTERNAL-IP 字段。分配 IP 可能需要几分钟时间。您将需要使用它来更新您域的 DNS 记录。转到 customers.ravendb.net 并将 DNS 记录编辑为您获得的新外部 IP。（或者如果您有自己的域，请与您的域提供商一起执行此操作）
当所有ravenpods准备就绪后，就可以去浏览器访问集群了。

对 kubernetes 上的 ravendb 部署进行故障排除

Troubleshooting a ravendb deployment on kubernetes

ravendb

ravenhq

kubernetes

google-kubernetes-engine

kubernetes-ingress