Kubernetes GKE ingress-nginx loki grafana prometheus 设置问题

Question

我正在尝试在 Google Kubernetes Engine AKA GKE 上设置新的集群。我正在使用 helm 并添加到 repo ingress-nginx https://kubernetes.github.io/ingress-nginx and grafana https://grafana.github.io/helm-charts

我正在关注此 guide but I was not able to access an internal ip or get the ingress working to where I could see the dashboard. I have then tried this guide 使用他们的 helm 部署用于 Loki Stack（Loki、Promtail、Grafana、Prometheus）并具有持久卷声明。

结合打开端口 8443，我仍然收到 503 错误。我做了这个入口：

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: ingress-resource
  annotations:
    kubernetes.io/ingress.class: "nginx"
    nginx.ingress.kubernetes.io/use-regex: "true"
spec:
  rules:
    - http:
        paths:
          - path: /pro
            pathType: Prefix
            backend:
              service:
                name: loki-prometheus-server
                port:
                  number: 9090
          - path: /graf
            pathType: Prefix
            backend:
              service:
                name: loki-grafana
                port:
                  number: 3000

我尝试了很多不同的端口，包括 80，但是不行。

loki-prometheus-server 服务：

Name:              loki-prometheus-server
Namespace:         ingress-nginx
Labels:            app=prometheus
                   app.kubernetes.io/managed-by=Helm
                   chart=prometheus-11.16.9
                   component=server
                   heritage=Helm
                   release=loki
Annotations:       cloud.google.com/neg: {"ingress":true}
                   meta.helm.sh/release-name: loki
                   meta.helm.sh/release-namespace: ingress-nginx
Selector:          app=prometheus,component=server,release=loki
Type:              ClusterIP
IP Families:       <none>
IP:                10.***
IPs:               10.***
Port:              http  80/TCP
TargetPort:        9090/TCP
Endpoints:         10.***:9090
Session Affinity:  None
Events:            <none>

loki-grafana 服务：

Name:              loki-grafana
Namespace:         ingress-nginx
Labels:            app.kubernetes.io/instance=loki
                   app.kubernetes.io/managed-by=Helm
                   app.kubernetes.io/name=grafana
                   app.kubernetes.io/version=7.5.0
                   helm.sh/chart=grafana-5.7.10
Annotations:       cloud.google.com/neg: {"ingress":true}
                   meta.helm.sh/release-name: loki
                   meta.helm.sh/release-namespace: ingress-nginx
Selector:          app.kubernetes.io/instance=loki,app.kubernetes.io/name=grafana
Type:              ClusterIP
IP Families:       <none>
IP:                10.***
IPs:               10.***
Port:              service  80/TCP
TargetPort:        3000/TCP
Endpoints:         10.***:3000
Session Affinity:  None
Events:            <none>

循序渐进：

在 gcloud 上创建新集群：

GKE 标准
默认池->节点->机器类型：e2-small
default-pool->Security-> Allow full access to all Cloud APIs selected 创建
创建。

连接到主机或云端shell

然后：

helm repo add ingress-nginx https://kubernetes.github.io/ingress-nginx
helm repo update

kubectl create namespace ingress-nginx
helm install ingress-nginx ingress-nginx/ingress-nginx --namespace ingress-nginx

到目前为止一切似乎都很正常。

进入VPC网络->防火墙->在Protocols/ports中找到80,443的那个然后加上8443。

安装 loki：

helm upgrade --install loki grafana/loki-stack --namespace ingress-nginx  --set grafana.enabled=true,prometheus.enabled=true,prometheus.alertmanager.persistentVolume.enabled=false,prometheus.server.persistentVolume.enabled=false,loki.persistence.enabled=true,loki.persistence.storageClassName=standard,loki.persistence.size=5Gi

一切似乎仍然很顺利，但如果我转到我的外部负载平衡器，我会收到 404 not found / nginx

我添加这个入口：

使用kubectl apply -f ingress.yml -n ingress-nginx

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: ingress-resource
  annotations:
    kubernetes.io/ingress.class: "nginx"
    prometheus.io/scrape: "true"
spec:
  rules:
    - http:
        paths:
          - path: /pro
            pathType: Prefix
            backend:
              service:
                name: loki-prometheus-server
                port:
                  number: 80
          - path: /graf
            pathType: Prefix
            backend:
              service:
                name: loki-grafana
                port:
                  number: 80

并得到一个 404 页面未在 /pro 上的文本中找到并在 /graf

处使用 404 not found nginx 页面重定向到 /login

我做错了什么？

Answer 1

运行:

helm upgrade ingress-nginx ingress-nginx/ingress-nginx --namespace ingress-nginx  --set controller.metrics.enabled=true --set-string controller.podAnnotations."prometheus\.io/scrape"="true" --set-string controller.podAnnotations."prometheus\.io/port"="10254"

并且使用这个入口我能够让它工作：

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: ingress-resource
  annotations:
    kubernetes.io/ingress.class: "nginx"
    nginx.ingress.kubernetes.io/use-regex: "true"
    nginx.ingress.kubernetes.io/ssl-redirect: "false"
    nginx.ingress.kubernetes.io/rewrite-target: /
    prometheus.io/port: "10254"
    prometheus.io/scrape: "true"
spec:
  rules:
    - http:
        paths:
          - path: /?(.*)
            pathType: Prefix
            backend:
              service:
                name: loki-grafana
                port:
                  number: 80

Kubernetes GKE ingress-nginx loki grafana prometheus 设置问题

Kubernetes GKE ingress-nginx loki grafana prometheus setup issues

nginx

kubernetes

google-kubernetes-engine

prometheus

kubernetes-ingress