GKE 上的 Prometheus 监控 Strimzi Kafka - 如何获取 Prometheus Pod IP

Question

我正在尝试在 GKE 上部署 Prometheus 以监控现有的 Strimzi Kafka GKE 集群，但遇到了问题。（参考 - https://strimzi.io/docs/operators/latest/deploying.html#proc-metrics-deploying-prometheus-operator-str）

这是完成的：

创建了命名空间-monitoring，而Kafka部署在命名空间-kafka
修改了 kafka-deployment.yaml 以包含 metricsConfig 和 KafkaExporter 在文件 https://github.com/strimzi/strimzi-kafka-operator/tree/0.26.0/examples/metrics/kafka-metrics.yaml

更改如下：

apiVersion: kafka.strimzi.io/v1beta2
kind: Kafka
metadata:
  name: prom-graf #1
spec:
  kafka:
    version: 3.0.0
    ........
     metricsConfig:
      type: jmxPrometheusExporter
      valueFrom:
        configMapKeyRef:
          name: kafka-metrics
          key: kafka-metrics-config.yml 
     zookeeper:
      ......
      metricsConfig:
      type: jmxPrometheusExporter
      valueFrom:
        configMapKeyRef:
          name: kafka-metrics
          key: zookeeper-metrics-config.yml    
    entityOperator: #11
    topicOperator: {}
    userOperator: {}
  kafkaExporter:
    topicRegex: ".*"
    groupRegex: ".*"

将更改应用于 Kafka 集群的命令：

kubectl apply -f kafka-deployment.yaml -n kafka

创建了 Prometheous 运算符

command :
curl -s https://raw.githubusercontent.com/coreos/prometheus-operator/master/bundle.yaml > bundle.yaml

在 bundle.yaml 中将命名空间更改为 'monitoring' 并创建了 Prometheous 运算符

kubectl create -f bundle.yaml -n monitoring

运行以下命令在 GKE 上创建 Prometheus & Grafana：文件与 github - https://github.com/strimzi/strimzi-kafka-operator/tree/0.26.0/examples/metrics 中的文件相同，尽管我对命名空间的更改很生气

kubectl apply -f strimzi-pod-monitor.yaml  
(changed namespace to kafka, since kafka is deployed in namespace kafka)
kubectl apply -f prometheus-rules.yaml -n monitoring
kubectl apply -f prometheus.yaml -n monitoring
kubectl apply -f grafana.yaml - monitoring

- created secret from file - prometheus-additional.yaml
kubectl create secret generic additional-scrape-configs --from-file=prometheus-additional.yaml -n monitoring

当我检查在名称空间 - 监控上创建的对象时，列出了以下内容：

Karans-MacBook-Pro:prometheus-yamls karanalang$ kc get all,prometheus -n monitoring
NAME                                       READY   STATUS    RESTARTS   AGE
pod/grafana-5ccfb68647-55mzk               0/1     Running   0          21s
pod/prometheus-operator-85d84bb848-57lrn   1/1     Running   0          46m

NAME                          TYPE        CLUSTER-IP    EXTERNAL-IP   PORT(S)    AGE
service/grafana               ClusterIP   10.100.3.10   <none>        3000/TCP   21s
service/prometheus-operator   ClusterIP   None          <none>        8080/TCP   46m

NAME                                  READY   UP-TO-DATE   AVAILABLE   AGE
deployment.apps/grafana               0/1     1            0           24s
deployment.apps/prometheus-operator   1/1     1            1           46m

NAME                                             DESIRED   CURRENT   READY   AGE
replicaset.apps/grafana-5ccfb68647               1         1         0       23s
replicaset.apps/prometheus-operator-85d84bb848   1         1         1       46m

NAME                                          VERSION   REPLICAS   AGE
prometheus.monitoring.coreos.com/prometheus             3          44m

问题是 - 我如何获取 Prometheus 安装的主机名（因为我需要将其添加为 Grafana 中的源）？不知何故，我没有看到 prometheus pod，难道不应该创建它吗？

有什么想法如何 debug/fix 这个吗？蒂亚！

Answer 1

假设 Prometheus pods 确实启动了，它们的集体主机名将通过服务发现找到，例如 prometheus.monitoring.svc.cluster.local

https://cloud.google.com/kubernetes-engine/docs/concepts/service-discovery

您可能也对 exposing Prometheus 本身感兴趣

Answer 2

此问题已解决，文件 prometheus-additional.yaml 已经是一个秘密 yaml 文件，应该使用命令创建秘密

kubectl create -f prometheus-additional.yaml

instead of :

kubectl create secret generic additional-scrape-configs --from-file=prometheus-additional.yaml

参考 - https://issueexplorer.com/issue/strimzi/strimzi-kafka-operator/5873

最近的 strimzi 版本似乎改变了文件格式

GKE 上的 Prometheus 监控 Strimzi Kafka - 如何获取 Prometheus Pod IP

Prometheus on GKE to monitor Strimzi Kafka - how to get the Prometheus Pod IP

monitoring

grafana

kubernetes

google-kubernetes-engine

prometheus