将 promethus `job` 标签从 `kubelet` 更改为 `kubernetes-cadvisor`

Question

我正在使用 Google Kubernetes Engine 集群，我希望我的 VerticalPodAutoscalers 到 use Prometheus as a history provider。说明说：

Configure your Prometheus to get metrics from cadvisor. Make sure that the metrics from the cadvisor have the label job=kubernetes-cadvisor.

我正在使用 kube-prometheus-stack 图表通过 Helm 在我的集群上安装 Prometheus。此图表中 cadvisor 指标的默认 job 标签似乎是 job='kubelet；我正在查看 source code for the chart，但我不明白如何操纵图表值以将工作标签更改为 kubernetes-cadvisor。正确的做法是什么？

在我的 Prometheus 配置下，我唯一能找到的 cadvisor 是：

- job_name: monitoring/prometheus-operator-kubelet/1
  honor_labels: true
  honor_timestamps: true
  scrape_interval: 30s
  scrape_timeout: 10s
  metrics_path: /metrics/cadvisor
  scheme: https
  kubernetes_sd_configs:
  - role: endpoints
    namespaces:
      names:
      - kube-system
  bearer_token_file: /var/run/secrets/kubernetes.io/serviceaccount/token
  tls_config:
    ca_file: /var/run/secrets/kubernetes.io/serviceaccount/ca.crt
    insecure_skip_verify: true
  relabel_configs:
  - source_labels: [__meta_kubernetes_service_label_k8s_app]
    separator: ;
    regex: kubelet
    replacement: 
    action: keep
  - source_labels: [__meta_kubernetes_endpoint_port_name]
    separator: ;
    regex: https-metrics
    replacement: 
    action: keep
  - source_labels: [__meta_kubernetes_endpoint_address_target_kind, __meta_kubernetes_endpoint_address_target_name]
    separator: ;
    regex: Node;(.*)
    target_label: node
    replacement: 
    action: replace
  - source_labels: [__meta_kubernetes_endpoint_address_target_kind, __meta_kubernetes_endpoint_address_target_name]
    separator: ;
    regex: Pod;(.*)
    target_label: pod
    replacement: 
    action: replace
  - source_labels: [__meta_kubernetes_namespace]
    separator: ;
    regex: (.*)
    target_label: namespace
    replacement: 
    action: replace
  - source_labels: [__meta_kubernetes_service_name]
    separator: ;
    regex: (.*)
    target_label: service
    replacement: 
    action: replace
  - source_labels: [__meta_kubernetes_pod_name]
    separator: ;
    regex: (.*)
    target_label: pod
    replacement: 
    action: replace
  - source_labels: [__meta_kubernetes_service_name]
    separator: ;
    regex: (.*)
    target_label: job
    replacement: 
    action: replace
  - source_labels: [__meta_kubernetes_service_label_k8s_app]
    separator: ;
    regex: (.+)
    target_label: job
    replacement: 
    action: replace
  - separator: ;
    regex: (.*)
    target_label: endpoint
    replacement: https-metrics
    action: replace
  - source_labels: [__metrics_path__]
    separator: ;
    regex: (.*)
    target_label: metrics_path
    replacement: 
    action: replace

Answer 1

将以下值传递给 Helm：

kubelet:
  serviceMonitor:
    cAdvisor: false

# Add a scrape job for cAdvisor in accordance with the Prometheus docs
prometheus:
  prometheusSpec:
    additionalScrapeConfigs:
      - job_name: 'kubernetes-cadvisor'
        scheme: https
        metrics_path: /metrics/cadvisor
        tls_config:
          ca_file: /var/run/secrets/kubernetes.io/serviceaccount/ca.crt
        bearer_token_file: /var/run/secrets/kubernetes.io/serviceaccount/token
        kubernetes_sd_configs:
        - role: node
        relabel_configs:
        - action: labelmap
          regex: __meta_kubernetes_node_label_(.+)

将 promethus `job` 标签从 `kubelet` 更改为 `kubernetes-cadvisor`

Changing promethus `job` label from `kubelet` to `kubernetes-cadvisor`

google-kubernetes-engine

prometheus

cadvisor