普罗米修斯警报管理器规则的自动标签

Question

我正在使用 prometheus-community/prometheus 图表

我想将以下标签自动添加到任何警报管理器规则触发

env=prod
cluster=project-prod-eks

这样我就不会手动为每个警报规则添加这些标签。

 - alert: NGINXTooMany400s
   expr: 100 * ( sum( nginx_ingress_controller_requests{status=~"4.+"} ) / sum(nginx_ingress_controller_requests) ) > 5
   for: 1m
   labels:
     severity: warning
     env: prod
     cluster: project-prod-eks              <---------------HOW to inject them?
   annotations:
     description: Too many 4XXs
     summary: More than 5% of all requests returned 4XX, this requires your attention

这样我就可以做类似

的事情

 - alert: NGINXTooMany400s
   expr: 100 * ( sum( nginx_ingress_controller_requests{status=~"4.+"} ) / sum(nginx_ingress_controller_requests) ) > 5
   for: 1m
   labels:
     severity: warning
   annotations:
     description: Too many 4XXs on {{ $labels.env }} / {{ $labels.cluster }}  <----- THIS
     summary: More than 5% of all requests returned 4XX, this requires your attention

有什么想法吗？

Answer 1

您可以将 external_labels 添加到您的 prometheus.yml:

global:
  # The labels to add to any time series or alerts when communicating with
  # external systems (federation, remote storage, Alertmanager).
  external_labels:
    env: prod
    cluster: project-prod-eks

社区图表在 values.yml 中：

serverFiles:
  prometheus.yml:
    global:
      external_labels:
        foo: bar
    ...

Answer 2

所以我对“serverFiles”的修改略有不同，见下文

server:
  nodeSelector: 
    prometheus: "true"
  baseURL: "https://prometheus.project.io"
  enabled: true
  retention: "30d"
  strategy:
    type: RollingUpdate
  global:
    scrape_interval: 30s
    external_labels:
      env: prod
      client: client-name
      project: project-name
      cluster: project-prod-eks

普罗米修斯警报管理器规则的自动标签

automatic labels to prometheus alertmanager rules

kubernetes

prometheus

prometheus-alertmanager

有什么想法吗？