SCDF 指标收集器 - 包括 Prometheus 指标

Question

我将 SCDF 与 Spring Boot 2.x 指标和 SCDF 指标收集器一起使用，以从我的 Spring Boot 应用程序收集指标。我真的不明白收集器关于 aggregateMetrics 数据的逻辑。

当我获取为我的流收集的指标列表时，我只有以 integration.channel.* 开头的指标，因此我只有 mean 值。我尝试了所有方法以查看其他指标是否与 /actuator/prometheus 端点公开的指标相似。

我想我误解了指标的聚合方式。我注意到 SCDF 会自动向指标添加一些属性，我想将这些属性应用于我公开的所有指标，以便将它们全部收集起来。

{
  "_embedded": {
    "streamMetricsList": [
      {
        "name": "poc-stream",
        "applications": [
          {
            "name": "poc-message-sink",
            "instances": [
              {
                "guid": "poc-stream-poc-message-sink-v7-75b8f4dcff-29fst",
                "key": "poc-stream.poc-message-sink.poc-stream-poc-message-sink-v7-75b8f4dcff-29fst",
                "properties": {
                  "spring.cloud.dataflow.stream.app.label": "poc-message-sink",
                  "spring.application.name": "poc-message-sink",
                  "spring.cloud.dataflow.stream.name": "poc-stream",
                  "spring.cloud.dataflow.stream.app.type": "sink",
                  "spring.cloud.application.guid": "poc-stream-poc-message-sink-v7-75b8f4dcff-29fst",
                  "spring.cloud.application.group": "poc-stream",
                  "spring.cloud.dataflow.stream.metrics.version": "2.0"
                },
                "metrics": [
                  {
                    "name": "integration.channel.input.send.mean",
                    "value": 0,
                    "timestamp": "2018-10-25T16:34:39.889Z"
                  }
                ]
              }
            ],
            "aggregateMetrics": [
              {
                "name": "integration.channel.input.send.mean",
                "value": 0,
                "timestamp": "2018-10-25T16:34:52.894Z"
              }
            ]
          },
...

我有一些 Micrometer 计数器，我想通过 Metrics 收集器获取这些值。我知道它们暴露得很好，因为我已经正确设置了所有属性，我什至进入了启动的 Docker 容器来检查端点。

我读过

When deploying applications, Data Flow sets the spring.cloud.stream.metrics.properties property, as shown in the following example:

spring.cloud.stream.metrics.properties=spring.application.name,spring.application.index,spring.cloud.application.*,spring.cloud.dataflow.*

The values of these keys are used as the tags to perform aggregation. In the case of 2.x applications, these key-values map directly onto tags in the Micrometer library. The property spring.cloud.application.guid can be used to track back to the specific application instance that generated the metric.

这是否意味着我需要自己专门将这些属性添加到我所有指标的标签中？我知道我可以通过让 Bean MeterRegistryCustomizer 返回以下内容来做到这一点：registry -> registry.config().commonTags(tags) 带有标记 SCDF 通常为 integration 指标设置的属性的标签。或者 SCDF 向所有指标添加属性 ?

谢谢！

Answer 1

虽然您对 MetricsCollector 的观察是 "generally" 正确的，但我相信有一种替代方法（也许更清洁）可以通过使用 SCDF Micrometer 指标集合来实现您一直在尝试的方法方法。我将在下面尝试解释这两种方法。

作为 MetricsCollector precedes in time the Micrometer framework they both implement a quite different metrics processing flows. The primary goal for the Metrics Collector 2.x was to ensure backward compatibility with SpringBoot 1.x metrics. The MetricsCollector 2.x allows mixing metrics coming from both SpringBoot 1.x (pre micrometer) and Spring Boot 2.x (e.g. micrometer) app starters. The consequence of this decision is that the Collector 2.x supports only the common denominator of metrics available in Boot 1.x and 2.x. This requirement is enforced by pre-filtering only the integration.channel.* metrics。目前，如果不修改指标收集器代码，您将无法添加更多指标。如果您认为支持不同的 Micrometer 指标比向后兼容 Boot 1.x 更重要，那么请在 Metrics Collector 项目中打开一个新问题。我仍然相信下面解释的方法更适合你的情况！

与 MetricsCollector 方法不同，"pure" Micrometer 指标直接发送到选定的指标注册表（例如 Prometheus、InfluxDB、Atlas 等）。如illustrated in the sample, the collected metrics can be analyzed and visualized with tools such as Grafana。按照 SCDF Metrics samples to setup your metrics collection via InfluxDB (or Prometheus) and Grafana. Later would allow you explore any out-of-the-box or custom Micrometer metrics. The downside of this approach (for the moment) is that you will not be able to visualize those metrics in the SCDF UI's pipeline. Still if you find it important to have such visualization inside the SCDF UI please open a new issue in the SCDF project（我有 Altals Micrometer Registry 的 WIP）。

我希望这对替代方法有所启发。我们很想听听您的反馈。

干杯！

SCDF 指标收集器 - 包括 Prometheus 指标

SCDF Metrics Collector - Include Prometheus metrics

spring-boot-actuator

spring-cloud-dataflow

spring-micrometer