没有自定义指标的水平 pod 自动缩放

Question

我们希望根据 Kafka 主题中的消息量横向扩展 pods。标准的解决方案是将指标发布到 Kubernetes 的自定义指标 API。但是，由于公司准则，我们不允许使用 Kubernetes 的自定义指标 API。我们只被允许使用非管理功能。是否有具有 kubernetes-nativ 功能的解决方案，或者我们是否需要实施定制解决方案？

Answer 1

我不确定这是否符合您的需求，但您可以使用 Autoscaling on metrics not related to Kubernetes objects。

Applications running on Kubernetes may need to autoscale based on metrics that don’t have an obvious relationship to any object in the Kubernetes cluster, such as metrics describing a hosted service with no direct correlation to Kubernetes namespaces. In Kubernetes 1.10 and later, you can address this use case with external metrics.

Using external metrics requires knowledge of your monitoring system; the setup is similar to that required when using custom metrics. External metrics allow you to autoscale your cluster based on any metric available in your monitoring system. Just provide a metric block with a name and selector, as above, and use the External metric type instead of Object. If multiple time series are matched by the metricSelector, the sum of their values is used by the HorizontalPodAutoscaler. External metrics support both the Value and AverageValue target types, which function exactly the same as when you use the Object type.

For example if your application processes tasks from a hosted queue service, you could add the following section to your HorizontalPodAutoscaler manifest to specify that you need one worker per 30 outstanding tasks.
- type: External
 external:
   metric:
     name: queue_messages_ready
     selector: "queue=worker_tasks"
   target:
     type: AverageValue
     averageValue: 30
When possible, it’s preferable to use the custom metric target types instead of external metrics, since it’s easier for cluster administrators to secure the custom metrics API. The external metrics API potentially allows access to any metric, so cluster administrators should take care when exposing it.

您也可以看看 zalando-incubator/kube-metrics-adapter and use Prometheus collector external metrics。

This is an example of an HPA configured to get metrics based on a Prometheus query. The query is defined in the annotation metric-config.external.prometheus-query.prometheus/processed-events-per-second where processed-events-per-second is the query name which will be associated with the result of the query. A matching query-name label must be defined in the matchLabels of the metric definition. This allows having multiple prometheus queries associated with a single HPA.
apiVersion: autoscaling/v2beta2
kind: HorizontalPodAutoscaler
metadata:
 name: myapp-hpa
 annotations:
   # This annotation is optional.
   # If specified, then this prometheus server is used,
   # instead of the prometheus server specified as the CLI argument `--prometheus-server`.
   metric-config.external.prometheus-query.prometheus/prometheus-server: http://prometheus.my->namespace.svc
   # metric-config.<metricType>.<metricName>.<collectorName>/<configKey>
   # <configKey> == query-name
   metric-config.external.prometheus-query.prometheus/processed-events-per-second: |
     scalar(sum(rate(event-service_events_count{application="event-service",processed="true"}[1m])))
spec:
 scaleTargetRef:
   apiVersion: apps/v1
   kind: Deployment
   name: custom-metrics-consumer
 minReplicas: 1
 maxReplicas: 10
 metrics:
 - type: External
   external:
     metric:
       name: prometheus-query
       selector:
         matchLabels:
           query-name: processed-events-per-second
     target:
       type: AverageValue
       averageValue: "10"

没有自定义指标的水平 pod 自动缩放

Horizontal pod Autoscaling without custom metrics

metrics

openshift

autoscaling

apache-kafka

kubernetes