KEDA 未扩展到 1 个 pod 以上
KEDA not scaling above 1 pod
我有一个 Kubernetes 部署,它部署了一个 pod,该 pod 将从 RabbitMQ 队列中拉下单个消息。我还使用 KEDA 根据当前队列中的 RabbitMQ 消息扩展部署。它正确地缩放到 0,然后在有消息时缩放到 1,但部署从未缩放到 1 以上。我当前的部署 YAML 文件:
apiVersion: apps/v1
kind: Deployment
metadata:
name: scale
labels:
app: scale
spec:
replicas: 1
selector:
matchLabels:
app: scale
template:
metadata:
labels:
app: scale
spec:
containers:
- name: scale-deployment
image: bharper7/scale:v1
imagePullPolicy: Never
我的 KEDA YAML 文件:
apiVersion: keda.sh/v1alpha1
kind: ScaledObject
metadata:
name: scale-keda-deployment
labels:
app: scale
deploymentName: scale
spec:
scaleTargetRef:
name: scale
pollingInterval: 5
minReplicaCount: 0
maxReplicaCount: 10
cooldownPeriod: 60
triggers:
- type: rabbitmq
metadata:
host: amqp://EmZn4ScuOPLEU1CGIsFKOaQSCQdjhzca:dJhLl2aVF78Gn07g2yGoRuwjXSc6tT11@192.168.49.2:30861
mode: QueueLength
value: '1'
queueName: scaleTest
KEDA操作日志文件:
2021-04-28T19:25:39.846Z INFO scaleexecutor Successfully updated ScaleTarget {"scaledobject.Name": "scale-keda-deployment", "scaledObject.Namespace": "default", "scaleTarget.Name": "scale", "Original Replicas Count": 0, "New Replicas Count": 1}
2021-04-28T19:25:40.272Z INFO controllers.ScaledObject Reconciling ScaledObject {"ScaledObject.Namespace": "default", "ScaledObject.Name": "scale-keda-deployment"}
就 RabbitMQ 连接的工作而言,我知道一切,而且 KEDA 知道要查看什么部署以及什么队列。 pod 缩放到 0 和 1 这一事实证明了所有这一切。但出于某种原因,即使队列中有 50 条消息,它也永远不会超过 1。
到目前为止,我尝试过使用 pollingInterval
和 cooldownPeriod
标签,但似乎都没有效果。有什么想法吗?
编辑:
我按照下面的建议从部署 YAML 文件中删除了 replicas
值。还查看了 HPA 日志。
生成的 HPA 日志:
Name: keda-hpa-scale-keda-deployment
Namespace: default
Labels: app=scale
app.kubernetes.io/managed-by=keda-operator
app.kubernetes.io/name=keda-hpa-scale-keda-deployment
app.kubernetes.io/part-of=scale-keda-deployment
app.kubernetes.io/version=2.1.0
deploymentName=scale
scaledObjectName=scale-keda-deployment
Annotations: <none>
CreationTimestamp: Wed, 28 Apr 2021 11:24:15 +0100
Reference: Deployment/scale
Metrics: ( current / target )
"rabbitmq-scaleTest" (target average value): 4 / 20
Min replicas: 1
Max replicas: 10
Deployment pods: 1 current / 1 desired
Conditions:
Type Status Reason Message
---- ------ ------ -------
AbleToScale True ReadyForNewScale recommended size matches current size
ScalingActive True ValidMetricFound the HPA was able to successfully calculate a replica count from external metric rabbitmq-scaleTest(&LabelSelector{MatchLabels:map[string]string{scaledObjectName: scale-keda-deployment,},MatchExpressions:[]LabelSelectorRequirement{},})
ScalingLimited False DesiredWithinRange the desired count is within the acceptable range
Events:
Type Reason Age From Message
---- ------ ---- ---- -------
Warning FailedGetScale 22m (x6 over 23m) horizontal-pod-autoscaler deployments/scale.apps "scale" not found
Normal SuccessfulRescale 15m horizontal-pod-autoscaler New size: 1; reason: All metrics below target
这是在将 5 条消息发送到队列之后。出于某种原因,即使我在 KEDA YAML 文件中将 value
设置为 1,它也只认为需要 1 个 pod。
为了防止其他人遇到这个问题,我设法找到了解决方法。根据 docs,不推荐使用 queueLength
,应该使用 mode
。但是改回已弃用的标签对我有用,由于某种原因,新标签不起作用。不是真正的正确修复,但至少它让我的部署按预期扩展。我的 KEDA 部署文件现在如下所示:
apiVersion: keda.sh/v1alpha1
kind: ScaledObject
metadata:
name: keda-deployment
labels:
apps: ffmpeg
deploymentName: ffmpeg
spec:
scaleTargetRef:
name: ffmpeg
pollingInterval: 10
cooldownPeriod: 1200
maxReplicaCount: 50
triggers:
- type: rabbitmq
metadata:
host: amqp://EmZn4ScuOPLEU1CGIsFKOaQSCQdjhzca:dJhLl2aVF78Gn07g2yGoRuwjXSc6tT11@192.168.49.2:30861
queueName: files
queueLength: '1'
请检查一次KEDA版本。如果您使用的版本低于或等于 2.1,则需要添加 queueLength
参数。 queueLength
的默认值为20,即一个pod可以执行20条消息。 ScaledObject 不会增加 pods 直到 21 条消息。您必须将 queueLength
设置为 1,以便为每条消息增加一个 pod 计数。
Link 到文档:https://keda.sh/docs/2.1/scalers/rabbitmq-queue/
KEDA 2.2 版本新增mode
和value
参数
我有一个 Kubernetes 部署,它部署了一个 pod,该 pod 将从 RabbitMQ 队列中拉下单个消息。我还使用 KEDA 根据当前队列中的 RabbitMQ 消息扩展部署。它正确地缩放到 0,然后在有消息时缩放到 1,但部署从未缩放到 1 以上。我当前的部署 YAML 文件:
apiVersion: apps/v1
kind: Deployment
metadata:
name: scale
labels:
app: scale
spec:
replicas: 1
selector:
matchLabels:
app: scale
template:
metadata:
labels:
app: scale
spec:
containers:
- name: scale-deployment
image: bharper7/scale:v1
imagePullPolicy: Never
我的 KEDA YAML 文件:
apiVersion: keda.sh/v1alpha1
kind: ScaledObject
metadata:
name: scale-keda-deployment
labels:
app: scale
deploymentName: scale
spec:
scaleTargetRef:
name: scale
pollingInterval: 5
minReplicaCount: 0
maxReplicaCount: 10
cooldownPeriod: 60
triggers:
- type: rabbitmq
metadata:
host: amqp://EmZn4ScuOPLEU1CGIsFKOaQSCQdjhzca:dJhLl2aVF78Gn07g2yGoRuwjXSc6tT11@192.168.49.2:30861
mode: QueueLength
value: '1'
queueName: scaleTest
KEDA操作日志文件:
2021-04-28T19:25:39.846Z INFO scaleexecutor Successfully updated ScaleTarget {"scaledobject.Name": "scale-keda-deployment", "scaledObject.Namespace": "default", "scaleTarget.Name": "scale", "Original Replicas Count": 0, "New Replicas Count": 1}
2021-04-28T19:25:40.272Z INFO controllers.ScaledObject Reconciling ScaledObject {"ScaledObject.Namespace": "default", "ScaledObject.Name": "scale-keda-deployment"}
就 RabbitMQ 连接的工作而言,我知道一切,而且 KEDA 知道要查看什么部署以及什么队列。 pod 缩放到 0 和 1 这一事实证明了所有这一切。但出于某种原因,即使队列中有 50 条消息,它也永远不会超过 1。
到目前为止,我尝试过使用 pollingInterval
和 cooldownPeriod
标签,但似乎都没有效果。有什么想法吗?
编辑:
我按照下面的建议从部署 YAML 文件中删除了 replicas
值。还查看了 HPA 日志。
生成的 HPA 日志:
Name: keda-hpa-scale-keda-deployment
Namespace: default
Labels: app=scale
app.kubernetes.io/managed-by=keda-operator
app.kubernetes.io/name=keda-hpa-scale-keda-deployment
app.kubernetes.io/part-of=scale-keda-deployment
app.kubernetes.io/version=2.1.0
deploymentName=scale
scaledObjectName=scale-keda-deployment
Annotations: <none>
CreationTimestamp: Wed, 28 Apr 2021 11:24:15 +0100
Reference: Deployment/scale
Metrics: ( current / target )
"rabbitmq-scaleTest" (target average value): 4 / 20
Min replicas: 1
Max replicas: 10
Deployment pods: 1 current / 1 desired
Conditions:
Type Status Reason Message
---- ------ ------ -------
AbleToScale True ReadyForNewScale recommended size matches current size
ScalingActive True ValidMetricFound the HPA was able to successfully calculate a replica count from external metric rabbitmq-scaleTest(&LabelSelector{MatchLabels:map[string]string{scaledObjectName: scale-keda-deployment,},MatchExpressions:[]LabelSelectorRequirement{},})
ScalingLimited False DesiredWithinRange the desired count is within the acceptable range
Events:
Type Reason Age From Message
---- ------ ---- ---- -------
Warning FailedGetScale 22m (x6 over 23m) horizontal-pod-autoscaler deployments/scale.apps "scale" not found
Normal SuccessfulRescale 15m horizontal-pod-autoscaler New size: 1; reason: All metrics below target
这是在将 5 条消息发送到队列之后。出于某种原因,即使我在 KEDA YAML 文件中将 value
设置为 1,它也只认为需要 1 个 pod。
为了防止其他人遇到这个问题,我设法找到了解决方法。根据 docs,不推荐使用 queueLength
,应该使用 mode
。但是改回已弃用的标签对我有用,由于某种原因,新标签不起作用。不是真正的正确修复,但至少它让我的部署按预期扩展。我的 KEDA 部署文件现在如下所示:
apiVersion: keda.sh/v1alpha1
kind: ScaledObject
metadata:
name: keda-deployment
labels:
apps: ffmpeg
deploymentName: ffmpeg
spec:
scaleTargetRef:
name: ffmpeg
pollingInterval: 10
cooldownPeriod: 1200
maxReplicaCount: 50
triggers:
- type: rabbitmq
metadata:
host: amqp://EmZn4ScuOPLEU1CGIsFKOaQSCQdjhzca:dJhLl2aVF78Gn07g2yGoRuwjXSc6tT11@192.168.49.2:30861
queueName: files
queueLength: '1'
请检查一次KEDA版本。如果您使用的版本低于或等于 2.1,则需要添加 queueLength
参数。 queueLength
的默认值为20,即一个pod可以执行20条消息。 ScaledObject 不会增加 pods 直到 21 条消息。您必须将 queueLength
设置为 1,以便为每条消息增加一个 pod 计数。
Link 到文档:https://keda.sh/docs/2.1/scalers/rabbitmq-queue/
KEDA 2.2 版本新增mode
和value
参数