GCP GKE Kubernetes HPA:horizontal-pod-autoscaler 无法获得内存利用率:缺少内存请求。如何解决?
GCP GKE Kubernetes HPA: horizontal-pod-autoscaler failed to get memory utilization: missing request for memory. How to fix it?
我在 Google Kubernetes Engine 中有一个集群,并希望其中一个部署可以通过内存自动扩展。
完成部署后,我使用以下命令检查水平扩展
kubectl describe hpa -n my-namespace
结果如下:
Name: myapi-api-deployment
Namespace: my-namespace
Labels: <none>
Annotations: <none>
CreationTimestamp: Tue, 15 Feb 2022 12:21:44 +0100
Reference: Deployment/myapi-api-deployment
Metrics: ( current / target )
resource memory on pods (as a percentage of request): <unknown> / 50%
Min replicas: 1
Max replicas: 5
Deployment pods: 1 current / 1 desired
Conditions:
Type Status Reason Message
---- ------ ------ -------
AbleToScale True ReadyForNewScale recommended size matches current size
ScalingActive False FailedGetResourceMetric the HPA was unable to compute the replica count: failed to get memory utilization: missing request for memory
ScalingLimited False DesiredWithinRange the desired count is within the acceptable range
Events:
Type Reason Age From Message
---- ------ ---- ---- -------
Warning FailedGetResourceMetric 2m22s (x314 over 88m) horizontal-pod-autoscaler failed to get memory utilization: missing request for memory
当我使用 kubectl top
命令时,我可以看到内存和 cpu 使用情况。这是我的部署,包括自动缩放:
apiVersion: apps/v1
kind: Deployment
metadata:
name: my-api-deployment
namespace: my-namespace
annotations:
reloader.stakater.com/auto: "true"
spec:
replicas: 1
selector:
matchLabels:
app: my-api
version: v1
template:
metadata:
labels:
app: my-api
version: v1
annotations:
sidecar.istio.io/rewriteAppHTTPProbers: "true"
spec:
serviceAccountName: my-api-sa
containers:
- name: esp
image: gcr.io/endpoints-release/endpoints-runtime:2
imagePullPolicy: Always
args: [
"--listener_port=9000",
"--backend=127.0.0.1:8080",
"--service=myproject.company.ai"
]
ports:
- containerPort: 9000
- name: my-api
image: gcr.io/myproject/my-api:24
ports:
- containerPort: 8080
livenessProbe:
httpGet:
path: "/healthcheck"
port: 8080
initialDelaySeconds: 30
periodSeconds: 10
readinessProbe:
httpGet:
path: "/healthcheck"
port: 8080
initialDelaySeconds: 30
periodSeconds: 10
resources:
limits:
cpu: 500m
memory: 2048Mi
requests:
cpu: 300m
memory: 1024Mi
---
apiVersion: autoscaling/v2beta2
kind: HorizontalPodAutoscaler
metadata:
name: my-api-deployment
namespace: my-namespace
spec:
scaleTargetRef:
apiVersion: apps/v1
kind: Deployment
name: my-api-deployment
minReplicas: 1
maxReplicas: 5
metrics:
- type: Resource
resource:
name: memory
target:
type: "Utilization"
averageUtilization: 50
---
使用 GKE 文档推荐的autoscaling/v2beta2
将 HPA 与内存或 CPU 一起使用时,您需要为 HPA 使用的任何指标设置资源请求。见How does a HorizontalPodAutoscaler work,具体
For per-pod resource metrics (like CPU), the controller fetches the
metrics from the resource metrics API for each Pod targeted by the
HorizontalPodAutoscaler. Then, if a target utilization value is set,
the controller calculates the utilization value as a percentage of the
equivalent resource request on the containers in each Pod. If a target
raw value is set, the raw metric values are used directly.
您的 HPA 设置为与具有两个容器的 my-api-deployment
匹配。您为 my-api
设置了资源请求,但没有为 esp
设置。所以你只需要在esp
.
中添加一个内存资源请求
我在 Google Kubernetes Engine 中有一个集群,并希望其中一个部署可以通过内存自动扩展。
完成部署后,我使用以下命令检查水平扩展
kubectl describe hpa -n my-namespace
结果如下:
Name: myapi-api-deployment
Namespace: my-namespace
Labels: <none>
Annotations: <none>
CreationTimestamp: Tue, 15 Feb 2022 12:21:44 +0100
Reference: Deployment/myapi-api-deployment
Metrics: ( current / target )
resource memory on pods (as a percentage of request): <unknown> / 50%
Min replicas: 1
Max replicas: 5
Deployment pods: 1 current / 1 desired
Conditions:
Type Status Reason Message
---- ------ ------ -------
AbleToScale True ReadyForNewScale recommended size matches current size
ScalingActive False FailedGetResourceMetric the HPA was unable to compute the replica count: failed to get memory utilization: missing request for memory
ScalingLimited False DesiredWithinRange the desired count is within the acceptable range
Events:
Type Reason Age From Message
---- ------ ---- ---- -------
Warning FailedGetResourceMetric 2m22s (x314 over 88m) horizontal-pod-autoscaler failed to get memory utilization: missing request for memory
当我使用 kubectl top
命令时,我可以看到内存和 cpu 使用情况。这是我的部署,包括自动缩放:
apiVersion: apps/v1
kind: Deployment
metadata:
name: my-api-deployment
namespace: my-namespace
annotations:
reloader.stakater.com/auto: "true"
spec:
replicas: 1
selector:
matchLabels:
app: my-api
version: v1
template:
metadata:
labels:
app: my-api
version: v1
annotations:
sidecar.istio.io/rewriteAppHTTPProbers: "true"
spec:
serviceAccountName: my-api-sa
containers:
- name: esp
image: gcr.io/endpoints-release/endpoints-runtime:2
imagePullPolicy: Always
args: [
"--listener_port=9000",
"--backend=127.0.0.1:8080",
"--service=myproject.company.ai"
]
ports:
- containerPort: 9000
- name: my-api
image: gcr.io/myproject/my-api:24
ports:
- containerPort: 8080
livenessProbe:
httpGet:
path: "/healthcheck"
port: 8080
initialDelaySeconds: 30
periodSeconds: 10
readinessProbe:
httpGet:
path: "/healthcheck"
port: 8080
initialDelaySeconds: 30
periodSeconds: 10
resources:
limits:
cpu: 500m
memory: 2048Mi
requests:
cpu: 300m
memory: 1024Mi
---
apiVersion: autoscaling/v2beta2
kind: HorizontalPodAutoscaler
metadata:
name: my-api-deployment
namespace: my-namespace
spec:
scaleTargetRef:
apiVersion: apps/v1
kind: Deployment
name: my-api-deployment
minReplicas: 1
maxReplicas: 5
metrics:
- type: Resource
resource:
name: memory
target:
type: "Utilization"
averageUtilization: 50
---
使用 GKE 文档推荐的autoscaling/v2beta2
将 HPA 与内存或 CPU 一起使用时,您需要为 HPA 使用的任何指标设置资源请求。见How does a HorizontalPodAutoscaler work,具体
For per-pod resource metrics (like CPU), the controller fetches the metrics from the resource metrics API for each Pod targeted by the HorizontalPodAutoscaler. Then, if a target utilization value is set, the controller calculates the utilization value as a percentage of the equivalent resource request on the containers in each Pod. If a target raw value is set, the raw metric values are used directly.
您的 HPA 设置为与具有两个容器的 my-api-deployment
匹配。您为 my-api
设置了资源请求,但没有为 esp
设置。所以你只需要在esp
.