ISTIO:在出口启用熔断
ISTIO: enable circuit breaking on egress
我无法通过出口配置让断路配置在我的 elb 上工作。
ELB
elb 的成功率为 25%(75% 500 错误和 25% 状态 200),
elb 有 4 个实例,只有 1 个 returns 成功响应,其他实例配置为 returns 500 错误用于测试目的。
设置
k8s: v1.7.4
istio: 0.5.0
- 环境:AWS 上的 K8S
出口规则
apiVersion: config.istio.io/v1alpha2
kind: EgressRule
metadata:
name: elb-egress-rule
spec:
destination:
service: xxxx.us-east-1.elb.amazonaws.com
ports:
- port: 80
protocol: http
目标策略
kind: DestinationPolicy
metadata:
name: elb-circuit-breaker
spec:
destination:
service: xxxx.us-east-1.elb.amazonaws.com
loadBalancing:
name: RANDOM
circuitBreaker:
simpleCb:
maxConnections: 100
httpMaxPendingRequests: 100
sleepWindow: 3m
httpDetectionInterval: 1s
httpMaxEjectionPercent: 100
httpConsecutiveErrors: 3
httpMaxRequestsPerConnection: 10
路由规则:未设置
测试
apiVersion: v1
kind: Service
metadata:
name: sleep
labels:
app: sleep
spec:
ports:
- port: 80
name: http
selector:
app: sleep
---
apiVersion: extensions/v1beta1
kind: Deployment
metadata:
name: sleep
spec:
replicas: 1
template:
metadata:
labels:
app: sleep
spec:
containers:
- name: sleep
image: tutum/curl
command: ["/bin/sleep","infinity"]
imagePullPolicy: IfNotPresent
.
export SOURCE_POD=$(kubectl get pod -l app=sleep -o jsonpath={.items..metadata.name})
kubectl exec -it $SOURCE_POD -c sleep bash
从 pod 并行发送请求
#!/bin/sh
set -m # Enable Job Control
for i in `seq 100`; do # start 100 jobs in parallel
curl xxxx.us-east-1.elb.amazonaws.com &
done
回应
目前,Istio 考虑 Egress Rule
来指定单个主机。由于 Envoy(Istio 的 sidecar 代理实现)的负载均衡器的 panic 阈值,这台主机不会被弹出。 Envoy 的默认恐慌阈值为 50%。也就是说至少需要两台主机才能弹出一台主机,所以一个Egress Rule
的单个主机不会被弹出
这实际上意味着 httpConsecutiveErrors
不会影响外部服务。这种功能缺失应该通过 Istio 的 External Services
部分解决,它将取代 Egress Rules
.
查看由多个端点支持的 Istio External Services
文档 -https://github.com/istio/api/blob/master/routing/v1alpha2/external_service.proto#L113
我无法通过出口配置让断路配置在我的 elb 上工作。
ELB elb 的成功率为 25%(75% 500 错误和 25% 状态 200), elb 有 4 个实例,只有 1 个 returns 成功响应,其他实例配置为 returns 500 错误用于测试目的。
设置
k8s: v1.7.4
istio: 0.5.0
- 环境:AWS 上的 K8S
出口规则
apiVersion: config.istio.io/v1alpha2
kind: EgressRule
metadata:
name: elb-egress-rule
spec:
destination:
service: xxxx.us-east-1.elb.amazonaws.com
ports:
- port: 80
protocol: http
目标策略
kind: DestinationPolicy
metadata:
name: elb-circuit-breaker
spec:
destination:
service: xxxx.us-east-1.elb.amazonaws.com
loadBalancing:
name: RANDOM
circuitBreaker:
simpleCb:
maxConnections: 100
httpMaxPendingRequests: 100
sleepWindow: 3m
httpDetectionInterval: 1s
httpMaxEjectionPercent: 100
httpConsecutiveErrors: 3
httpMaxRequestsPerConnection: 10
路由规则:未设置
测试
apiVersion: v1
kind: Service
metadata:
name: sleep
labels:
app: sleep
spec:
ports:
- port: 80
name: http
selector:
app: sleep
---
apiVersion: extensions/v1beta1
kind: Deployment
metadata:
name: sleep
spec:
replicas: 1
template:
metadata:
labels:
app: sleep
spec:
containers:
- name: sleep
image: tutum/curl
command: ["/bin/sleep","infinity"]
imagePullPolicy: IfNotPresent
.
export SOURCE_POD=$(kubectl get pod -l app=sleep -o jsonpath={.items..metadata.name})
kubectl exec -it $SOURCE_POD -c sleep bash
从 pod 并行发送请求
#!/bin/sh
set -m # Enable Job Control
for i in `seq 100`; do # start 100 jobs in parallel
curl xxxx.us-east-1.elb.amazonaws.com &
done
回应
目前,Istio 考虑 Egress Rule
来指定单个主机。由于 Envoy(Istio 的 sidecar 代理实现)的负载均衡器的 panic 阈值,这台主机不会被弹出。 Envoy 的默认恐慌阈值为 50%。也就是说至少需要两台主机才能弹出一台主机,所以一个Egress Rule
的单个主机不会被弹出
这实际上意味着 httpConsecutiveErrors
不会影响外部服务。这种功能缺失应该通过 Istio 的 External Services
部分解决,它将取代 Egress Rules
.
查看由多个端点支持的 Istio External Services
文档 -https://github.com/istio/api/blob/master/routing/v1alpha2/external_service.proto#L113