Kubernetes AWS NLB 服务创建过多的安全组规则
Kubernetes AWS NLB Services Create Excessive Amount Of Security Group Rules
Kubernetes Service doc 下面说明了如何为每个 NLB 服务更新节点安全组。
不幸的是，我的 VPC 有 3 个不同的 CIDR。这意味着对于服务上的每个端口，节点的安全组中都会添加 4 个新规则。有一个团队拥有带 5 个端口的 NLB 服务，这意味着它会导致 20 个新规则添加到节点的安全组中。其他团队通常有 2 个端口，这导致 8 个规则被添加到节点的安全组中。最终结果是我们有时会达到一个安全组允许的最大数量 64 条规则。
有什么想法可以围绕此进行设计，以便团队可以根据需要创建具有尽可能多的端口的尽可能多的 NLB 服务？
EKS documentation 对此有所说明。
For each NLB that you create Amazon EKS adds one inbound rule to the
node's security group for client traffic and one rule for each load
balancer subnet in the VPC for health checks. Deployment of a service
of type LoadBalancer can fail if Amazon EKS attempts to create rules
that exceed the quota for the maximum number of rules allowed for a
security group. For more information, see Security groups in Amazon
VPC quotas in the Amazon VPC User Guide. Consider the following
options to minimize the chances of exceeding the maximum number of
rules for a security group.
Request an increase in your rules per security group quota. For more
information, see Requesting a quota increase in the Service Quotas
User Guide.
Use Create a network load balancer, rather than instance targets. With
IP targets, rules can potentially be shared for the same target ports.
Load balancer subnets can be manually specified with an annotation.
For more information, see Annotations on GitHub.
Use an Ingress, instead of a Service of type LoadBalancer to send
traffic to your service. The AWS Application Load Balancer (ALB)
requires fewer rules than NLBs. An ALB can also be shared across
multiple Ingresses. For more information, see Application load
balancing on Amazon EKS.
Deploy your clusters to multiple accounts.
如果这些选项中的 none 适合您，并且您也有 ALB，您可以最小化那些 ALB 的规则，强制它们使用特定的安全组，而不是将它们的规则添加到节点的安全组.注释为service.beta.kubernetes.io/aws-load-balancer-security-groups。这样做，您将几个规则替换为一个规则，为 NLB 规则留下更多 space。
Kubernetes Service doc 下面说明了如何为每个 NLB 服务更新节点安全组。
不幸的是，我的 VPC 有 3 个不同的 CIDR。这意味着对于服务上的每个端口，节点的安全组中都会添加 4 个新规则。有一个团队拥有带 5 个端口的 NLB 服务，这意味着它会导致 20 个新规则添加到节点的安全组中。其他团队通常有 2 个端口，这导致 8 个规则被添加到节点的安全组中。最终结果是我们有时会达到一个安全组允许的最大数量 64 条规则。
有什么想法可以围绕此进行设计，以便团队可以根据需要创建具有尽可能多的端口的尽可能多的 NLB 服务？
EKS documentation 对此有所说明。
For each NLB that you create Amazon EKS adds one inbound rule to the node's security group for client traffic and one rule for each load balancer subnet in the VPC for health checks. Deployment of a service of type LoadBalancer can fail if Amazon EKS attempts to create rules that exceed the quota for the maximum number of rules allowed for a security group. For more information, see Security groups in Amazon VPC quotas in the Amazon VPC User Guide. Consider the following options to minimize the chances of exceeding the maximum number of rules for a security group.
Request an increase in your rules per security group quota. For more information, see Requesting a quota increase in the Service Quotas User Guide.
Use Create a network load balancer, rather than instance targets. With IP targets, rules can potentially be shared for the same target ports. Load balancer subnets can be manually specified with an annotation. For more information, see Annotations on GitHub.
Use an Ingress, instead of a Service of type LoadBalancer to send traffic to your service. The AWS Application Load Balancer (ALB) requires fewer rules than NLBs. An ALB can also be shared across multiple Ingresses. For more information, see Application load balancing on Amazon EKS.
Deploy your clusters to multiple accounts.
如果这些选项中的 none 适合您，并且您也有 ALB，您可以最小化那些 ALB 的规则，强制它们使用特定的安全组，而不是将它们的规则添加到节点的安全组.注释为service.beta.kubernetes.io/aws-load-balancer-security-groups。这样做，您将几个规则替换为一个规则，为 NLB 规则留下更多 space。