如何使用 MetalLB 和入口控制器在 bare-metal 集群中传递传入流量?

How to pass incoming traffic in bare-metal cluster with MetalLB and Ingress Controllers?

我正在努力在裸机 k8s 集群上将内部负载平衡暴露给外部世界。

假设我们有一个基本集群:

  1. 一些主节点和一些工作节点,有两个接口,一个public面向(eth0)和一个本地(eth1),ip在192.168.0.0/16网络内

  2. 已部署 MetalLB 并为其内部 ips 配置 192.168.200.200-192.168.200.254 范围

  3. 入口控制器及其类型为 LoadBalancer 的服务

据我目前的理解,MetalLB 现在应该将 192.168.200.200-192.168.200.254 中的一个 ip 分配给入口服务。

但我有以下问题:

在每个节点上,我可以卷曲入口控制器外部 IP(只要它们可以在 eth1 上访问)附加主机 header 并从相应入口资源中配置的服务获得响应,或者它是否仅有效在当前放置 Ingress pods 的节点上?

我有哪些选项可以将传入的外部流量传递到 eth0 到在 eth1 网络上侦听的入口?

是否可以转发请求保存源 ip 地址或附加 X-Forwarded-For header 是唯一的选择?

假设我们正在谈论 Metallb 使用 Layer2

解决以下问题:

On every node I could curl ingress controller externalIP (as long as they are reachable on eth1) with host header attached and get a response from a service thats configured in coresponding ingress resource or is it valid only on node where Ingress pods are currently placed?

Is it possible to forward requests saving source ip address or attaching X-Forwarded-For header is the only option?

在保留源IP的前提下分解,本题可以双向:


保留源 IP 地址

为此,您需要通过设置(在您的 YAML 清单中)将 Ingress controllerService of type LoadBalancer 设置为支持“本地流量策略”:

  • .spec.externalTrafficPolicy: Local

只要在每个 Node 上都有您的 Ingress controller 的副本,此设置就会有效,因为进入您的控制器的所有网络都将包含在一个 Node 中.

引用官方文档:

With the Local traffic policy, kube-proxy on the node that received the traffic sends it only to the service’s pod(s) that are on the same node. There is no “horizontal” traffic flow between nodes.

Because kube-proxy doesn’t need to send traffic between cluster nodes, your pods can see the real source IP address of incoming connections.

The downside of this policy is that incoming traffic only goes to some pods in the service. Pods that aren’t on the current leader node receive no traffic, they are just there as replicas in case a failover is needed.

Metallb.universe.tf: Usage: Local traffic policy


不保留源 IP 地址

如果您的用例不需要您保留源 IP 地址,您可以使用:

  • .spec.externalTrafficPolicy: Cluster

此设置不需要您的 Ingress controller 的副本出现在每个 Node 上。

引用官方文档:

With the default Cluster traffic policy, kube-proxy on the node that received the traffic does load-balancing, and distributes the traffic to all the pods in your service.

This policy results in uniform traffic distribution across all pods in the service. However, kube-proxy will obscure the source IP address of the connection when it does load-balancing, so your pod logs will show that external traffic appears to be coming from the service’s leader node.

Metallb.universe.tf: Usage: Cluster traffic policy


解决第二个问题:

What are my options to pass incoming external traffic to eth0 to an ingress listening on eth1 network?

Metallb 默认侦听所有接口,您需要做的就是在 Metallb 配置中从这个 eth 指定地址池。

您可以通过以下方式找到有关此主题的更多参考资料:

此类配置的示例如下:

apiVersion: v1
kind: ConfigMap
metadata:
  namespace: metallb-system
  name: config
data:
  config: |
    address-pools: # HERE
    - name: my-ip-space
      protocol: layer2
      addresses:
      - 192.168.1.240/28