nginx ingress：在 nginx-controller 日志中忽略了 yaml 定义中的冲突主机，但造成了非工作情况

Question

已安装：https://helm.nginx.com/稳定版

[root@node1 ~]# helm search repo nginx-stable 
NAME                        CHART VERSION   APP VERSION DESCRIPTION             
nginx-stable/nginx-ingress  0.6.1           1.8.1       NGINX Ingress Controller

kubectl run nginx --image=nginx 
kubectl run apache --image=httpd 
kubectl expose pod nginx --port=80 --name=nginx-svc
kubectl expose pod apache --port=80

作为一种测试方法，pods 可以通过 nodeport 成功访问

我创建了两个 yaml 文件来让 ingress 正常工作

apiVersion: extensions/v1beta1
kind: Ingress
metadata:
  name: nginx-ing-rule
  annotations:
    nginx.org/rewrites: "serviceName=nginx-svc rewrite=/"
spec:
  rules:
  - host: kubernetes.somelan.lan
    http:
      paths:
      - path: /nginx
        backend:
          serviceName: nginx-svc
          servicePort: 80

对于 apache

apiVersion: extensions/v1beta1
kind: Ingress
metadata:
  name: apache-ingress
  annotations:
    nginx.org/rewrites: "serviceName=apache rewrite=/"
spec:
  rules:
  - host: kubernetes.somelan.lan
    http:
      paths:
      - path: /apache
        backend:
          serviceName: apache
          servicePort: 80

当我分别应用两个 yaml 时，它们被 nginx-controller 接受，因为它记录

conflicting server name "kubernetes.somelan.lan" on 0.0.0.0:80, ignored

结果： 我可以成功访问http://kubernetes.somelan.lan/apache, but i am unable to access http://kubernetes.somelan.lan/nginx。以不同的顺序应用 yaml 会呈现相同的结果如果我删除 apache yaml，nginx ingress 开始工作；我可以访问 http://kubernetes.somelan.lan/nginx

错误代码和日志为：

020/10/27 20:34:28 [error] 62#62: *20 open() "/etc/nginx/html/nginx" failed (2: No such file or directory), client: 10.10.70.22, server: kubernetes.somelan.lan, request: "GET /nginx HTTP/1.1", host: "kubernetes.somelan.lan"
10.10.70.22 - - [27/Oct/2020:20:34:28 +0000] "GET /nginx HTTP/1.1" 404 153 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:82.0) Gecko/20100101 Firefox/82.0" "-"
2020/10/27 20:34:28 [error] 62#62: *20 open() "/etc/nginx/html/favicon.ico" failed (2: No such file or directory), client: 10.10.70.22, server: kubernetes.somelan.lan, request: "GET /favicon.ico HTTP/1.1", host: "kubernetes.somelan.lan", referrer: "kubernetes.somelan.lan/nginx"
10.10.70.22 - - [27/Oct/2020:20:34:28 +0000] "GET /favicon.ico HTTP/1.1" 404 153 "kubernetes.somelan.lan/nginx" "Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:82.0) Gecko/20100101 Firefox/82.0" "-"

但是，如果我创建一个 yaml：

apiVersion: extensions/v1beta1
kind: Ingress
metadata:
  name: nginx-ing-rule
  annotations:
    nginx.org/rewrites: "serviceName=nginx-svc rewrite=/; serviceName=apache rewrite=/"
spec:
  rules:
  - host: kubernetes.somelan.lan
    http:
      paths:
      - path: /nginx
        backend:
          serviceName: nginx-svc
          servicePort: 80
      - path: /apache
        backend:
          serviceName: apache
          servicePort: 80

一切正常，我可以访问 /nginx 和 /apache

在 nginx 控制器上，使用单个 yaml 在控制器上提供以下配置：

# configuration for default/nginx-ing-rule
  
upstream default-nginx-ing-rule-kubernetes.somelan.lan-apache-80 {
    zone default-nginx-ing-rule-kubernetes.somelan.lan-apache-80 256k;
    random two least_conn;
    server 10.233.112.2:80 max_fails=1 fail_timeout=10s max_conns=0;
}
upstream default-nginx-ing-rule-kubernetes.somelan.lan-nginx-svc-80 {
    zone default-nginx-ing-rule-kubernetes.somelan.lan-nginx-svc-80 256k;
    random two least_conn;
    server 10.233.64.2:80 max_fails=1 fail_timeout=10s max_conns=0;
}
server {
    listen 80;
    server_tokens on;
    server_name kubernetes.somelan.lan;
    location /nginx {
        proxy_http_version 1.1;
        proxy_connect_timeout 60s;
        proxy_read_timeout 60s;
        proxy_send_timeout 60s;
        client_max_body_size 1m;
        proxy_set_header Host $host;
        proxy_set_header X-Real-IP $remote_addr;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_set_header X-Forwarded-Host $host;
        proxy_set_header X-Forwarded-Port $server_port;
        proxy_set_header X-Forwarded-Proto $scheme;
        proxy_buffering on;

        proxy_pass http://default-nginx-ing-rule-kubernetes.somelan.lan-nginx-svc-80/       
        
    }
    location /apache {
        proxy_http_version 1.1;
        proxy_connect_timeout 60s;
        proxy_read_timeout 60s;
        proxy_send_timeout 60s;
        client_max_body_size 1m;
        proxy_set_header Host $host;
        proxy_set_header X-Real-IP $remote_addr;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_set_header X-Forwarded-Host $host;
        proxy_set_header X-Forwarded-Port $server_port;
        proxy_set_header X-Forwarded-Proto $scheme;
        proxy_buffering on;

        proxy_pass http://default-nginx-ing-rule-kubernetes.somelan.lan-apache-80/;         
    }

如果我应用我之前指定的 2 个 yaml，nginx 控制器有 2 个配置

apache 入口的配置

# configuration for default/apache-ingress

upstream default-apache-ingress-kubernetes.somelan.lan-apache-80 {
    zone default-apache-ingress-kubernetes.somelan.lan-apache-80 256k;
    random two least_conn;
    
    server 10.233.112.2:80 max_fails=1 fail_timeout=10s max_conns=0;    
}

server {    
    listen 80;
    server_tokens on;
    server_name kubernetes.somelan.lan;
    location /apache {
        proxy_http_version 1.1;
        proxy_connect_timeout 60s;
        proxy_read_timeout 60s;
        proxy_send_timeout 60s;
        client_max_body_size 1m;
        proxy_set_header Host $host;
        proxy_set_header X-Real-IP $remote_addr;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_set_header X-Forwarded-Host $host;
        proxy_set_header X-Forwarded-Port $server_port;
        proxy_set_header X-Forwarded-Proto $scheme;
        proxy_buffering on;
        
        proxy_pass http://default-apache-ingress-kubernetes.somelan.lan-apache-80/;
    }
}

    }

nginx 入口的配置

# configuration for default/nginx-ing-rule

upstream default-nginx-ing-rule-kubernetes.somelan.lan-nginx-svc-80 {
    zone default-nginx-ing-rule-kubernetes.somelan.lan-nginx-svc-80 256k;
    random two least_conn;
    
    server 10.233.64.2:80 max_fails=1 fail_timeout=10s max_conns=0;
    
}


server {
    listen 80;
    server_tokens on;
    server_name kubernetes.somelan.lan;
    location /nginx {
        proxy_http_version 1.1;
        proxy_connect_timeout 60s;
        proxy_read_timeout 60s;
        proxy_send_timeout 60s;
        client_max_body_size 1m;
        proxy_set_header Host $host;
        proxy_set_header X-Real-IP $remote_addr;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_set_header X-Forwarded-Host $host;
        proxy_set_header X-Forwarded-Port $server_port;
        proxy_set_header X-Forwarded-Proto $scheme;
        proxy_buffering on;
        
        proxy_pass http://default-nginx-ing-rule-kubernetes.somelan.lan-nginx-svc-80/;
        
    }   
}

我是否认为使用两个 yaml 的方法应该呈现与使用一个 yaml 文件相同的结果？

Answer 1

这里要解释的东西很少。

您看到的错误：

conflicting server name "kubernetes.somelan.lan" on 0.0.0.0:80, ignored

是因为你不能有两个具有相同 server_name 并监听相同端口的服务器块。

了解 nginx model 中的操作是如何工作的很重要。这些如下：

Order Ingress rules by CreationTimestamp field, i.e., old rules first.

If the same path for the same host is defined in more than one Ingress, the oldest rule wins.

If more than one Ingress contains a TLS section for the same host, the oldest rule wins.

If multiple Ingresses define an annotation that affects the configuration of the Server block, the oldest rule wins.

Create a list of NGINX Servers (per hostname)

Create a list of NGINX Upstreams

If multiple Ingresses define different paths for the same host, the ingress controller will merge the definitions.

Annotations are applied to all the paths in the Ingress.

Multiple Ingresses can define different annotations. These definitions are not shared between Ingresses.

此外，请记住When a reload is required：

New Ingress Resource Created.

TLS section is added to existing Ingress.

Change in Ingress annotations that impacts more than just upstream configuration. For instance load-balance annotation does not require a reload.

A path is added/removed from an Ingress.

An Ingress, Service, Secret is removed.

Some missing referenced object from the Ingress is available, like a Service or Secret.

A Secret is updated.

现在，根据上面的信息，如果您创建了两个单独的入口，在您的用例中应该会发生以下情况：

If multiple Ingresses define different paths for the same host, the ingress controller will merge the definitions.

如果没有发生这种情况，您需要检查您的 Ingress Controller 及其版本以确保它支持合并操作。您可以查看有关 here.

的更多详细信息

终于回答你的主要问题了：

Am i wrong in expecting the method of using two yaml should render the same result as using one yaml file?

这取决于您的配置和您使用的入口控制器。在您的特定用例中，由于 server_name 冲突，使用两个 yaml 与使用一个 yaml 的效果不同。您发现启用合并选项很困难，请考虑使用 Merge Ingress Controller:

Merge Ingress Controller combines multiple ingress resources into a new one.

nginx ingress：在 nginx-controller 日志中忽略了 yaml 定义中的冲突主机，但造成了非工作情况

nginx ingress: conflicting host in yaml definition ignored in nginx-controller log, but creating non working situation

nginx

kubernetes

nginx-ingress

metallb