kubernetes 中的入口 nginx 路由（从 docker-compose 转换为 nginx 反向代理）
Ingress nginx routing in kubernetes (convertign from docker-compose with nginx reverse proxy)
我正在尝试将 运行 中的某些服务从 docker-compose 迁移到 kubernetes，并且努力从 nginx 反向代理迁移到入口 nginx。该服务使用具有 Web 界面的 h2o ai。我无法在 k8s 版本中调出 Web 界面。
我认为我需要修改 ingress 文件中的规范以进行适当的路由，感谢任何有关如何执行此操作的指示。
在使用 nginx 作为反向代理的 docker 解决方案中，当我访问 http://k8s-master:3002 它重定向到 http://k8s-master:3002/flow/index.html 并正确显示
对于 k8s，（我在 32000 上有 ingress-nginx-controller 运行 作为 NodePort），我尝试访问 https://k8s-master:32000/h2otest 并收到一条错误消息如下：
{"__meta":{"schema_version":3,"schema_name":"H2OErrorV3","schema_type":"H2OError"},"timestamp":1602674406013,"error_url":"Resource /h2otest","msg":"\n\nERROR MESSAGE:\n\nResource /h2otest not found\n\n","dev_msg":"\n\nERROR MESSAGE:\n\nResource /h2otest not found\n\n","http_status":404,"values":{},"exception_type":"water.exceptions.H2ONotFoundArgumentException","exception_msg":"\n\nERROR MESSAGE:\n\nResource /h2otest not found\n\n","stacktrace":["water.exceptions.H2ONotFoundArgumentException: Resource /h2otest not found"," water.api.RequestServer.response404(RequestServer.java:743)"," water.api.RequestServer.getResource(RequestServer.java:846)"," water.api.RequestServer.serve(RequestServer.java:465)"," water.api.RequestServer.doGeneric(RequestServer.java:301)"," water.api.RequestServer.doGet(RequestServer.java:225)"," javax.servlet.http.HttpServlet.service(HttpServlet.java:687)"," javax.servlet.http.HttpServlet.service(HttpServlet.java:790)"," org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:865)"," org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:535)"," org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:255)"," org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1317)"," org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:203)"," org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:473)"," org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:201)"," org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1219)"," org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:144)"," org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:126)"," org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:132)"," water.webserver.jetty9.Jetty9ServerAdapter$LoginHandler.handle(Jetty9ServerAdapter.java:130)"," org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:126)"," org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:132)"," org.eclipse.jetty.server.Server.handle(Server.java:531)"," org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:352)"," org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:260)"," org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:281)"," org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:102)"," org.eclipse.jetty.io.ChannelEndPoint.run(ChannelEndPoint.java:118)"," org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:762)"," org.eclipse.jetty.util.thread.QueuedThreadPool.run(QueuedThreadPool.java:680)"," java.lang.Thread.run(Thread.java:748)"]}
更新：基于 Ken Chen 的评论
我已经将下面的入口更改为使用 nginx.ingress.kubernetes.io/rewrite-target: /
现在，当我输入 https://k8s-master:32000/h2otest 时，它会重定向到：https://k8s-master:32000/flow/index.html
直接输入https://k8s-master:32000/h2otest/flow/index.html即可。 如何修改规则以便 https://k8s-master:32000/h2otest 直接进入 https://k8s-master:32000/h2otest/flow/index.html ?
以下包含每个版本的文件：
docker 版本
docker-compose.yml
services :
test-h2o-svc:
build:
context: h2o
dockerfile: Dockerfile
image: test-h2o:3.30.1.1
restart: always
networks:
- testnet
# ports:
# - 54323:54321
test-reverse-proxy:
image: nginx:alpine
hostname: reverse-proxy
restart: always
ports:
- 3002:3002
volumes:
- "./nginx_files/nginx.conf:/etc/nginx/nginx.conf:rw"
- "./nginx_files/sites-available/test.conf:/etc/nginx/conf.d/sites-available/test.conf:rw"
- "./nginx_files/sites-available/test.conf:/etc/nginx/conf.d/sites-enabled/test.conf:rw"
networks:
- testnet
networks:
testnet:
version: "3.5"
nginx.conf
user nginx;
worker_processes 4;
error_log /var/log/nginx/error.log warn;
pid /var/run/nginx.pid;
events {
worker_connections 1024;
}
http {
include /etc/nginx/mime.types;
default_type application/octet-stream;
log_format main '$remote_addr - $remote_user [$time_local] "$request" '
'$status $body_bytes_sent "$http_referer" '
'"$http_user_agent" "$http_x_forwarded_for"';
access_log /var/log/nginx/access.log main;
sendfile on;
#tcp_nopush on;
keepalive_timeout 180;
#gzip on;
include /etc/nginx/conf.d/sites-enabled/*.conf;
client_body_buffer_size 50M;
client_max_body_size 2000M;
}
test.conf
upstream link_h2o-svc {
server test-h2o-svc:54321;
}
server {
listen 3002;
location / {
proxy_pass http://link_h2o-svc;
}
}
Dockerfile 容器：test-h2o:3.30.1.1
FROM ubuntu:latest
ARG H2O_VERSION=3.30.1.1
RUN apt-get update \
&& apt-get install default-jdk unzip wget -y
RUN wget https://h2o-release.s3.amazonaws.com/h2o/rel-zeno/1/h2o-${H2O_VERSION}.zip \
&& unzip h2o-${H2O_VERSION}.zip
ENV H2O_VERSION ${H2O_VERSION}
CMD java -jar h2o-${H2O_VERSION}/h2o.jar
k8s版本
kind: Namespace
apiVersion: v1
metadata:
name: ns-test
labels:
name: ns-test
---
apiVersion: apps/v1
kind: Deployment
metadata:
annotations:
kompose.cmd: /snap/kompose/19/kompose-linux-amd64 convert
kompose.version: 1.21.0 (992df58d8)
creationTimestamp: null
labels:
app: test-h2o
name: test-h2o
namespace: ns-test
spec:
replicas: 1
selector:
matchLabels:
app: test-h2o
strategy: {}
template:
metadata:
annotations:
kompose.cmd: /snap/kompose/19/kompose-linux-amd64 convert
kompose.version: 1.21.0 (992df58d8)
creationTimestamp: null
labels:
app: test-h2o
spec:
containers:
- image: test-h2o:3.30.1.1
imagePullPolicy: ""
name: test-h2o
resources: {}
restartPolicy: Always
serviceAccountName: ""
volumes: null
status: {}
---
apiVersion: v1
kind: Service
metadata:
annotations:
kompose.cmd: /snap/kompose/19/kompose-linux-amd64 convert
kompose.version: 1.21.0 (992df58d8)
creationTimestamp: null
name: test-h2o-svc
namespace: ns-test
spec:
ports:
- name: "54321"
port: 54321
targetPort: 54321
selector:
app: test-h2o
status:
loadBalancer: {}
---
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: test-ingress
namespace: ns-test
annotations:
#ingress.kubernetes.io/rewrite-target: /
nginx.ingress.kubernetes.io/rewrite-target: /
spec:
rules:
- http:
paths:
- path: /h2otest(/|$)(.*)
pathType: Prefix
backend:
service:
name: test-h2o-svc
port:
number: 54321
Nginx 反向代理将 /h2otest 重写为 /，但 h2o 将 / 重定向为 /flow/index.html。此时，nginx 没有处理规则 /flow 所以结果为 404.
尝试删除路径并改用根 /。
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: test-ingress
namespace: ns-test
annotations:
nginx.ingress.kubernetes.io/rewrite-target: /
spec:
rules:
- http:
paths:
- path: /
pathType: Prefix
backend:
serviceName: test-h2o-svc
servicePort: 54321
How do I amend the rule so that https://k8s-master:30439/h2otest goes directly to https://k8s-master:30439/h2otest/flow/index.html
这不是 nginx-ingress 问题。与大多数其他入口一样 nginx-ingress 是基于名称的虚拟主机和 SNI（或基于 SSL 名称的虚拟主机）和 SSL 卸载。开箱即用，数量有限。
虽然 nginx-ingress 可以做一些基本的 re-writing，但仅限于将 /path/(.*) 重写为 /(.*) 并将其传递给下一个容器以处理来自那里的流量。如果您希望它做更多的事情，您可以为 nginx-ingress 编写配置映射。
是的，您需要为客户端体型等设置注释，但应用程序本身（不是入口）需要处理 /h2otest -> /h2otest/flow/index.html 重写（或 / -> /flow/index.html).
根据您所描述的内容，您的布局看起来像
(nginx)ingress -> app
您可能需要将布局更改为 (nginx)ingress -> nginx ->app 中间的 nginx 执行 / -> /flow/index.html重写
忠告。如果您有任何机会使用基于名称的虚拟主机而不是基于路径的虚拟主机（除非您有充分的理由，例如 cors）基于路径。现在很多应用程序只是假设它们在 / 上，如果你在其他地方
，配置会变得“有趣”
我正在尝试将 运行 中的某些服务从 docker-compose 迁移到 kubernetes，并且努力从 nginx 反向代理迁移到入口 nginx。该服务使用具有 Web 界面的 h2o ai。我无法在 k8s 版本中调出 Web 界面。
我认为我需要修改 ingress 文件中的规范以进行适当的路由，感谢任何有关如何执行此操作的指示。
在使用 nginx 作为反向代理的 docker 解决方案中，当我访问 http://k8s-master:3002 它重定向到 http://k8s-master:3002/flow/index.html 并正确显示
对于 k8s，（我在 32000 上有 ingress-nginx-controller 运行 作为 NodePort），我尝试访问 https://k8s-master:32000/h2otest 并收到一条错误消息如下：
{"__meta":{"schema_version":3,"schema_name":"H2OErrorV3","schema_type":"H2OError"},"timestamp":1602674406013,"error_url":"Resource /h2otest","msg":"\n\nERROR MESSAGE:\n\nResource /h2otest not found\n\n","dev_msg":"\n\nERROR MESSAGE:\n\nResource /h2otest not found\n\n","http_status":404,"values":{},"exception_type":"water.exceptions.H2ONotFoundArgumentException","exception_msg":"\n\nERROR MESSAGE:\n\nResource /h2otest not found\n\n","stacktrace":["water.exceptions.H2ONotFoundArgumentException: Resource /h2otest not found"," water.api.RequestServer.response404(RequestServer.java:743)"," water.api.RequestServer.getResource(RequestServer.java:846)"," water.api.RequestServer.serve(RequestServer.java:465)"," water.api.RequestServer.doGeneric(RequestServer.java:301)"," water.api.RequestServer.doGet(RequestServer.java:225)"," javax.servlet.http.HttpServlet.service(HttpServlet.java:687)"," javax.servlet.http.HttpServlet.service(HttpServlet.java:790)"," org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:865)"," org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:535)"," org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:255)"," org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1317)"," org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:203)"," org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:473)"," org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:201)"," org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1219)"," org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:144)"," org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:126)"," org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:132)"," water.webserver.jetty9.Jetty9ServerAdapter$LoginHandler.handle(Jetty9ServerAdapter.java:130)"," org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:126)"," org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:132)"," org.eclipse.jetty.server.Server.handle(Server.java:531)"," org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:352)"," org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:260)"," org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:281)"," org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:102)"," org.eclipse.jetty.io.ChannelEndPoint.run(ChannelEndPoint.java:118)"," org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:762)"," org.eclipse.jetty.util.thread.QueuedThreadPool.run(QueuedThreadPool.java:680)"," java.lang.Thread.run(Thread.java:748)"]}
更新：基于 Ken Chen 的评论
我已经将下面的入口更改为使用 nginx.ingress.kubernetes.io/rewrite-target: /
现在，当我输入 https://k8s-master:32000/h2otest 时，它会重定向到：https://k8s-master:32000/flow/index.html
直接输入https://k8s-master:32000/h2otest/flow/index.html即可。 如何修改规则以便 https://k8s-master:32000/h2otest 直接进入 https://k8s-master:32000/h2otest/flow/index.html ?
以下包含每个版本的文件：
docker 版本docker-compose.yml
services :
test-h2o-svc:
build:
context: h2o
dockerfile: Dockerfile
image: test-h2o:3.30.1.1
restart: always
networks:
- testnet
# ports:
# - 54323:54321
test-reverse-proxy:
image: nginx:alpine
hostname: reverse-proxy
restart: always
ports:
- 3002:3002
volumes:
- "./nginx_files/nginx.conf:/etc/nginx/nginx.conf:rw"
- "./nginx_files/sites-available/test.conf:/etc/nginx/conf.d/sites-available/test.conf:rw"
- "./nginx_files/sites-available/test.conf:/etc/nginx/conf.d/sites-enabled/test.conf:rw"
networks:
- testnet
networks:
testnet:
version: "3.5"
nginx.conf
user nginx;
worker_processes 4;
error_log /var/log/nginx/error.log warn;
pid /var/run/nginx.pid;
events {
worker_connections 1024;
}
http {
include /etc/nginx/mime.types;
default_type application/octet-stream;
log_format main '$remote_addr - $remote_user [$time_local] "$request" '
'$status $body_bytes_sent "$http_referer" '
'"$http_user_agent" "$http_x_forwarded_for"';
access_log /var/log/nginx/access.log main;
sendfile on;
#tcp_nopush on;
keepalive_timeout 180;
#gzip on;
include /etc/nginx/conf.d/sites-enabled/*.conf;
client_body_buffer_size 50M;
client_max_body_size 2000M;
}
test.conf
upstream link_h2o-svc {
server test-h2o-svc:54321;
}
server {
listen 3002;
location / {
proxy_pass http://link_h2o-svc;
}
}
Dockerfile 容器：test-h2o:3.30.1.1
FROM ubuntu:latest
ARG H2O_VERSION=3.30.1.1
RUN apt-get update \
&& apt-get install default-jdk unzip wget -y
RUN wget https://h2o-release.s3.amazonaws.com/h2o/rel-zeno/1/h2o-${H2O_VERSION}.zip \
&& unzip h2o-${H2O_VERSION}.zip
ENV H2O_VERSION ${H2O_VERSION}
CMD java -jar h2o-${H2O_VERSION}/h2o.jar
k8s版本
kind: Namespace
apiVersion: v1
metadata:
name: ns-test
labels:
name: ns-test
---
apiVersion: apps/v1
kind: Deployment
metadata:
annotations:
kompose.cmd: /snap/kompose/19/kompose-linux-amd64 convert
kompose.version: 1.21.0 (992df58d8)
creationTimestamp: null
labels:
app: test-h2o
name: test-h2o
namespace: ns-test
spec:
replicas: 1
selector:
matchLabels:
app: test-h2o
strategy: {}
template:
metadata:
annotations:
kompose.cmd: /snap/kompose/19/kompose-linux-amd64 convert
kompose.version: 1.21.0 (992df58d8)
creationTimestamp: null
labels:
app: test-h2o
spec:
containers:
- image: test-h2o:3.30.1.1
imagePullPolicy: ""
name: test-h2o
resources: {}
restartPolicy: Always
serviceAccountName: ""
volumes: null
status: {}
---
apiVersion: v1
kind: Service
metadata:
annotations:
kompose.cmd: /snap/kompose/19/kompose-linux-amd64 convert
kompose.version: 1.21.0 (992df58d8)
creationTimestamp: null
name: test-h2o-svc
namespace: ns-test
spec:
ports:
- name: "54321"
port: 54321
targetPort: 54321
selector:
app: test-h2o
status:
loadBalancer: {}
---
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: test-ingress
namespace: ns-test
annotations:
#ingress.kubernetes.io/rewrite-target: /
nginx.ingress.kubernetes.io/rewrite-target: /
spec:
rules:
- http:
paths:
- path: /h2otest(/|$)(.*)
pathType: Prefix
backend:
service:
name: test-h2o-svc
port:
number: 54321
Nginx 反向代理将 /h2otest 重写为 /，但 h2o 将 / 重定向为 /flow/index.html。此时，nginx 没有处理规则 /flow 所以结果为 404.
尝试删除路径并改用根 /。
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: test-ingress
namespace: ns-test
annotations:
nginx.ingress.kubernetes.io/rewrite-target: /
spec:
rules:
- http:
paths:
- path: /
pathType: Prefix
backend:
serviceName: test-h2o-svc
servicePort: 54321
How do I amend the rule so that https://k8s-master:30439/h2otest goes directly to https://k8s-master:30439/h2otest/flow/index.html
这不是 nginx-ingress 问题。与大多数其他入口一样 nginx-ingress 是基于名称的虚拟主机和 SNI（或基于 SSL 名称的虚拟主机）和 SSL 卸载。开箱即用，数量有限。
虽然 nginx-ingress 可以做一些基本的 re-writing，但仅限于将 /path/(.*) 重写为 /(.*) 并将其传递给下一个容器以处理来自那里的流量。如果您希望它做更多的事情，您可以为 nginx-ingress 编写配置映射。
是的，您需要为客户端体型等设置注释，但应用程序本身（不是入口）需要处理 /h2otest -> /h2otest/flow/index.html 重写（或 / -> /flow/index.html).
根据您所描述的内容，您的布局看起来像
(nginx)ingress -> app
您可能需要将布局更改为 (nginx)ingress -> nginx ->app 中间的 nginx 执行 / -> /flow/index.html重写
忠告。如果您有任何机会使用基于名称的虚拟主机而不是基于路径的虚拟主机（除非您有充分的理由，例如 cors）基于路径。现在很多应用程序只是假设它们在 / 上，如果你在其他地方