nginx 中的动态 proxy_pass 到 Kubernetes 中的另一个 pod

Question

我正在尝试创建一个 nginx 代理，将请求转发到 /<service> 到 http://<service>。我首先尝试了以下方法：

location ~ ^/(.+)$ {
    set $backend "http://:80";
    proxy_pass $backend;
}

但是它没有说类似的话（当调用 /myservice）：

[error] 7741#0: *1 no resolver defined to resolve http://myservice

由于无法从外部访问 myservice 我已尝试安装 go-dnsmasq as a sidecar in the same pod and I try to use it for DNS resolution (like I've seen in this 示例）并将我的 nginx 配置更改为如下所示：

location ~ ^/(.+)$ {
        resolver 127.0.0.1:53;
        set $backend "http://:80";
        proxy_pass $backend;
}

但是现在 nginx 失败了：

[error] 9#9: *734 myservice could not be resolved (2: Server failure), client: 127.0.0.1, server: nginx-proxy, request: "GET /myservice HTTP/1.1", host: "localhost:8080"
127.0.0.1 - xxx [30/May/2016:10:34:23 +0000] "GET /myservice HTTP/1.1" 502 173 "-" "curl/7.38.0" "-"

我的 Kubernetes pod 如下所示：

spec:
  containers:
    - name: nginx
      image: "nginx:1.10.0"
      ports:
        - containerPort: 8080
          name: "external"
          protocol: "TCP"
    - name: dnsmasq
      image: "janeczku/go-dnsmasq:release-1.0.5"
      args:
        - --listen
        - "0.0.0.0:53"

运行 netstat -ntlp 在 dnsmasq 容器中给我：

Proto Recv-Q Send-Q Local Address           Foreign Address         State       PID/Program name
tcp        0      0 0.0.0.0:8080            0.0.0.0:*               LISTEN      -
tcp        0      0 :::53                   :::*                    LISTEN      1/go-dnsmasq

和运行nmap --min-parallelism 100 -sT -sU localhost在nginx容器中：

Starting Nmap 6.47 ( http://nmap.org ) at 2016-05-30 10:33 UTC
Nmap scan report for localhost (127.0.0.1)
Host is up (0.00055s latency).
Other addresses for localhost (not scanned): 127.0.0.1
Not shown: 1997 closed ports
PORT     STATE SERVICE
53/tcp   open  domain
8080/tcp open  http-proxy
53/udp   open  domain

看来dnsmasq和nginx确实是起来了运行？我可能做错了什么？

Answer 1

经过大量研究和反复试验，我设法解决了这个问题。首先，我将 pod 规格更改为：

spec:
  containers:
    - name: nginx
      image: "nginx:1.10.0"
      ports:
        - containerPort: 8080
          name: "external"
          protocol: "TCP"
    - name: dnsmasq
      image: "janeczku/go-dnsmasq:release-1.0.5"
      args:
        - --listen
        - "127.0.0.1:53"
        - --default-resolver
        - --append-search-domains
        - --hostsfile=/etc/hosts
        - --verbose

然后我还必须为 nginx 中的解析器禁用 ipv6：

location ~ ^/(.+)$ {
        resolver 127.0.0.1:53 ipv6=off;
        set $backend "http://:80";
        proxy_pass $backend;
}

然后它按预期工作！

Answer 2

我通过coredns解决了这个问题 docker ：我的 nginx 和 coredns 都部署在主机上

第一步：配置核心文件 在 Corefile 中，也许你应该更改 k8s 主配置参考：https://coredns.io/plugins/kubernetes/

sudo mkdir /etc/coredns; sudo tee /etc/coredns/Corefile   <<-'EOF' .:53 {
    log
    errors
    health
    kubernetes cluster.local in-addr.arpa ip6.arpa {
       endpoint http://172.31.88.71:8080
       pods insecure
       upstream
       fallthrough in-addr.arpa ip6.arpa
       ttl 30
    }
    forward . /etc/resolv.conf
    cache 30
    loop
    reload
    loadbalance } EOF

step2:config docker 然后启动

tee coreos.sh   <<-'EOF'
 docker run --restart=always  -idt --name coredns \
 -v /etc/coredns/Corefile:/etc/coredns/Corefile \
 -v /home/ec2-user/.kube/config:/etc/coredns/kubeconfig \
 -p 53:53/udp \
 coredns/coredns:1.6.9 \
 -conf /etc/coredns/Corefile
EOF

step3：配置nginx然后重新加载

resolver 127.0.0.1 valid=60s ipv6=off;

nginx 中的动态 proxy_pass 到 Kubernetes 中的另一个 pod

Dynamic proxy_pass in nginx to another pod in Kubernetes

nginx

dns

kubernetes

dnsmasq