无法在 Cloud Foundry 中创建共享域 - 504 网关超时
Can not create shared-domain in Cloud Foundry - 504 Gateway Time-out
我无法在 Cloud Foundry 中创建共享域，任何推送的应用程序的运行状况检查连接都被拒绝。
我有基于 OpenStack IaaS 的 Cloud Foundry 工作环境。一切都按预期进行。我获取了我的部署文件，并在一段时间后成功地将其部署在 IaaS Vmware VSphere 7 中。问题是，我推送的每个应用程序都存在健康检查问题：
2020-10-29T16:55:01.43+0000 [CELL/0] OUT Cell 938b869c-5a68-40cc-9486-c5bc0d53a73a successfully destroyed container for instance 44e9c2a6-b54d-4fc4-4118-6d6d
2020-10-29T16:55:36.55+0000 [CELL/0] OUT Cell 938b869c-5a68-40cc-9486-c5bc0d53a73a creating container for instance 17f161a2-9788-426d-414d-6c33
2020-10-29T16:55:37.18+0000 [CELL/0] OUT Cell 938b869c-5a68-40cc-9486-c5bc0d53a73a successfully created container for instance 17f161a2-9788-426d-414d-6c33
2020-10-29T16:55:37.47+0000 [CELL/0] OUT Downloading droplet...
2020-10-29T16:55:37.75+0000 [CELL/0] OUT Downloaded droplet
2020-10-29T16:55:37.75+0000 [CELL/0] OUT Starting health monitoring of container
2020-10-29T16:56:38.45+0000 [HEALTH/0] ERR Failed to make TCP connection to port 8080: connection refused
2020-10-29T16:56:38.45+0000 [CELL/0] ERR Timed out after 1m0s: health check never passed.
2020-10-29T16:56:38.46+0000 [CELL/SSHD/0] OUT Exit status 0
2020-10-29T16:56:38.48+0000 [APP/PROC/WEB/0] OUT Exit status 143
我也无法创建任何共享域：
bash-5.0# cf create-shared-domain tcp.cf.test-env.net --router-group default-tcp -v
REQUEST: [2020-10-29T17:03:33Z]
GET /v2/info HTTP/1.1
Host: api.cf.test-env.net
Accept: application/json
User-Agent: cf/6.47.2+d526c2cb3.2019-11-05 (go1.12.12; amd64 linux)
RESPONSE: [2020-10-29T17:03:33Z]
HTTP/1.1 200 OK
Content-Length: 561
Content-Type: application/json;charset=utf-8
Date: Thu, 29 Oct 2020 17:03:33 GMT
Server: nginx
X-Content-Type-Options: nosniff
X-Vcap-Request-Id: 4badb79b-2faf-4623-6c3c-ce5fa3223cd5::dc43d2c9-c902-4429-9d65-d9a0060983c5
{
"api_version": "2.144.0",
"app_ssh_endpoint": "ssh.cf.test-env.net:2222",
"app_ssh_host_key_fingerprint": "ae:a3:ed:ad:37:d3:8a:7b:ed:b4:e5:d2:25:e5:8c:d0",
"app_ssh_oauth_client": "ssh-proxy",
"authorization_endpoint": "https://login.cf.test-env.net",
"build": "",
"description": "",
"doppler_logging_endpoint": "wss://doppler.cf.test-env.net:443",
"min_cli_version": null,
"min_recommended_cli_version": null,
"name": "",
"osbapi_version": "2.15",
"routing_endpoint": "https://api.cf.test-env.net/routing",
"support": "",
"token_endpoint": "https://uaa.cf.test-env.net",
"version": 0
}
REQUEST: [2020-10-29T17:03:33Z]
GET /login HTTP/1.1
Host: login.cf.test-env.net
Accept: application/json
Connection: close
User-Agent: cf/6.47.2+d526c2cb3.2019-11-05 (go1.12.12; amd64 linux)
RESPONSE: [2020-10-29T17:03:34Z]
HTTP/1.1 200 OK
Cache-Control: no-store
Content-Language: en-US
Content-Length: 384
Content-Type: application/json;charset=UTF-8
Date: Thu, 29 Oct 2020 17:03:34 GMT
Set-Cookie: X-Uaa-Csrf=NJlSPAjspn7m8oWuQdKsVD; Max-Age=86400; Expires=Fri, 30-Oct-2020 17:03:34 GMT; Path=/; Secure; HttpOnly
Strict-Transport-Security: max-age=31536000
X-Content-Type-Options: nosniff
X-Frame-Options: DENY
X-Vcap-Request-Id: 577d4d31-ec30-477e-6f44-c0dd9306270d
X-Xss-Protection: 1; mode=block
{
"app": {
"version": "74.12.0"
},
"commit_id": "7311e68",
"entityID": "login.cf.test-env.net",
"idpDefinitions": {},
"links": {
"login": "https://login.cf.test-env.net",
"passwd": "/forgot_password",
"register": "/create_account",
"uaa": "https://uaa.cf.test-env.net"
},
"prompts": {
"password": "[PRIVATE DATA HIDDEN]",
"username": [
"text",
"Email"
]
},
"timestamp": "2019-12-02T22:53:03+0000",
"zone_name": "uaa"
}
Creating shared domain tcp.cf.test-env.net as admin...
REQUEST: [2020-10-29T17:03:34Z]
GET /routing/v1/router_groups?name=default-tcp HTTP/1.1
Host: api.cf.test-env.net
Accept: application/json
Authorization: [PRIVATE DATA HIDDEN]
Connection: close
Content-Type: application/json
User-Agent: cf/6.47.2+d526c2cb3.2019-11-05 (go1.12.12; amd64 linux)
[application/json Content Hidden]
RESPONSE: [2020-10-29T17:03:34Z]
HTTP/1.1 200 OK
Content-Length: 114
Content-Type: application/json
Date: Thu, 29 Oct 2020 17:03:34 GMT
X-Vcap-Request-Id: 9459b068-0987-4f5e-7dee-1efdb5ca6fb8
[
{
"guid": "343ba1e8-88a7-4003-6db6-4feabedd072b",
"name": "default-tcp",
"reservable_ports": "1024-2048",
"type": "tcp"
}
]
REQUEST: [2020-10-29T17:03:34Z]
POST /v2/shared_domains HTTP/1.1
Host: api.cf.test-env.net
Accept: application/json
Authorization: [PRIVATE DATA HIDDEN]
Content-Type: application/json
User-Agent: cf/6.47.2+d526c2cb3.2019-11-05 (go1.12.12; amd64 linux)
{
"internal": false,
"name": "tcp.cf.test-env.net",
"router_group_guid": "343ba1e8-88a7-4003-6db6-4feabedd072b"
}
RESPONSE: [2020-10-29T17:04:04Z]
HTTP/1.0 504 Gateway Time-out
Cache-Control: no-cache
Connection: close
Content-Type: text/html
<html><body><h1>504 Gateway Time-out</h1>
The server didn't respond in time.
</body></html>
Error unmarshalling the following into a cloud controller error: <html><body><h1>504 Gateway Time-out</h1>
The server didn't respond in time.
</body></html>
FAILED
我怀疑网络配置问题，阻止了某些内部 CF 部件的连接。 VMware 中没有任何防火墙或任何规则。我还可以在 bosh 创建的 VM 之间执行 ping 和建立 ssh 连接。
任何想法，我还能做什么？
问题出在 VmWare NSX-T 上的 DNAT 和 SNAT 规则上。如果任何内部 VM 询问 dns 名称“api.cf.test-env.net”，它会获取远程 (public) IP 地址作为答案。当应该建立连接时，内部 VM 一直在通过 public IP 地址询问 api.cf.test-env.net，并通过 TCP 第二阶段 three-way-handshake 获取本地地址- 是什么导致了 TCP RST。正确创建 DNAT 和 SNAT 规则后，一切正常。我仍然想知道为什么 api.cf.test-env.net 没有被带有内部地址的 bosh-dns 回答。有谁知道为什么会这样以及如何更改它？
我无法在 Cloud Foundry 中创建共享域，任何推送的应用程序的运行状况检查连接都被拒绝。
我有基于 OpenStack IaaS 的 Cloud Foundry 工作环境。一切都按预期进行。我获取了我的部署文件，并在一段时间后成功地将其部署在 IaaS Vmware VSphere 7 中。问题是，我推送的每个应用程序都存在健康检查问题：
2020-10-29T16:55:01.43+0000 [CELL/0] OUT Cell 938b869c-5a68-40cc-9486-c5bc0d53a73a successfully destroyed container for instance 44e9c2a6-b54d-4fc4-4118-6d6d
2020-10-29T16:55:36.55+0000 [CELL/0] OUT Cell 938b869c-5a68-40cc-9486-c5bc0d53a73a creating container for instance 17f161a2-9788-426d-414d-6c33
2020-10-29T16:55:37.18+0000 [CELL/0] OUT Cell 938b869c-5a68-40cc-9486-c5bc0d53a73a successfully created container for instance 17f161a2-9788-426d-414d-6c33
2020-10-29T16:55:37.47+0000 [CELL/0] OUT Downloading droplet...
2020-10-29T16:55:37.75+0000 [CELL/0] OUT Downloaded droplet
2020-10-29T16:55:37.75+0000 [CELL/0] OUT Starting health monitoring of container
2020-10-29T16:56:38.45+0000 [HEALTH/0] ERR Failed to make TCP connection to port 8080: connection refused
2020-10-29T16:56:38.45+0000 [CELL/0] ERR Timed out after 1m0s: health check never passed.
2020-10-29T16:56:38.46+0000 [CELL/SSHD/0] OUT Exit status 0
2020-10-29T16:56:38.48+0000 [APP/PROC/WEB/0] OUT Exit status 143
我也无法创建任何共享域：
bash-5.0# cf create-shared-domain tcp.cf.test-env.net --router-group default-tcp -v
REQUEST: [2020-10-29T17:03:33Z]
GET /v2/info HTTP/1.1
Host: api.cf.test-env.net
Accept: application/json
User-Agent: cf/6.47.2+d526c2cb3.2019-11-05 (go1.12.12; amd64 linux)
RESPONSE: [2020-10-29T17:03:33Z]
HTTP/1.1 200 OK
Content-Length: 561
Content-Type: application/json;charset=utf-8
Date: Thu, 29 Oct 2020 17:03:33 GMT
Server: nginx
X-Content-Type-Options: nosniff
X-Vcap-Request-Id: 4badb79b-2faf-4623-6c3c-ce5fa3223cd5::dc43d2c9-c902-4429-9d65-d9a0060983c5
{
"api_version": "2.144.0",
"app_ssh_endpoint": "ssh.cf.test-env.net:2222",
"app_ssh_host_key_fingerprint": "ae:a3:ed:ad:37:d3:8a:7b:ed:b4:e5:d2:25:e5:8c:d0",
"app_ssh_oauth_client": "ssh-proxy",
"authorization_endpoint": "https://login.cf.test-env.net",
"build": "",
"description": "",
"doppler_logging_endpoint": "wss://doppler.cf.test-env.net:443",
"min_cli_version": null,
"min_recommended_cli_version": null,
"name": "",
"osbapi_version": "2.15",
"routing_endpoint": "https://api.cf.test-env.net/routing",
"support": "",
"token_endpoint": "https://uaa.cf.test-env.net",
"version": 0
}
REQUEST: [2020-10-29T17:03:33Z]
GET /login HTTP/1.1
Host: login.cf.test-env.net
Accept: application/json
Connection: close
User-Agent: cf/6.47.2+d526c2cb3.2019-11-05 (go1.12.12; amd64 linux)
RESPONSE: [2020-10-29T17:03:34Z]
HTTP/1.1 200 OK
Cache-Control: no-store
Content-Language: en-US
Content-Length: 384
Content-Type: application/json;charset=UTF-8
Date: Thu, 29 Oct 2020 17:03:34 GMT
Set-Cookie: X-Uaa-Csrf=NJlSPAjspn7m8oWuQdKsVD; Max-Age=86400; Expires=Fri, 30-Oct-2020 17:03:34 GMT; Path=/; Secure; HttpOnly
Strict-Transport-Security: max-age=31536000
X-Content-Type-Options: nosniff
X-Frame-Options: DENY
X-Vcap-Request-Id: 577d4d31-ec30-477e-6f44-c0dd9306270d
X-Xss-Protection: 1; mode=block
{
"app": {
"version": "74.12.0"
},
"commit_id": "7311e68",
"entityID": "login.cf.test-env.net",
"idpDefinitions": {},
"links": {
"login": "https://login.cf.test-env.net",
"passwd": "/forgot_password",
"register": "/create_account",
"uaa": "https://uaa.cf.test-env.net"
},
"prompts": {
"password": "[PRIVATE DATA HIDDEN]",
"username": [
"text",
"Email"
]
},
"timestamp": "2019-12-02T22:53:03+0000",
"zone_name": "uaa"
}
Creating shared domain tcp.cf.test-env.net as admin...
REQUEST: [2020-10-29T17:03:34Z]
GET /routing/v1/router_groups?name=default-tcp HTTP/1.1
Host: api.cf.test-env.net
Accept: application/json
Authorization: [PRIVATE DATA HIDDEN]
Connection: close
Content-Type: application/json
User-Agent: cf/6.47.2+d526c2cb3.2019-11-05 (go1.12.12; amd64 linux)
[application/json Content Hidden]
RESPONSE: [2020-10-29T17:03:34Z]
HTTP/1.1 200 OK
Content-Length: 114
Content-Type: application/json
Date: Thu, 29 Oct 2020 17:03:34 GMT
X-Vcap-Request-Id: 9459b068-0987-4f5e-7dee-1efdb5ca6fb8
[
{
"guid": "343ba1e8-88a7-4003-6db6-4feabedd072b",
"name": "default-tcp",
"reservable_ports": "1024-2048",
"type": "tcp"
}
]
REQUEST: [2020-10-29T17:03:34Z]
POST /v2/shared_domains HTTP/1.1
Host: api.cf.test-env.net
Accept: application/json
Authorization: [PRIVATE DATA HIDDEN]
Content-Type: application/json
User-Agent: cf/6.47.2+d526c2cb3.2019-11-05 (go1.12.12; amd64 linux)
{
"internal": false,
"name": "tcp.cf.test-env.net",
"router_group_guid": "343ba1e8-88a7-4003-6db6-4feabedd072b"
}
RESPONSE: [2020-10-29T17:04:04Z]
HTTP/1.0 504 Gateway Time-out
Cache-Control: no-cache
Connection: close
Content-Type: text/html
<html><body><h1>504 Gateway Time-out</h1>
The server didn't respond in time.
</body></html>
Error unmarshalling the following into a cloud controller error: <html><body><h1>504 Gateway Time-out</h1>
The server didn't respond in time.
</body></html>
FAILED
我怀疑网络配置问题，阻止了某些内部 CF 部件的连接。 VMware 中没有任何防火墙或任何规则。我还可以在 bosh 创建的 VM 之间执行 ping 和建立 ssh 连接。
任何想法，我还能做什么？
问题出在 VmWare NSX-T 上的 DNAT 和 SNAT 规则上。如果任何内部 VM 询问 dns 名称“api.cf.test-env.net”，它会获取远程 (public) IP 地址作为答案。当应该建立连接时，内部 VM 一直在通过 public IP 地址询问 api.cf.test-env.net，并通过 TCP 第二阶段 three-way-handshake 获取本地地址- 是什么导致了 TCP RST。正确创建 DNAT 和 SNAT 规则后，一切正常。我仍然想知道为什么 api.cf.test-env.net 没有被带有内部地址的 bosh-dns 回答。有谁知道为什么会这样以及如何更改它？