Apache websocket 重定向到 Tomcat：mod_proxy 和 mod_proxy_wstunnel
Apache websocket redirection to Tomcat: mod_proxy and mod_proxy_wstunnel
我正在尝试使用 mod_proxy 和 mod_proxy_wstunnel 模块将流量从 Apache 重定向到 Tomcat。 HTTP 流量重定向没有问题，但我无法使用目前尝试的任何配置成功重定向 websocket 流量。
我正在使用 Apache 2.4.28 和 Tomcat 8.5.13
我必须说，当我在没有 Apache 的情况下使用 Tomcat 时，websockets 工作得非常好：
下一个适用于此配置的 Tomcat 连接器：
<Connector URIEncoding="UTF-8"
compressableMimeType="text/html,text/xml,text/plain,text/css,text/javascript,application/javascript"
compression="on"
compressionMinSize="1024"
connectionTimeout="20000"
noCompressionUserAgents="gozilla, traviata"
port="443"
protocol="org.apache.coyote.http11.Http11AprProtocol"
SSLEnabled="true"
scheme="https"
secure="true">
<UpgradeProtocol className="org.apache.coyote.http2.Http2Protocol" />
<SSLHostConfig>
<Certificate certificateKeyFile="/opt/tomcat/cert/privkey.pem"
certificateFile="/opt/tomcat/cert/cert.pem"
certificateChainFile="/opt/tomcat/cert/chain.pem"
type="RSA" />
</SSLHostConfig>
</Connector>
到此为止一切都清楚了。现在我在 Tomcat 前面启动一个 Apache 服务器，我首先更改的是 Tomcat 连接器，如下所示：
<Connector URIEncoding="UTF-8"
compressableMimeType="text/html,text/xml,text/plain,text/css,text/javascript,application/javascript"
compression="on"
compressionMinSize="1024"
connectionTimeout="20000"
noCompressionUserAgents="gozilla, traviata"
port="8080"
protocol="org.apache.coyote.http11.Http11AprProtocol">
<UpgradeProtocol className="org.apache.coyote.http2.Http2Protocol" />
</Connector>
在 Apache 中我成功加载了下一个模块（我检查过它们是否真的加载了）：
LoadModule proxy_module modules/mod_proxy.so
LoadModule proxy_http_module modules/mod_proxy_http.so
LoadModule proxy_wstunnel_module modules/mod_proxy_wstunnel.so
LoadModule ssl_module modules/mod_ssl.so
这是我在 vhosts.conf 文件中尝试的配置之一：
<VirtualHost *:443>
ServerName www.example.com
ServerAdmin server@example.com
LogFormat "%h %l %u %t \"%r\" %>s %b \"%{Referer}i\" \"%{User-agent}i\"" combined
CustomLog /var/log/httpd/lavnet_access.log combined
ErrorLog /var/log/httpd/lavnet_error.log
SSLProxyEngine on
#websocket
RewriteEngine on
RewriteCond %{HTTP:Upgrade} "(?i)websocket"
RewriteRule ^/(.*)$ wss://www.example.com/ [P]
#rest
ProxyPass "/" "http://www.example.com:8080/"
ProxyPassReverse "/" "http://www.example.com:8080/"
SSLEngine on
SSLCertificateFile /etc/letsencrypt/live/example.com/cert.pem
SSLCertificateKeyFile /etc/letsencrypt/live/example.com/privkey.pem
SSLCertificateChainFile /etc/letsencrypt/live/example.com/chain.pem
</VirtualHost>
这是我取得的最佳效果的配置，但仍然无法正常工作。当我尝试建立此连接时，lavnet_error.log 中的日志跟踪似乎相当不错：
[Tue Oct 10 16:46:39.014980 2017] [proxy:debug] [pid 10558:tid 47319680603904] proxy_util.c(2209): [client XX.XX.XX.109:11208] AH00944: connecting wss://www.example.com:443/rest/notify/675/fgcw02lm/websocket to www.example.com:443
[Tue Oct 10 16:46:39.016495 2017] [proxy:debug] [pid 10558:tid 47319680603904] proxy_util.c(2418): [client XX.XX.XX.109:11208] AH00947: connected /rest/notify/675/fgcw02lm/websocket to www.example.com:443
[Tue Oct 10 16:46:39.016567 2017] [proxy:debug] [pid 10558:tid 47319680603904] proxy_util.c(2887): AH02824: WSS: connection established with XX.XX.XX.109:443 (*)
[Tue Oct 10 16:46:39.016590 2017] [proxy:debug] [pid 10558:tid 47319680603904] proxy_util.c(3054): AH00962: WSS: connection complete to XX.XX.XX.109:443 (www.example.com)
[Tue Oct 10 16:46:39.016603 2017] [ssl:info] [pid 10558:tid 47319680603904] [remote 217.61.129.109:443] AH01964: Connection to child 0 established (server www.example.com:443)
[Tue Oct 10 16:46:39.026370 2017] [proxy:debug] [pid 10558:tid 47319680603904] proxy_util.c(2171): AH00943: WSS: has released connection for (*)
但这是 Chrome 显示的错误：
我也尝试过其他配置：
<VirtualHost *:443>
ServerName www.example.com
ServerAdmin server@example.com
LogFormat "%h %l %u %t \"%r\" %>s %b \"%{Referer}i\" \"%{User-agent}i\"" combined
CustomLog /var/log/httpd/lavnet_access.log combined
ErrorLog /var/log/httpd/lavnet_error.log
SSLProxyEngine on
ProxyPass "/rest/notify/" "wss://www.example.com:8080/rest/notify"
ProxyPassReverse "/rest/notify/" "wss://www.example.com:8080/rest/notify"
ProxyPass "/" "http://www.example.com:8080/"
ProxyPassReverse "/" "http://www.example.com:8080/"
SSLEngine on
SSLCertificateFile /etc/letsencrypt/live/example.com/cert.pem
SSLCertificateKeyFile /etc/letsencrypt/live/example.com/privkey.pem
SSLCertificateChainFile /etc/letsencrypt/live/example.com/chain.pem
</VirtualHost>
但在这种情况下，我收到“500（内部服务器错误）”并且我还可以在 lavnet_error.log:
中看到下一个跟踪
[Tue Oct 10 17:14:14.778824 2017] [proxy:warn] [pid 11924:tid 47694559057664] [client XX.XX.XXX.189:11665] AH01144: No protocol handler was valid for the URL /rest/notify/info (scheme 'wss'). If you are using a DSO version of mod_proxy, make sure the proxy submodules are included in the configuration using LoadModule., referer: https://www.example.com/equipment-command-panel/8287/8482
我已经尝试了很多配置，但我无法让它工作。我希望你能帮助我。
谢谢。
经过多次尝试终于解决了。接下来是工作配置，以备不时之需：
这是 Apache 的 vhost.conf 文件：
<VirtualHost *:443>
ServerName www.example.com
ServerAdmin admin@example.com
LogFormat "%h %l %u %t \"%r\" %>s %b \"%{Referer}i\" \"%{User-agent}i\"" combined
CustomLog /var/log/httpd/lavnet_access.log combined
ErrorLog /var/log/httpd/lavnet_error.log
ProxyPreserveHost On
ProxyPass / http://www.example.com:8080/
ProxyPassReverse / http://www.example.com:8080/
ProxyRequests Off
RewriteEngine on
RewriteCond %{HTTP:UPGRADE} ^WebSocket$ [NC]
RewriteCond %{HTTP:CONNECTION} Upgrade$ [NC]
RewriteRule .* ws://www.example.com:8080%{REQUEST_URI} [P]
SSLEngine on
SSLCertificateFile /etc/letsencrypt/live/example.com/cert.pem
SSLCertificateKeyFile /etc/letsencrypt/live/example.com/privkey.pem
SSLCertificateChainFile /etc/letsencrypt/live/example.com/chain.pem
</VirtualHost>
而这个是 Tomcat 中 server.xml 中定义的连接器：
<Connector URIEncoding="UTF-8"
connectionTimeout="20000"
port="8080"
protocol="org.apache.coyote.http11.Http11AprProtocol">
<UpgradeProtocol className="org.apache.coyote.http2.Http2Protocol" />
</Connector>
谢谢。
接受的解决方案适用于单个实例。但是我有多个服务器的集群，我有带 mod_jk 模块的负载平衡器。在这种情况下，经过数小时的搜索，我可以找到解决问题的方法。在 mod_jk 中，服务器名称附加到 JSESSIONID 的末尾，如下所示：JSESSIONID=25B9813E079BA3543C242438FD74CA84.banana.
所以我可以编写一个规则，使用以下代码将 websocket 请求代理到正确的服务器：
SSLProxyEngine On
RewriteEngine on
RewriteCond %{HTTP:UPGRADE} ^WebSocket$ [NC]
RewriteCond %{HTTP:CONNECTION} Upgrade$ [NC]
RewriteCond %{HTTP_COOKIE} (^|;\ *)JSESSIONID=.*\.(.+?)(?=;|$) [NC]
RewriteRule .* wss://%2:8443%{REQUEST_URI} [P]
正如您在上一个 RewriteCond 中看到的那样，我编写了一个正则表达式，它在第二个正则表达式组中查找服务器的名称。所以我写了一个代理，将请求路由到 wss://%2:8443%{REQUEST_URI} 这是正确的服务器。
p.s: 不要忘记将服务器名称和 IP 添加到 /etc/hosts
我正在尝试使用 mod_proxy 和 mod_proxy_wstunnel 模块将流量从 Apache 重定向到 Tomcat。 HTTP 流量重定向没有问题，但我无法使用目前尝试的任何配置成功重定向 websocket 流量。
我正在使用 Apache 2.4.28 和 Tomcat 8.5.13
我必须说，当我在没有 Apache 的情况下使用 Tomcat 时，websockets 工作得非常好：
下一个适用于此配置的 Tomcat 连接器：
<Connector URIEncoding="UTF-8"
compressableMimeType="text/html,text/xml,text/plain,text/css,text/javascript,application/javascript"
compression="on"
compressionMinSize="1024"
connectionTimeout="20000"
noCompressionUserAgents="gozilla, traviata"
port="443"
protocol="org.apache.coyote.http11.Http11AprProtocol"
SSLEnabled="true"
scheme="https"
secure="true">
<UpgradeProtocol className="org.apache.coyote.http2.Http2Protocol" />
<SSLHostConfig>
<Certificate certificateKeyFile="/opt/tomcat/cert/privkey.pem"
certificateFile="/opt/tomcat/cert/cert.pem"
certificateChainFile="/opt/tomcat/cert/chain.pem"
type="RSA" />
</SSLHostConfig>
</Connector>
到此为止一切都清楚了。现在我在 Tomcat 前面启动一个 Apache 服务器，我首先更改的是 Tomcat 连接器，如下所示：
<Connector URIEncoding="UTF-8"
compressableMimeType="text/html,text/xml,text/plain,text/css,text/javascript,application/javascript"
compression="on"
compressionMinSize="1024"
connectionTimeout="20000"
noCompressionUserAgents="gozilla, traviata"
port="8080"
protocol="org.apache.coyote.http11.Http11AprProtocol">
<UpgradeProtocol className="org.apache.coyote.http2.Http2Protocol" />
</Connector>
在 Apache 中我成功加载了下一个模块（我检查过它们是否真的加载了）：
LoadModule proxy_module modules/mod_proxy.so
LoadModule proxy_http_module modules/mod_proxy_http.so
LoadModule proxy_wstunnel_module modules/mod_proxy_wstunnel.so
LoadModule ssl_module modules/mod_ssl.so
这是我在 vhosts.conf 文件中尝试的配置之一：
<VirtualHost *:443>
ServerName www.example.com
ServerAdmin server@example.com
LogFormat "%h %l %u %t \"%r\" %>s %b \"%{Referer}i\" \"%{User-agent}i\"" combined
CustomLog /var/log/httpd/lavnet_access.log combined
ErrorLog /var/log/httpd/lavnet_error.log
SSLProxyEngine on
#websocket
RewriteEngine on
RewriteCond %{HTTP:Upgrade} "(?i)websocket"
RewriteRule ^/(.*)$ wss://www.example.com/ [P]
#rest
ProxyPass "/" "http://www.example.com:8080/"
ProxyPassReverse "/" "http://www.example.com:8080/"
SSLEngine on
SSLCertificateFile /etc/letsencrypt/live/example.com/cert.pem
SSLCertificateKeyFile /etc/letsencrypt/live/example.com/privkey.pem
SSLCertificateChainFile /etc/letsencrypt/live/example.com/chain.pem
</VirtualHost>
这是我取得的最佳效果的配置，但仍然无法正常工作。当我尝试建立此连接时，lavnet_error.log 中的日志跟踪似乎相当不错：
[Tue Oct 10 16:46:39.014980 2017] [proxy:debug] [pid 10558:tid 47319680603904] proxy_util.c(2209): [client XX.XX.XX.109:11208] AH00944: connecting wss://www.example.com:443/rest/notify/675/fgcw02lm/websocket to www.example.com:443
[Tue Oct 10 16:46:39.016495 2017] [proxy:debug] [pid 10558:tid 47319680603904] proxy_util.c(2418): [client XX.XX.XX.109:11208] AH00947: connected /rest/notify/675/fgcw02lm/websocket to www.example.com:443
[Tue Oct 10 16:46:39.016567 2017] [proxy:debug] [pid 10558:tid 47319680603904] proxy_util.c(2887): AH02824: WSS: connection established with XX.XX.XX.109:443 (*)
[Tue Oct 10 16:46:39.016590 2017] [proxy:debug] [pid 10558:tid 47319680603904] proxy_util.c(3054): AH00962: WSS: connection complete to XX.XX.XX.109:443 (www.example.com)
[Tue Oct 10 16:46:39.016603 2017] [ssl:info] [pid 10558:tid 47319680603904] [remote 217.61.129.109:443] AH01964: Connection to child 0 established (server www.example.com:443)
[Tue Oct 10 16:46:39.026370 2017] [proxy:debug] [pid 10558:tid 47319680603904] proxy_util.c(2171): AH00943: WSS: has released connection for (*)
但这是 Chrome 显示的错误：
我也尝试过其他配置：
<VirtualHost *:443>
ServerName www.example.com
ServerAdmin server@example.com
LogFormat "%h %l %u %t \"%r\" %>s %b \"%{Referer}i\" \"%{User-agent}i\"" combined
CustomLog /var/log/httpd/lavnet_access.log combined
ErrorLog /var/log/httpd/lavnet_error.log
SSLProxyEngine on
ProxyPass "/rest/notify/" "wss://www.example.com:8080/rest/notify"
ProxyPassReverse "/rest/notify/" "wss://www.example.com:8080/rest/notify"
ProxyPass "/" "http://www.example.com:8080/"
ProxyPassReverse "/" "http://www.example.com:8080/"
SSLEngine on
SSLCertificateFile /etc/letsencrypt/live/example.com/cert.pem
SSLCertificateKeyFile /etc/letsencrypt/live/example.com/privkey.pem
SSLCertificateChainFile /etc/letsencrypt/live/example.com/chain.pem
</VirtualHost>
但在这种情况下，我收到“500（内部服务器错误）”并且我还可以在 lavnet_error.log:
中看到下一个跟踪[Tue Oct 10 17:14:14.778824 2017] [proxy:warn] [pid 11924:tid 47694559057664] [client XX.XX.XXX.189:11665] AH01144: No protocol handler was valid for the URL /rest/notify/info (scheme 'wss'). If you are using a DSO version of mod_proxy, make sure the proxy submodules are included in the configuration using LoadModule., referer: https://www.example.com/equipment-command-panel/8287/8482
我已经尝试了很多配置，但我无法让它工作。我希望你能帮助我。
谢谢。
经过多次尝试终于解决了。接下来是工作配置，以备不时之需：
这是 Apache 的 vhost.conf 文件：
<VirtualHost *:443>
ServerName www.example.com
ServerAdmin admin@example.com
LogFormat "%h %l %u %t \"%r\" %>s %b \"%{Referer}i\" \"%{User-agent}i\"" combined
CustomLog /var/log/httpd/lavnet_access.log combined
ErrorLog /var/log/httpd/lavnet_error.log
ProxyPreserveHost On
ProxyPass / http://www.example.com:8080/
ProxyPassReverse / http://www.example.com:8080/
ProxyRequests Off
RewriteEngine on
RewriteCond %{HTTP:UPGRADE} ^WebSocket$ [NC]
RewriteCond %{HTTP:CONNECTION} Upgrade$ [NC]
RewriteRule .* ws://www.example.com:8080%{REQUEST_URI} [P]
SSLEngine on
SSLCertificateFile /etc/letsencrypt/live/example.com/cert.pem
SSLCertificateKeyFile /etc/letsencrypt/live/example.com/privkey.pem
SSLCertificateChainFile /etc/letsencrypt/live/example.com/chain.pem
</VirtualHost>
而这个是 Tomcat 中 server.xml 中定义的连接器：
<Connector URIEncoding="UTF-8"
connectionTimeout="20000"
port="8080"
protocol="org.apache.coyote.http11.Http11AprProtocol">
<UpgradeProtocol className="org.apache.coyote.http2.Http2Protocol" />
</Connector>
谢谢。
接受的解决方案适用于单个实例。但是我有多个服务器的集群，我有带 mod_jk 模块的负载平衡器。在这种情况下，经过数小时的搜索，我可以找到解决问题的方法。在 mod_jk 中，服务器名称附加到 JSESSIONID 的末尾，如下所示：JSESSIONID=25B9813E079BA3543C242438FD74CA84.banana.
所以我可以编写一个规则，使用以下代码将 websocket 请求代理到正确的服务器：
SSLProxyEngine On
RewriteEngine on
RewriteCond %{HTTP:UPGRADE} ^WebSocket$ [NC]
RewriteCond %{HTTP:CONNECTION} Upgrade$ [NC]
RewriteCond %{HTTP_COOKIE} (^|;\ *)JSESSIONID=.*\.(.+?)(?=;|$) [NC]
RewriteRule .* wss://%2:8443%{REQUEST_URI} [P]
正如您在上一个 RewriteCond 中看到的那样，我编写了一个正则表达式，它在第二个正则表达式组中查找服务器的名称。所以我写了一个代理，将请求路由到 wss://%2:8443%{REQUEST_URI} 这是正确的服务器。
p.s: 不要忘记将服务器名称和 IP 添加到 /etc/hosts