如何通过 TLS 使用 HTTP/2 发送 HTTP 请求？ (Java 11 个 HttpClient)
How can I send an HTTP request using HTTP/2 over TLS? (Java 11 HttpClient)
我正在尝试在 Java 应用程序中创建一个代理，允许我修改 HTTP 请求的某些方面。
为此，我在端口 8080 上打开一个 ServerSocket，在 Mozilla Firefox 中在该端口上配置一个代理，并且对于每个连接，在单独的线程中执行 ServerSocket 的 accept() 方法。目前一切正常。
为了从浏览器向相应的网站发送请求，我使用了 Java 11 中包含的 HttpClient 库。这是我使用该库的代码段：
private void obtainResponse(Socket socket, IHttpRequest req, String uri) {
HttpClient client = null;
if (req.isSSL()) {
SSLContext sslContext = ((SecureConnectionHandler)connHandler).createSSLContext( req.getHost() );
client = HttpClient.newBuilder()
.connectTimeout(Duration.ofSeconds(30))
.priority(1)
.version(HttpClient.Version.HTTP_2)
.followRedirects(Redirect.NORMAL)
.sslContext( sslContext )
.build();
}
else
client = HttpClient.newBuilder()
.connectTimeout(Duration.ofSeconds(30))
.priority(1)
.version(HttpClient.Version.HTTP_2)
.followRedirects(Redirect.NORMAL)
.build();
String protocolAndHost = ((req.isSSL()) ? "https://" : "http://") + req.getHost();
if (uri == null)
uri = protocolAndHost + req.getRequestedResource();
else {
if (uri.startsWith("/"))
uri = protocolAndHost + uri;
System.out.println("Aqui:" + uri);
}
HttpRequest.Builder preRequest=null;
if (req.getMethod().equalsIgnoreCase("GET")) {
preRequest = HttpRequest.newBuilder() // GET request!
.uri(URI.create( uri ))
.GET();
}
else if (req.getMethod().equalsIgnoreCase("POST")) {
preRequest = HttpRequest.newBuilder() // POST request!
.uri(URI.create( uri ))
.POST(BodyPublishers.ofString(req.getBody()));
}
for (Header header : req.getHeaders()) {
if (!header.getKey().equalsIgnoreCase("Host") &&
!header.getKey().equalsIgnoreCase("Connection") &&
!header.getKey().equalsIgnoreCase("Content-Length") &&
!header.getKey().equalsIgnoreCase("Upgrade") )
{
preRequest.setHeader(header.getKey(), header.getValues());
}
}
HttpRequest request = preRequest.build();
System.err.println("Request to: " + uri);
HttpResponse<byte[]> response;
try {
response = client.sendAsync(request, BodyHandlers.ofByteArray())
.join();
} catch (CompletionException ce) {
System.err.println("Address " + uri + " is unreachable!");
return ;
}
HttpHeaders httpHeaders = response.headers();
Optional<String> locationHeader = httpHeaders.firstValue("Location"); // When resource has been permanently moved
if ( !locationHeader.isEmpty() ) {
System.out.println("Moved permanently to " + locationHeader.get());
obtainResponse( socket, req, locationHeader.get() );
}
else {
Map<String, List<String>> headers = httpHeaders.map();
String protocol = response.version().toString().replace("_", ".").replaceFirst("\.", "/");
int code = response.statusCode();
String reasonPhrase = HttpStatus.getStatusText( code );
var crlf = "\r\n";
var responseString = protocol + " " + code + " " + reasonPhrase + crlf;
for (String key : headers.keySet()) {
responseString += key + ":";
for (String valor : headers.get(key)) {
responseString += " " + valor;
}
responseString += crlf;
}
responseString += crlf; // espacio cabeceras y cuerpo
writeResponse(socket, response.body(), responseString);
}
}
private void writeResponse(Socket socket, byte[] streamResponse, String responseHeaders) {
OutputStream outputStream = null;
try {
outputStream = socket.getOutputStream();
outputStream.write(responseHeaders.getBytes());
outputStream.write(streamResponse);
outputStream.flush();
} catch (IOException e) {
// TODO Auto-generated catch block
e.printStackTrace();
} finally {
try {
if (!socket.isOutputShutdown()) {
socket.shutdownOutput();
}
outputStream.close();
} catch (IOException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
}
}
注意：IHttpRequest 是我创建的一个class，它包含从套接字收集的所有信息（目标主机，headers，body 如果它存在等)
到目前为止，我已经成功拦截了所有指向 HTTP 网站的请求。但是，我遇到了问题，例如 https://www.google.com/，它实现了 HTTP/2 协议并使用了 TLS。当我 运行 Java 应用程序并访问上一个 Google 页面时，它没有向我显示网站，而是在浏览器中向我显示以下内容（忽略 headers 之间的空白） :
HTTP/2 200 OK
:status: 200
alt-svc: quic = ":443"; ma = 2592000; v = "46,43,39"
cache-control: private
content-encoding: gzip
content-length: 46058
content-type: text/html; charset = UTF-8
date: Tue, 03 Sep 2019 09:47:34 GMT
Expires: Tue, 03 Sep 2019 09:47:34 GMT
p3p: CP = "This is not a P3P policy! See g.co/p3phelp for more info."
server: gws
set-cookie: 1P_JAR = 2019-09-03-09; expires = Thu, 03-Oct-2019 09:47:34 GMT;
path = /; domain = .google.com; SameSite = none NID = 188 = XOJkffugf5G8rxNLov_iqqxo-Cq5RCvhwJPNu9tvtzLesZ4q8CE0IDVt9VgCEHZsw-AV0EYaaL8D4d_2Qwb6jXCcss7RydfV9PqQFemN_Ezz0kUjyseDDbJXfrHpmqPR6GIQCnR7bjukfasxg883K9fjnhAaqz6IpUYxoguZx-vazWc; expires = Wed, 04-Mar-2020 09:47:34 GMT; path = /; domain = .google.com; HttpOnly CONSENT = WP.27dd1a; expires = Fri, 01-Jan-2038 00:00:00 GMT; path = /; domain = .google.com
x-frame-options: SAMEORIGIN
x-xss-protection: 0
(↓↓BODY↓↓)
‹����� ÿÔ½ézâȲ (ú¿Ÿ‚ ¢ örÁ²À or PªÚ ° çyÜÞ¾ © $ Æ.ÞåžG¸ßýwþ® »™ š ¶« »× ùöíê ¢ ¤T '' '' ™ 'ß¿ († l / Æj ¢ or õ ßñ7¡ “QOLªV ÞU ¢ üø> Tm' ûÄ´T [L ^] îd * I7Õ Ùê Rçšb ÷ EE i²š¡ / ÜP iÃé0cÉDWE! ËsCò I ™ ZªI_ ‰) ## ™ '* & gš: ¦½ÒŽêê¸oŒ ‚™» † 9 $ vFQmU¶5c´Rˆ (Š © ZÖï 1L§ Üì ¦ÚUMS5W² Û $ # K'¶êæí FOWWrniCÒS- ò + Ú¨ · Åòõ¶ „÷ ñɲá 1 • 'ÙÐ
<< And much more information in form of bytes >>
你在我的代码中看到什么奇怪的地方了吗？我知道 HTTP / 2 在 Frames 中压缩 headers，但我假设 HTTPClient 是在内部完成的...
如果您需要更多信息，请告诉我:)
提前致谢。
HTTP/2协议是一个二进制协议。您发送回浏览器的是 HTTP/1.1 响应（即使 HttpClient 用于获取响应的协议是 HTTP/2）。
但是 - 您的状态行是 HTTP/2 200 OK，您的浏览器将无法理解。您需要以格式正确的 HTTP/1.1 回复形式发送您的回复。
这包括过滤掉 headers 其键以 ':' 开头的 - 就像 ':status' 因为这些是 HTTP/2 特定响应 headers。在不理解它们的情况下转发回所有响应 headers 可能根本行不通：编写一个完整的 HTTP 代理很难。
我正在尝试在 Java 应用程序中创建一个代理，允许我修改 HTTP 请求的某些方面。
为此，我在端口 8080 上打开一个 ServerSocket，在 Mozilla Firefox 中在该端口上配置一个代理，并且对于每个连接，在单独的线程中执行 ServerSocket 的 accept() 方法。目前一切正常。
为了从浏览器向相应的网站发送请求，我使用了 Java 11 中包含的 HttpClient 库。这是我使用该库的代码段：
private void obtainResponse(Socket socket, IHttpRequest req, String uri) {
HttpClient client = null;
if (req.isSSL()) {
SSLContext sslContext = ((SecureConnectionHandler)connHandler).createSSLContext( req.getHost() );
client = HttpClient.newBuilder()
.connectTimeout(Duration.ofSeconds(30))
.priority(1)
.version(HttpClient.Version.HTTP_2)
.followRedirects(Redirect.NORMAL)
.sslContext( sslContext )
.build();
}
else
client = HttpClient.newBuilder()
.connectTimeout(Duration.ofSeconds(30))
.priority(1)
.version(HttpClient.Version.HTTP_2)
.followRedirects(Redirect.NORMAL)
.build();
String protocolAndHost = ((req.isSSL()) ? "https://" : "http://") + req.getHost();
if (uri == null)
uri = protocolAndHost + req.getRequestedResource();
else {
if (uri.startsWith("/"))
uri = protocolAndHost + uri;
System.out.println("Aqui:" + uri);
}
HttpRequest.Builder preRequest=null;
if (req.getMethod().equalsIgnoreCase("GET")) {
preRequest = HttpRequest.newBuilder() // GET request!
.uri(URI.create( uri ))
.GET();
}
else if (req.getMethod().equalsIgnoreCase("POST")) {
preRequest = HttpRequest.newBuilder() // POST request!
.uri(URI.create( uri ))
.POST(BodyPublishers.ofString(req.getBody()));
}
for (Header header : req.getHeaders()) {
if (!header.getKey().equalsIgnoreCase("Host") &&
!header.getKey().equalsIgnoreCase("Connection") &&
!header.getKey().equalsIgnoreCase("Content-Length") &&
!header.getKey().equalsIgnoreCase("Upgrade") )
{
preRequest.setHeader(header.getKey(), header.getValues());
}
}
HttpRequest request = preRequest.build();
System.err.println("Request to: " + uri);
HttpResponse<byte[]> response;
try {
response = client.sendAsync(request, BodyHandlers.ofByteArray())
.join();
} catch (CompletionException ce) {
System.err.println("Address " + uri + " is unreachable!");
return ;
}
HttpHeaders httpHeaders = response.headers();
Optional<String> locationHeader = httpHeaders.firstValue("Location"); // When resource has been permanently moved
if ( !locationHeader.isEmpty() ) {
System.out.println("Moved permanently to " + locationHeader.get());
obtainResponse( socket, req, locationHeader.get() );
}
else {
Map<String, List<String>> headers = httpHeaders.map();
String protocol = response.version().toString().replace("_", ".").replaceFirst("\.", "/");
int code = response.statusCode();
String reasonPhrase = HttpStatus.getStatusText( code );
var crlf = "\r\n";
var responseString = protocol + " " + code + " " + reasonPhrase + crlf;
for (String key : headers.keySet()) {
responseString += key + ":";
for (String valor : headers.get(key)) {
responseString += " " + valor;
}
responseString += crlf;
}
responseString += crlf; // espacio cabeceras y cuerpo
writeResponse(socket, response.body(), responseString);
}
}
private void writeResponse(Socket socket, byte[] streamResponse, String responseHeaders) {
OutputStream outputStream = null;
try {
outputStream = socket.getOutputStream();
outputStream.write(responseHeaders.getBytes());
outputStream.write(streamResponse);
outputStream.flush();
} catch (IOException e) {
// TODO Auto-generated catch block
e.printStackTrace();
} finally {
try {
if (!socket.isOutputShutdown()) {
socket.shutdownOutput();
}
outputStream.close();
} catch (IOException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
}
}
注意：IHttpRequest 是我创建的一个class，它包含从套接字收集的所有信息（目标主机，headers，body 如果它存在等)
到目前为止，我已经成功拦截了所有指向 HTTP 网站的请求。但是，我遇到了问题，例如 https://www.google.com/，它实现了 HTTP/2 协议并使用了 TLS。当我 运行 Java 应用程序并访问上一个 Google 页面时，它没有向我显示网站，而是在浏览器中向我显示以下内容（忽略 headers 之间的空白） :
HTTP/2 200 OK
:status: 200
alt-svc: quic = ":443"; ma = 2592000; v = "46,43,39"
cache-control: private
content-encoding: gzip
content-length: 46058
content-type: text/html; charset = UTF-8
date: Tue, 03 Sep 2019 09:47:34 GMT
Expires: Tue, 03 Sep 2019 09:47:34 GMT
p3p: CP = "This is not a P3P policy! See g.co/p3phelp for more info."
server: gws
set-cookie: 1P_JAR = 2019-09-03-09; expires = Thu, 03-Oct-2019 09:47:34 GMT; path = /; domain = .google.com; SameSite = none NID = 188 = XOJkffugf5G8rxNLov_iqqxo-Cq5RCvhwJPNu9tvtzLesZ4q8CE0IDVt9VgCEHZsw-AV0EYaaL8D4d_2Qwb6jXCcss7RydfV9PqQFemN_Ezz0kUjyseDDbJXfrHpmqPR6GIQCnR7bjukfasxg883K9fjnhAaqz6IpUYxoguZx-vazWc; expires = Wed, 04-Mar-2020 09:47:34 GMT; path = /; domain = .google.com; HttpOnly CONSENT = WP.27dd1a; expires = Fri, 01-Jan-2038 00:00:00 GMT; path = /; domain = .google.com
x-frame-options: SAMEORIGIN
x-xss-protection: 0
(↓↓BODY↓↓)
‹����� ÿÔ½ézâȲ (ú¿Ÿ‚ ¢ örÁ²À or PªÚ ° çyÜÞ¾ © $ Æ.ÞåžG¸ßýwþ® »™ š ¶« »× ùöíê ¢ ¤T '' '' ™ 'ß¿ († l / Æj ¢ or õ ßñ7¡ “QOLªV ÞU ¢ üø> Tm' ûÄ´T [L ^] îd * I7Õ Ùê Rçšb ÷ EE i²š¡ / ÜP iÃé0cÉDWE! ËsCò I ™ ZªI_ ‰) ## ™ '* & gš: ¦½ÒŽêê¸oŒ ‚™» † 9 $ vFQmU¶5c´Rˆ (Š © ZÖï 1L§ Üì ¦ÚUMS5W² Û $ # K'¶êæí FOWWrniCÒS- ò + Ú¨ · Åòõ¶ „÷ ñɲá 1 • 'ÙÐ
<< And much more information in form of bytes >>
你在我的代码中看到什么奇怪的地方了吗？我知道 HTTP / 2 在 Frames 中压缩 headers，但我假设 HTTPClient 是在内部完成的...
如果您需要更多信息，请告诉我:)
提前致谢。
HTTP/2协议是一个二进制协议。您发送回浏览器的是 HTTP/1.1 响应（即使 HttpClient 用于获取响应的协议是 HTTP/2）。
但是 - 您的状态行是 HTTP/2 200 OK，您的浏览器将无法理解。您需要以格式正确的 HTTP/1.1 回复形式发送您的回复。
这包括过滤掉 headers 其键以 ':' 开头的 - 就像 ':status' 因为这些是 HTTP/2 特定响应 headers。在不理解它们的情况下转发回所有响应 headers 可能根本行不通：编写一个完整的 HTTP 代理很难。