使用 InputStreamResponseListener 复制带有 Jetty HTTP 客户端的 InputStream 块

Question

我正在使用 Jetty 9.4.8 HTTP 客户端并希望将传入数据流写入文件。目前我正在使用 InputStreamResponseListener 和 IOUtils.copy(..) 写入 FileOutputStream。我也试过 Files.copy().

InputStreamResponseListener streamResponseListener = new InputStreamResponseListener();

request.send(streamResponseListener);

if(streamResponseListener.get(5, TimeUnit.MINUTES).getStatus() == 200) {
    OutputStream outputStream = null;
    try {
        TMP_FILE.toFile().createNewFile();
        outputStream = new FileOutputStream(TMP_FILE.toFile());
        IOUtils.copy(inputStream, outputStream);
    } catch(IOException e) {
        this.getLogService().log(..)
    } finally {
        IOUtils.closeQuietly(inputStream);
        IOUtils.closeQuietly(outputStream);
    }

    // NOT REACHED IN CASE InputStream is BLOCKED FOR SOME REASON
}

但是，在接收到所有字节后，复制方法似乎会阻塞。为什么会发生这种情况，我该如何避免这种情况？

Headers 请求的 HTTP 内容：

Date: Wed, 23 May 2018 16:46:06 GMT
Content-Type: application/octet-stream
Content-Disposition: attachment; filename=".."
Content-Length: 613970044
Server: Jetty(9.4.8.v20171121)

来自 Apache Commons IO 版本 2.4 的 IOUtils

Answer 1

这是您的代码库的一个工作示例，仅使用 Java 和 Jetty。

这是从已知符合 HTTP 规范的服务器请求内容。

package demo.jettyclient;

import java.io.IOException;
import java.io.InputStream;
import java.io.OutputStream;
import java.net.URI;
import java.net.URL;
import java.nio.file.Files;
import java.nio.file.Path;
import java.util.Properties;
import java.util.concurrent.TimeUnit;

import org.eclipse.jetty.client.HttpClient;
import org.eclipse.jetty.client.api.Request;
import org.eclipse.jetty.client.api.Response;
import org.eclipse.jetty.client.util.InputStreamResponseListener;
import org.eclipse.jetty.http.HttpStatus;
import org.eclipse.jetty.util.IO;
import org.eclipse.jetty.util.StringUtil;
import org.eclipse.jetty.util.ssl.SslContextFactory;

public class DownloadUrl
{
    public static void main(String[] args) throws Exception
    {
        String uriString = "https://upload.wikimedia.org/wikipedia/en/a/a1/Jetty_logo.png?download";

        if (args.length >= 1)
            uriString = args[0];

        URI srcUri = URI.create(uriString);

        SslContextFactory ssl = new SslContextFactory(true);

        HttpClient client = new HttpClient(ssl);
        try
        {
            client.start();

            Request request = client.newRequest(srcUri);

            System.out.printf("Using HttpClient v%s%n", getHttpClientVersion());
            System.out.printf("Requesting: %s%n", srcUri);
            InputStreamResponseListener streamResponseListener = new InputStreamResponseListener();
            request.send(streamResponseListener);
            Response response = streamResponseListener.get(5, TimeUnit.SECONDS);

            if (response.getStatus() != HttpStatus.OK_200)
            {
                throw new IOException(
                        String.format("Failed to GET URI [%d %s]: %s",
                                response.getStatus(),
                                response.getReason(),
                                srcUri));
            }

            Path tmpFile = Files.createTempFile("tmp", ".dl");

            try (InputStream inputStream = streamResponseListener.getInputStream();
                 OutputStream outputStream = Files.newOutputStream(tmpFile))
            {
                IO.copy(inputStream, outputStream);
            }

            System.out.printf("Downloaded %s%n", srcUri);
            System.out.printf("Destination: %s (%,d bytes)%n", tmpFile.toString(), Files.size(tmpFile));
        }
        finally
        {
            client.stop();
        }
    }

    private static String getHttpClientVersion()
    {
        ClassLoader cl = HttpClient.class.getClassLoader();

        // Attempt to use maven pom properties first
        String pomResource = "/META-INF/maven/org/eclipse/jetty/jetty-client/pom.properties";
        URL url = cl.getResource(pomResource);
        if (url != null)
        {
            try (InputStream in = url.openStream())
            {
                Properties props = new Properties();
                props.load(in);
                String version = props.getProperty("version");
                if (StringUtil.isNotBlank(version))
                    return version;
            }
            catch (IOException ignore)
            {
            }
        }

        // Attempt to use META-INF/MANIFEST.MF
        String version = HttpClient.class.getPackage().getImplementationVersion();
        if (StringUtil.isNotBlank(version))
            return version;

        return "<unknown>";
    }
}

当运行时，这会导致 ...

2018-05-23 10:52:08.401:INFO::main: Logging initialized @325ms to org.eclipse.jetty.util.log.StdErrLog
Using HttpClient v9.4.9.v20180320
Requesting: https://upload.wikimedia.org/wikipedia/en/a/a1/Jetty_logo.png?download
Downloaded https://upload.wikimedia.org/wikipedia/en/a/a1/Jetty_logo.png?download
Destination: C:\Users\joakim\AppData\Local\Temp\tmp2166600286896937563.dl (11,604 bytes)

Process finished with exit code 0

以下一项（或多项）可能导致您的问题。

您的服务器有问题，不符合 HTTP 规范。
HTTP 交换尚未完成（从协议的角度来看）。捕获流量并验证行为。
您正在使用的 IOUtil 库（您没有说是哪个）有一个错误。

wget（或 curl）起作用的事实可能是因为它们对 Content-Length 不严格（根据 RFC7230 中的建议）并且会显示/下载在物理连接 EOF/disconnect 之前收到的所有内容。虽然 HTTP/1.1 协议具有连接持久性和关于请求（和响应）内容何时结束的严格规则。

使用 InputStreamResponseListener 复制带有 Jetty HTTP 客户端的 InputStream 块

Copy of InputStream blocks with Jetty HTTP client using an InputStreamResponseListener

java

jetty