收集日志 ZIP 并上传到没有临时文件的 Http 服务

Question

我需要创建一个实用程序来收集一组日志，将它们压缩并附加到 jira。

我打算使用 httpclient 4.5。（有 NIO 支持）jira API 在这里不是问题。

我想通过 zip 流直接从文件系统流式传输文件并上传到 jira，而无需在文件系统上创建临时 zip 文件。

我发现 java API 在这种情况下令人困惑，无法弄清楚用法。细心看来，这种情况是 java NIO 使用的一个很好的候选者（也许是一个频道？）但未能找到一个好的例子。

作为起点，我创建了一个示例，该示例从中上传了一个文件，但我未能将其带到 Zip 流的下一步。

HttpClient httpClient = createHttpClient();


final String issueUriAttach = issueUri + "/" + "attachments";

HttpPost postRequest = new HttpPost(issueUriAttach);
final FileInputStream fileInputStream = new FileInputStream("README.md");
InputStream in = new InputStream() {
    @Override
    public int read() throws IOException {
        return 0;
    }

    @Override
    public int read(byte[] b, int off, int len) throws IOException {
        return fileInputStream.read(b,off,len);
    }
};
InputStreamBody inputStreamBody = new InputStreamBody(in,"README.csv");
HttpEntity reqEntity = MultipartEntityBuilder.create()
        .setContentType(ContentType.MULTIPART_FORM_DATA)
        .addPart("file", inputStreamBody)
        .build();
postRequest.setHeader(getAuthHeader());
postRequest.setHeader(getNocheckHeader());

postRequest.setEntity(reqEntity);
final HttpResponse response = httpClient.execute(postRequest);

正如我所说，我不限于 java I/O java NIO 也是一个选项..

Answer 1

您必须在单独的线程中将数据写入 ZipOutputStream 并将其通过管道传输到 InputStream 以供 HttpClient 从中读取。

/**
 * Zip a collection of files and pipe the output to an InputStream 
 */
public static InputStream zipSource(Collection<Path> paths) throws IOException {
    // Pipe pair. We zip to the output stream in a separate thread. The data is
    // then available to be read from the input stream.
    PipedOutputStream out = new PipedOutputStream();
    PipedInputStream in = new PipedInputStream(out);

    // Zip the file to the input stream in another thread
    zipExecutor.execute(() -> {
        try {
            zip(paths, out);
        } catch (IOException e) {
            System.err.println("Zip failed " + e);
        }
    });

    return in;
}

/**
 * Zip a collection of files to an output stream
 */
public static void zip(Collection<Path> paths, OutputStream out) throws IOException {
    try (ZipOutputStream zout = new ZipOutputStream(out)) {
        for (Path path : paths) {
            zout.putNextEntry(new ZipEntry(path.getFileName().toString()));
            Files.copy(path, zout);
            zout.closeEntry();
        }
    }
}

private static final Executor zipExecutor = Executors.newCachedThreadPool(r -> {
    Thread t = new Thread(r, "zip task");
    t.setDaemon(true);
    return t;
});

收集日志 ZIP 并上传到没有临时文件的 Http 服务

Collect logs ZIP and upload to Http service without temp file

java

nio

jira

apache-httpclient-4.x