Java 压缩 nio 路径列表时出现内存不足问题

Question

我正在异步运行下面的方法来压缩给定的 nio 路径集。有多个任务时运行，java遇到heap out of memory异常

public InputStream compressToZip(String s3folderName, Set<Path> paths) throws Exception {
    try {
        ByteArrayOutputStream byteArrayOutputStream = new ByteArrayOutputStream();
        ZipOutputStream zos = new ZipOutputStream(byteArrayOutputStream);
        paths.forEach(path -> {
            try {
                System.out.println("Zipping " + path.getFileName());
                zos.putNextEntry(new ZipEntry(path.getFileName().toString()));
                FileInputStream ObjectInputStream = new FileInputStream(path.toFile());
                IOUtils.copy(ObjectInputStream, zos);
                zos.closeEntry();
            } catch (Exception e) {
                ...
            }
        });
        zos.close();
        return new ByteArrayInputStream(byteArrayOutputStream.toByteArray());
    } catch (Exception e) {
        ...
    }
}

从此文件返回的输入流将写入 sftp 位置。

org.springframework.integration.file.remote.session.Session session = this.createSession(deliveryType, deliveryLocation);    
zippedIpStream = fileCompressionSvc.compressToZip(s3folderName, fileDir);
session.write(zippedIpStream, deliveryLocation.getLocation().getFolder() + "/"
                                + getfileNameFormat(fileNameFormat, masterId) + ".zip");

我不确定发生了什么问题 java 堆问题。你能帮帮我吗

Answer 1

更改了将文件写入本地路径中的文件的实现，然后将该文件发送到 sftp，然后删除临时 zip 文件。

public void compressToZip(String s3folderName, Set<Path> distinctPaths, String efsPathWithFileName) throws Exception {
    try(FileOutputStream fos = new FileOutputStream(efsPathWithFileName);
        ZipOutputStream zos = new ZipOutputStream(fos)) {
        distinctPaths.forEach(path -> {
            try {
                zos.putNextEntry(new ZipEntry(path.getFileName().toString()));
                final FileInputStream fis = new FileInputStream(path.toFile());
                IOUtils.copy(fis, zos);
                zos.closeEntry();
            } catch (IOException e) {
                ...
            }
        });
    } catch (Exception e) {
        ...
        throw e;
    }
}

调用方法：

InputStream zippedIpStream = new FileInputStream(tempCompressedFilePath);
session.write(zippedIpStream, deliveryLocation.getLocation().getFolder() + "/" + fileNameToWrite);
...
...                     
zippedIpStream.close();
...
...
Files.deleteIfExists(Paths.get(tempCompressedFilePath));

Java 压缩 nio 路径列表时出现内存不足问题

Java heap out of memory issue while zipping list of nio paths

java

nio

inputstream

bytearrayinputstream

gzipoutputstream