两种方法之间 IO 速度的意外巨大差异

Question

我必须从 google 云存储中检索一个文件，将其解压缩，然后使用 Wire 将其解析为 proto-buf 对象。

首先，我尝试执行以下操作：

final GcsInputChannel inputChannel;
final ByteArrayInputStream byteArrayInputStream;
final GZIPInputStream compressedData;

final MusicList musicList;

logger.info("Receiving the object");
inputChannel = gcsService.openReadChannel(gcsFilename, 0);

final int fileSize = (int) gcsService.getMetadata(gcsFilename).getLength();
final ByteBuffer result = ByteBuffer.allocate(fileSize);
inputChannel.read(result);

byteArrayInputStream = new ByteArrayInputStream(result.array());
compressedData = new GZIPInputStream(byteArrayInputStream);
musicList = new Wire(MusicList.class).parseFrom(compressedData, MusicList.class);
closeQuietly(byteArrayInputStream, inputChannel, compressedData);

这基本上是首先将整个文件提取到一个字节缓冲区中，然后将其传递给 UN-compressor。我想为什么不分摊将实际输入流传递给 UN 压缩器所花费的时间。

final GcsInputChannel inputChannel = gcsService.openPrefetchingReadChannel(gcsFilename, 0, 1024 * 1024);
final InputStream download = Channels.newInputStream(inputChannel);
final GZIPInputStream compressedData;

final MusicList musicList;

logger.info("Receiving the object");
compressedData = new GZIPInputStream(download);
musicList = new Wire(MusicList.class).parseFrom(compressedData, MusicList.class);
closeQuietly(download, inputChannel, compressedData);

令人惊讶的是，第一种方法的运行速度几乎是第二种方法的 6 倍。非常感谢澄清这个问题。

-----EDIT-----

文件大小不会超过 1 兆字节。

Answer 1

缓冲是这里的关键。用缓冲输入流装饰由 Channels.newInputStream 返回的默认输入流是迄今为止完成这项工作最快的方法。

这是最终代码

final GcsInputChannel inputChannel;
final BufferedInputStream bufferedInputStream;
final GZIPInputStream compressedData;
final InputStream inputStream;

final MusicList musicList;

logger.info("Receiving the object");
try {

    inputChannel = gcsService.openReadChannel(gcsFilename, 0);
    bufferedInputStream = new BufferedInputStream(inputStream = Channels.newInputStream(inputChannel));
    compressedData = new GZIPInputStream(bufferedInputStream);
    musicList = new Wire(MusicList.class).parseFrom(compressedData, MusicList.class);
} catch (IOException e) {
    e.printStackTrace();
    return Collections.emptyList(); //fail
}

closeQuietly(bufferedInputStream, inputChannel, compressedData, inputStream);

据我了解，默认输入流缺乏有效的跳过方法，而且没有缓冲，导致解压缩速度极慢。

两种方法之间 IO 速度的意外巨大差异

Unexpected huge difference in IO speed between 2 methods

java

io

google-app-engine

google-cloud-storage