使用 Java NIO 读写大文件

Question

如何使用 Java NIO 框架有效地读取大文件并将批量数据写入文件。

我正在与 ByteBuffer 和 FileChannel 合作，并尝试过类似下面的方法：

public static void main(String[] args) 
{
    String inFileStr = "screen.png";
    String outFileStr = "screen-out.png";
    long startTime, elapsedTime; 
    int bufferSizeKB = 4;
    int bufferSize = bufferSizeKB * 1024;

    // Check file length
    File fileIn = new File(inFileStr);
    System.out.println("File size is " + fileIn.length() + " bytes");
    System.out.println("Buffer size is " + bufferSizeKB + " KB");
    System.out.println("Using FileChannel with an indirect ByteBuffer of " + bufferSizeKB + " KB");

    try (   FileChannel in = new FileInputStream(inFileStr).getChannel();
            FileChannel out = new FileOutputStream(outFileStr).getChannel() ) 
    {
        // Allocate an indirect ByteBuffer
        ByteBuffer bytebuf = ByteBuffer.allocate(bufferSize);

        startTime = System.nanoTime();

        int bytesCount = 0;
        // Read data from file into ByteBuffer
        while ((bytesCount = in.read(bytebuf)) > 0) { 
            // flip the buffer which set the limit to current position, and position to 0.
            bytebuf.flip();
            out.write(bytebuf); // Write data from ByteBuffer to file
            bytebuf.clear(); // For the next read
        }

        elapsedTime = System.nanoTime() - startTime;
        System.out.println("Elapsed Time is " + (elapsedTime / 1000000.0) + " msec");
    } 
    catch (IOException ex) {
        ex.printStackTrace();
    }
}

谁能告诉我，如果我的文件大小超过 2 GB，我是否应该遵循相同的步骤？

写作业批量写的时候想做类似的事情应该怎么办？

Answer 1

while ((bytesCount = in.read(bytebuf)) > 0) { 
        // flip the buffer which set the limit to current position, and position to 0.
        bytebuf.flip();
        out.write(bytebuf); // Write data from ByteBuffer to file
        bytebuf.clear(); // For the next read
    }

您的复制循环不正确。应该是：

while ((bytesCount = in.read(bytebuf)) > 0 || bytebuf.position() > 0) { 
        // flip the buffer which set the limit to current position, and position to 0.
        bytebuf.flip();
        out.write(bytebuf); // Write data from ByteBuffer to file
        bytebuf.compact(); // For the next read
    }

Can anybody tell, should I follow the same procedure if my file size [is] more than 2 GB?

是的。文件大小没有任何区别。

Answer 2

请注意，您可以简单地使用 Files.copy(Paths.get(inFileStr),Paths.get(outFileStr), StandardCopyOption.REPLACE_EXISTING) 来复制文件，就像您的示例代码那样，可能速度更快，而且只需要一行代码。

否则，如果你已经打开了两个文件通道，你可以直接使用
in.transferTo(0, in.size(), out) to transfer the entire contents of the in channel to the out channel. Note that this method allows to specify a range within the source file that will be transferred to the target channel’s current position (which is initially zero) and that there’s also a method for the opposite way, i.e. out.transferFrom(in, 0, in.size()) 将数据从源通道的当前位置传输到目标文件中的绝对范围。

一起，它们允许以高效的方式进行几乎所有可以想象的重要批量传输，而无需将数据复制到 Java 侧缓冲区。如果这不能解决您的需求，则您的问题必须更具体。

顺便说一句，自 Java 7.

以来，您可以 open a FileChannel directly 而无需 FileInputStream/FileOutputStream 弯路

使用 Java NIO 读写大文件

Reading and writting a large file using Java NIO

java

nio

file-processing