缓冲区大小如何影响 NIO 通道性能？

Question

我正在阅读 Hadoop IPC 实现。 https://github.com/apache/hadoop/blob/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/ipc/Server.java

/**
 * When the read or write buffer size is larger than this limit, i/o will be 
 * done in chunks of this size. Most RPC requests and responses would be
 * be smaller.
 */
private static int NIO_BUFFER_LIMIT = 8*1024; //should not be more than 64KB.

/**
 * This is a wrapper around {@link WritableByteChannel#write(ByteBuffer)}.
 * If the amount of data is large, it writes to channel in smaller chunks. 
 * This is to avoid jdk from creating many direct buffers as the size of 
 * buffer increases. This also minimizes extra copies in NIO layer
 * as a result of multiple write operations required to write a large 
 * buffer.  
 *
 * @see WritableByteChannel#write(ByteBuffer)
 */
private int channelWrite(WritableByteChannel channel, 
                         ByteBuffer buffer) throws IOException {

  int count =  (buffer.remaining() <= NIO_BUFFER_LIMIT) ?
               channel.write(buffer) : channelIO(null, channel, buffer);
  if (count > 0) {
    rpcMetrics.incrSentBytes(count);
  }
  return count;
}


/**
 * This is a wrapper around {@link ReadableByteChannel#read(ByteBuffer)}.
 * If the amount of data is large, it writes to channel in smaller chunks. 
 * This is to avoid jdk from creating many direct buffers as the size of 
 * ByteBuffer increases. There should not be any performance degredation.
 * 
 * @see ReadableByteChannel#read(ByteBuffer)
 */
private int channelRead(ReadableByteChannel channel, 
                        ByteBuffer buffer) throws IOException {

  int count = (buffer.remaining() <= NIO_BUFFER_LIMIT) ?
              channel.read(buffer) : channelIO(channel, null, buffer);
  if (count > 0) {
    rpcMetrics.incrReceivedBytes(count);
  }
  return count;
}

逻辑是，如果缓冲区很小，它将 read/write 通道一次。如果缓冲区很大，它会做很多次，每次 read/write 8kb.

我不明白 javadoc 以及它为什么这样做。为什么 "This is to avoid jdk from creating many direct buffers as the size of buffer increases."？缓冲区大小是否也会影响读取性能？

我了解缓冲区大小如何影响 FileInputStream 性能 (link)。但是这里是 SocketChannel。所以这是无关的。

Answer 1

好问题。 sun.nio.ch.IOUtil 是在通道写入时使用的，它的 write(..) 函数中有以下几行

int var7 = var5 <= var6?var6 - var5:0;
ByteBuffer var8 = Util.getTemporaryDirectBuffer(var7);

这里是Util.getTemporaryDirectBuffer

static ByteBuffer getTemporaryDirectBuffer(int var0) {
    Util.BufferCache var1 = (Util.BufferCache)bufferCache.get();
    ByteBuffer var2 = var1.get(var0);
    if(var2 != null) {
        return var2;
    } else {
        if(!var1.isEmpty()) {
            var2 = var1.removeFirst();
            free(var2);
        }

        return ByteBuffer.allocateDirect(var0);
    }
}

在重负载下，当 int var0 处于较大范围时，它会创建大量新缓冲区和 free(..) 旧缓冲区。因为 bufferCache 的长度有限（等于系统配置中定义的 IOUtil.IOV_MAX。On modern Linux systems, the limit is 1024）并且不会存储每个长度的缓冲区。
我认为这是 This is to avoid jdk from creating many direct buffers as the size of buffer increases. 中的意思。

缓冲区大小如何影响 NIO 通道性能？

How does buffer size affect NIO Channel performance?

java

performance

buffer

nio

socketchannel