为什么 Java 文件写入消耗 CPU？

Question

我正在使用单独线程上的队列将数据写入文件，但该进程消耗了大约 25% 的 CPU，如本测试主图所示。

我能做些什么来解决这个问题吗？

也许我应该在某处使用 flush()？

测试显示主要方法启动和运行队列线程，然后将创建的数据发送给它。队列线程将数据写入 BufferedWriter，后者负责将数据写入文件。

import java.io.BufferedWriter;
import java.io.File;
import java.io.FileWriter;

import java.io.IOException;
import java.util.ArrayList;
import java.util.List;
import java.util.concurrent.ConcurrentLinkedQueue;
import java.util.logging.Level;
import java.util.logging.Logger;
import uk.co.moonsit.utils.timing.Time;

public class OutputFloatQueueReceiver extends Thread {
private static final Logger LOG = Logger.getLogger(OutputFloatQueueReceiver.class.getName());

private ConcurrentLinkedQueue<List<Float>> queue = null;
private boolean running = true;
private final BufferedWriter outputWriter;
private int ctr = 0;

private final int LIMIT = 1000;

public OutputFloatQueueReceiver(String outputFile, String header, ConcurrentLinkedQueue<List<Float>> q) throws IOException {

    queue = q;

    File f = new File(outputFile);
    FileWriter fstream = null;
    if (!f.exists()) {
        try {
            f.getParentFile().mkdirs();
            if (!f.createNewFile()) {
                throw new IOException("Exception when trying to create file " + f.getAbsolutePath());
            }

            fstream = new FileWriter(outputFile, false);
        } catch (IOException ex) {
            //Logger.getLogger(ControlHierarchy.class.getName()).log(Level.SEVERE, null, ex);
            throw new IOException("Exception when trying to create file " + f.getAbsolutePath());
        }
    }

    fstream = new FileWriter(outputFile, true);
    outputWriter = new BufferedWriter(fstream);
    outputWriter.append(header);

}

public synchronized void setRunning(boolean running) {
    this.running = running;
}

@Override
public void run() {

    while (running) {
        while (queue.peek() != null) {
            if (ctr++ % LIMIT == 0) {
                LOG.log(Level.INFO, "Output Queue size = {0} '{'ctr={1}'}'", new Object[]{queue.size(), ctr});
            }

            List<Float> list = queue.poll();
            if (list == null) {
                continue;
            }
            try {
                StringBuilder sbline = new StringBuilder();
                Time t = new Time(list.get(0));
                sbline.append(t.HMSS()).append(",");
                for (Float f : list) {
                    sbline.append(f).append(",");
                }
                sbline.append("\n");
                outputWriter.write(sbline.toString());

            } catch (IOException ex) {
                LOG.info(ex.toString());
                break;
            }
        }
    }
    if (outputWriter != null) {
        try {
            outputWriter.close();
            LOG.info("Closed outputWriter");
        } catch (IOException ex) {
            Logger.getLogger(OutputFloatQueueReceiver.class.getName()).log(Level.SEVERE, null, ex);
        }
    }

}

public static void main(String[] args) {

    try {

        String outputFile = "c:\tmp\qtest.csv";
        File f = new File(outputFile);
        f.delete();

        StringBuilder header = new StringBuilder();
        header.append("1,2,3,4,5,6,7,8,9");
        header.append("\n");
        ConcurrentLinkedQueue<List<Float>> outputQueue = null;

        OutputFloatQueueReceiver outputQueueReceiver = null;
        outputQueue = new ConcurrentLinkedQueue<>();

        outputQueueReceiver = new OutputFloatQueueReceiver(outputFile, header.toString(), outputQueue);
        outputQueueReceiver.start();

        for (int i = 1; i < 100000; i++) {
            List<Float> list = new ArrayList<>();

            //list.set(0, (float) i); // causes exception
            list.add((float) i);
            for (int j = 1; j < 10; j++) {
                list.add((float) j);
            }

            outputQueue.add(list);                                
        }

        try {
            Thread.sleep(5000);
        } catch (InterruptedException ex) {
            Logger.getLogger(OutputFloatQueueReceiver.class.getName()).log(Level.SEVERE, null, ex);
        }

        outputQueueReceiver.setRunning(false);

    } catch (IOException ex) {
        Logger.getLogger(OutputFloatQueueReceiver.class.getName()).log(Level.SEVERE, null, ex);
    }

}

}

Answer 1

此代码是您的代码使用如此多的原因 CPU:

while (running) {
    while (queue.peek() != null) {
        // logging
        List<Float> list = queue.poll();
        if (list == null) {
            continue;
        }
        // do stuff with list
    }
}

基本上，您的代码正在忙等待，重复 "peeking" 直到队列条目可用。它可能在那里紧紧地旋转着。

您应该将队列 class 替换为 BlockingQueue，然后简单地使用 take() ... 像这样：

while (running) {
    List<Float> list = queue.take();
    // do stuff with list
}

take() 调用无限期地阻塞，只有在有可用元素时才返回，并将该元素作为结果返回。如果无限期阻塞是一个问题，您可以使用 poll(...) 和超时，或者您可以安排其他线程中断被阻塞的线程。

为什么 Java 文件写入消耗 CPU？

Why does Java file write consume CPU?

java

queue

bufferedwriter