Python

Question

我正在编写一个 Python 脚本，并希望将 Flexible I/O (FIO) 数据文本文件的特定行复制到我的输出文本文件中。

我尝试使用 this tutorial 从我的数据文本文件中复制数据，但遗憾的是，我没有得到理想的结果。在我执行我的代码后，我连接到 output.txt、output.txt returns 什么都没有，而不是 cpu 或来自 data.txt 的 lat (usec) 的文本行。

这是我获取data.txt:

的代码

#!/usr/bin/python
# -*- coding: utf-8 -*-

import subprocess
import os

cmd = subprocess.Popen(["sudo", "fio", "--filename=/dev/sdg", "--name=test1", "--ioengine=libaio",
    "--bs=4k", "--rw=read", "--iodepth=32", "--runtime=10"],
    stdout = subprocess.PIPE, stderr = subprocess.PIPE, preexec_fn = os.setpgrp)

f = open("logfile.txt", "w+")
g = open("logs.txt", "w+")

f.write(cmd.stdout.read().decode())

with g as f:
    for line in f:
        # finds the leading text in the text file that tarts with "", and also removes all
        # leading and trailing spaces from a strin.
        if line.strip().startswith("lat (usec)") or line.strip().startswith("cpu"):
            # adds items to end of list
            g.write(line + "\n")

这是我的数据文本文件 (data.txt):

test1: (g=0): rw=read, bs=4K-4K/4K-4K/4K-4K, ioengine=libaio, iodepth=32
fio-2.16
Starting 1 process

test1: (groupid=0, jobs=1): err= 0: pid=9401: Tue Jul  2 14:28:30 2019
  read : io=4066.2MB, bw=416418KB/s, iops=104104, runt= 10001msec
    slat (usec): min=1, max=2650, avg= 7.75, stdev=41.93
    clat (usec): min=19, max=2790, avg=298.85, stdev=180.63
     lat (usec): min=101, max=2792, avg=306.59, stdev=181.00
    clat percentiles (usec):
     |  1.00th=[  105],  5.00th=[  112], 10.00th=[  118], 20.00th=[  120],
     | 30.00th=[  129], 40.00th=[  137], 50.00th=[  153], 60.00th=[  454],
     | 70.00th=[  474], 80.00th=[  490], 90.00th=[  502], 95.00th=[  516],
     | 99.00th=[  548], 99.50th=[  580], 99.90th=[  756], 99.95th=[  932],
     | 99.99th=[ 1592]
    lat (usec) : 20=0.01%, 100=0.01%, 250=51.56%, 500=36.57%, 750=11.76%
    lat (usec) : 1000=0.06%
    lat (msec) : 2=0.04%, 4=0.01%
  cpu          : usr=18.48%, sys=38.28%, ctx=16300, majf=0, minf=40
  IO depths    : 1=0.1%, 2=0.1%, 4=0.1%, 8=0.1%, 16=0.1%, 32=100.0%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.1%, 64=0.0%, >=64=0.0%
     issued    : total=r=1041149/w=0/d=0, short=r=0/w=0/d=0, drop=r=0/w=0/d=0
     latency   : target=0, window=0, percentile=100.00%, depth=32

Run status group 0 (all jobs):
   READ: io=4066.2MB, aggrb=416417KB/s, minb=416417KB/s, maxb=416417KB/s, mint=10001msec, maxt=10001msec

Disk stats (read/write):
  sdg: ios=16094/0, merge=0/0, ticks=14376/0, in_queue=14368, util=99.01%

现在我希望数据文本文件中的第 8、15 和 18 行进入输出文本文件，但它没有返回任何内容。这是输出文本文件的样子 (output.txt):

     lat (usec): min=101, max=2792, avg=306.59, stdev=181.00
    lat (usec) : 250=5.09%, 500=67.38%, 750=23.29%, 1000=3.87%
  cpu          : usr=18.48%, sys=38.28%, ctx=16300, majf=0, minf=40

是否有类似的方法可以使用上面的教程将数据文本文件中的数据提取到结果输出文件中？

谢谢！

Answer 1

是这样的吗？

with open("/tmp/log.txt") as f:
        to_keep = []
        for line in f:
                if line.strip().startswith("lat (usec)") or line.strip().startswith("cpu"):
                        to_keep.append(line)
        print(to_keep)

顺便说一句：这并不是最干净的方法，但我想是最简单的方法（在 python 中）

更简单的方法是使用简单的 linux 命令，例如：

   cat log.txt | grep "lat (usec)"

Answer 2

我解决了我的问题，谢谢@AlessioM 和@pkfm 的帮助！

这是我的回答：

#!/usr/bin/python
# -*- coding: utf-8 -*-

import subprocess
import os

def fioCall():
    cmd = subprocess.Popen(["sudo", "fio", "--filename=/dev/sdg", "--name=test1", "--ioengine=libaio",
    "--bs=4k", "--rw=read", "--iodepth=32", "--runtime=10"],
    stdout = subprocess.PIPE, stderr = subprocess.PIPE, preexec_fn = os.setpgrp)

    f = open("data.txt", "w+")
    f.write(cmd.stdout.read().decode())
    f.close()

def copyFiles():
    with open("data.txt") as f:
        with open("output.txt", "w") as f1:
            for line in f:
                if line.strip().startswith("lat (usec)") or line.strip().startswith("cpu"):
                    f1.write(line)

def main():
    fioCall()
    copyFiles()

if __name__ == "__main__":
    main()

这是我获得的输出 (output.txt):

     lat (usec): min=161, max=10624, avg=458.78, stdev=148.10
    lat (usec) : 250=5.09%, 500=67.38%, 750=23.29%, 1000=3.87%
  cpu          : usr=17.24%, sys=63.28%, ctx=81571, majf=0, minf=40

Python - 从一个文本文件复制到另一个

Python - Copying from one text file to another

io

benchmarking

text-extraction

copy