如何在 python 中创建多个分隔文件？

Question

我有一个程序可以将 CSV 文件转换为竖线分隔文件，还可以计算总行数。但在这种情况下，如果总行数超过 7000，我想创建一个新的输出文件。这种情况将在一个文件中仅调整 7000 行，并为每隔 7000 行创建所有其他输出文件。

我们将不胜感激任何建议、想法或修改。

上一个转换为单个文件的代码：

import csv
input_file = input("Enter input file")
output_file = input("Enter Output file")

# count number of lines
def total_lines(input_file):
    with open(input_file) as f:
        return sum(1 for line in f)

# convert input files to output
def file_conversion(input_file, output_file):
    with open(input_file) as fin:
        with open(output_file, 'w', newline='') as fout:
            reader = csv.DictReader(fin, delimiter=',')
            writer = csv.DictWriter(fout, reader.fieldnames, delimiter='|')
            writer.writeheader()
            writer.writerows(reader)
            print("Successfully converted into", output_file)

Answer 1

more-itertools 让这变得简单。

from more_itertools import chunked

def file_conversion(input_file, output_file_pattern, chunksize):
    with open(input_file) as fin:
        reader = csv.DictReader(fin, delimiter=',')
        for i, chunk in enumerate(chunked(reader, chunksize)):
            with open(output_file_pattern.format(i), 'w', newline='') as fout:
                writer = csv.DictWriter(fout, reader.fieldnames, delimiter='|')
                writer.writeheader()
                writer.writerows(chunk)
                print("Successfully converted into", output_file)

用法示例：

file_conversion('in.csv', 'out{:03}.csv', 7000)

这将生成文件 out000.csv、out001.csv 等

如何在 python 中创建多个分隔文件？

How to create multiple delimited files in python?

python

csv

file-handling

delimiter

python-3.x