Python 将 Json 转换为 CSV 文件时为 CSV 文件插入 headers

Question

目前正在编写一个脚本，将文件从 jsonl 格式转换为 CSV 格式，最后我写了一行代码为 csv 文件包含一个 header 来标识每个转换的变量.但是，似乎从脚本生成的 CSV 文件似乎对每个读取的 json 行都有一个 header，我只想要一个在第 1 行具有 header 的文件下面的其余值，而不是为每个单独的 json 行读取一个单独的 header。希望有人能帮我解决这个问题，谢谢！

示例 jsonl:

{"symbol": "DOGE-PERP", "timestamp": 1621948955550, "datetime": "2021-05-25T13:22:35.550Z", "high": null, "low": null, "bid": 0.342372, "bidVolume": null, "ask": 0.3424855, "askVolume": null, "vwap": null, "open": null, "close": 0.3424025, "last": 0.3424025, "previousClose": null, "change": null, "percentage": 0.039249281423858244, "average": null, "baseVolume": null, "quoteVolume": 433162290.0506585, "info": {"name": "DOGE-PERP", "enabled": true, "postOnly": false, "priceIncrement": "5e-7", "sizeIncrement": "1.0", "minProvideSize": "1.0", "last": "0.3424025", "bid": "0.342372", "ask": "0.3424855", "price": "0.3424025", "type": "future", "baseCurrency": null, "quoteCurrency": null, "underlying": "DOGE", "restricted": false, "highLeverageFeeExempt": false, "change1h": "0.023470298206100425", "change24h": "0.039249281423858244", "changeBod": "-0.07136396489976689", "quoteVolume24h": "433162290.0506585", "volumeUsd24h": "433162290.0506585"}}
{"symbol": "DOGE-PERP", "timestamp": 1621948955976, "datetime": "2021-05-25T13:22:35.976Z", "high": null, "low": null, "bid": 0.3424955, "bidVolume": null, "ask": 0.3427185, "askVolume": null, "vwap": null, "open": null, "close": 0.3427185, "last": 0.3427185, "previousClose": null, "change": null, "percentage": 0.04020839466903005, "average": null, "baseVolume": null, "quoteVolume": 433162290.0506585, "info": {"name": "DOGE-PERP", "enabled": true, "postOnly": false, "priceIncrement": "5e-7", "sizeIncrement": "1.0", "minProvideSize": "1.0", "last": "0.3427185", "bid": "0.3424955", "ask": "0.3427185", "price": "0.3427185", "type": "future", "baseCurrency": null, "quoteCurrency": null, "underlying": "DOGE", "restricted": false, "highLeverageFeeExempt": false, "change1h": "0.024414849178225707", "change24h": "0.04020839466903005", "changeBod": "-0.07050693556414092", "quoteVolume24h": "433162290.0506585", "volumeUsd24h": "433162290.0506585"}}

CSV 文件当前的样子：

我想达到的目标：

我的脚本：

import glob
import json
import csv

import time


start = time.time()
#import pandas as pd
from flatten_json import flatten

#Path of jsonl file
File_path = (r'C:\Users\Natthanon\Documents\Coding 101\Python\JSONL')
#reading all jsonl files
files = [f for f in glob.glob( File_path + "**/*.jsonl", recursive=True)]
i=0
for f in files:
    with open(f, 'r') as F:
        for line in F:
#flatten json files 
            data = json.loads(line)
            data_1=flatten(data)
#creating csv files  
            with open(r'C:\Users\Natthanon\Documents\Coding 101\Python\CSV\' + f.split("\")[-1] +".csv", 'a' , newline='') as csv_file:
                thewriter = csv.writer(csv_file)
                thewriter.writerow(["symbol", "timestamp", "datetime","high","low","bid","bidVolume","ask","askVolume","vwap","open","close","last","previousClose","change","percentage","average","baseVolume","quoteVolume"])
#headers should be the Key values from json files that make Coulmn header                    
                thewriter.writerow([data_1['symbol'],data_1['timestamp'],data_1['datetime'],data_1['high'],data_1['low'],data_1['bid'],data_1['bidVolume'],data_1['ask'],data_1['askVolume'],data_1['vwap'],data_1['open'],data_1['close'],data_1['last'],data_1['previousClose'],data_1['change'],data_1['percentage'],data_1['average'],data_1['baseVolume'],data_1['quoteVolume']])

Answer 1

您需要将输出 CSV 文件 open() 移动到开始解析行之前，例如：

import glob
import json
import csv
import time


start = time.time()
#import pandas as pd
from flatten_json import flatten

#Path of jsonl file
File_path = (r'C:\Users\Natthanon\Documents\Coding 101\Python\JSONL')
#reading all jsonl files
files = [f for f in glob.glob( File_path + "**/*.jsonl", recursive=True)]
i = 0

for f in files:
    with open(f, 'r') as F:
        #creating csv files  
        with open(r'C:\Users\Natthanon\Documents\Coding 101\Python\CSV\' + f.split("\")[-1] + ".csv", 'w' , newline='') as csv_file:
            thewriter = csv.writer(csv_file)
            thewriter.writerow(["symbol", "timestamp", "datetime","high","low","bid","bidVolume","ask","askVolume","vwap","open","close","last","previousClose","change","percentage","average","baseVolume","quoteVolume"])

            for line in F:
                #flatten json files 
                data = json.loads(line)
                data_1 = flatten(data)
                #headers should be the Key values from json files that make Column header                    
                thewriter.writerow([data_1['symbol'],data_1['timestamp'],data_1['datetime'],data_1['high'],data_1['low'],data_1['bid'],data_1['bidVolume'],data_1['ask'],data_1['askVolume'],data_1['vwap'],data_1['open'],data_1['close'],data_1['last'],data_1['previousClose'],data_1['change'],data_1['percentage'],data_1['average'],data_1['baseVolume'],data_1['quoteVolume']])

在您的代码中，您为每一行打开和关闭文件，并且每次都添加 header。

Python 将 Json 转换为 CSV 文件时为 CSV 文件插入 headers

Python Inserting headers for CSV file when converting JsonL to CSV File

python

csv

json

type-conversion

jsonlines