python 将文件分成两部分的脚本，分别命名每个部分

Question

我正在尝试使用 d3.js 和交叉过滤器编写地图可视化代码，现在我有一个大文件和一些破坏整个事情的有害行。

我想创建一个文件来将我的输入数据分成两半，这样我就可以缩小问题的根源，从而在保持理智的同时消除它。

输入数据如下所示：

http://www.edsa-project.eu/adzuna/eyJhbGciOiJIUzI1NiJ9.eyJzIjoia0EtLWlpVHhUMUNtSFM0SzE4TUVzUSIsImkiOiIzMzI2ODE3ODgifQ.1u6YvzMuu_HbWqRaMwFd8zYNP43w7wYFnRbl5r2qSoY,C# Developer,Connectus,Chesterton,52.202499,0.131237,United Kingdom,statistics,1
http://www.edsa-project.eu/adzuna/eyJhbGciOiJIUzI1NiJ9.eyJzIjoia0EtLWlpVHhUMUNtSFM0SzE4TUVzUSIsImkiOiIzMzI2ODk1ODIifQ.jxcx56YcDm-4nmB8VvoIGQKew4yquszeaPon60hcDKs,Senior Java Developer,Redhill,Godstow,51.784375,-1.308003,United Kingdom,java|metadata,1
http://www.edsa-project.eu/adzuna/eyJhbGciOiJIUzI1NiJ9.eyJzIjoia0EtLWlpVHhUMUNtSFM0SzE4TUVzUSIsImkiOiIzMzI2OTEyMjIifQ.qK3xtYQDxRpKJkNargPu6Jef4njm2fSZnNIVulRHoqA,Software Development Manager,Spring Technology ,Woolstone,52.042198,-0.7047,United Kingdom,software development|sdlc|data analysis,1
http://www.edsa-project.eu/adzuna/eyJhbGciOiJIUzI1NiJ9.eyJzIjoia0EtLWlpVHhUMUNtSFM0SzE4TUVzUSIsImkiOiIzMzI4NDM1MzgifQ.pYnBX-APPdB3edTRC_M8x6usmBq_GfIxcdZOXSLJN04,Data Scientists Python R Scala Java or Matlab,Aspire Data Recruitment,East Boldon,54.94452,-1.42815,United Kingdom,data science|java|python|scala|matlab|analysis,1
http://www.edsa-project.eu/adzuna/eyJhbGciOiJIUzI1NiJ9.eyJzIjoia0EtLWlpVHhUMUNtSFM0SzE4TUVzUSIsImkiOiIzMzI4NzM4NTMifQ.mgRKEZh-0GLUXQmZ9Bp6H10haZNAieIKAH1uoWV63YU,Data Analyst - Programmatic Tech Company,Ultimate Asset Limited,London,51.50853,-0.12574,United Kingdom,data analysis|analysis|statistics,1

那么，在我的想法中，我会将其平均分配，这样随后我将拥有：

http://www.edsa-project.eu/adzuna/eyJhbGciOiJIUzI1NiJ9.eyJzIjoia0EtLWlpVHhUMUNtSFM0SzE4TUVzUSIsImkiOiIzMzI2ODE3ODgifQ.1u6YvzMuu_HbWqRaMwFd8zYNP43w7wYFnRbl5r2qSoY,C# Developer,Connectus,Chesterton,52.202499,0.131237,United Kingdom,statistics,1
http://www.edsa-project.eu/adzuna/eyJhbGciOiJIUzI1NiJ9.eyJzIjoia0EtLWlpVHhUMUNtSFM0SzE4TUVzUSIsImkiOiIzMzI2ODk1ODIifQ.jxcx56YcDm-4nmB8VvoIGQKew4yquszeaPon60hcDKs,Senior Java Developer,Redhill,Godstow,51.784375,-1.308003,United Kingdom,java|metadata,1
http://www.edsa-project.eu/adzuna/eyJhbGciOiJIUzI1NiJ9.eyJzIjoia0EtLWlpVHhUMUNtSFM0SzE4TUVzUSIsImkiOiIzMzI2OTEyMjIifQ.qK3xtYQDxRpKJkNargPu6Jef4njm2fSZnNIVulRHoqA,Software Development Manager,Spring Technology ,Woolstone,52.042198,-0.7047,United Kingdom,software development|sdlc|data analysis,1

还有这个：

http://www.edsa-project.eu/adzuna/eyJhbGciOiJIUzI1NiJ9.eyJzIjoia0EtLWlpVHhUMUNtSFM0SzE4TUVzUSIsImkiOiIzMzI2OTEyMjIifQ.qK3xtYQDxRpKJkNargPu6Jef4njm2fSZnNIVulRHoqA,Software Development Manager,Spring Technology ,Woolstone,52.042198,-0.7047,United Kingdom,software development|sdlc|data analysis,1
http://www.edsa-project.eu/adzuna/eyJhbGciOiJIUzI1NiJ9.eyJzIjoia0EtLWlpVHhUMUNtSFM0SzE4TUVzUSIsImkiOiIzMzI4NDM1MzgifQ.pYnBX-APPdB3edTRC_M8x6usmBq_GfIxcdZOXSLJN04,Data Scientists Python R Scala Java or Matlab,Aspire Data Recruitment,East Boldon,54.94452,-1.42815,United Kingdom,data science|java|python|scala|matlab|analysis,1
http://www.edsa-project.eu/adzuna/eyJhbGciOiJIUzI1NiJ9.eyJzIjoia0EtLWlpVHhUMUNtSFM0SzE4TUVzUSIsImkiOiIzMzI4NzM4NTMifQ.mgRKEZh-0GLUXQmZ9Bp6H10haZNAieIKAH1uoWV63YU,Data Analyst - Programmatic Tech Company,Ultimate Asset Limited,London,51.50853,-0.12574,United Kingdom,data analysis|analysis|statistics,1

例如。

按照惯例命名它们，例如 starting_input.csv 变成：

starting_input_a.csv

和

starting_input_b.csv

然后当我想再次运行时：

starting_input_aa.csv

和

starting_input_ab.csv

等等。

你能按照我的想法做吗？

我试过这个：

splitLen = 20         # 20 lines per file
outputBase = 'output' # output.1.txt, output.2.txt, etc.

# This is shorthand and not friendly with memory
# on very large files, but it works.
input = open('input.txt', 'r').read().split('\n')

at = 1
for lines in range(0, len(input), splitLen):
    # First, get the list slice
    outputData = input[lines:lines+splitLen]

    # Now open the output file, join the new slice with newlines
    # and write it out. Then close the file.
    output = open(outputBase + str(at) + '.txt', 'w')
    output.write('\n'.join(outputData))
    output.close()

    # Increment the counter
    at += 1

但是没用

Answer 1

这是一个提示。

刚刚读了两遍文件。一次获取行数，然后再次获取上半部分和下半部分。

简单的例子。鉴于您的 5 行示例输入：

$ cat /tmp/f1.txt
http://www.edsa-project.eu/adzuna/eyJhbGciOiJIUzI1NiJ9.eyJzIjoia0EtLWlpVHhUMUNtSFM0SzE4TUVzUSIsImkiOiIzMzI2ODE3ODgifQ.1u6YvzMuu_HbWqRaMwFd8zYNP43w7wYFnRbl5r2qSoY,C# Developer,Connectus,Chesterton,52.202499,0.131237,United Kingdom,statistics,1
http://www.edsa-project.eu/adzuna/eyJhbGciOiJIUzI1NiJ9.eyJzIjoia0EtLWlpVHhUMUNtSFM0SzE4TUVzUSIsImkiOiIzMzI2ODk1ODIifQ.jxcx56YcDm-4nmB8VvoIGQKew4yquszeaPon60hcDKs,Senior Java Developer,Redhill,Godstow,51.784375,-1.308003,United Kingdom,java|metadata,1
http://www.edsa-project.eu/adzuna/eyJhbGciOiJIUzI1NiJ9.eyJzIjoia0EtLWlpVHhUMUNtSFM0SzE4TUVzUSIsImkiOiIzMzI2OTEyMjIifQ.qK3xtYQDxRpKJkNargPu6Jef4njm2fSZnNIVulRHoqA,Software Development Manager,Spring Technology ,Woolstone,52.042198,-0.7047,United Kingdom,software development|sdlc|data analysis,1
http://www.edsa-project.eu/adzuna/eyJhbGciOiJIUzI1NiJ9.eyJzIjoia0EtLWlpVHhUMUNtSFM0SzE4TUVzUSIsImkiOiIzMzI4NDM1MzgifQ.pYnBX-APPdB3edTRC_M8x6usmBq_GfIxcdZOXSLJN04,Data Scientists Python R Scala Java or Matlab,Aspire Data Recruitment,East Boldon,54.94452,-1.42815,United Kingdom,data science|java|python|scala|matlab|analysis,1
http://www.edsa-project.eu/adzuna/eyJhbGciOiJIUzI1NiJ9.eyJzIjoia0EtLWlpVHhUMUNtSFM0SzE4TUVzUSIsImkiOiIzMzI4NzM4NTMifQ.mgRKEZh-0GLUXQmZ9Bp6H10haZNAieIKAH1uoWV63YU,Data Analyst - Programmatic Tech Company,Ultimate Asset Limited,London,51.50853,-0.12574,United Kingdom,data analysis|analysis|statistics,1

你可以这样做：

def divide(fn):
    # get total lines in file
    with open(fn) as f:
        lc=sum(1 for _ in f)

    with open(fn) as fin:
        # top half of file:
        for i, line in enumerate(fin):
            print line
            if i>=lc/2:
                break
        # middle
        print "======="
        # remainder
        for line in fin:
            print line

这将从文件顶部开始打印 3 行，然后是 '=======' 分隔符，然后是示例的最后 2 行。

您可以将 'a' 和 'b' 添加到基本名称，而不是打印，写入两个文件。重新应用于生成的文件，直到完成。

python 将文件分成两部分的脚本，分别命名每个部分

python script to split a file in two parts, name each one separately

python

data-processing