如何根据多个列值拆分文件

Question

我需要使用这个 test_file 并将其拆分为 col5 和 col6 的每个独特组合都有一个单独的文件。另一个注意事项是此文件需要在 150,000 条记录后拆分。还需要从文件中提取命名规则： "$5"_"$6"_P"sysdate"_IU"$4"60""[文件号]".zip

test_file.csv

col1, col2, col3, col4, col5, col6 ..... col32
1234, 6789, 1, 01/31/2017 00:00:00, 1000, 1234 ..... col32
1235, 1233, 1, 01/31/2017 00:00:00, 1000, 1234 ..... col32
1236, 4423, 1, 01/31/2017 00:00:00, 1000, 5678 ..... col32
1237, 3323, 1, 01/31/2017 00:00:00, 1000, 1234 ..... col32
1238, 0808, 1, 01/31/2017 00:00:00, 1000, 1234 ..... col32
1239, 2222, 1, 01/31/2017 00:00:00, 2000, 1234 ..... col32
1231, 4535, 1, 01/31/2017 00:00:00, 2000, 1234 ..... col32
1232, 8080, 1, 01/31/2017 00:00:00, 2000, 5678 ..... col32
1233, 7878, 1, 01/31/2017 00:00:00, 2000, 5678 ..... col32

结果应如下所示：

1000_1234_P20170203_IU20170131_60_1.ZIP
col1, col2, col3, col4, col5, col6 ..... col32
1234, 6789, 1, 01/31/2017 00:00:00, 1000, 1234 ..... col32
1235, 1233, 1, 01/31/2017 00:00:00, 1000, 1234 ..... col32
1237, 3323, 1, 01/31/2017 00:00:00, 1000, 1234 ..... col32
1238, 0808, 1, 01/31/2017 00:00:00, 1000, 1234 ..... col32

1000_5678_P20170203_IU20170131_60_1.ZIP
col1, col2, col3, col4, col5, col6 ..... col32
1236, 4423, 1, 01/31/2017 00:00:00, 1000, 5678 ..... col32

2000_1234_P20170203_IU20170131_60_1.ZIP
col1, col2, col3, col4, col5, col6 ..... col32
1239, 2222, 1, 01/31/2017 00:00:00, 2000, 1234 ..... col32
1231, 4535, 1, 01/31/2017 00:00:00, 2000, 1234 ..... col32

2000_5678_P20170203_IU20170131_60_1.ZIP
col1, col2, col3, col4, col5, col6 ..... col32
1232, 8080, 1, 01/31/2017 00:00:00, 2000, 5678 ..... col32
1233, 7878, 1, 01/31/2017 00:00:00, 2000, 5678 ..... col32

Answer 1

从这里开始：

awk -F', *' -v sysdate="$(date +'%Y%m%d')" '
NR==1 { hdr = [=10=]; next }
(cnt[,]++ % 150000) == 0 { sfx[,]++ }
{
    split(,d,/[\/ ]/)
    out =  "_"  "_P" sysdate "_IU" d[3] d[1] d[2] "_60_" sfx[,] ".zip"
    if (!seen[out]++) {
        print hdr > out
    }
    print > out
}
' file

并进行适当的按摩。如果您不使用 GNU awk，您可能必须 close() 文件以避免 "too many files open" 错误。

如何根据多个列值拆分文件

How can I split a file based on multiple column values

unix

aix

awk

split

multiple-columns