使用 sed 将 tomcat 日志转换为 CSV

Question

我在 Linux 服务器上有一些应用程序日志数据，如下所示：

Jan 11 14:24:42 AttackSimulator.abcd [1587566256,49294,"ryan.wright@abcd.com",3237159933,1,0,0,3,"2314","https",443,2899903330,"https://googleads.g.doubleclick.net/pagead/ads",0,"","","","","1 - Default Policy","","googleads.g.doubleclick.net","GET",4,0,5]
Jan 11 14:24:42 AttackSimulator.abcd [1587566256,49294,"melisa.zeunert@abcd.com",3237159933,1,0,0,3,"2339,37788","http",80,387803624,"http://ping.citrix.com",0,"","","","","3 - Extended Policy High","","ping.citrix.com","HEAD",3,0,4]

我想将其输出到 CSV 文件中。 [ ] 之间的所有内容都应该是 CSV 的一部分，列名包括 DateandTime、AccountID、UserID、ClientIP 等。看起来像这样：

DateandTime, AccountID, UserID, ClientIP
1587566256,49294,"ryan.wright@abcd.com",3237159933
1587566256,49294,"melisa.zeunert@abcd.com",3237159933

（这只是一个只有 4 列的示例。实际输出将有 25）

任何意见都会有所帮助

Answer 1

使用sed

$ sed 's/.*\[\(.*\)]//;1iDateandTime, AccountID, UserID, ClientIP ' file
DateandTime, AccountID, UserID, ClientIP
1587566256,49294,"ryan.wright@scnx.com",3237159933,1,0,0,3,"2314","https",443,2899903330,"https://googleads.g.doubleclick.net/pagead/ads",0,"","","","","1 - Default Policy","","googleads.g.doubleclick.net","GET",4,0,5
1587566256,49294,"melisa.zeunert@scnx.com",3237159933,1,0,0,3,"2339,37788","http",80,387803624,"http://ping.citrix.com",0,"","","","","3 - Extended Policy High","","ping.citrix.com","HEAD",3,0,4

.*\[ - 排除直到最后一个 [ 方括号的所有内容。

\(.*\)] - 包括括号内的所有内容，直到最后一个 ] 方括号

</code> - Return 向后引用括号内存储的内容</p> <p><code>1i - 在第 1 行插入

使用 sed 将 tomcat 日志转换为 CSV

Converting tomcat logs to CSV using sed

csv

logging

awk

sed

scripting