从 Tshark JSON 输出中过滤特定值
Filter specific value from Tshark JSON output
我想提取 "tls.handshake.certificate_raw" 值，但到目前为止没有成功。大JSON快死我了
这是我的 JSON 文件：Download
提前致谢！
下面假设如果文件中的一个JSON对象不止一次有一个特定的键，那么只有最后一个是相关的。如果这个假设不成立，那么一种选择是使用 jq 的流解析器。
下面显示了一个简单的 jq 检索命名密钥的应用程序需要大约 26MB RAM。这在当今时代是个问题吗？
/usr/bin/time -lp jq '.. | objects | .["tls.handshake.certificate_raw"] // empty' Tshark.json
输出：
[
"308204633082034ba00302010202100182f8098ea2e626b91a3b27841fb9af300d06092a864886f70d01010b0500305a310b300906035504061302494531123010060355040a130942616c74696d6f726531133011060355040b130a43796265725472757374312230200603550403131942616c74696d6f7265204379626572547275737420526f6f74301e170d3135313230383132303530375a170d3235303531303132303030305a3064310b300906035504061302555331153013060355040a130c446967694365727420496e6331193017060355040b13107777772e64696769636572742e636f6d312330210603550403131a44696769436572742042616c74696d6f72652043412d3220473230820122300d06092a864886f70d01010105000382010f003082010a0282010100bbe700fe000173ef9b88f05c0875f05c70731fffb57a1f8d0823fdc7c1133fb12106bd4dccdb2db8871931deda6b0dc15dd251d5b05e72d4f066d484f2911841b50926b2039604699553fa074f3ebe863a432bb259b5abd1baf45940725a67b6cde655fc092ba498b2bfcca91fd4bf878f8225fd4580a9ec9d8cf97490cf1cf86550afcdf845abc82f8d8e361ddeea89cdb252d9e509cd36712bdac6d09bc6f6a45b5b39b61f001f19a4fbd3be0ce6fb8fdd0e80e6a4e75ac4c02f72ee91f38367c9cdcace4ab02081fde472af0918e74f42ec4d27424d07a0ddd50d75f73660cc265498a18837f083f7ad295920cfbb25e1d59ee6f3aec101ebb50689e20a8f0203010001a382011930820115301d0603551d0e04160414c012b22874684667e97025741a00455b067d5c44301f0603551d23041830168014e59d5930824758ccacfa085436867b3ab5044df030120603551d130101ff040830060101ff020100300e0603551d0f0101ff040403020186303406082b0601050507010104283026302406082b060105050730018618687474703a2f2f6f6373702e64696769636572742e636f6d303a0603551d1f04333031302fa02da02b8629687474703a2f2f63726c332e64696769636572742e636f6d2f4f6d6e69726f6f74323032352e63726c303d0603551d200436303430320604551d2000302a302806082b06010505070201161c68747470733a2f2f7777772e64696769636572742e636f6d2f435053300d06092a864886f70d01010b050003820101002fe23766c31acf9155ee2914fa5010be998de228f1741607693b44303df5b949f56836feb7308da9e0074942634652b569c69b49f21a0f57c6f6b7beab758d0da2ab33e462854354ab63940ca353a0ea1de4a6a6ac950e6578ec338cbe943ed870abeac7e178ce3395bbfb2c58e08a40a66d41d8d8d9b3f92f02c51b706be23a857b1d18b0d14b9ac16f171029e80ec3e1549654a5919508208d692c9e2f3f50b42ee5571fafa473d2a124bb0d90fe45131da7a129d8660bc3d16cb34fb42b844c4790ec8be2418913b607467742fdbb1e17faff254b8e5057eb8ecc4e153fb125dd410ca09c626d6fba97078bd9f30610c529d4f7ba1690a8d054e14b49c798",
1778,
1127,
0,
30
]
real 1.42
user 1.34
sys 0.03
26693632 maximum resident set size
由于 JSON 具有带重复键的对象，因此不使用 jq 的正常对象语义的方法是必要的。
jq 的流解析器（使用 --stream 选项调用）允许处理具有重复键的对象：
jq --stream -c '(.[0]|index("tls.handshake.certificate_raw")) as $ix
| select($ix) | .[0] |= .[$ix+1:]' Tshark.json |
jq -nc 'fromstream(inputs)'
这会产生 503 个实体，第一个是：
["308208df308206c7a00302010202132d000491fc30ee99b24196c0d30000000491fc300d06092a864886f70d01010b050030818b310b3009060355040613025553311330110603550408130a57617368696e67746f6e3110300e060355040713075265646d6f6e64311e301c060355040a13154d6963726f736f667420436f72706f726174696f6e31153013060355040b130c4d6963726f736f6674204954311e301c060355040313154d6963726f736f667420495420544c532043412035301e170d3138313130323139353233375a170d3230313130323139353233375a3021311f301d06035504030c162a2e636c6f2e666f6f747072696e74646e732e636f6d30820122300d06092a864886f70d01010105000382010f003082010a0282010100ab93c55b9e948806364d71adf61ed1ae88c3f3531d8c03df279da3c618391d59636aaeaf9981ac75aaf95e98a97b5bf0fcbc4d819a6c896b29d12652ffa2b603a7edb61eb921a02e2c6730275f3ec23998aab45c5dd4d00cb91ac0cbae0dd8f897cd0b34d81056429c4d854d45b309644df10e7d7fe5a99643eb935263d862a5f4d6750b8202d16c0f89a6b888e8361dcd5314bdb3093b7d6b55106f874824c4a220657f72f8a8ab0c3da27f3f76d343ef85482c88f5423b88cc008c2d9e4ea3b3c373ff22e881491ad2922b6cf5076786b886b1c0e8c603d28d014dd0cd1216c59657c9806b2f1d34795286f729ce712728c12b80197df61790fe629d4732490203010001a38204a33082049f308201f7060a2b06010401d679020402048201e7048201e301e1007600a4b90990b418581487bb13a2cc67700a3c359804f91bdfb8e377cd0ec80ddc1000000166d6053ea50000040300473045022100865c52fbcc0846f7826de85fe28ee5192c706671e208359cc884ef8f862c384a022003dfd681fb2488ccbd8452733eaa7defafe80e1b834fe2a1bb8810cf492883b8007700bbd9dfbc1f8a71b593942397aa927b473857950aab52e81a909664368e1ed18500000166d6053f9a0000040300483046022100d0d6894ad6c1c2ff24a426483e354f21a2172ed4d2c3e544d887d2021f16b6e6022100eb61a67dd9e1b00c244750d43acac412ef9bd47ce77e85db86a6f83ee656c47e0077005ea773f9df56c0e7b536487dd049e0327a919a0c84a11212841875968171455800000166d6053eb20000040300483046022100f81aae0caef665d9071e5ebb949ccfc4033038c9df3227f3e90d46d77ed43574022100f76f1a30eb8b7ff4175cba86e07f46b6f625d1698c91b348a8e8ede7752c2456007500f095a459f200d18240102d2f93888ead4bfe1d47e399e1d034a6b0a8aa8eb27300000166d6053ebd0000040300463044022038393db91d24a1a0121f64446fdb7297406f89ae6d1abe39714073d8dd6bf79202203fec31afceb6a00663f7e3b5e4d9930c0b533dd938bc2ce82117502e97a30863302706092b060104018237150a041a3018300a06082b06010505070302300a06082b06010505070301303e06092b06010401823715070431302f06272b060104018237150887da867583eed90182c9851b81b59e6185f4eb60815d84d2df4282e7937a02016402011d30818506082b0601050507010104793077305106082b060105050730028645687474703a2f2f7777772e6d6963726f736f66742e636f6d2f706b692f6d73636f72702f4d6963726f736f66742532304954253230544c532532304341253230352e637274302206082b060105050730018616687474703a2f2f6f6373702e6d736f6373702e636f6d301d0603551d0e04160414e8e1323d172500a9588138faa2d310619350ac06300b0603551d0f0404030204b030470603551d110440303e82162a2e636c6f2e666f6f747072696e74646e732e636f6d8211632d72696e672e6d73656467652e6e65748211712d72696e672e6d73656467652e6e65743081ac0603551d1f0481a43081a130819ea0819ba08198864b687474703a2f2f6d7363726c2e6d6963726f736f66742e636f6d2f706b692f6d73636f72702f63726c2f4d6963726f736f66742532304954253230544c532532304341253230352e63726c8649687474703a2f2f63726c2e6d6963726f736f66742e636f6d2f706b692f6d73636f72702f63726c2f4d6963726f736f66742532304954253230544c532532304341253230352e63726c304d0603551d2004463044304206092b0601040182372a013035303306082b060105050702011627687474703a2f2f7777772e6d6963726f736f66742e636f6d2f706b692f6d73636f72702f637073301f0603551d2304183016801408fe259f74ea8704c2bcbb8ea8385f33c6d16c65301d0603551d250416301406082b0601050507030206082b06010505070301300d06092a864886f70d01010b050003820201007873266ca28f9b88b653ce3d9eff1d8d61f3cd6513b10228b4c9ea7276f8bca602812c10b45d7055b593880b632bbd8853b945b62033b159c4ad521304f5c2d55edbc027ce3d45aff48fa0c02d4f1b935ae1f2e3633c40bdaa0e618f08ec58996a52b2e0e1a6beb94d481255f5ee70a459ffb5f0dad8b9f739fcc6a7cb97a70229190864a70100e39e0e1c33383413a2e527aab47d1c9930a7f43082478e644ed6e4a3d4625a0fe8bda7fb731e569b38ec1fb75a5b7a51535af0bc96280da67d775fb9131027a13242a39d62e93fc140b7bcf201483d8d6a3ad1326386738dd8c707982bf5003da56a9c096b90d0d3827150e2b1d66393016bbb5b180aba7d98098fbf63ed9b8c85c64cd6414689e684424c68514bbb9f6da3d73165fbd58d972a7fba91fccda255b88c421b30d0961bb431bd2c6bfcb7b5a680acff3777ad911f030af3d552d69a7a6f27ca10ebbcab7195d6a363e769c872da596fd0641889c47f881b155df2ac045a5a016ac769d611e6c0cfcfcbaf9039d13ff6d321fff5abd5f922e8623162192eed42170eb090356549fa0867fa890bdbaf82f2fc41028494a133ccc97e7b6b4a21f6ad1aba032bfb1aa67b9247c97065dcd171270ea387862d92d8acd22994efdf03247d17c8b033f68d88231b655e4d22f30696aaf9e5667cebeb6ed001a3ea0e48a6231d1488a46755828c93a746e4c33a645db0d4",109,2275,0,30]
说明
jq 的第一次调用使用 jq 的流解析器来提取感兴趣的 [path, atomicValue] 对。第二次调用重构相关实体。
资源
以下是 运行 在 Mac Mini 上使用 /usr/bin/time -lp 的主要统计数据：
real 5.35
user 5.31
sys 0.05
13279232 maximum resident set size
我最后的命令：
jq --stream -c '(.[0]|index("tls.handshake.certificate_raw")) as $ix
| select($ix) | .[0] |= .[$ix+1:]' Tshark.json |
jq -r -nc 'fromstream(inputs)[0]' | sort -u > Certificates.txt
我只从数组中获取第一个值并删除双引号。在排序 uniq 之后，我将 SSL 证书逐行写入文件。
非常感谢您的大力协助！
我想提取 "tls.handshake.certificate_raw" 值，但到目前为止没有成功。大JSON快死我了
这是我的 JSON 文件：Download
提前致谢！
下面假设如果文件中的一个JSON对象不止一次有一个特定的键，那么只有最后一个是相关的。如果这个假设不成立，那么一种选择是使用 jq 的流解析器。
下面显示了一个简单的 jq 检索命名密钥的应用程序需要大约 26MB RAM。这在当今时代是个问题吗？
/usr/bin/time -lp jq '.. | objects | .["tls.handshake.certificate_raw"] // empty' Tshark.json
输出：
[
"308204633082034ba00302010202100182f8098ea2e626b91a3b27841fb9af300d06092a864886f70d01010b0500305a310b300906035504061302494531123010060355040a130942616c74696d6f726531133011060355040b130a43796265725472757374312230200603550403131942616c74696d6f7265204379626572547275737420526f6f74301e170d3135313230383132303530375a170d3235303531303132303030305a3064310b300906035504061302555331153013060355040a130c446967694365727420496e6331193017060355040b13107777772e64696769636572742e636f6d312330210603550403131a44696769436572742042616c74696d6f72652043412d3220473230820122300d06092a864886f70d01010105000382010f003082010a0282010100bbe700fe000173ef9b88f05c0875f05c70731fffb57a1f8d0823fdc7c1133fb12106bd4dccdb2db8871931deda6b0dc15dd251d5b05e72d4f066d484f2911841b50926b2039604699553fa074f3ebe863a432bb259b5abd1baf45940725a67b6cde655fc092ba498b2bfcca91fd4bf878f8225fd4580a9ec9d8cf97490cf1cf86550afcdf845abc82f8d8e361ddeea89cdb252d9e509cd36712bdac6d09bc6f6a45b5b39b61f001f19a4fbd3be0ce6fb8fdd0e80e6a4e75ac4c02f72ee91f38367c9cdcace4ab02081fde472af0918e74f42ec4d27424d07a0ddd50d75f73660cc265498a18837f083f7ad295920cfbb25e1d59ee6f3aec101ebb50689e20a8f0203010001a382011930820115301d0603551d0e04160414c012b22874684667e97025741a00455b067d5c44301f0603551d23041830168014e59d5930824758ccacfa085436867b3ab5044df030120603551d130101ff040830060101ff020100300e0603551d0f0101ff040403020186303406082b0601050507010104283026302406082b060105050730018618687474703a2f2f6f6373702e64696769636572742e636f6d303a0603551d1f04333031302fa02da02b8629687474703a2f2f63726c332e64696769636572742e636f6d2f4f6d6e69726f6f74323032352e63726c303d0603551d200436303430320604551d2000302a302806082b06010505070201161c68747470733a2f2f7777772e64696769636572742e636f6d2f435053300d06092a864886f70d01010b050003820101002fe23766c31acf9155ee2914fa5010be998de228f1741607693b44303df5b949f56836feb7308da9e0074942634652b569c69b49f21a0f57c6f6b7beab758d0da2ab33e462854354ab63940ca353a0ea1de4a6a6ac950e6578ec338cbe943ed870abeac7e178ce3395bbfb2c58e08a40a66d41d8d8d9b3f92f02c51b706be23a857b1d18b0d14b9ac16f171029e80ec3e1549654a5919508208d692c9e2f3f50b42ee5571fafa473d2a124bb0d90fe45131da7a129d8660bc3d16cb34fb42b844c4790ec8be2418913b607467742fdbb1e17faff254b8e5057eb8ecc4e153fb125dd410ca09c626d6fba97078bd9f30610c529d4f7ba1690a8d054e14b49c798",
1778,
1127,
0,
30
]
real 1.42
user 1.34
sys 0.03
26693632 maximum resident set size
由于 JSON 具有带重复键的对象，因此不使用 jq 的正常对象语义的方法是必要的。
jq 的流解析器（使用 --stream 选项调用）允许处理具有重复键的对象：
jq --stream -c '(.[0]|index("tls.handshake.certificate_raw")) as $ix
| select($ix) | .[0] |= .[$ix+1:]' Tshark.json |
jq -nc 'fromstream(inputs)'
这会产生 503 个实体，第一个是：
["308208df308206c7a00302010202132d000491fc30ee99b24196c0d30000000491fc300d06092a864886f70d01010b050030818b310b3009060355040613025553311330110603550408130a57617368696e67746f6e3110300e060355040713075265646d6f6e64311e301c060355040a13154d6963726f736f667420436f72706f726174696f6e31153013060355040b130c4d6963726f736f6674204954311e301c060355040313154d6963726f736f667420495420544c532043412035301e170d3138313130323139353233375a170d3230313130323139353233375a3021311f301d06035504030c162a2e636c6f2e666f6f747072696e74646e732e636f6d30820122300d06092a864886f70d01010105000382010f003082010a0282010100ab93c55b9e948806364d71adf61ed1ae88c3f3531d8c03df279da3c618391d59636aaeaf9981ac75aaf95e98a97b5bf0fcbc4d819a6c896b29d12652ffa2b603a7edb61eb921a02e2c6730275f3ec23998aab45c5dd4d00cb91ac0cbae0dd8f897cd0b34d81056429c4d854d45b309644df10e7d7fe5a99643eb935263d862a5f4d6750b8202d16c0f89a6b888e8361dcd5314bdb3093b7d6b55106f874824c4a220657f72f8a8ab0c3da27f3f76d343ef85482c88f5423b88cc008c2d9e4ea3b3c373ff22e881491ad2922b6cf5076786b886b1c0e8c603d28d014dd0cd1216c59657c9806b2f1d34795286f729ce712728c12b80197df61790fe629d4732490203010001a38204a33082049f308201f7060a2b06010401d679020402048201e7048201e301e1007600a4b90990b418581487bb13a2cc67700a3c359804f91bdfb8e377cd0ec80ddc1000000166d6053ea50000040300473045022100865c52fbcc0846f7826de85fe28ee5192c706671e208359cc884ef8f862c384a022003dfd681fb2488ccbd8452733eaa7defafe80e1b834fe2a1bb8810cf492883b8007700bbd9dfbc1f8a71b593942397aa927b473857950aab52e81a909664368e1ed18500000166d6053f9a0000040300483046022100d0d6894ad6c1c2ff24a426483e354f21a2172ed4d2c3e544d887d2021f16b6e6022100eb61a67dd9e1b00c244750d43acac412ef9bd47ce77e85db86a6f83ee656c47e0077005ea773f9df56c0e7b536487dd049e0327a919a0c84a11212841875968171455800000166d6053eb20000040300483046022100f81aae0caef665d9071e5ebb949ccfc4033038c9df3227f3e90d46d77ed43574022100f76f1a30eb8b7ff4175cba86e07f46b6f625d1698c91b348a8e8ede7752c2456007500f095a459f200d18240102d2f93888ead4bfe1d47e399e1d034a6b0a8aa8eb27300000166d6053ebd0000040300463044022038393db91d24a1a0121f64446fdb7297406f89ae6d1abe39714073d8dd6bf79202203fec31afceb6a00663f7e3b5e4d9930c0b533dd938bc2ce82117502e97a30863302706092b060104018237150a041a3018300a06082b06010505070302300a06082b06010505070301303e06092b06010401823715070431302f06272b060104018237150887da867583eed90182c9851b81b59e6185f4eb60815d84d2df4282e7937a02016402011d30818506082b0601050507010104793077305106082b060105050730028645687474703a2f2f7777772e6d6963726f736f66742e636f6d2f706b692f6d73636f72702f4d6963726f736f66742532304954253230544c532532304341253230352e637274302206082b060105050730018616687474703a2f2f6f6373702e6d736f6373702e636f6d301d0603551d0e04160414e8e1323d172500a9588138faa2d310619350ac06300b0603551d0f0404030204b030470603551d110440303e82162a2e636c6f2e666f6f747072696e74646e732e636f6d8211632d72696e672e6d73656467652e6e65748211712d72696e672e6d73656467652e6e65743081ac0603551d1f0481a43081a130819ea0819ba08198864b687474703a2f2f6d7363726c2e6d6963726f736f66742e636f6d2f706b692f6d73636f72702f63726c2f4d6963726f736f66742532304954253230544c532532304341253230352e63726c8649687474703a2f2f63726c2e6d6963726f736f66742e636f6d2f706b692f6d73636f72702f63726c2f4d6963726f736f66742532304954253230544c532532304341253230352e63726c304d0603551d2004463044304206092b0601040182372a013035303306082b060105050702011627687474703a2f2f7777772e6d6963726f736f66742e636f6d2f706b692f6d73636f72702f637073301f0603551d2304183016801408fe259f74ea8704c2bcbb8ea8385f33c6d16c65301d0603551d250416301406082b0601050507030206082b06010505070301300d06092a864886f70d01010b050003820201007873266ca28f9b88b653ce3d9eff1d8d61f3cd6513b10228b4c9ea7276f8bca602812c10b45d7055b593880b632bbd8853b945b62033b159c4ad521304f5c2d55edbc027ce3d45aff48fa0c02d4f1b935ae1f2e3633c40bdaa0e618f08ec58996a52b2e0e1a6beb94d481255f5ee70a459ffb5f0dad8b9f739fcc6a7cb97a70229190864a70100e39e0e1c33383413a2e527aab47d1c9930a7f43082478e644ed6e4a3d4625a0fe8bda7fb731e569b38ec1fb75a5b7a51535af0bc96280da67d775fb9131027a13242a39d62e93fc140b7bcf201483d8d6a3ad1326386738dd8c707982bf5003da56a9c096b90d0d3827150e2b1d66393016bbb5b180aba7d98098fbf63ed9b8c85c64cd6414689e684424c68514bbb9f6da3d73165fbd58d972a7fba91fccda255b88c421b30d0961bb431bd2c6bfcb7b5a680acff3777ad911f030af3d552d69a7a6f27ca10ebbcab7195d6a363e769c872da596fd0641889c47f881b155df2ac045a5a016ac769d611e6c0cfcfcbaf9039d13ff6d321fff5abd5f922e8623162192eed42170eb090356549fa0867fa890bdbaf82f2fc41028494a133ccc97e7b6b4a21f6ad1aba032bfb1aa67b9247c97065dcd171270ea387862d92d8acd22994efdf03247d17c8b033f68d88231b655e4d22f30696aaf9e5667cebeb6ed001a3ea0e48a6231d1488a46755828c93a746e4c33a645db0d4",109,2275,0,30]
说明
jq 的第一次调用使用 jq 的流解析器来提取感兴趣的 [path, atomicValue] 对。第二次调用重构相关实体。
资源
以下是 运行 在 Mac Mini 上使用 /usr/bin/time -lp 的主要统计数据：
real 5.35
user 5.31
sys 0.05
13279232 maximum resident set size
我最后的命令：
jq --stream -c '(.[0]|index("tls.handshake.certificate_raw")) as $ix
| select($ix) | .[0] |= .[$ix+1:]' Tshark.json |
jq -r -nc 'fromstream(inputs)[0]' | sort -u > Certificates.txt
我只从数组中获取第一个值并删除双引号。在排序 uniq 之后，我将 SSL 证书逐行写入文件。
非常感谢您的大力协助！