如何将已解析的文本拆分为 "blocks" 文本？

Question

我正在编写一个 bash 脚本来运行网络的 Nmap 扫描。在此之后，需要检查扫描并提取相关位。

我需要从完成的扫描中提取 IP、MAC 和 OS。问题是 Nmap 并不总是从扫描中获得 OS，因此不会将其放入结果中。我需要在最终结果中关联 IP、MAC 和 OS。

这是测试扫描的示例：

Nmap scan report for 192.168.0.1
Host is up (0.0029s latency).
Not shown: 990 closed ports
PORT      STATE SERVICE
PORT#    STATE    XXXXXXX
MAC Address: MA:CA:DR:ES:S0:03 (Unknown)
Device type: general purpose
Running: Linux 2.6.X|3.X
OS CPE: cpe:/o:linux:linux_kernel:2.6 cpe:/o:linux:linux_kernel:3
OS details: Linux 2.6.32 - 3.13
Network Distance: 1 hop

Nmap scan report for 192.168.0.102
Host is up (0.0044s latency).
Not shown: 999 closed ports
PORT     STATE    SERVICE
PORT#    STATE    XXXXXXX
MAC Address: MA:CA:DR:ES:S0:02 (Sony Mobile Communications AB)
Too many fingerprints match this host to give specific OS details
Network Distance: 1 hop

Nmap scan report for 192.168.0.104
Host is up (0.00024s latency).
Not shown: 995 filtered ports
PORT     STATE SERVICE
PORT#    STATE XXXXXX
MAC Address: MA:CA:DR:ES:S0:01 (Micro-star Intl)
Warning: OSScan results may be unreliable because we could not find at least 1 open and 1 closed port
Device type: general purpose
Running (JUST GUESSING): Microsoft Windows 2008 (91%)
OS CPE: cpe:/o:microsoft:windows_server_2008::sp1 cpe:/o:microsoft:windows_server_2008:r2
Aggressive OS guesses: Microsoft Windows Server 2008 SP1 or Windows Server 2008 R2 (91%)
No exact OS matches for host (test conditions non-ideal).
Network Distance: 1 hop

另请注意上例中的最后一个如何找不到 OS，在这种情况下需要攻击性猜测

最终结果需要是一个包含如下内容的文本文件：

192.168.0.1 - MA:CA:DR:ES:S0:03 - Linux 2.6.32 - 3.13
192.168.0.102 - MA:CA:DR:ES:S0:02 - Not found
192.168.0.104 - MA:CA:DR:ES:S0:01 - Microsoft Windows Server 2008 SP1 or Windows Server 2008 R2

我做了一些研究，但找不到任何解释我如何将 IP 与文本块中的 mac 地址和 os 相关联的内容。

我有以下命令可用于简单扫描，其中 IP 和 Mac 地址彼此相邻

  while read line; do
    Mac="$(grep -oE '[A-Z0-9]{2}:[A-Z0-9]{2}:[A-Z0-9]{2}:[A-Z0-9]{2}:[A-Z0-9]{2}:[A-Z0-9]{2}' <<< "$line")"
    ip="$(grep -oE '[0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}' <<< "$line")"
    echo -e $ip'\t-\t '$Mac >>/path/to/results.txt
  done </path/to/testscan.txt

我对 bash 脚本编写还很陌生，如果我遗漏了一些明显的东西，我深表歉意。

任何感兴趣的人的 nmap 命令是：

nmap -O --osscan-guess 192.168.0.0/24 -oN /path/to/testscan.txt

抱歉文字墙，我觉得信息越多越好！

Answer 1

这很容易用 awk:

解析

BEGIN {os_details="Not found"}

/^Nmap scan report/      {target=}
/^MAC Address/           {mac_address=}
/^OS details/            {os_details=substr([=10=], length("OS details: "))}
/^Aggressive OS guesses/ {
    os_details=substr([=10=], length("Aggressive OS guesses: "))
}

# This matches the blank lines between hosts
/^$/ {
    printf "%s - %s - %s\n", target, mac_address, os_details
    target=""
    mac_address=""
    os_details="Not found"
}

END {
    printf "%s - %s - %s\n", target, mac_address, os_details
}

运行您的样本数据上的这个让您：

192.168.0.1 - MA:CA:DR:ES:S0:03 -  Linux 2.6.32 - 3.13
192.168.0.102 - MA:CA:DR:ES:S0:02 - Not found
192.168.0.104 - MA:CA:DR:ES:S0:01 -  Microsoft Windows Server 2008 SP1 or Windows Server 2008 R2 (91%)

我不得不更正我认为是您的示例数据中的错误...我在此处删除了 MAC Address 行之前的空白行：

Nmap scan report for 192.168.0.104
Host is up (0.00024s latency).
Not shown: 995 filtered ports
PORT     STATE SERVICE
PORT#    STATE XXXXXX

MAC Address: MA:CA:DR:ES:S0:01 (Micro-star Intl)

Answer 2

使用 nmap 的选项 -oX（输出为 XML 格式）解析可能更准确：

nmap -oX /path/to/testscan.xml ...
# or
nmap -oX - ... > /path/to/testscan.xml

然后你可以使用，例如，xmllint 来用 XPath 解析这个 XML：

file="/path/to/testscan.xml"

get_details() {
    local file addr mac os
    file=""
    addr=
    mac=$(xmllint --xpath "string(//address[../address[@addr='$addr']][@addrtype='mac']/@addr)" "$file")
    os=$(xmllint --xpath "string(//os[../address[@addr='$addr']]/osmatch/@name)" "$file")
    : ${mac:="No data"}
    : ${os:="No data"}
    printf "%s - %s - %s\n" "$addr" "$mac" "$os"
}   

for a in $(xmllint --xpath "//address[@addrtype='ipv4']/@addr" "$file" | grep -Po '\d+\.\d+\.\d+\.\d+'); do
    get_details "$file" $a
done

如何将已解析的文本拆分为 "blocks" 文本？

How can I split parsed text by "blocks" of text?

bash

parsing

text

nmap