Netstat TCP 状态数据记录脚本
Netstat TCP status data logging script
我有一个脚本可以执行 netstat -an 调用以显示两个端口（8080 和 5555）的 TCP 状态。我让它每分钟将它打印到一行中到一个日志文件中。这一切都很好，但由于流量的性质，状态值经常变化。我需要获取这些状态的计数并能够将它们插入 excel 并为每个状态绘制图表。我需要静态数据，这意味着我还需要未显示的状态（等于 0 的计数）。
带排序 | uniq -c 它显然只会获得积极的结果。我的问题是如何为未显示的状态填写空白以便获得完整数据？
这是我的脚本（它在 运行 到下午 2 点的 while 循环中）：
#!/bin/bash
TS=$(date '+%Y-%m-%d %H:%M:%S')
LOG=_$(hostname)_TCP.log
LOGTS=$(date '+%Y%m%d')
HR=$(date '+%H')
while [ "$HR" != "14" ]; do
TS=$(date '+%Y-%m-%d %H:%M:%S')
echo "$(echo $TS) $(printf "Port 8080 ")( $(netstat -an | grep 8080 | awk '{print }' | sort -k1 | uniq -c | awk '{print " " ","}' | xargs)) $(printf "Port 5555 ")( $(netstat -an | grep 5555 | awk '{print }' | sort -k1 | uniq -c | awk '{print " " ","}' | xargs)) " | tee -a $LOGTS$LOG
# sleep 3600
sleep 60
HR=$(date '+%H')
done
echo "Past 14:00 so script is finished"
这是我目前的结果：
2015-08-13 09:55:27 Port 8080 ( ESTABLISHED 7, FIN_WAIT2 1, LISTEN 1,) Port 5555 ( CLOSE_WAIT 1, ESTABLISHED 2,)
2015-08-13 09:56:27 Port 8080 ( ESTABLISHED 1, LISTEN 1,) Port 5555 ( CLOSE_WAIT 1, ESTABLISHED 1,)
如您所见，我可以很好地计算出计数。
但是，如果我将它导入 excel，数据将不统一，我将不得不为无计数填充空白，以便能够绘制图形。
除非有另一种方法或方式可以用 excel?
很好地做到这一点
我的想法可能是使用一个带有 tcp 状态的数组来保持 table 的结果命中率并计算零。这样的想法对吗？
抱歉这么久 post。提前谢谢你。
我制作了一个 bash 脚本来转换您的文件输出：
2015-08-13 09:55:27 Port 8080 ( ESTABLISHED 7, FIN_WAIT2 1, LISTEN 1,) Port 5555 ( CLOSE_WAIT 1, ESTABLISHED 2,)
2015-08-13 09:56:27 Port 8080 ( ESTABLISHED 1, LISTEN 1,) Port 5555 ( CLOSE_WAIT 1, ESTABLISHED 1,)
至：
2015-08-13,09:55:27,8080,7,,,,1,,,,,1,,5555,2,,,,,,,1,,,,0
2015-08-13,09:56:27,8080,1,,,,,,,,,1,,5555,1,,,,,,,1,,,,1
在 linux 系统中为您拥有的每个端口定义所有会话状态：
declare -a arr=("ESTABLISHED" "SYN_SENT" "SYN_RECV" "FIN_WAIT1" "FIN_WAIT2" "TIME_WAIT" "CLOSED" "CLOSE_WAIT" "LAST_ACK" "LISTEN" "CLOSING")
每一行的最后一个数字是行计数变量。
用法：
netstat_format.sh your_output.txt formatted_output.txt
netstat_format.sh的完整代码：
#!/bin/bash
#title :format_netstat.sh
#author :Bertrand Martel
#date :13/08/2015
#declare a list of all session state you may find in linux system
declare -a arr=("ESTABLISHED" "SYN_SENT" "SYN_RECV" "FIN_WAIT1" "FIN_WAIT2" "TIME_WAIT" "CLOSED" "CLOSE_WAIT" "LAST_ACK" "LISTEN" "CLOSING")
IFS=$'\n' #line delimiter
set -f #Disable file name generation (globbing)
count_line=0 #line counter
#empty your output file
cp /dev/null ""
for i in $(cat ""); do
#test="2015-08-13 09:55:27 Port 8080 ( ESTABLISHED 7, FIN_WAIT2 1, LISTEN 1,) Port 5555 ( CLOSE_WAIT 1, ESTABLISHED 2,)"
main_part=$i
new_line=""
#extract first,second and fourth column with ' ' delimiter
date_val=`echo $main_part | cut -d' ' -f1`
time_val=`echo $main_part | cut -d' ' -f2`
port_val=`echo $main_part | cut -d' ' -f4`
#append these fields to new line output var
new_line="$date_val,$time_val,$port_val"
for i in {0..10}
{
#here extract all that is between parenthesis and process it independently with replacing "," with ' ', looking for session state in arr defined in the beginning.
# awk '{print }' => will finally print the second argument eg the value of the key found in arr
result=`echo $main_part | awk -v FS="([(]|[)])" '{print }' | sed 's/,/ /g' | grep -o "${arr[i]} [^ ]*" | awk '{print }'`
if [ -z "$result" ]; then
result=""
fi
new_line="$new_line,$result"
}
#cut all before " Port"
second_part=`echo $main_part | sed 's/.*) Port //'`
#second port in line
port2_val=`echo $second_part | cut -d' ' -f1`
#add port2 value to line output
new_line="$new_line,$port2_val"
for i in {0..10}
{
result=`echo $second_part | awk -v FS="([(]|[)])" '{print }' | sed 's/,/ /g' | grep -o "${arr[i]} [^ ]*" | awk '{print }'`
if [ -z "$result" ]; then
result=""
fi
new_line="$new_line,$result"
}
#############################################
#cut all before " Port"
third_part=`echo $second_part | sed 's/.*) Port //'`
#second port in line
port3_val=`echo $third_part | cut -d' ' -f1`
#add port2 value to line output
new_line="$new_line,$port3_val"
for i in {0..10}
{
result=`echo $third_part | awk -v FS="([(]|[)])" '{print }' | sed 's/,/ /g' | grep -o "${arr[i]} [^ ]*" | awk '{print }'`
if [ -z "$result" ]; then
result=""
fi
new_line="$new_line,$result"
}
############################################
#add line count
new_line="$new_line,$count_line"
#increment line count
count_line=$((count_line+1))
#append content of new line to output file
echo $new_line >> ""
done
cat ""
我创建了一个要点，你可以拿走文件：
https://gist.github.com/bertrandmartel/5f1c0c0c84db44e85ca8#file-netstat_format-sh
然而，它只处理 2 系列的 Port XXX (....) 字符串，如果你希望有更多，你必须稍微修改脚本
我试图修改您的脚本以处理具有 3 个端口结果的字符串输出。
使用 $second_part 和 $third_part 变量我无法使用 sed 完成此操作，因此改用 awk。我没有合适的分隔符，所以我修改了我的原始脚本以包含逗号分隔符，这样 awk 就可以工作了。
这是我为处理 3 个端口字符串而修改的脚本。
再次感谢您花时间编写和注释此 bash 脚本。从中学到了一些东西。 :)
#!/bin/bash
#title :format_netstat.sh
#author :Bertrand Martel
#date :13/08/2015
#declare a list of all session state you may find in linux system
declare -a arr=("ESTABLISHED" "SYN_SENT" "SYN_RECV" "FIN_WAIT1" "FIN_WAIT2" "TIME_WAIT" "CLOSED" "CLOSE_WAIT" "LAST_ACK" "LISTEN" "CLOSING")
IFS=$'\n' #line delimiter
set -f #Disable file name generation (globbing)
count_line=0 #line counter
#empty your output file
cp /dev/null ""
for i in $(cat ""); do
#test="2015-08-13 09:55:27 Port 8080 ( ESTABLISHED 7, FIN_WAIT2 1, LISTEN 1,) Port 5555 ( CLOSE_WAIT 1, ESTABLISHED 2,)"
main_part=$i
new_line=""
#extract first,second and fourth column with ' ' delimiter
date_val=`echo $main_part | cut -d' ' -f1`
time_val=`echo $main_part | cut -d' ' -f2`
port_val=`echo $main_part | cut -d' ' -f4`
#append these fields to new line output var
new_line="$date_val,$time_val,$port_val"
for i in {0..10}
{
#here extract all that is between parenthesis and process it independently with replacing "," with ' ', looking for session state in arr defined in the beginning.
# awk '{print }' => will finally print the second argument eg the value of the key found in arr
result=`echo $main_part | awk -v FS="([(]|[)])" '{print }' | sed 's/,/ /g' | grep -o "${arr[i]} [^ ]*" | awk '{print }'`
if [ -z "$result" ]; then
result="0"
fi
new_line="$new_line,$result"
}
echo $main_part >> main.txt
#cut all before " Port"
# second_part=`echo $main_part | sed 's/.*) Port //g'`
second_part=`echo $main_part | awk -F'.' '{print }'`
echo $second_part >> main.txt
#second port in line
port2_val=`echo $second_part | cut -d' ' -f3`
#add port2 value to line output
new_line="$new_line,$port2_val"
for i in {0..10}
{
result=`echo $second_part | awk -v FS="([(]|[)])" '{print }' | sed 's/,/ /g' | grep -o "${arr[i]} [^ ]*" | awk '{print }'`
if [ -z "$result" ]; then
result="0"
fi
new_line="$new_line,$result"
}
#cut all before " Port"
#third_part=`echo $main_part | sed 's/*) Port //'`
third_part=`echo $main_part | awk -F'.' '{print }'`
#third port in line
port3_val=`echo $third_part | cut -d' ' -f3`
#add port3 value to line output
new_line="$new_line,$port3_val"
for i in {0..10}
{
result=`echo $third_part | awk -v FS="([(]|[)])" '{print }' | sed 's/,/ /g' | grep -o "${arr[i]} [^ ]*" | awk '{print }'`
if [ -z "$result" ]; then
result="0"
fi
new_line="$new_line,$result"
}
#add line count
new_line="$new_line,$count_line"
#increment line count
count_line=$((count_line+1))
#append content of new line to output file
echo $new_line >> ""
done
cat ""
这里是完成的工作脚本，仅供参考：
declare -a arr=("LISTEN" "SYN_SENT" "SYN_RECV" "ESTABLISHED" "FIN-WAIT1" "FIN-WAIT2" "CLOSE_WAIT" "CLOSING" "LAST_ACK" "TIME_WAIT" "CLOSED")
IFS=$'\n' #line delimiter
set -f #Disable file name generation (globbing)
count_line=0 #line counter
#empty your output file
cp /dev/null ""
for i in $(cat ""); do
#test="2015-08-13 09:55:27 Port 8080 ( ESTABLISHED 7, FIN_WAIT2 1, LISTEN 1,) Port 5555 ( CLOSE_WAIT 1, ESTABLISHED 2,)"
main_part=$i
new_line=""
#extract first,second and fourth column with ' ' delimiter
date_val=`echo $main_part | cut -d' ' -f1`
time_val=`echo $main_part | cut -d' ' -f2`
port_val=`echo $main_part | cut -d' ' -f4`
#append these fields to new line output var
new_line="$date_val,$time_val,$port_val"
for i in {0..10}
{
#here extract all that is between parenthesis and process it independently with replacing "," with ' ', looking for session state in arr defined in the beginning.
# awk '{print }' => will finally print the second argument eg the value of the key found in arr
result=`echo $main_part | awk -v FS="([(]|[)])" '{print }' | sed 's/,/ /g' | grep -o "${arr[i]} [^ ]*" | awk '{print }'`
if [ -z "$result" ]; then
result="0"
fi
new_line="$new_line,$result"
}
#cut all before " Port"
second_part=`echo $main_part | sed 's/.*) Port //'`
#second port in line
port2_val=`echo $second_part | cut -d' ' -f1`
#add port2 value to line output
new_line="$new_line,$port2_val"
for i in {0..10}
{
result=`echo $second_part | awk -v FS="([(]|[)])" '{print }' | sed 's/,/ /g' | grep -o "${arr[i]} [^ ]*" | awk '{print }'`
if [ -z "$result" ]; then
result="0"
fi
new_line="$new_line,$result"
}
#add line count
new_line="$new_line,$count_line"
#increment line count
count_line=$((count_line+1))
#append content of new line to output file
echo $new_line >> ""
done
cat ""
我有一个脚本可以执行 netstat -an 调用以显示两个端口（8080 和 5555）的 TCP 状态。我让它每分钟将它打印到一行中到一个日志文件中。这一切都很好，但由于流量的性质，状态值经常变化。我需要获取这些状态的计数并能够将它们插入 excel 并为每个状态绘制图表。我需要静态数据，这意味着我还需要未显示的状态（等于 0 的计数）。 带排序 | uniq -c 它显然只会获得积极的结果。我的问题是如何为未显示的状态填写空白以便获得完整数据？
这是我的脚本（它在 运行 到下午 2 点的 while 循环中）：
#!/bin/bash
TS=$(date '+%Y-%m-%d %H:%M:%S')
LOG=_$(hostname)_TCP.log
LOGTS=$(date '+%Y%m%d')
HR=$(date '+%H')
while [ "$HR" != "14" ]; do
TS=$(date '+%Y-%m-%d %H:%M:%S')
echo "$(echo $TS) $(printf "Port 8080 ")( $(netstat -an | grep 8080 | awk '{print }' | sort -k1 | uniq -c | awk '{print " " ","}' | xargs)) $(printf "Port 5555 ")( $(netstat -an | grep 5555 | awk '{print }' | sort -k1 | uniq -c | awk '{print " " ","}' | xargs)) " | tee -a $LOGTS$LOG
# sleep 3600
sleep 60
HR=$(date '+%H')
done
echo "Past 14:00 so script is finished"
这是我目前的结果：
2015-08-13 09:55:27 Port 8080 ( ESTABLISHED 7, FIN_WAIT2 1, LISTEN 1,) Port 5555 ( CLOSE_WAIT 1, ESTABLISHED 2,)
2015-08-13 09:56:27 Port 8080 ( ESTABLISHED 1, LISTEN 1,) Port 5555 ( CLOSE_WAIT 1, ESTABLISHED 1,)
如您所见，我可以很好地计算出计数。 但是，如果我将它导入 excel，数据将不统一，我将不得不为无计数填充空白，以便能够绘制图形。 除非有另一种方法或方式可以用 excel?
很好地做到这一点我的想法可能是使用一个带有 tcp 状态的数组来保持 table 的结果命中率并计算零。这样的想法对吗？
抱歉这么久 post。提前谢谢你。
我制作了一个 bash 脚本来转换您的文件输出：
2015-08-13 09:55:27 Port 8080 ( ESTABLISHED 7, FIN_WAIT2 1, LISTEN 1,) Port 5555 ( CLOSE_WAIT 1, ESTABLISHED 2,)
2015-08-13 09:56:27 Port 8080 ( ESTABLISHED 1, LISTEN 1,) Port 5555 ( CLOSE_WAIT 1, ESTABLISHED 1,)
至：
2015-08-13,09:55:27,8080,7,,,,1,,,,,1,,5555,2,,,,,,,1,,,,0
2015-08-13,09:56:27,8080,1,,,,,,,,,1,,5555,1,,,,,,,1,,,,1
在 linux 系统中为您拥有的每个端口定义所有会话状态：
declare -a arr=("ESTABLISHED" "SYN_SENT" "SYN_RECV" "FIN_WAIT1" "FIN_WAIT2" "TIME_WAIT" "CLOSED" "CLOSE_WAIT" "LAST_ACK" "LISTEN" "CLOSING")
每一行的最后一个数字是行计数变量。
用法：
netstat_format.sh your_output.txt formatted_output.txt
netstat_format.sh的完整代码：
#!/bin/bash
#title :format_netstat.sh
#author :Bertrand Martel
#date :13/08/2015
#declare a list of all session state you may find in linux system
declare -a arr=("ESTABLISHED" "SYN_SENT" "SYN_RECV" "FIN_WAIT1" "FIN_WAIT2" "TIME_WAIT" "CLOSED" "CLOSE_WAIT" "LAST_ACK" "LISTEN" "CLOSING")
IFS=$'\n' #line delimiter
set -f #Disable file name generation (globbing)
count_line=0 #line counter
#empty your output file
cp /dev/null ""
for i in $(cat ""); do
#test="2015-08-13 09:55:27 Port 8080 ( ESTABLISHED 7, FIN_WAIT2 1, LISTEN 1,) Port 5555 ( CLOSE_WAIT 1, ESTABLISHED 2,)"
main_part=$i
new_line=""
#extract first,second and fourth column with ' ' delimiter
date_val=`echo $main_part | cut -d' ' -f1`
time_val=`echo $main_part | cut -d' ' -f2`
port_val=`echo $main_part | cut -d' ' -f4`
#append these fields to new line output var
new_line="$date_val,$time_val,$port_val"
for i in {0..10}
{
#here extract all that is between parenthesis and process it independently with replacing "," with ' ', looking for session state in arr defined in the beginning.
# awk '{print }' => will finally print the second argument eg the value of the key found in arr
result=`echo $main_part | awk -v FS="([(]|[)])" '{print }' | sed 's/,/ /g' | grep -o "${arr[i]} [^ ]*" | awk '{print }'`
if [ -z "$result" ]; then
result=""
fi
new_line="$new_line,$result"
}
#cut all before " Port"
second_part=`echo $main_part | sed 's/.*) Port //'`
#second port in line
port2_val=`echo $second_part | cut -d' ' -f1`
#add port2 value to line output
new_line="$new_line,$port2_val"
for i in {0..10}
{
result=`echo $second_part | awk -v FS="([(]|[)])" '{print }' | sed 's/,/ /g' | grep -o "${arr[i]} [^ ]*" | awk '{print }'`
if [ -z "$result" ]; then
result=""
fi
new_line="$new_line,$result"
}
#############################################
#cut all before " Port"
third_part=`echo $second_part | sed 's/.*) Port //'`
#second port in line
port3_val=`echo $third_part | cut -d' ' -f1`
#add port2 value to line output
new_line="$new_line,$port3_val"
for i in {0..10}
{
result=`echo $third_part | awk -v FS="([(]|[)])" '{print }' | sed 's/,/ /g' | grep -o "${arr[i]} [^ ]*" | awk '{print }'`
if [ -z "$result" ]; then
result=""
fi
new_line="$new_line,$result"
}
############################################
#add line count
new_line="$new_line,$count_line"
#increment line count
count_line=$((count_line+1))
#append content of new line to output file
echo $new_line >> ""
done
cat ""
我创建了一个要点，你可以拿走文件：
https://gist.github.com/bertrandmartel/5f1c0c0c84db44e85ca8#file-netstat_format-sh
然而，它只处理 2 系列的 Port XXX (....) 字符串，如果你希望有更多，你必须稍微修改脚本
我试图修改您的脚本以处理具有 3 个端口结果的字符串输出。
使用 $second_part 和 $third_part 变量我无法使用 sed 完成此操作，因此改用 awk。我没有合适的分隔符，所以我修改了我的原始脚本以包含逗号分隔符，这样 awk 就可以工作了。
这是我为处理 3 个端口字符串而修改的脚本。
再次感谢您花时间编写和注释此 bash 脚本。从中学到了一些东西。 :)
#!/bin/bash
#title :format_netstat.sh
#author :Bertrand Martel
#date :13/08/2015
#declare a list of all session state you may find in linux system
declare -a arr=("ESTABLISHED" "SYN_SENT" "SYN_RECV" "FIN_WAIT1" "FIN_WAIT2" "TIME_WAIT" "CLOSED" "CLOSE_WAIT" "LAST_ACK" "LISTEN" "CLOSING")
IFS=$'\n' #line delimiter
set -f #Disable file name generation (globbing)
count_line=0 #line counter
#empty your output file
cp /dev/null ""
for i in $(cat ""); do
#test="2015-08-13 09:55:27 Port 8080 ( ESTABLISHED 7, FIN_WAIT2 1, LISTEN 1,) Port 5555 ( CLOSE_WAIT 1, ESTABLISHED 2,)"
main_part=$i
new_line=""
#extract first,second and fourth column with ' ' delimiter
date_val=`echo $main_part | cut -d' ' -f1`
time_val=`echo $main_part | cut -d' ' -f2`
port_val=`echo $main_part | cut -d' ' -f4`
#append these fields to new line output var
new_line="$date_val,$time_val,$port_val"
for i in {0..10}
{
#here extract all that is between parenthesis and process it independently with replacing "," with ' ', looking for session state in arr defined in the beginning.
# awk '{print }' => will finally print the second argument eg the value of the key found in arr
result=`echo $main_part | awk -v FS="([(]|[)])" '{print }' | sed 's/,/ /g' | grep -o "${arr[i]} [^ ]*" | awk '{print }'`
if [ -z "$result" ]; then
result="0"
fi
new_line="$new_line,$result"
}
echo $main_part >> main.txt
#cut all before " Port"
# second_part=`echo $main_part | sed 's/.*) Port //g'`
second_part=`echo $main_part | awk -F'.' '{print }'`
echo $second_part >> main.txt
#second port in line
port2_val=`echo $second_part | cut -d' ' -f3`
#add port2 value to line output
new_line="$new_line,$port2_val"
for i in {0..10}
{
result=`echo $second_part | awk -v FS="([(]|[)])" '{print }' | sed 's/,/ /g' | grep -o "${arr[i]} [^ ]*" | awk '{print }'`
if [ -z "$result" ]; then
result="0"
fi
new_line="$new_line,$result"
}
#cut all before " Port"
#third_part=`echo $main_part | sed 's/*) Port //'`
third_part=`echo $main_part | awk -F'.' '{print }'`
#third port in line
port3_val=`echo $third_part | cut -d' ' -f3`
#add port3 value to line output
new_line="$new_line,$port3_val"
for i in {0..10}
{
result=`echo $third_part | awk -v FS="([(]|[)])" '{print }' | sed 's/,/ /g' | grep -o "${arr[i]} [^ ]*" | awk '{print }'`
if [ -z "$result" ]; then
result="0"
fi
new_line="$new_line,$result"
}
#add line count
new_line="$new_line,$count_line"
#increment line count
count_line=$((count_line+1))
#append content of new line to output file
echo $new_line >> ""
done
cat ""
这里是完成的工作脚本，仅供参考：
declare -a arr=("LISTEN" "SYN_SENT" "SYN_RECV" "ESTABLISHED" "FIN-WAIT1" "FIN-WAIT2" "CLOSE_WAIT" "CLOSING" "LAST_ACK" "TIME_WAIT" "CLOSED")
IFS=$'\n' #line delimiter
set -f #Disable file name generation (globbing)
count_line=0 #line counter
#empty your output file
cp /dev/null ""
for i in $(cat ""); do
#test="2015-08-13 09:55:27 Port 8080 ( ESTABLISHED 7, FIN_WAIT2 1, LISTEN 1,) Port 5555 ( CLOSE_WAIT 1, ESTABLISHED 2,)"
main_part=$i
new_line=""
#extract first,second and fourth column with ' ' delimiter
date_val=`echo $main_part | cut -d' ' -f1`
time_val=`echo $main_part | cut -d' ' -f2`
port_val=`echo $main_part | cut -d' ' -f4`
#append these fields to new line output var
new_line="$date_val,$time_val,$port_val"
for i in {0..10}
{
#here extract all that is between parenthesis and process it independently with replacing "," with ' ', looking for session state in arr defined in the beginning.
# awk '{print }' => will finally print the second argument eg the value of the key found in arr
result=`echo $main_part | awk -v FS="([(]|[)])" '{print }' | sed 's/,/ /g' | grep -o "${arr[i]} [^ ]*" | awk '{print }'`
if [ -z "$result" ]; then
result="0"
fi
new_line="$new_line,$result"
}
#cut all before " Port"
second_part=`echo $main_part | sed 's/.*) Port //'`
#second port in line
port2_val=`echo $second_part | cut -d' ' -f1`
#add port2 value to line output
new_line="$new_line,$port2_val"
for i in {0..10}
{
result=`echo $second_part | awk -v FS="([(]|[)])" '{print }' | sed 's/,/ /g' | grep -o "${arr[i]} [^ ]*" | awk '{print }'`
if [ -z "$result" ]; then
result="0"
fi
new_line="$new_line,$result"
}
#add line count
new_line="$new_line,$count_line"
#increment line count
count_line=$((count_line+1))
#append content of new line to output file
echo $new_line >> ""
done
cat ""