用于 Ping 大量主机的 Perl 脚本

Question

我有很多主机（大约 40000+）需要使用 perl 脚本测试 ping。要求在 3-4 分钟之前完成此验证，我需要 write/dump ping results/output 到一个平面文件。另一个集成将读取此输出（可能是尾部）并生成一些警报。

我将在一台有 16 个 CPU（1 个套接字，每个套接字 8 个核心，每个核心 2 个线程）的机器上运行这个 perl 脚本。我有 perl 5，版本 14，颠覆 1 (v5.14.1) 我使用了 'nmap' 实用程序，我的时间如下所示：

Nmap 完成：在 356.54 秒~6 分钟内扫描了 50233 个 IP 地址（39313 个主机）

Nmap 完成：在 641.93 秒~11 分钟内扫描了 79195 个 IP 地址（28178 个主机）

我可以使用多线程 perl 进一步加速吗？我安装了 Net::Ping 模块，但我对安装更多自定义模块不感兴趣。

更新##1

根据“Sobrique”的回答和示例代码，我有如下所示的有效代码。非常感谢您提供示例，因为它帮助我了解了 perl 中的 Thread::Queue 模块。

#!/usr/bin/perl

# Run under Oracle Perl, for DBI.

    BEGIN {
        my $ORACLE_HOME = "/opt/oracle/app/oracle/product/12.1.0/client_1";
        $ENV{ORACLE_HOME}=$ORACLE_HOME;

        die "ORACLE_HOME not set\n" unless $ENV{ORACLE_HOME};
        unless ($ENV{OrAcLePeRl}) {
        $ENV{OrAcLePeRl} = "$ENV{ORACLE_HOME}/perl";
        $ENV{PERL5LIB} = 
       "$ENV{PERL5LIB}:$ENV{OrAcLePeRl}/lib:$ENV{OrAcLePeRl}/lib/site_perl";
        $ENV{LD_LIBRARY_PATH} = 
      "$ENV{LD_LIBRARY_PATH}:$ENV{ORACLE_HOME}/lib32:$ENV{ORACLE_HOME}/lib";
        exec "$ENV{OrAcLePeRl}/bin/perl", [=10=], @ARGV;
    }
  }

use strict;
use warnings;
use threads;
use Thread::Queue;

my $process_q = Thread::Queue->new();

#insert tasks into thread queue.
open( my $input_fh, "<", "server_list" ) or die $!;
$process_q->enqueue(<$input_fh>);
close($input_fh);

my $OutputFile = "PingOutput.mout";
open (OUTPUT_FILEH, ">>$OutputFile") or die "Can't Open $OutputFile : $!";

  # Worker threads
  my $thread_limit = 25;

  # send markers.
  $process_q->enqueue(undef) for 1..$thread_limit;

   my @thr = map {
      threads->create(sub {
        while (defined (my $server = $process_q->dequeue())) {
                       chomp($server);
                       print OUTPUT_FILEH threads->self()->tid() . ":pinging$server\n";
                       my $result = `/usr/bin/ping -c 1 $server`;
                       print OUTPUT_FILEH $result;
        }
    });
} 1..$thread_limit;

# terminate.
$_->join() for @thr;

Answer 1

您确实可以使用多线程来进一步加快速度。这是使用工作队列的完美场景：

#!/usr/bin/perl

use strict;
use warnings;

use threads;

use Thread::Queue;

my $nthreads = 5;

my $process_q = Thread::Queue->new();
my $failed_q  = Thread::Queue->new();

#this is a subroutine, but that runs 'as a thread'.
#when it starts, it inherits the program state 'as is'. E.g.
#the variable declarations above all apply - but changes to
#values within the program are 'thread local' unless the
#variable is defined as 'shared'.
#Behind the scenes - Thread::Queue are 'shared' arrays.

sub worker {
    #NB - this will sit a loop indefinitely, until you close the queue.
    #using $process_q -> end
    #we do this once we've queued all the things we want to process
    #and the sub completes and exits neatly.
    #however if you _don't_ end it, this will sit waiting forever.
    while ( my $server = $process_q->dequeue() ) {
        chomp($server);
        print threads->self()->tid() . ": pinging $server\n";
        my $result = `/bin/ping -c 1 $server`;
        if ($?) { $failed_q->enqueue($server) }
        print $result;
    }
}

#insert tasks into thread queue.
open( my $input_fh, "<", "server_list" ) or die $!;
$process_q->enqueue(<$input_fh>);
close($input_fh);

#we 'end' process_q  - when we do, no more items may be inserted,
#and 'dequeue' returns 'undefined' when the queue is emptied.
#this means our worker threads (in their 'while' loop) will then exit.
$process_q->end();

#start some threads
for ( 1 .. $nthreads ) {
    threads->create( \&worker );
}

#Wait for threads to all finish processing.
foreach my $thr ( threads->list() ) {
    $thr->join();
}

#collate results. ('synchronise' operation)
while ( my $server = $failed_q->dequeue_nb() ) {
    print "$server failed to ping\n";
}

(注：我的代码，来自我给出的另一个答案：Perl daemonize with child daemons)

但是 - 我认为您不会发现比 nmap 更快的东西，因为它已经以非常适合网络扫描场景的方式进行并行处理。

用于 Ping 大量主机的 Perl 脚本

Perl Script to Ping lots of hosts

multithreading

perl

ping