将 Log::Log4perl 与 MCE（众核引擎）Perl 一起使用

Question

我正在使用 MCE 对我的一个项目进行并行处理。我在记录 MCE 的输出时遇到问题（为了记录，我使用的是 Log4perl）。

我查看了提供的示例，发现它们打印到 STDOUT/STDERR 或当时提供的某些日志文件。有 MCE->sendto() 和 MCE->print() 选项可用，但我不确定如何将它与 log4perl 一起使用。

package ABC;
use strict;
use warnings;
use MCE;

sub new {
    my $class = shift;
    my ($self) = {@_};
    return bless $self, $class;
}

sub initialize_mce {
     my $mce = MCE->new(
        max_workers => 5,
        input_data => \@input_data,
        on_post_exit => \&on_post_exit,
        user_begin => \&user_begin,
        user_end   => \&user_end,
        user_func => \&run_function
    );

}
sub on_post_exit {
    my ($self, $e) = @_;
    # I want to write something like this -
    # I want to log to my global log file using Log4perl which is initialized in some other package and passed to this package

    # $self->{'logger'}->info("$e->{wid}: $e->{pid}: $e->{status}: $e->{msg}: $e->{id}");

    print "$e->{wid}: $e->{pid}: $e->{status}: $e->{msg}: $e->{id}\n";
}

sub user_begin {                   ## Called once at the beginning
    my ($self, $e) = @_;
    print "$$ start";
}

sub user_end {                     ## Called once at the end
    my $self = shift;
    #$self->{'logger'}->info("$$ end");
    print "$$ end";
}

sub run_function {
    my ($self) = @_;
    my $wid = MCE->wid;
    $self->{'logger'}->info("Running...$wid");
    my $input_data = $_;
    ###
    #Rest of subroutine 
    ####
}
1;

Answer 1

我找到了解决办法。如果有人也遇到这个问题，他们可以试试这个-

您必须初始化 'user_output' 和 'user_error'。来自 MCE 文档 -

    user_error => \&user_error,         ## Default undef
    user_output => \&user_output,       ## Default undef

    # MCE will forward data to user_error/user_output,
    # when defined, for the following methods.

    # MCE->sendto(\*STDERR, "sent to user_error\n");
    # MCE->printf(\*STDERR, "%s\n", "sent to user_error");
    # MCE->print(\*STDERR, "sent to user_error\n");
    # MCE->say(\*STDERR, "sent to user_error");

    # MCE->sendto(\*STDOUT, "sent to user_output\n");
    # MCE->printf("%s\n", "sent to user_output");
    # MCE->print("sent to user_output\n");
    # MCE->say("sent to user_output");

所以就我而言 -

sub initialize_mce {
     my $mce = MCE->new(
        max_workers => 5,
        input_data => \@input_data,
        on_post_exit => \&on_post_exit,
        user_begin => \&user_begin,
        user_end   => \&user_end,
        user_func => \&run_function,
        user_output => sub {
            $logger->info($_[0]);            # Log4perl Obj
        },
        user_error => sub {
            $logger->error($_[0]);          # Log4perl Obj
        }
    );
}

而在其他功能中，您只需 'print' 到 STDOUT/STDERR 例如-

sub user_begin {                   ## Called once at the beginning
    my ($self, $e) = @_;
    $mce->print("Process Id : ".$$." start");
    # OR to log error 
    $mce->print(\*STDERR, "Exception/Error Found");
}

我不确定这是唯一的方法还是有其他有效的方法，但现在它完成了工作。

将 Log::Log4perl 与 MCE（众核引擎）Perl 一起使用

Using Log::Log4perl with MCE (Many-core Engine) Perl

parallel-processing

perl

multicore

log4perl