Perl6多线程突发问题
Perl6 multithreaded spurt problems
我编写的程序生成 "random" 文本文件，其中 3 个单词替换为存储在 $keysfilename 文件中的密钥。密钥文件结构非常简单，如
ASD123ASD
QWE123QWE
XZC123ZXC
当我使用多个线程时出现问题，例如
my @threads = (^32).map({
如果在任意文件上失败，错误
started
Thread<17>(14) got 1
Thread<18>(15) got 2
Thread<20>(17) got 17
Thread<5>(2) got 3
Thread<16>(13) got 4
Thread<21>(18) got 5
Thread<3>(0) got 6
Thread<8>(5) got 7
Thread<12>(9) got 10
Thread<11>(8) got 8
Thread<9>(6) got 9
Thread<14>(11) got 11
Thread<15>(12) got 12
Unhandled exception: Failed to open file C:\c\perltests[=12=]000017.txt: no such file or directory
Thread<10>(7) got 13
Thread<13>(10) got 14
Thread<7>(4) got 15
Thread<19>(16) got 16
Thread<4>(1) got 0
Thread<6>(3) got 18
Thread<22>(19) got 19
at <unknown>:1 (C:\rakudo/share/perl6/runtime/CORE.setting.moarvm:throw:4294967295)
Thread<23>(20) got 20
Thread<24>(21) got 21
from gen/moar/m-CORE.setting:22337 (C:\rakudo/share/perl6/runtime/CORE.setting.moarvm:throw:34)
Thread<26>(23) got 22
Thread<25>(22) got 23
Thread<27>(24) got 24
from gen/moar/m-CORE.setting:22374 (C:\rakudo/share/perl6/runtime/CORE.setting.moarvm:FALLBACK:35)
Thread<28>(25) got 25
Thread<29>(26) got 26
from gen/moar/m-Metamodel.nqp:3041 (C:\rakudo\share\nqp\lib/Perl6/Metamodel.moarvm::13)
Thread<30>(27) got 27
Thread<16>(13) got 28
Thread<17>(14) got 29
Thread<5>(2) got 30
Thread<18>(15) got 31
Thread<14>(11) got 32
Thread<15>(12) got 33
from gen/moar/m-CORE.setting:25189 (C:\rakudo/share/perl6/runtime/CORE.setting.moarvm:spurt:226)
Thread<30>(27) got 58
Thread<29>(26) got 57
Unhandled exception: Failed to open file C:\c\perltests[=12=]000028.txt: no such file or directory
Thread<28>(25) got 56
from gen/moar/m-CORE.setting:25203 (C:\rakudo/share/perl6/runtime/CORE.setting.moarvm:spurt:92)
Thread<25>(22) got 55
from gen/moar/m-CORE.setting:25199 (C:\rakudo/share/perl6/runtime/CORE.setting.moarvm:spurt:44)
Thread<27>(24) got 54
from gen/moar/m-CORE.setting:25506 (C:\rakudo/share/perl6/runtime/CORE.setting.moarvm:spurt:96)
Thread<26>(23) got 53
from gentexts.pl:54 (<ephemeral file>::189)
Thread<24>(21) got 52
Thread<23>(20) got 51
Unhandled exception: Failed to open file C:\c\perltests[=12=]000058.txt: no such file or directory
Thread<6>(3) got 50
Thread<22>(19) got 49
at <unknown>:1 (C:\rakudo/share/perl6/runtime/CORE.setting.moarvm:throw:4294967295)
Thread<34>(31) got 48
from gen/moar/m-CORE.setting:22337 (C:\rakudo/share/perl6/runtime/CORE.setting.moarvm:throw:34)
Thread<33>(30) got 47
Thread<4>(1) got 46
from gen/moar/m-CORE.setting:22374 (C:\rakudo/share/perl6/runtime/CORE.setting.moarvm:FALLBACK:35)
Thread<7>(4) got 45
from gen/moar/m-Metamodel.nqp:3041 (C:\rakudo\share\nqp\lib/Perl6/Metamodel.moarvm::13)
Thread<19>(16) got 44
Thread<11>(8) got 43
at <unknown>:1 (C:\rakudo/share/perl6/runtime/CORE.setting.moarvm:throw:4294967295)
Thread<8>(5) got 42
from gen/moar/m-CORE.setting:22337 (C:\rakudo/share/perl6/runtime/CORE.setting.moarvm:throw:34)
Thread<12>(9) got 41
Thread<10>(7) got 40
from gen/moar/m-CORE.setting:25189 (C:\rakudo/share/perl6/runtime/CORE.setting.moarvm:spurt:226)
Thread<13>(10) got 39
from gen/moar/m-CORE.setting:25203 (C:\rakudo/share/perl6/runtime/CORE.setting.moarvm:spurt:92)
Thread<31>(28) got 37
from gen/moar/m-CORE.setting:25199 (C:\rakudo/share/perl6/runtime/CORE.setting.moarvm:spurt:44)
Thread<32>(29) got 38
Thread<9>(6) got 36
from gentexts.pl:44 (<ephemeral file>::15)
Thread<3>(0) got 35
Thread<21>(18) got 34
from gen/moar/m-CORE.setting:22374 (C:\rakudo/share/perl6/runtime/CORE.setting.moarvm:FALLBACK:35)
Thread<17>(14) got 59
from gen/moar/m-CORE.setting:25506 (C:\rakudo/share/perl6/runtime/CORE.setting.moarvm:spurt:96)
Thread<5>(2) got 60
Thread<18>(15) got 61
from gentexts.pl:54 (<ephemeral file>::189)
Thread<9>(6) got 85
Thread<32>(29) got 84
from gen/moar/m-CORE.setting:30638 (C:\rakudo/share/perl6/runtime/CORE.setting.moarvm:THREAD-ENTRY:44)
系统 Win 10 x32，Rakudo 6.c
my $keysfilename := 'C:/c/keysfile.txt';
my $output := 'C:/c/perltests';
my @keys = ();
for $keysfilename.IO.words {
@keys.push($_);
}
my $len := elems @keys;
my $lorem = q:to/END/;
Lorem ipsum dolor sit amet, consectetur adipiscing elit. Donec malesuada purus vel tincidunt eleifend. Fusce sollicitudin augue augue, et gravida dolor varius a. Vestibulum iaculis, dui iaculis iaculis molestie, tellus ante hendrerit massa, at volutpat risus metus vitae nisi. Integer neque magna, ultrices eu erat at, efficitur sollicitudin sem. Aliquam sed purus malesuada, porta est eu, rutrum neque. Quisque dolor leo, condimentum non mollis eget, tristique eget odio. Donec dignissim magna nec imperdiet iaculis. Vestibulum lorem ligula, euismod ac porttitor faucibus, rutrum eu ex.
Donec scelerisque nisi eget risus condimentum ultrices. Integer porta maximus quam, in lobortis quam fermentum eu. Morbi eu ligula consequat, aliquam sem eget, sollicitudin eros. Suspendisse potenti. Cras finibus metus et eros accumsan, id vehicula libero lobortis. Aenean vulputate lacinia urna at fringilla. Nulla id tincidunt lectus, quis accumsan lorem. In posuere magna non purus hendrerit rutrum. Maecenas in mattis tellus. Maecenas vel feugiat enim. Lorem ipsum dolor sit amet, consectetur adipiscing elit. Proin convallis dapibus tellus vitae euismod. Nam eleifend dui quam, eget lobortis quam pulvinar id. Cras euismod posuere dolor non ultricies.
Ut dapibus porta faucibus. Duis velit ante, tincidunt id velit id, imperdiet egestas velit. Morbi efficitur enim dignissim interdum egestas. Vivamus eu urna condimentum, aliquam orci non, ullamcorper est. Phasellus egestas at tellus nec tristique. Fusce feugiat commodo faucibus. In hac habitasse platea dictumst. Quisque dignissim, mauris a pellentesque dictum, mauris velit tincidunt lorem, sed tincidunt libero enim vitae orci. Nam interdum, ante nec consequat vulputate, nisi turpis euismod nibh, sit amet elementum nunc diam non eros. Proin quis viverra risus. Vestibulum vestibulum diam in velit consectetur, eu elementum lacus sagittis. Morbi accumsan ac ante eget faucibus. In nec elit bibendum, tristique enim non, sodales ex. Donec sodales erat vitae odio cursus commodo.
Vestibulum felis lacus, mattis eget porta eget, mattis ut felis. Pellentesque aliquet, purus eu semper suscipit, sem ipsum euismod nunc, sed dapibus augue sem vel elit. Etiam tincidunt arcu ut nisi tempor commodo. Mauris at eros tincidunt, fringilla erat nec, sagittis ante. Integer et malesuada quam. Cras vel porta erat, sit amet efficitur erat. Praesent blandit purus quis urna consectetur, eget ultricies ipsum pulvinar. Phasellus ac molestie elit. Vestibulum et tincidunt velit. Aliquam a venenatis ipsum, nec commodo libero. Nullam eget consectetur lectus. Morbi placerat interdum erat nec interdum.
Morbi bibendum dui eu turpis pretium, eget aliquet augue aliquam. Aliquam eu dignissim mauris, vitae placerat augue. Ut sed tortor sit amet augue imperdiet rutrum. Aliquam erat volutpat. Morbi a turpis in sapien ultrices tristique. Proin quis vestibulum lorem, ut pharetra ex. Quisque tempor bibendum purus ac vehicula. Suspendisse tellus ipsum, imperdiet id sodales vel, congue a leo. Nulla gravida tincidunt nisi eu tempor. Mauris imperdiet tempor ante eget rutrum. Nam ut dui at augue laoreet mollis. Sed metus elit, viverra ac fringilla vel, fermentum et magna. Nam ligula purus, pretium vel dignissim vitae, fermentum at urna. Nullam ac ullamcorper felis. Maecenas dapibus consequat mi.
END
my @words = $lorem.split(' ');
my $wordlen = @words.elems;
my &getNext = sub {
my $counter = 0;
my Lock $lock .= new;
return sub (@filename) {
$lock.lock;
if ($counter < 100_000) {
@filename[0] = $counter;
$counter++;
}
$lock.unlock;
};
}();
say "started";
my @threads = (^1).map({
Thread.start(
name => $_,
sub {
loop {
my @counter = (-1);
getNext(@counter);
last if @counter[0] < 0;
say $*THREAD ~ " got " ~ @counter[0];
my @copy = @words.clone;
for (^3) {
my $pos = $wordlen.rand.round;
@copy[$pos] = @keys[$len.rand.round];
}
spurt sprintf($output ~ '/%08d.txt', @counter[0]), @copy.join(' ');
}
}
);
}).join;
几乎没有理由使用 Threads directly in Perl 6, there are plenty of features that do the hard work for you. In most cases the simplest improvement is to just use start instead of Thread.start, and await LIST 而不是 LIST».join 或 LIST.map(*.join)。
您正在呼叫 List.join not Thread.join，这可能至少是您遇到问题的部分原因。
还有很多东西使本文难以阅读和理解。
根据你的写作方式 getNext 看来你只知道或大部分只知道 JavaScript。我就是这样写的。
sub get-next (@filename) {
state Int $counter = 0; # set to zero only the first time it is called
state Lock $lock .= new;
$lock.protect: ->{
if $counter < 100_000 {
@filename[0] = $counter++;
}
}
}
除了你只使用 @filename 来绕过参数的默认只读语义。
sub get-next ($filename is rw) {
state Int $counter = 0;
state Lock $lock .= new;
$lock.protect: ->{
if $counter < 100_000 {
$filename = $counter++;
}
}
}
只是 return 值更有意义。
sub get-next () {
state Int $counter = 0;
state Lock $lock .= new;
$lock.protect: ->{
$counter++ if $counter < 100_000
} // Nil
# the `if` will return `Empty` when $counter gets too big
# but we want `Nil` instead
}
这可以是 return 计数器或 Nil 这是一个未定义的值，因此您可以像这样使用它：
loop {
my $counter = get-next() orelse last;
…
}
因为你只在循环中使用 sub，所以它可以调用 last 本身。
或者更好的是，如果您改为将声明计数器的行更改为
state Int $counter = 0 but True;
然后您可以将 loop 更改为 while 循环，无需在任何地方添加 last。
while get-next() -> $counter { … }
既然我已经向您展示了如何改进 getNext，我打算放弃它并使用 Perl 6 更好的功能。
（顺便说一句，使用 Channel 会更有意义）
# will probably still work with `use v6.c;`
# but v6.d has a better system backing `await`
use v6.d.PREVIEW;
# override the original default number of threads
# (16 threads currently)
BEGIN %*ENV<RAKUDO_MAX_THREADS> //= 32;
# the "correct" way to do this is setting $*SCHEDULER
# but this is easier
my $keys-filename = 'C:/c/keysfile.txt';
my $output-dir = 'C:/c/perltests';
my @keys = $keys-filename.IO.words;
my @lorem = q:to/EOF/.split(' ');
…
EOF
say 'started';
END say 'finished';
for race ^100_000 -> $counter {
say $*THREAD, " got ", $counter;
my @copy = @lorem; # no need for .clone
for (^+@copy).pick(3) -> $pos {
@copy[$pos] = @keys.roll;
}
spurt $*SPEC.catfile($output-dir, $counter.fmt('/%08d.txt')), @copy.join(' ');
}
( 我 运行 一个测试，它从来没有给出高于 6 的线程 id )
我使用 race rather than hyper 因为 return 值无论如何都会被丢弃。
如果它不起作用，则说明您的 Rakudo 版本有问题。如果是这样，我建议更新到最新版本 v2017.02
(say $*PERL.compiler.version;)
+@array 等同于 @array.elems
^ NUMBER 与 0 ..^ NUMBER 相同，后者是
的糖分
Range.new( 0, NUMBER, :excludes-max )
这个还是有一些“单词”有多余字符的问题
amet, elit. 和 ex.Donec 例如
…
my @pos = ($lorem ~~ m:ex/ « \w+: /).map: { .from, .chars }
# @pos looks something like [(0,5),(6,5),(12,5),(18,3),…]
…
for race ^100_000 -> $counter {
say $*THREAD, " got ", $counter;
# vvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvv
my $copy = $lorem;
# sort so that the transforms are done from the end of the string
# towards the beginning of the string
for @pos.pick(3).sort.reverse {
$copy.substr-rw( |$_ ) = @keys.roll;
}
# ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
spurt $*SPEC.catfile($output-dir, $counter.fmt('/%08d.txt')), $copy;
}
一些注意事项。
这个：
my @keys = ();
for $keysfilename.IO.words {
@keys.push($_);
}
真的只是这样：my @keys = $keysfilename.IO.words;
还有这个：@keys[$len.rand.round]
真的只是这样：@keys.pick
我不太确定你用计数器数组做什么，看来
就像你只想要文件 0..99999.
您正在为每个线程在子例程内进行锁定。如果你
想要防止多个线程访问某些东西，锁一个
外面，让他们每个人都用它来锁定独占密码。
您也在使用 Thread 制作线程，但这确实是
相当低级的界面。尝试使用更高级别的构造
如果可能的话，在下降到那个水平之前。
在这种情况下，您似乎只想拆分 100,000 个文件
创建多个线程只是为了获得一些并行性。那里
有一些构造可以使这变得非常容易，hyper 和
race。因为你甚至不关心文件的顺序
创建于，你可以只使用 race 它会尽可能快地产生它们
尽可能。
你可以用'degree'参数来race表示有多少
你想使用的线程。 （您可能需要将 RAKUDO_MAX_THREADS 设置为
得到你想要的行为）。
my $keysfilename := 'C:/c/keysfile.txt';
my $output := 'C:/c/perltests';
my @keys = $keysfilename.IO.words;
my $lorem = q:to/END/;
Lorem ipsum ...
END
my @words = $lorem.split(' ');
my $wordlen = @words.elems;
[^100000].race(degree => 32).map({
# say $*THREAD ~ " got " ~ $_;
my @copy = @words;
for (^3) {
my $pos = $wordlen.rand.round;
@copy[$pos] = @keys.pick;
}
spurt sprintf($output ~ '/%08d.txt', $_), @copy.join(' ');
});
我编写的程序生成 "random" 文本文件，其中 3 个单词替换为存储在 $keysfilename 文件中的密钥。密钥文件结构非常简单，如
ASD123ASD
QWE123QWE
XZC123ZXC
当我使用多个线程时出现问题，例如
my @threads = (^32).map({
如果在任意文件上失败，错误
started
Thread<17>(14) got 1
Thread<18>(15) got 2
Thread<20>(17) got 17
Thread<5>(2) got 3
Thread<16>(13) got 4
Thread<21>(18) got 5
Thread<3>(0) got 6
Thread<8>(5) got 7
Thread<12>(9) got 10
Thread<11>(8) got 8
Thread<9>(6) got 9
Thread<14>(11) got 11
Thread<15>(12) got 12
Unhandled exception: Failed to open file C:\c\perltests[=12=]000017.txt: no such file or directory
Thread<10>(7) got 13
Thread<13>(10) got 14
Thread<7>(4) got 15
Thread<19>(16) got 16
Thread<4>(1) got 0
Thread<6>(3) got 18
Thread<22>(19) got 19
at <unknown>:1 (C:\rakudo/share/perl6/runtime/CORE.setting.moarvm:throw:4294967295)
Thread<23>(20) got 20
Thread<24>(21) got 21
from gen/moar/m-CORE.setting:22337 (C:\rakudo/share/perl6/runtime/CORE.setting.moarvm:throw:34)
Thread<26>(23) got 22
Thread<25>(22) got 23
Thread<27>(24) got 24
from gen/moar/m-CORE.setting:22374 (C:\rakudo/share/perl6/runtime/CORE.setting.moarvm:FALLBACK:35)
Thread<28>(25) got 25
Thread<29>(26) got 26
from gen/moar/m-Metamodel.nqp:3041 (C:\rakudo\share\nqp\lib/Perl6/Metamodel.moarvm::13)
Thread<30>(27) got 27
Thread<16>(13) got 28
Thread<17>(14) got 29
Thread<5>(2) got 30
Thread<18>(15) got 31
Thread<14>(11) got 32
Thread<15>(12) got 33
from gen/moar/m-CORE.setting:25189 (C:\rakudo/share/perl6/runtime/CORE.setting.moarvm:spurt:226)
Thread<30>(27) got 58
Thread<29>(26) got 57
Unhandled exception: Failed to open file C:\c\perltests[=12=]000028.txt: no such file or directory
Thread<28>(25) got 56
from gen/moar/m-CORE.setting:25203 (C:\rakudo/share/perl6/runtime/CORE.setting.moarvm:spurt:92)
Thread<25>(22) got 55
from gen/moar/m-CORE.setting:25199 (C:\rakudo/share/perl6/runtime/CORE.setting.moarvm:spurt:44)
Thread<27>(24) got 54
from gen/moar/m-CORE.setting:25506 (C:\rakudo/share/perl6/runtime/CORE.setting.moarvm:spurt:96)
Thread<26>(23) got 53
from gentexts.pl:54 (<ephemeral file>::189)
Thread<24>(21) got 52
Thread<23>(20) got 51
Unhandled exception: Failed to open file C:\c\perltests[=12=]000058.txt: no such file or directory
Thread<6>(3) got 50
Thread<22>(19) got 49
at <unknown>:1 (C:\rakudo/share/perl6/runtime/CORE.setting.moarvm:throw:4294967295)
Thread<34>(31) got 48
from gen/moar/m-CORE.setting:22337 (C:\rakudo/share/perl6/runtime/CORE.setting.moarvm:throw:34)
Thread<33>(30) got 47
Thread<4>(1) got 46
from gen/moar/m-CORE.setting:22374 (C:\rakudo/share/perl6/runtime/CORE.setting.moarvm:FALLBACK:35)
Thread<7>(4) got 45
from gen/moar/m-Metamodel.nqp:3041 (C:\rakudo\share\nqp\lib/Perl6/Metamodel.moarvm::13)
Thread<19>(16) got 44
Thread<11>(8) got 43
at <unknown>:1 (C:\rakudo/share/perl6/runtime/CORE.setting.moarvm:throw:4294967295)
Thread<8>(5) got 42
from gen/moar/m-CORE.setting:22337 (C:\rakudo/share/perl6/runtime/CORE.setting.moarvm:throw:34)
Thread<12>(9) got 41
Thread<10>(7) got 40
from gen/moar/m-CORE.setting:25189 (C:\rakudo/share/perl6/runtime/CORE.setting.moarvm:spurt:226)
Thread<13>(10) got 39
from gen/moar/m-CORE.setting:25203 (C:\rakudo/share/perl6/runtime/CORE.setting.moarvm:spurt:92)
Thread<31>(28) got 37
from gen/moar/m-CORE.setting:25199 (C:\rakudo/share/perl6/runtime/CORE.setting.moarvm:spurt:44)
Thread<32>(29) got 38
Thread<9>(6) got 36
from gentexts.pl:44 (<ephemeral file>::15)
Thread<3>(0) got 35
Thread<21>(18) got 34
from gen/moar/m-CORE.setting:22374 (C:\rakudo/share/perl6/runtime/CORE.setting.moarvm:FALLBACK:35)
Thread<17>(14) got 59
from gen/moar/m-CORE.setting:25506 (C:\rakudo/share/perl6/runtime/CORE.setting.moarvm:spurt:96)
Thread<5>(2) got 60
Thread<18>(15) got 61
from gentexts.pl:54 (<ephemeral file>::189)
Thread<9>(6) got 85
Thread<32>(29) got 84
from gen/moar/m-CORE.setting:30638 (C:\rakudo/share/perl6/runtime/CORE.setting.moarvm:THREAD-ENTRY:44)
系统 Win 10 x32，Rakudo 6.c
my $keysfilename := 'C:/c/keysfile.txt';
my $output := 'C:/c/perltests';
my @keys = ();
for $keysfilename.IO.words {
@keys.push($_);
}
my $len := elems @keys;
my $lorem = q:to/END/;
Lorem ipsum dolor sit amet, consectetur adipiscing elit. Donec malesuada purus vel tincidunt eleifend. Fusce sollicitudin augue augue, et gravida dolor varius a. Vestibulum iaculis, dui iaculis iaculis molestie, tellus ante hendrerit massa, at volutpat risus metus vitae nisi. Integer neque magna, ultrices eu erat at, efficitur sollicitudin sem. Aliquam sed purus malesuada, porta est eu, rutrum neque. Quisque dolor leo, condimentum non mollis eget, tristique eget odio. Donec dignissim magna nec imperdiet iaculis. Vestibulum lorem ligula, euismod ac porttitor faucibus, rutrum eu ex.
Donec scelerisque nisi eget risus condimentum ultrices. Integer porta maximus quam, in lobortis quam fermentum eu. Morbi eu ligula consequat, aliquam sem eget, sollicitudin eros. Suspendisse potenti. Cras finibus metus et eros accumsan, id vehicula libero lobortis. Aenean vulputate lacinia urna at fringilla. Nulla id tincidunt lectus, quis accumsan lorem. In posuere magna non purus hendrerit rutrum. Maecenas in mattis tellus. Maecenas vel feugiat enim. Lorem ipsum dolor sit amet, consectetur adipiscing elit. Proin convallis dapibus tellus vitae euismod. Nam eleifend dui quam, eget lobortis quam pulvinar id. Cras euismod posuere dolor non ultricies.
Ut dapibus porta faucibus. Duis velit ante, tincidunt id velit id, imperdiet egestas velit. Morbi efficitur enim dignissim interdum egestas. Vivamus eu urna condimentum, aliquam orci non, ullamcorper est. Phasellus egestas at tellus nec tristique. Fusce feugiat commodo faucibus. In hac habitasse platea dictumst. Quisque dignissim, mauris a pellentesque dictum, mauris velit tincidunt lorem, sed tincidunt libero enim vitae orci. Nam interdum, ante nec consequat vulputate, nisi turpis euismod nibh, sit amet elementum nunc diam non eros. Proin quis viverra risus. Vestibulum vestibulum diam in velit consectetur, eu elementum lacus sagittis. Morbi accumsan ac ante eget faucibus. In nec elit bibendum, tristique enim non, sodales ex. Donec sodales erat vitae odio cursus commodo.
Vestibulum felis lacus, mattis eget porta eget, mattis ut felis. Pellentesque aliquet, purus eu semper suscipit, sem ipsum euismod nunc, sed dapibus augue sem vel elit. Etiam tincidunt arcu ut nisi tempor commodo. Mauris at eros tincidunt, fringilla erat nec, sagittis ante. Integer et malesuada quam. Cras vel porta erat, sit amet efficitur erat. Praesent blandit purus quis urna consectetur, eget ultricies ipsum pulvinar. Phasellus ac molestie elit. Vestibulum et tincidunt velit. Aliquam a venenatis ipsum, nec commodo libero. Nullam eget consectetur lectus. Morbi placerat interdum erat nec interdum.
Morbi bibendum dui eu turpis pretium, eget aliquet augue aliquam. Aliquam eu dignissim mauris, vitae placerat augue. Ut sed tortor sit amet augue imperdiet rutrum. Aliquam erat volutpat. Morbi a turpis in sapien ultrices tristique. Proin quis vestibulum lorem, ut pharetra ex. Quisque tempor bibendum purus ac vehicula. Suspendisse tellus ipsum, imperdiet id sodales vel, congue a leo. Nulla gravida tincidunt nisi eu tempor. Mauris imperdiet tempor ante eget rutrum. Nam ut dui at augue laoreet mollis. Sed metus elit, viverra ac fringilla vel, fermentum et magna. Nam ligula purus, pretium vel dignissim vitae, fermentum at urna. Nullam ac ullamcorper felis. Maecenas dapibus consequat mi.
END
my @words = $lorem.split(' ');
my $wordlen = @words.elems;
my &getNext = sub {
my $counter = 0;
my Lock $lock .= new;
return sub (@filename) {
$lock.lock;
if ($counter < 100_000) {
@filename[0] = $counter;
$counter++;
}
$lock.unlock;
};
}();
say "started";
my @threads = (^1).map({
Thread.start(
name => $_,
sub {
loop {
my @counter = (-1);
getNext(@counter);
last if @counter[0] < 0;
say $*THREAD ~ " got " ~ @counter[0];
my @copy = @words.clone;
for (^3) {
my $pos = $wordlen.rand.round;
@copy[$pos] = @keys[$len.rand.round];
}
spurt sprintf($output ~ '/%08d.txt', @counter[0]), @copy.join(' ');
}
}
);
}).join;
几乎没有理由使用 Threads directly in Perl 6, there are plenty of features that do the hard work for you. In most cases the simplest improvement is to just use start instead of Thread.start, and await LIST 而不是 LIST».join 或 LIST.map(*.join)。
您正在呼叫 List.join not Thread.join，这可能至少是您遇到问题的部分原因。
还有很多东西使本文难以阅读和理解。
根据你的写作方式 getNext 看来你只知道或大部分只知道 JavaScript。我就是这样写的。
sub get-next (@filename) {
state Int $counter = 0; # set to zero only the first time it is called
state Lock $lock .= new;
$lock.protect: ->{
if $counter < 100_000 {
@filename[0] = $counter++;
}
}
}
除了你只使用 @filename 来绕过参数的默认只读语义。
sub get-next ($filename is rw) {
state Int $counter = 0;
state Lock $lock .= new;
$lock.protect: ->{
if $counter < 100_000 {
$filename = $counter++;
}
}
}
只是 return 值更有意义。
sub get-next () {
state Int $counter = 0;
state Lock $lock .= new;
$lock.protect: ->{
$counter++ if $counter < 100_000
} // Nil
# the `if` will return `Empty` when $counter gets too big
# but we want `Nil` instead
}
这可以是 return 计数器或 Nil 这是一个未定义的值，因此您可以像这样使用它：
loop {
my $counter = get-next() orelse last;
…
}
因为你只在循环中使用 sub，所以它可以调用 last 本身。
或者更好的是，如果您改为将声明计数器的行更改为
state Int $counter = 0 but True;
然后您可以将 loop 更改为 while 循环，无需在任何地方添加 last。
while get-next() -> $counter { … }
既然我已经向您展示了如何改进 getNext，我打算放弃它并使用 Perl 6 更好的功能。
（顺便说一句，使用 Channel 会更有意义）
# will probably still work with `use v6.c;`
# but v6.d has a better system backing `await`
use v6.d.PREVIEW;
# override the original default number of threads
# (16 threads currently)
BEGIN %*ENV<RAKUDO_MAX_THREADS> //= 32;
# the "correct" way to do this is setting $*SCHEDULER
# but this is easier
my $keys-filename = 'C:/c/keysfile.txt';
my $output-dir = 'C:/c/perltests';
my @keys = $keys-filename.IO.words;
my @lorem = q:to/EOF/.split(' ');
…
EOF
say 'started';
END say 'finished';
for race ^100_000 -> $counter {
say $*THREAD, " got ", $counter;
my @copy = @lorem; # no need for .clone
for (^+@copy).pick(3) -> $pos {
@copy[$pos] = @keys.roll;
}
spurt $*SPEC.catfile($output-dir, $counter.fmt('/%08d.txt')), @copy.join(' ');
}
( 我 运行 一个测试，它从来没有给出高于 6 的线程 id )
我使用 race rather than hyper 因为 return 值无论如何都会被丢弃。
如果它不起作用，则说明您的 Rakudo 版本有问题。如果是这样，我建议更新到最新版本 v2017.02
(say $*PERL.compiler.version;)
+@array 等同于 @array.elems
^ NUMBER 与 0 ..^ NUMBER 相同，后者是
Range.new( 0, NUMBER, :excludes-max )
这个还是有一些“单词”有多余字符的问题
amet, elit. 和 ex.Donec 例如
…
my @pos = ($lorem ~~ m:ex/ « \w+: /).map: { .from, .chars }
# @pos looks something like [(0,5),(6,5),(12,5),(18,3),…]
…
for race ^100_000 -> $counter {
say $*THREAD, " got ", $counter;
# vvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvv
my $copy = $lorem;
# sort so that the transforms are done from the end of the string
# towards the beginning of the string
for @pos.pick(3).sort.reverse {
$copy.substr-rw( |$_ ) = @keys.roll;
}
# ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
spurt $*SPEC.catfile($output-dir, $counter.fmt('/%08d.txt')), $copy;
}
一些注意事项。
这个：
my @keys = ();
for $keysfilename.IO.words {
@keys.push($_);
}
真的只是这样：my @keys = $keysfilename.IO.words;
还有这个：@keys[$len.rand.round]
真的只是这样：@keys.pick
我不太确定你用计数器数组做什么，看来
就像你只想要文件 0..99999.
您正在为每个线程在子例程内进行锁定。如果你 想要防止多个线程访问某些东西，锁一个 外面，让他们每个人都用它来锁定独占密码。
您也在使用 Thread 制作线程，但这确实是
相当低级的界面。尝试使用更高级别的构造
如果可能的话，在下降到那个水平之前。
在这种情况下，您似乎只想拆分 100,000 个文件
创建多个线程只是为了获得一些并行性。那里
有一些构造可以使这变得非常容易，hyper 和
race。因为你甚至不关心文件的顺序
创建于，你可以只使用 race 它会尽可能快地产生它们
尽可能。
你可以用'degree'参数来race表示有多少
你想使用的线程。 （您可能需要将 RAKUDO_MAX_THREADS 设置为
得到你想要的行为）。
my $keysfilename := 'C:/c/keysfile.txt';
my $output := 'C:/c/perltests';
my @keys = $keysfilename.IO.words;
my $lorem = q:to/END/;
Lorem ipsum ...
END
my @words = $lorem.split(' ');
my $wordlen = @words.elems;
[^100000].race(degree => 32).map({
# say $*THREAD ~ " got " ~ $_;
my @copy = @words;
for (^3) {
my $pos = $wordlen.rand.round;
@copy[$pos] = @keys.pick;
}
spurt sprintf($output ~ '/%08d.txt', $_), @copy.join(' ');
});