python 使用池进行多重处理

Question

我在 Python 的课程有作业。

问题是这样的：

您将编写一个创建多个进程的程序 (p=10)。每个进程都会收到整数值 (y) 并计算 y 乘以 4 (y*4)。 y 值的集合是一个列表整数 [0, 1, 2, 3, 4, 5]。必须使用进程池，所以需要映射这10个进程获取 y 整数列表。执行以下操作：

你的程序应该打印出以下三个。 (1) 打印出原始清单。 (2)按随机顺序打印每个运行进程名及其y*4的输出执行

(1) 的输出：输入列表：[0, 1, 2, 3, 4, 5]

(2) 的输出：随机输出：

PoolWorker-10 输出：[0, 4, 8, 12, 16, 20]

PoolWorker-11 输出：[0, 4, 8, 12, 16, 20]

PoolWorker-12 输出：[0, 4, 8, 12, 16, 20]

我的第一次尝试是：

import time
from multiprocessing import Pool
from multiprocessing import Process, current_process
def f(number):
    result=4*number
    Process_name=current_process().name
    print(f"Poolworker -{Process_name} output: {result} ")

    return result

if __name__=='__main__':
    p= Pool(processes=10)
    numbers=[0,1,2,3,4,5]
    result=p.map(f,numbers)

输出为：

Poolworker -SpawnPoolWorker-2 output: 0 
Poolworker -SpawnPoolWorker-2 output: 4 
Poolworker -SpawnPoolWorker-2 output: 8 
Poolworker -SpawnPoolWorker-2 output: 12 
Poolworker -SpawnPoolWorker-2 output: 16 
Poolworker -SpawnPoolWorker-2 output: 20

我的第二次尝试：

import time
from multiprocessing import Pool
from multiprocessing import Process, current_process
def f(*number):
 numbers=list(number)
 i=0
 for x in numbers:
  numbers[i]=4*numbers[i]
  i+=1
 Process_name=current_process().name
 print(f"Poolworker -{Process_name}output: {numbers} ")

 return numbers

if __name__ == '__main__':
  array=[0,1,2,3,4,5]
  p=Pool(processes=10)
 
  result=p.map(f,array)

我仍然有更改后的错误输出。

如何获得所需的输出？

Answer 1

问题是 f 应用于列表中的每个数字，而您希望每个工作人员对 整体进行计算列表。对您的代码进行了一些小的修改以获得这些结果。

from itertools import repeat
from multiprocessing import Pool
from multiprocessing import Process, current_process

def f(numbers):
    result= [4 * num for num in numbers]
    name=current_process().name
    print(f"Poolworker - {name} output: {result} ")
    return result

if __name__=='__main__':
    WORKERS = 10
    p = Pool(processes=WORKERS)
    numbers = [0, 1, 2, 3, 4, 5]
    print(f"Original - {numbers}")
    result = p.map(f, repeat(numbers, WORKERS))

这个returns以下：

Original - [0, 1, 2, 3, 4, 5]
Poolworker - ForkPoolWorker-1 output: [0, 4, 8, 12, 16, 20] 
Poolworker - ForkPoolWorker-2 output: [0, 4, 8, 12, 16, 20] 
Poolworker - ForkPoolWorker-3 output: [0, 4, 8, 12, 16, 20] 
Poolworker - ForkPoolWorker-4 output: [0, 4, 8, 12, 16, 20] 
Poolworker - ForkPoolWorker-5 output: [0, 4, 8, 12, 16, 20] 
Poolworker - ForkPoolWorker-6 output: [0, 4, 8, 12, 16, 20] 
Poolworker - ForkPoolWorker-7 output: [0, 4, 8, 12, 16, 20] 
Poolworker - ForkPoolWorker-8 output: [0, 4, 8, 12, 16, 20] 
Poolworker - ForkPoolWorker-9 output: [0, 4, 8, 12, 16, 20] 
Poolworker - ForkPoolWorker-10 output: [0, 4, 8, 12, 16, 20]

Answer 2

根据您的描述，您希望所有进程都将数组乘以 4。

因此，使用并行进程，每个进程都给定一个数字数组进行乘法运算。

Reference

代码

from multiprocessing import Process

import multiprocessing as mp

def f(numbers, output):
    " Places process name and numbers multiplied by 4 into output queue "
    output.put((mp.current_process().name, [4*n for n in numbers]))

# Define an output queue
output = mp.Queue()

# Setup a list of processes that we want to run
numbers = [1, 2, 3, 4, 5]
processes = [mp.Process(target=f, args=([1, 2, 3, 4, 5], output)) for x in range(10)]

# Run processes
for p in processes:
    p.start()

# Exit the completed processes
for p in processes:
    p.join()

# Get process results from the output queue
results = [output.get() for p in processes]


for id, values in results:
  print(f'PoolWorker {id} output: {values}')

输出

PoolWorker Process-1 output: [4, 8, 12, 16, 20]
PoolWorker Process-3 output: [4, 8, 12, 16, 20]
PoolWorker Process-9 output: [4, 8, 12, 16, 20]
PoolWorker Process-6 output: [4, 8, 12, 16, 20]
PoolWorker Process-7 output: [4, 8, 12, 16, 20]
PoolWorker Process-5 output: [4, 8, 12, 16, 20]
PoolWorker Process-4 output: [4, 8, 12, 16, 20]
PoolWorker Process-8 output: [4, 8, 12, 16, 20]
PoolWorker Process-2 output: [4, 8, 12, 16, 20]
PoolWorker Process-10 output: [4, 8, 12, 16, 20]

python 使用池进行多重处理

python multiple processing using pool

python

multiprocessing

pool