5 个进程的多处理比旧 2 核上的单个进程慢 CPU

Question

我正在尝试运行 5 个并行进程并将列表分成 5 个块，但它比单个进程中的线性处理需要更多时间，我在 Whosebug 上尝试了其他解决方案，但结果是相同的。如果我运行代码，这就是我得到的：

import multiprocessing as mp
import time


def raiseNum(pwr, nums):
    for v in nums:
        x = v**pwr 


if __name__ == '__main__':

    # Let's get the cube of all numbers in the list nums
    nums = [i for i in range(0, 100000000, 1)]
    pwr = 3

    ## linear processing
    start = time.time()

    results_l = raiseNum(pwr, nums)

    end = time.time()

    print('Linear Processing time: ', str(end-start), 'Seconds')
    


    ## Parallel processing: 5 Processes
    
    # divide nums list in 5 parts
    blockSize = len(nums)//5
    
    numsBlocks = [nums[i: i+blockSize] for i in range(0, len(nums), blockSize)]

    processList = []

    p0 = mp.Process(target=raiseNum, args=(pwr, numsBlocks[0]))
    p1 = mp.Process(target=raiseNum, args=(pwr, numsBlocks[1]))
    p2 = mp.Process(target=raiseNum, args=(pwr, numsBlocks[2]))
    p3 = mp.Process(target=raiseNum, args=(pwr, numsBlocks[3]))
    p4 = mp.Process(target=raiseNum, args=(pwr, numsBlocks[4]))

    start = time.time()

    p0.start()
    p1.start()
    p2.start()
    p3.start()
    p4.start()

    p0.join()
    p1.join()
    p2.join()
    p3.join()
    p4.join()

    end = time.time()

    print('5 Parallel Processes time: ', str(end-start), 'Seconds')

Answer 1

优化取决于您的硬件和计算。当我运行在我的机器上使用相同的代码时，我得到了以下结果：

Linear Processing time:  27.75643253326416 sec
5 Parallel Processes total time:  7.949779510498047 sec
Computation time:  1.7256593704223633 sec
Join time:  6.224120140075684 sec

注意两点：

并行执行几乎快了 3.5 倍。（我在24核的机器上测试了你的代码）
当我们将并行处理分成两部分时，我们可以注意到连接需要相当长的时间（因为计算不是那么密集）。

这是代码，我在最后添加了一些额外的打印语句：

import multiprocessing as mp
import time


def raiseNum(power, numbers):
    for v in numbers:
        x = v ** power


if __name__ == '__main__':
    # Let's get the cube of all numbers in the list nums
    nums = [i for i in range(0, 100000000, 1)]
    pwr = 3

    # linear processing
    start = time.time()
    raiseNum(pwr, nums)
    end = time.time()
    print('Linear Processing time: ', str(end - start), 'sec')

    # Parallel processing: 5 Processes
    # divide nums list in 5 parts
    blockSize = len(nums) // 5
    numsBlocks = [nums[i: i + blockSize] for i in range(0, len(nums), blockSize)]
    processList = []
    p0 = mp.Process(target=raiseNum, args=(pwr, numsBlocks[0]))
    p1 = mp.Process(target=raiseNum, args=(pwr, numsBlocks[1]))
    p2 = mp.Process(target=raiseNum, args=(pwr, numsBlocks[2]))
    p3 = mp.Process(target=raiseNum, args=(pwr, numsBlocks[3]))
    p4 = mp.Process(target=raiseNum, args=(pwr, numsBlocks[4]))

    start1 = time.time()

    p0.start()
    p1.start()
    p2.start()
    p3.start()
    p4.start()

    start2 = time.time()

    p0.join()
    p1.join()
    p2.join()
    p3.join()
    p4.join()

    end = time.time()

    print('5 Parallel Processes total time: ', str(end - start1), 'sec')
    print('Computation time: ', str(start2 - start1), 'sec')
    print('Join time: ', str(end - start2), 'sec')

希望这有助于更好的优化。

5 个进程的多处理比旧 2 核上的单个进程慢 CPU

multiprocessing with 5 processes slower than single process on an old 2-core CPU

python

parallel-processing

multiprocessing