当将 stderr 重定向到 /dev/null 并由 multiprocessing.map_async 调用时，子进程 check_output 抛出异常

Question

考虑这段代码：

#!/usr/bin/python                                                               

import os                                                                       
import multiprocessing                                                          
import subprocess                                                               

def search(name):                                                               
    devnull=open(os.devnull, 'w')                                               
    res = subprocess.check_output(                                              
            ['sleep', name]                                                     
            #, stderr=devnull # works fine when comment out                                                  
            )                                                                   

class Manager(object):                                                          
    def __init__(self):                                             
        self.pool = multiprocessing.Pool(4)                                     

    def clean(self):                                                            
        self.pool.close()                                                       
        self.pool.join()                                                        

    def fetch(self, names):                                                     
        res = self.pool.map_async(search, names)                                
        return res.get(10)                                                      


if __name__ ==  '__main__':                                                                                                              
    manager = Manager()                                                       
    manager.fetch([1, 2, 3])                                                    
    manager.clean()

如果我将 stderr 更改为指向一个临时文件或根本不更改 stderr，代码将正常执行，否则我会收到以下异常：

Traceback (most recent call last):
  File "./z.py", line 71, in <module>
    manager.fetch([1, 2, 3])
  File "./z.py", line 64, in fetch
    return res.get(10)
  File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/multiprocessing/pool.py", line 554, in get
    raise self._value
TypeError: execv() arg 2 must contain only strings

谁能解释一下为什么？

Answer 1

根据异常提示，你的问题出在execv()，实际上是subprocess.check_output到运行的参数。它告诉您参数数组必须是字符串。以下代码会出现相同的错误：

devnull=open(os.devnull, 'w') 
subprocess.check_output(["sleep", 1], stderr=devnull)

解决方法很简单——把manager.fetch([1, 2, 3])改成manager.fetch(["1", "2", "3"])。

当将 stderr 重定向到 /dev/null 并由 multiprocessing.map_async 调用时，子进程 check_output 抛出异常

subprocess check_output throw exception when redirect stderr to /dev/null and called by multiprocessing.map_async

python

subprocess

python-multiprocessing