为什么子流程模块不在 for 循环中执行多个命令？

Question

我有一个保存在文本文件中的 IP 地址列表，我想使用子进程模块和 for 循环检查它们的代码编号（404、200、504，...）。对于一个 IP 地址，我的代码工作正常，但是当我尝试一个包含多个 IP 的列表时，它会给我一些错误。

这是我的文本文件：

77.87.19.114
143.21.15.91
17.63.33.21
24.44.12.181

我使用的代码：

from subprocess import check_output

def ipcheck200(ip_list_file):
   with open(ip_list_file) as f:
      content = f.readlines()
   for item in content:

    # url generator for each ip
       url = "http://" + item + "/"
    #print(content[0])

    # command generator for each ip
       command = "python -c " + '"'
       command += "import urllib ;"
       command += "a = urllib.urlopen('%s') ;print(a.getcode())" % url
       command += '"'
       proc = check_output(command)
       print(proc)

结果将是：

> Traceback (most recent call last):
  File "<pyshell#0>", line 1, in <module>
    ipcheck200('test.txt')
  File "C:\Users\XXXX\XXXX\XXXX\file.py", line 17, in ipcheck200
    proc = check_output(command)
  File "C:\Python27\lib\subprocess.py", line 573, in check_output
    raise CalledProcessError(retcode, cmd, output=output)
CalledProcessError: Command 'python -c "import urllib ;a = urllib.urlopen('http://77.87.19.114
/') ;print(a.getcode())"' returned non-zero exit status 1

Answer 1

当存在未处理的异常时，

Python 以 1 退出。

你没有错误处理；如果 IP 地址不存在，包含额外的空格，或者即使机器拒绝连接或只是超时，urllib.urlopen() 将引发异常：

>>> urllib.urlopen('http://127.0.0.1:80')
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/Users/mjpieters/Development/Library/buildout.python/parts/opt/lib/python2.7/urllib.py", line 87, in urlopen
    return opener.open(url)
  File "/Users/mjpieters/Development/Library/buildout.python/parts/opt/lib/python2.7/urllib.py", line 213, in open
    return getattr(self, name)(url)
  File "/Users/mjpieters/Development/Library/buildout.python/parts/opt/lib/python2.7/urllib.py", line 350, in open_http
    h.endheaders(data)
  File "/Users/mjpieters/Development/Library/buildout.python/parts/opt/lib/python2.7/httplib.py", line 1053, in endheaders
    self._send_output(message_body)
  File "/Users/mjpieters/Development/Library/buildout.python/parts/opt/lib/python2.7/httplib.py", line 897, in _send_output
    self.send(msg)
  File "/Users/mjpieters/Development/Library/buildout.python/parts/opt/lib/python2.7/httplib.py", line 859, in send
    self.connect()
  File "/Users/mjpieters/Development/Library/buildout.python/parts/opt/lib/python2.7/httplib.py", line 836, in connect
    self.timeout, self.source_address)
  File "/Users/mjpieters/Development/Library/buildout.python/parts/opt/lib/python2.7/socket.py", line 575, in create_connection
    raise err
IOError: [Errno socket error] [Errno 61] Connection refused

您的代码从文件中获取行而不做进一步处理；它至少会在每一行附加一个换行符，所以你想要 strip the line:

for item in content:
    item = item.strip()

您还在执行序列中的所有子流程，因为check_output()首先等待流程完成；这比在同一进程中尝试运行这段代码要慢。不要在这里重新发明多处理轮，而是使用multiprocessing module：

from multiprocessing import Pool
import urllib

def check(ipaddress):
     try:
         response = urllib.urlopen('http://{}/'.format(url))
     except IOError:
         return 0  # connection unsuccessful (timeout, connection refused, etc)
     return response.getcode()

if __name__ == '__main__':
    p = Pool(10)
    with open(ip_list_file) as f:
        ipaddresses = [address.strip() for address in f if address.strip()]
    print(p.map(check, ipaddresses))

此示例将使用 10 个子进程并行测试 ip 地址。

为什么子流程模块不在 for 循环中执行多个命令？

why sub-process module does not execute multiple commands in a for loop?

python

subprocess

for-loop