multiprocessing python 模块无法放回处理后的结果

Question

我正在编写一个 TCP SYN 扫描器来检查所有打开的端口。该脚本能够通过使用多个内核来获取所有打开的端口。在脚本结束时，当尝试使用 get() 方法获取结果时，脚本变得无法运行。在进行键盘中断时，会出现代码下方提到的 Traceback。当我使用 2 个内核时，脚本运行没问题，但是当对运行进行循环 3 次或更多次（使用 3 个或更多内核）时，脚本会卡住。关于如何更进一步的任何建议？

==============代码如下============================ =========

#!/usr/bin/python

import multiprocessing as mp
from scapy.all import *
import sys
import time


results = []
output = mp.Queue()
processes = []

def portScan(ports,output):

    ip = sys.argv[1]
    for port in range(ports-100,ports):
         response = sr1(IP(dst=ip)/TCP(dport=port, flags="S"), verbose=False, timeout=.2)
         if response:
               if response[TCP].flags == 18 :
                     print "port number ======> %d <====== Status: OPEN" %(port)

                     output.put(port)



ports = 0

for loop in range(4):
    ports += 100
    print "Ports %d sent as the argument"%ports
    processes.append(mp.Process(target=portScan,args=(ports,output)))


for p in processes:
        p.start()


for p in processes:
        p.join()



results = [output.get() for p in processes]

===========输出======================

 ./tcpSynmultiprocess.py 10.0.2.1 

WARNING: No route found for IPv6 destination :: (no default route?)
  Ports 100 sent as the argument
  Ports 200 sent as the argument
  Ports 300 sent as the argument
  port number ======> 23 <====== Status: OPEN
  port number ======> 80 <====== Status: OPEN
  ^CTraceback (most recent call last):

===========TraceBack===================

 ^CTraceback (most recent call last):
 File "./tcpSynmultiprocess.py", line 43, in <module>
 results = [output.get() for p in processes]
 File "/usr/lib/python2.7/multiprocessing/queues.py", line 117, in get
 res = self._recv()

键盘中断

Answer 1

默认情况下，Queue.get() 会阻塞，直到它有数据到 return，如果所有进程都已经结束，则不会。

您可以使用 output.get(False) 不阻塞 return 没有的进程（您必须处理 Queue.Empty 异常）。

或者，由于队列大小也可以大于进程数，您应该使用 Queue.qsize() 而不是 processes:

results = [output.get() for x in range(output.qsize())]

Answer 2

#!/usr/bin/python

import multiprocessing as mp
from scapy.all import *
import sys
import time


results = []
output = mp.Queue()
processes = []

def portScan(ports,output):

    ip = sys.argv[1]
    for port in range(ports-100,ports):
    response = sr1(IP(dst=ip)/TCP(dport=port, flags="S"), verbose=False, timeout=.2)
     if response:
           if response[TCP].flags == 18 :
                 print "port number ======> %d <====== Status: OPEN" %(port)

                 output.put(port)



ports = 0

for loop in range(4):
    ports += 100
    print "Ports %d sent as the argument"%ports
    processes.append(mp.Process(target=portScan,args=(ports,output)))


for p in processes:
    p.start()


for p in processes:
    p.join()



for size in range(output.qsize()):
    try: 
        results.append(output.get())
    except:
        print "Nothing fetched from the Queue..."

print results

multiprocessing python 模块无法放回处理后的结果

The multiprocessing python module is unable to put back the processed results

python

multiprocessing

python-multithreading

python-2.7