Control-C 处理多处理、池和读取文件

Question

我刚刚开始使用 Multiprocessing & Python，我需要一些帮助在我的程序中捕获 Control-C。我正在制作的脚本将读入一个文件，然后在每一行上执行一些任务。在有人评论 I/O 和多处理的 advantages/disadvantages 之前，我知道 :) 这些任务本身对多线程非常友好。

我有以下代码，根据文档，我希望它能工作，但它没有捕捉到我的键盘异常！啊……请帮忙

运行在 Win10 上，如果有任何区别：

from multiprocessing import cpu_count
from multiprocessing.dummy import Pool as ThreadPool
import argparse
from time import sleep
import signal
import sys


def readfile(file):
  with open(file, 'r') as file:
    data = file.readlines()
    file.close()
  return data

def work(line):
  while(True):
    try:
      print(f"\rgoing to do some work on {line}")
      countdown(5)
    except (KeyboardInterrupt, SystemExit):
      print("Exiting...")
      break

def countdown(time=30):
  sleep(time)

def parseArgs(args):
  if args.verbose:
    verbose = True
    print("[+] Verbosity turned on")
  else:
    verbose = False
  if args.threads:
    threads = args.threads
  else:
    threads = cpu_count()
  print(f'[+] Using {threads} threads')
  return threads, verbose, args.file


if __name__ == '__main__':
  parser = argparse.ArgumentParser()
  parser.add_argument("-f", "--file", required = True, help="Insert the flie you plan on parsing")
  parser.add_argument("-t", "--threads", help="Number of threads, by default will use all available processors")
  parser.add_argument("-v", "--verbose", help="increase output verbosity",
                       action="store_true")
  threads, verbose, filename = parseArgs(parser.parse_args())
  #read the entire file and store it in a variable:
  data = readfile(filename)
  #Init the data pool
  pool = ThreadPool(threads) # Number of threads going to use
  try:
    pool.map(work,data) # This launches the workers at the function to do work
  except KeyboardInterrupt:
    print("Exiting...")
  finally:
    pool.close()
    pool.join()

Answer 1

当您使用 Control-C 时，程序可能正在 pool.join() 等待所有线程完成。 pool.map 函数只是启动所有进程但不会阻塞。这意味着在 KeyboardInterrupt 发生时它没有被捕获，因为程序不在 try 块内。

我不太确定这里的最佳做法，但我会尝试：

try: 
    pool.map(work, data) # This launches the workers at the function to do work 
    pool.close() 
    pool.join()
except KeyboardInterrupt: 
    print("Exiting...")

Control-C 处理多处理、池和读取文件

Control-C Handling with Multiprocessing, Pool, and reading files

python

multiprocessing

pool