Control-C处理多进程、Pool和读取文件的方法。

Question

我刚刚开始使用多处理& Python，我需要一些帮助，在我的程序中捕捉Control-C。 我正在做的脚本要读取一个文件，然后在每一行上执行一些任务。在有人评论IO和多处理的优缺点之前，我意识到:)这些任务对多线程非常友好。

我有下面的代码，从文档中看，我希望它能正常工作，但是它没有捕捉到我的键盘异常。 ARRGH... 请帮助我

运行在Win10上，如果这有什么不同的话。

from multiprocessing import cpu_count
from multiprocessing.dummy import Pool as ThreadPool
import argparse
from time import sleep
import signal
import sys


def readfile(file):
  with open(file, 'r') as file:
    data = file.readlines()
    file.close()
  return data

def work(line):
  while(True):
    try:
      print(f"\rgoing to do some work on {line}")
      countdown(5)
    except (KeyboardInterrupt, SystemExit):
      print("Exiting...")
      break

def countdown(time=30):
  sleep(time)

def parseArgs(args):
  if args.verbose:
    verbose = True
    print("[+] Verbosity turned on")
  else:
    verbose = False
  if args.threads:
    threads = args.threads
  else:
    threads = cpu_count()
  print(f'[+] Using {threads} threads')
  return threads, verbose, args.file


if __name__ == '__main__':
  parser = argparse.ArgumentParser()
  parser.add_argument("-f", "--file", required = True, help="Insert the flie you plan on parsing")
  parser.add_argument("-t", "--threads", help="Number of threads, by default will use all available processors")
  parser.add_argument("-v", "--verbose", help="increase output verbosity",
                       action="store_true")
  threads, verbose, filename = parseArgs(parser.parse_args())
  #read the entire file and store it in a variable:
  data = readfile(filename)
  #Init the data pool
  pool = ThreadPool(threads) # Number of threads going to use
  try:
    pool.map(work,data) # This launches the workers at the function to do work
  except KeyboardInterrupt:
    print("Exiting...")
  finally:
    pool.close()
    pool.join()

Answer 1

在你使用Control-C的时候，程序可能在 pool.join() 等待所有的线程完成。该 pool.map 函数只是启动所有的进程，但并不阻塞。这意味着在键盘中断发生时，由于程序不在尝试块内，所以没有被捕获。

我不太清楚这里的最佳实践，但我会尝试。

try: 
    pool.map(work, data) # This launches the workers at the function to do work 
    pool.close() 
    pool.join()
except KeyboardInterrupt: 
    print("Exiting...")