如何在 Windows 上解决多线程多进程 RuntimeError 问题？

2024-03-21 17:42:16

使用多线程和多进程解决 Windows 上的 RuntimeError

问题

在 Windows 机器上使用多线程和多进程时，Python 会抛出以下错误：

RuntimeError: 
            Attempt to start a new process before the current process
            has finished its bootstrapping phase.
            This probably means that you are on Windows and you have
            forgotten to use the proper idiom in the main module:
                if __name__ == '__main__':
                    freeze_support()
                    ...
            The "freeze_support()" line can be omitted if the program
            is not going to be frozen to produce a Windows executable.

原因

此错误通常出现在 Windows 系统中，当在主模块之外启动进程时。Windows 需要在主进程完成引导阶段之前启动新进程，而没有使用正确的习惯用法会导致此错误。

解决方法

要解决此问题，需要在主模块中使用以下代码片段：

if __name__ == '__main__':
    multiprocessing.freeze_support()

通过添加 multiprocessing.freeze_support()，Windows 特定的设置将应用于 Python 多处理，允许在主进程完成引导阶段之前启动新进程。

示例

以下是一个修复后的代码示例：

testMain.py：

import parallelTestModule

if __name__ == '__main__':
    multiprocessing.freeze_support()

extractor = parallelTestModule.ParallelExtractor()
extractor.runInParallel(numProcesses=2, numThreads=4)

parallelTestModule.py：

import multiprocessing
from multiprocessing import Process
import threading

class ThreadRunner(threading.Thread):
    """ This class represents a single instance of a running thread"""
    def __init__(self, name):
        threading.Thread.__init__(self)
        self.name = name
    def run(self):
        print self.name,'\n'

class ProcessRunner:
    """ This class represents a single instance of a running process """
    def runp(self, pid, numThreads):
        mythreads = []
        for tid in range(numThreads):
            name = "Proc-"+str(pid)+"-Thread-"+str(tid)
            th = ThreadRunner(name)
            mythreads.append(th) 
        for i in mythreads:
            i.start()
        for i in mythreads:
            i.join()

class ParallelExtractor:    
    def runInParallel(self, numProcesses, numThreads):
        myprocs = []
        prunner = ProcessRunner()
        for pid in range(numProcesses):
            pr = Process(target=prunner.runp, args=(pid, numThreads)) 
            myprocs.append(pr) 
        for i in myprocs:
            i.start()

        for i in myprocs:
            i.join()