64

I am confused about using freeze_support() for multiprocessing and I get a Runtime Error without it. I am only running a script, not defining a function or a module. Can I still use it? Or should the packages I'm importing be using it?

Here is the documentation.

Note that the specific issue is about scikit-learn calling GridSearchCV which tries to spawn processes in parallel. I am not sure if my script needs to be frozen for this, or the some code that's called (from the Anaconda distro). If details are relevant to this question, please head over to the more specific question.

geca
  • 2,711
  • 2
  • 17
  • 26
László
  • 3,914
  • 8
  • 34
  • 49
  • 2
    How is your script executed? – dano Jun 23 '14 at 20:16
  • 1
    @dano OK, then the specific question is relevant. I have my entire code pasted there. (But the link pointed to the wrong question, I fixed it.) Executed in a dedicated Python interpreter (from Spyder). – László Jun 23 '14 at 20:25

1 Answers1

105

On Windows all of your multiprocessing-using code must be guarded by if __name__ == "__main__":

So to be safe, I would put all of your the code currently at the top-level of your script in a main() function, and then just do this at the top-level:

if __name__ == "__main__":
    main()

See the "Safe importing of main module" sub-section here for an explanation of why this is necessary. You probably don't need to call freeze_support at all, though it won't hurt anything to include it.

Note that it's a best practice to use the if __name__ == "__main__" guard for scripts anyway, so that code isn't unexpectedly executed if you find you need to import your script into another script at some point in the future.

dano
  • 91,354
  • 19
  • 222
  • 219
  • 1
    Sounds great, thanks, I'll let you know how it ran, and then I'm happy to accept. – László Jun 23 '14 at 20:44
  • 13
    the only scenario this doesn't cover is when someone creates an object that inherits from Process, i.e: class myProcess(Process), where do you put then freeze_support in that scenario? – Har Feb 18 '15 at 11:16
  • 2
    Thank you. This error of not using multiprocessing inside `if __name__ == "__main__":` came as a surprise to me. I was unaware of the multiprocessing requirement though I always used multiprocessing functions inside the `if` clause. This was the first time I used the code outside of the `if` clause and it gave the error `RuntimeError: Attempt to start a new process before the current process has finished its bootstrapping phase` – Gary Nov 18 '21 at 04:30