7

In setting the number of processes, I'd be keen to see how many threads I can actually use on my machine - how do I find this? Is there a way to determine the number of threads available to me?

Yann
  • 33,811
  • 9
  • 79
  • 70
disruptive
  • 5,687
  • 15
  • 71
  • 135

1 Answers1

7

Do you want to know the CPU count?

According to the docs, when you start a pool and don't sepecify the number of processes, the default number is the number of cpu's on the system:

processes is the number of worker processes to use. If processes is None then the number returned by cpu_count() is used. If initializer is not None then each worker process will call initializer(*initargs) when it starts.

Yann
  • 33,811
  • 9
  • 79
  • 70
  • 1
    Multiprocessing does not create threads, it starts processes: http://stackoverflow.com/questions/200469/what-is-the-difference-between-a-process-and-a-thread Regardless, you can have more processes/threads than the number of CPUs, but you usually don't want to for performance reasons. – Yann Jul 31 '14 at 11:59
  • The exception to the "one thread/process per CPU" rule would be if you're doing I/O-bound work in most of your threads/processes. In that case, it makes sense to start more threads than you have CPUs, because while your thread are blocking on I/O they won't be using any CPU cycles. – dano Jul 31 '14 at 14:49
  • @dano, I agree, but usually with pool.map you're performing homogeneous tasks. – Yann Jul 31 '14 at 14:59