Freeze/Fail when using functional with OpenMP [Pybind11/OpenMP]

Question

I have a problem with the functional feature of Pybind11 when I use it with a for-loop with OpenMP. I've done some research and my problem sounds pretty similar to the one in this Pull Request from 2 years ago, but although this PR is closed and the issue seems to be fixed I still have this issue. A code example I created will hopefully explain my problem better:

b.h

#include <pybind11/pybind11.h>
#include <pybind11/functional.h>
#include <omp.h>

namespace py = pybind11;

class B {
  public:
    B(int n, const int& initial_value);
    void map(const std::function<int(int)> &f);
  private:
    int n;
    int* elements;
};

b.cpp

#include <pybind11/pybind11.h>
#include <pybind11/functional.h>
#include "b.h"

namespace py = pybind11;

B::B(int n, const int& v)
    : n(n) {
  elements = new int[n];
  #pragma omp parallel for
  for (int i = 0; i < n; i++) {
    elements[i] = v;
  }
}

void B::map(const std::function<int(int)> &f) {
  #pragma omp parallel for
  for (int i = 0; i < n; i++) {
    elements[i] = f(elements[i]);
  }
}

PYBIND11_MODULE(m, handle) {
  handle.doc() = "Example Module";
  py::class_<B>(handle, "B")
  .def(py::init<int, int>())
  .def("map", &B::map)
  ;
}

CMakeLists.txt

cmake_minimum_required(VERSION 3.4...3.18)
project(example)

find_package(OpenMP)

add_subdirectory(pybind11)
pybind11_add_module(m b.cpp)

if(OpenMP_CXX_FOUND)
  target_link_libraries(m PUBLIC OpenMP::OpenMP_CXX)
else()
  message( FATAL_ERROR "Your compiler does not support OpenMP" )
endif()

test.py

from build.m import *


def test(i):
    return i * 20


b = B(2, 2)
b.map(test)

I basically have an array where I want to apply a Python function to every element using a for-loop. I know that it is an issue with functional and OpenMP specifically because in other parts of my project I am using OpenMP successfully and functional is also working if I am not using OpenMP.

Edit: It freezes at the map function and has to be terminated. I am using Ubuntu 21.10, Python 3.9, GCC 11.2.0, OpenMP 4.5, and the newest version of the pybind11 repo.

"Freeze/Crash/Fail" -- well, which one is it? Does it stop responding, so you have to kill it? Or it terminates with some error message? Something else, like computer catching fire? Be specific. | What does the debugger tell you? | What platform? What compiler? What specific library versions? — Dan Mašek, Jan 11 '22 at 00:28
Sorry for not being specific enough. It freezes at the map function and I have to terminate it. I have tried it in 2 different environments (repl.it and Ubuntu 21.10). On Ubuntu the compiler CMake uses is GCC 11.2.0, the newest pybind11 repo and OpenMP 4.5. I haven't set up a debugger yet but I will try and come back with results. Also thanks for the heads up on the leakage (I am fairly new to C++) :) — chicom, Jan 11 '22 at 09:08
Thinking about this a bit more, I don't think there's a point of parallelizing `map` in the case when the function you're calling is defined in Python. The only expensive (by far) thing in the loop is that function call, and since it's a Python function it runs in the interpreter, and due to the GIL, only one thread can use the interpreter at one time. — Dan Mašek, Jan 11 '22 at 18:16

score 2 · Accepted Answer · answered Jan 11 '22 at 14:42

You're likely experiencing a deadlock between OpenMP's scheduler and Python's GIL (Global Interpreter Lock).

I suggest attaching gdb to your process and looking at where the threads are to verify that's really the problem.

IMHO mixing Python functions and OpenMP like that is asking for trouble. If you want multi-threading of Python functions you can use multiprocessing.pool.ThreadPool. But unless your functions release the GIL most of the time you won't benefit from multi-threading.

Freeze/Fail when using functional with OpenMP [Pybind11/OpenMP]

1 Answers1