Embedding Python in C++ crashes during running-time

Question

I've been around this question for way too long. Now I searched this and it was working for a while but it then crashed at a random place. I think that the problem is a deadlock or something.

So please, someone tell me what I'm doing wrong:

1. I make a global variable:

   static int gil_init = 0; 
   

2. At the start of the program I call Py_Initialize(). This start function is called multiple times in one session but since Py_Initialize() is a non-op if Python is already initialized I don't think there's a problem here.

3. From three functions I make lots of calls into the Python/C API function -py_embed(...) (there's only one at a time calling). The next code is present in the py_embed() function that is called from each thread:

   if (!gil_init) {
       gil_init = 1;
       PyEval_InitThreads();
       PyEval_SaveThread();
   }
   state = PyGILState_Ensure();
   // Call Python/C API functions...
   //pValue= PyObject_CallObject(pFunc, pArgs2);  Crash is always given here
   PyGILState_Release(state);
   

Oh and the crash takes place in one of the functions, but not the first time I call it which is strange. That's why I said its at a random place during running time.

In theory, and based in the reference manual of the C Api it should be working.. What am I doing wrong?

Edit: I don't call Py_Finalize() because of this problem that I had

Answer 1

Take a look at this post, it helped me with some threading issues I was having. In particular this part:

That said, however, I have found that this can lead to intermittent deadlocking. It seems that each PyThreadState used by PyGILState_Ensure() is not unique, leading to multiple threads attempting to restore the same PyThreadState, resulting in a deadlock. This is simple enough to overcome by guaranteeing that each thread has its own PyThreadState. A partial solution is as follows:

 // Once in each thread
 m_state = PyThreadState_New(m_interpreterState);
 PyEval_RestoreThread(m_state);

 // Perform some Python actions here

 // Release Python GIL
 PyEval_SaveThread();

Answer 2

I think I solved it (can't say for sure but until now it hasn't crashed).

The problem was that I was dereferenciating some borrowed references. I noticed this because the reference count of some objects were 2 or 3.