1

This is a followup question concerning this one. As pointed in the former question comments, my initial attempt for importing the module was completely wrong, and instead I should have done this : 

auto module = PyImport_ImportModule("Package1.Module1");

so I did so : 

auto module = PyImport_ImportModule("FV.F_V");

After doing this, now I get much more meaningful output as you can see: 

module: 0000021306FF9B38
size: 30
FaceVerification: C:\Users\Master\Anaconda3\Lib\site-packages\FV\__pycache__\F_V.cpython-36.pyc
Image: C:\Users\Master\Anaconda3\Lib\site-packages\FV\__pycache__\F_V.cpython-36.pyc
Path: C:\Users\Master\Anaconda3\Lib\site-packages\FV\__pycache__\F_V.cpython-36.pyc
__builtins__: C:\Users\Master\Anaconda3\Lib\site-packages\FV\__pycache__\F_V.cpython-36.pyc
__cached__: C:\Users\Master\Anaconda3\Lib\site-packages\FV\__pycache__\F_V.cpython-36.pyc
__doc__: FV.F_V
__file__: C:\Users\Master\Anaconda3\Lib\site-packages\FV\F_V.py
__loader__: FV
__name__: FV.F_V
__package__: FV
__spec__: FV
__warningregistry__ : C:\Users\Master\Anaconda3\Lib\site-packages\FV\__pycache__\F_V.cpython-36.pyc
align_face: C:\Users\Master\Anaconda3\Lib\site-packages\FV\__pycache__\F_V.cpython-36.pyc
cv2: C:\Users\Master\Anaconda3\Lib\site-packages\FV\__pycache__\F_V.cpython-36.pyc
data_transforms: C:\Users\Master\Anaconda3\Lib\site-packages\FV\__pycache__\F_V.cpython-36.pyc
get_central_face_attributes: C:\Users\Master\Anaconda3\Lib\site-packages\FV\__pycache__\F_V.cpython-36.pyc
math: C:\Users\Master\Anaconda3\Lib\site-packages\FV\__pycache__\F_V.cpython-36.pyc
nn: C:\Users\Master\Anaconda3\Lib\site-packages\FV\__pycache__\F_V.cpython-36.pyc
np: C:\Users\Master\Anaconda3\Lib\site-packages\FV\__pycache__\F_V.cpython-36.pyc
os: C:\Users\Master\Anaconda3\Lib\site-packages\FV\__pycache__\F_V.cpython-36.pyc
pickle: C:\Users\Master\Anaconda3\Lib\site-packages\FV\__pycache__\F_V.cpython-36.pyc
plt: C:\Users\Master\Anaconda3\Lib\site-packages\FV\__pycache__\F_V.cpython-36.pyc
resnet101: C:\Users\Master\Anaconda3\Lib\site-packages\FV\__pycache__\F_V.cpython-36.pyc
resnet18: C:\Users\Master\Anaconda3\Lib\site-packages\FV\__pycache__\F_V.cpython-36.pyc
resnet50: C:\Users\Master\Anaconda3\Lib\site-packages\FV\__pycache__\F_V.cpython-36.pyc
scipy: C:\Users\Master\Anaconda3\Lib\site-packages\FV\__pycache__\F_V.cpython-36.pyc
time: C:\Users\Master\Anaconda3\Lib\site-packages\FV\__pycache__\F_V.cpython-36.pyc
torch: C:\Users\Master\Anaconda3\Lib\site-packages\FV\__pycache__\F_V.cpython-36.pyc
tqdm: C:\Users\Master\Anaconda3\Lib\site-packages\FV\__pycache__\F_V.cpython-36.pyc
transforms : C:\Users\Master\Anaconda3\Lib\site-packages\FV\__pycache__\F_V.cpython-36.pyc

However, when ever I try to use a class defined in the module (i.e FaceVerificationand instantiate it, it fails. The class seems OK, but the calls that try to instantiate the object, return a null object!: I tried both PyEval_CallObject and PyObject_CallObject:

auto python_class = PyDict_GetItemString(pdict, "FaceVerification");

if (PyCallable_Check(python_class)) 
{
    object = PyEval_CallObject(python_class, args);
    # or 
    //object = PyObject_CallObject(python_class, args);
    //Py_DECREF(python_class);
}
else 
{
    cout << "Cannot instantiate the Python class" << endl;
    //Py_DECREF(python_class);
    //return 1;
}

to no avail. This is strange, I therefore added a new CONSTANT field to the F_V.py and tried to see if I can access that (to see if the module was indeed valid). It actually successfully retrieved the constant. However, I noticed, on the re-running of the c++ application, now when I print the map which holds the output of PyModule_GetDict(module), I see a different result : 

module: 000001F7E07E8C28
size: 31
CONSTANT: THIS IS A TEST. REMOVE IT
FaceVerification: THIS IS A TEST. REMOVE IT
Image: C:\Users\Master\Anaconda3\Lib\site-packages\FV\__pycache__\F_V.cpython-36.pyc
Path: C:\Users\Master\Anaconda3\Lib\site-packages\FV\__pycache__\F_V.cpython-36.pyc
__builtins__: C:\Users\Master\Anaconda3\Lib\site-packages\FV\__pycache__\F_V.cpython-36.pyc
__cached__: C:\Users\Master\Anaconda3\Lib\site-packages\FV\__pycache__\F_V.cpython-36.pyc
__doc__: FV.F_V
__file__: C:\Users\Master\Anaconda3\Lib\site-packages\FV\F_V.py
__loader__: FV
__name__: FV.F_V
__package__: FV
__spec__: FV
__warningregistry__: THIS IS A TEST. REMOVE IT
align_face: C:\Users\Master\Anaconda3\Lib\site-packages\FV\__pycache__\F_V.cpython-36.pyc
cv2: C:\Users\Master\Anaconda3\Lib\site-packages\FV\__pycache__\F_V.cpython-36.pyc
data_transforms: THIS IS A TEST. REMOVE IT
get_central_face_attributes: C:\Users\Master\Anaconda3\Lib\site-packages\FV\__pycache__\F_V.cpython-36.pyc
math: C:\Users\Master\Anaconda3\Lib\site-packages\FV\__pycache__\F_V.cpython-36.pyc
nn: C:\Users\Master\Anaconda3\Lib\site-packages\FV\__pycache__\F_V.cpython-36.pyc
np: C:\Users\Master\Anaconda3\Lib\site-packages\FV\__pycache__\F_V.cpython-36.pyc
os: C:\Users\Master\Anaconda3\Lib\site-packages\FV\__pycache__\F_V.cpython-36.pyc
pickle: C:\Users\Master\Anaconda3\Lib\site-packages\FV\__pycache__\F_V.cpython-36.pyc
plt: C:\Users\Master\Anaconda3\Lib\site-packages\FV\__pycache__\F_V.cpython-36.pyc
resnet101: C:\Users\Master\Anaconda3\Lib\site-packages\FV\__pycache__\F_V.cpython-36.pyc
resnet18: C:\Users\Master\Anaconda3\Lib\site-packages\FV\__pycache__\F_V.cpython-36.pyc
resnet50: C:\Users\Master\Anaconda3\Lib\site-packages\FV\__pycache__\F_V.cpython-36.pyc
scipy: C:\Users\Master\Anaconda3\Lib\site-packages\FV\__pycache__\F_V.cpython-36.pyc
time: C:\Users\Master\Anaconda3\Lib\site-packages\FV\__pycache__\F_V.cpython-36.pyc
torch: C:\Users\Master\Anaconda3\Lib\site-packages\FV\__pycache__\F_V.cpython-36.pyc
tqdm: C:\Users\Master\Anaconda3\Lib\site-packages\FV\__pycache__\F_V.cpython-36.pyc
transforms: C:\Users\Master\Anaconda3\Lib\site-packages\FV\__pycache__\F_V.cpython-36.pyc

most notably the FaceVerification which should be a class, is not showing the CONSTANT field's value!

What am I doing wrong here?

Hossein
  • 24,202
  • 35
  • 119
  • 224
  • I digged a bit trough your code (and more through the one of the previous question). Not sure whether this is helpful but: [PyDict_GetItemString()](https://docs.python.org/3/c-api/dict.html#c.PyDict_GetItemString) returns a borrowed reference. Hence, the `Py_DECREF(python_class);` looks very questionable to me. – Scheff's Cat Mar 25 '20 at 07:43
  • FYI: [SO: Create an object using Python's C API](https://stackoverflow.com/q/4163018/7478597) – Scheff's Cat Mar 25 '20 at 07:46
  • @Scheff Thanks, but that snippet where it is used is not executed (it was borrowed from other codes, and I forgot to comment that part). I looked at that link, it seems it uses `PyObject_CallObject` which I also used. this also returns a null object. – Hossein Mar 25 '20 at 08:07
  • 1
    I agree concerning `PyObject_CallObject()`. I've no real idea what could be broken as well. In our code (C++ extensions for Python which is in turn used as "macro engine" in our C++ appl.), I call the "constructor" functions directly and never faced such problems. What if the constructor fails in your case. (Thrown exceptions are usually signaled in Python C code by returning `NULL`.) I would try with an even simpler MCVE if I was in your situation. (Something like this: [SO: How to list all function names of a Python module in C++?](https://stackoverflow.com/a/58078872/7478597).) ;-) – Scheff's Cat Mar 25 '20 at 08:25
  • Thanks a lot. I'll have a look at it. really appreciate your kind help – Hossein Mar 25 '20 at 10:46

1 Answers1

0

As it turns out(many thanks to @Scheff), the reason why the object was null, was because an exception was happening inside the constructor and thus resulted in the object being null. The reason for the exception was a malformed path and problematic (wrong) arguments.
basically I had set the arguments like this : 

auto model_name = "r189";
auto model_checkpoint_path = PyUnicode_DecodeFSDefault("L:\\test_procs\\trained_models\BEST_checkpoint_r189.tar");
auto align_fn = Py_None;
auto img_bank_folder_root = PyUnicode_DecodeFSDefault("L:\\test_procs\\data\\facebank_2");
auto cache_folder = Py_None;
auto postfix = Py_None;
auto rebuild_cache = Py_False;
auto use_jit = Py_False;
auto threshold = 65; 
auto device = "cpu";

and the arguments thus like this : 

auto args = Py_BuildValue("sOOOOONNiN", model_name, model_checkpoint_path, align_fn,
                                  img_bank_folder_root, cache_folder, postfix, 
                                  rebuild_cache, use_jit, threshold, device);

which was wrong. I changed them to this : 

auto model_name = "r189";
auto model_checkpoint_path = "L:\\test_procs\\trained_models\BEST_checkpoint_r189.tar";
auto align_fn = "";
auto img_bank_folder_root = "L:\\test_procs\\data\\facebank_2";
auto cache_folder = "";
auto postfix = "";
auto rebuild_cache = 0;
auto use_jit = 0;
auto threshold = 65; 
auto device = "cpu";

and simplified the args like this : 

auto args = Py_BuildValue("ssssssiiis", model_name, model_checkpoint_path, align_fn,
                          img_bank_folder_root, cache_folder, postfix, 
                          rebuild_cache, use_jit, threshold, device);

after doing all of this, I still faced the exception and found out : model_checkpoint_path needed a fixed as its path lacked a single \ in its path: 

auto model_checkpoint_path = "L:\\test_procs\\trained_models\BEST_checkpoint_r189.tar";

fixing it : 

auto model_checkpoint_path = "L:\\test_procs\\trained_models\\BEST_checkpoint_r189.tar";

thus solved everything.

Hossein
  • 24,202
  • 35
  • 119
  • 224