python multiprocess problem with OpenCV returning key points

Question

I'm using multiprocess Python module to parallelise processing with OpenCV algorithms (e.g. ORB detector/descriptor). The multiprocess module works fine in most cases, but when it comes to returning a list of cv2.KeyPoint objects there is a problem - all fields of each key point are set to 0 when returned to the caller process, although inside the worker process all key points are correct (as returned by OpenCV).

Here is minimal example that can be used to reproduce the error (you will need an image file called lena.png to make it work):

import numpy as np

from cv2 import ORB_create, imread, cvtColor, COLOR_BGR2GRAY
from multiprocess import Pool

feature = ORB_create(nfeatures=4)

def proc(img):
    return feature.detect(img)

def good(feat, frames):
    return map(proc, frames)

def bad(feat, frames):
    # this starts a worker process
    # and then collects result
    # but something is lost on the way
    pool = Pool(4)
    return pool.map(proc, frames)

if __name__ == '__main__':
    # it doesn't matter how many images
    # a list of images is required to make use of
    # pool from multiprocess module
    rgb_images = map(lambda fn: imread(fn), ['lena.png'])
    grey_images = map(lambda img: cvtColor(img, COLOR_BGR2GRAY), rgb_images)
    good_kp = good(feature, grey_images)
    bad_kp = bad(feature, grey_images)

    # this will fail because elements in
    # bad_kp will all contain zeros
    for i in range(len(grey_images)):
    for x, y in zip(good_kp[i], bad_kp[i]):
            # these should be the same
            print('good: pt=%s angle=%s size=%s - bad: pt=%s angle=%s size=%s' % (x.pt, x.angle, x.size, y.pt, y.angle, y.size))
            assert x.pt == y.pt

Platforms: both CentOS 7.6 and Windows 10 x64

Versions:

Python version: 2.7.15
multiprocess: 0.70.6.1
opencv-python-headless: 3.4.5.20 and 4.0.0.21

Is there a way to work around this? Use of standard multiprocessing module is not an option because of heavy usage of lambdas and callable objects which "can't be pickled".

scrutari · Answer 1 · 2019-02-01T13:49:07.253

After some analysis it turned out that the problem is caused by something about cv2.KeyPoint class. This is suggested in a related question and corresponding answer. The problems is that pickle which is apparently used by dill is unable to work with this class.

A simple solution is to avoid sending instances of cv2.KeyPoint between worker and main process. If this is not convenient, then one should wrap data of each keypoint in a simple Python structure or dictionary and pass it.

An example of wrapper could be:

import cv2
class KeyPoint(object):

    def __init__(self, kp):
        # type: (cv2.KeyPoint) -> None
        x, y = kp.pt
        self.pt = float(x), float(y)
        self.angle = float(kp.angle) if kp.angle is not None else None
        self.size = float(kp.size) if kp.size is not None else None
        self.response = float(kp.response) if kp.response is not None else None
        self.class_id = int(kp.class_id) if kp.class_id is not None else None
        self.octave = int(kp.octave) if kp.octave is not None else None

   def to_opencv(self):
        # type: () -> cv2.KeyPoint
        kp = cv2.KeyPoint()
        kp.pt = self.pt
        kp.angle = self.angle
        kp.size = self.size
        kp.response = self.response
        kp.octave = self.octave
        kp.class_id = self.class_id
        return kp

python multiprocess problem with OpenCV returning key points

1 Answers1