How to better identify what causes segmentation fault in python?

Question

I am having trouble with segmentation fault in Python. Sometimes it happens, sometimes it doesn't but when it does I used gdb python as explained in this answer and I get this result:

[... a lot more lines before this ...]
[New Thread 0x7fff097fa700 (LWP 23244)]

Thread 31 "python" received signal SIGSEGV, Segmentation fault.                                                                                                                                  
[Switching to Thread 0x7fff4affd700 (LWP 23227)]                                                                                                                                                 
__GI___libc_free (mem=0x10035) at malloc.c:3103                                                                                                                                                  
3103    malloc.c: No such file or directory.
(gdb) backtrace
#0  __GI___libc_free (mem=0x10035) at malloc.c:3103
#1  0x00007fff70cc2a39 in ?? () from /<miniconda env path>/lib/python3.6/site-packages/cv2/cv2.cpython-36m-x86_64-linux-gnu.so
#2  0x00007fff70939ef5 in ?? () from /<miniconda env path>/lib/python3.6/site-packages/cv2/cv2.cpython-36m-x86_64-linux-gnu.so
#3  0x00007fff7103a6c5 in ?? () from /<miniconda env path>/lib/python3.6/site-packages/cv2/cv2.cpython-36m-x86_64-linux-gnu.so
#4  0x00007fff70d8263e in ?? () from /<miniconda env path>/lib/python3.6/site-packages/cv2/cv2.cpython-36m-x86_64-linux-gnu.so
#5  0x00007fff70d8304a in ?? () from /<miniconda env path>/lib/python3.6/site-packages/cv2/cv2.cpython-36m-x86_64-linux-gnu.so
#6  0x00007fff70d83d1d in ?? () from /<miniconda env path>/lib/python3.6/site-packages/cv2/cv2.cpython-36m-x86_64-linux-gnu.so
#7  0x00007fff70d83e89 in ?? () from /<miniconda env path>/lib/python3.6/site-packages/cv2/cv2.cpython-36m-x86_64-linux-gnu.so
#8  0x00007ffff7bbd6db in start_thread (arg=0x7fff4affd700) at pthread_create.c:463
#9  0x00007ffff78e688f in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:95

So the problem seems to be something related to cv2 if I uderstood the output correctly.

I am not sure if it is valid to say that, but the logs printed using Python Logging stop when I call a function from the module Albumentations which kinda makes sense since they use OpenCV.

The code I am using from Albumentations is this (not sure if it helps):

import cv2
from albumentations import ElasticTransform, GridDistortion, OneOf, Compose
import logging


class DataAug:
    def __init__(self):
        self.logger = logging.getLogger(__name__)

    @staticmethod
    def get_augs():
        grid = GridDistortion(
            distort_limit=0.2,
            interpolation=cv2.INTER_NEAREST,
            border_mode=cv2.BORDER_CONSTANT,
            value=0,
            always_apply=True
        )
        elast = ElasticTransform(
            alpha_affine=20,
            interpolation=cv2.INTER_NEAREST,
            border_mode=cv2.BORDER_CONSTANT,
            value=0,
            always_apply=True
        )
        return OneOf([
            OneOf([grid, elast], p=1),
            Compose([grid, elast], p=1)
        ], p=1)

    def apply(self, image, mask):
        data = {
            'image': image.astype('uint8'),
            'mask': mask.astype('uint8')
        }
        aug = self.get_augs()
        self.logger.debug('Apply augmentations')
        res = aug(**data)
        return res['image'], res['mask']

The last line printed to the log file is the Apply augmentations.

Is there any other method to better identify what could be the problem causing segmentation fault? Is it actually a problem with the module or am I doing something wrong that is causing it?

EDIT: Changing the apply method to this does not fix the problem

def apply(self, image, mask):
    aug = self.get_augs()
    self.logger.debug('Apply augmentations')
    res = aug(image=image.astype('uint8'), mask=mask.astype('uint8'))
    return res['image'], res['mask']

EDIT: Another thing I noticed is the seg fault only happens when I am loading pickle files instead of creating the objects in code. I do this to avoid applying this process everytime to save time the next execution.

EDIT: Minimal Reproducible Example (seg fault might not happen on every execution, out of 21 executions, 8 were seg fault)

import pickle
import cv2
import numpy as np
from albumentations import ElasticTransform, GridDistortion, OneOf, Compose


class DataAug:
    @staticmethod
    def get_augs():
        grid = GridDistortion(
            distort_limit=0.2,
            interpolation=cv2.INTER_NEAREST,
            border_mode=cv2.BORDER_CONSTANT,
            value=0,
            always_apply=True
        )
        elast = ElasticTransform(
            alpha_affine=20,
            interpolation=cv2.INTER_NEAREST,
            border_mode=cv2.BORDER_CONSTANT,
            value=0,
            always_apply=True
        )
        return OneOf([
            OneOf([grid, elast], p=1),
            Compose([grid, elast], p=1)
        ], p=1)

    def apply(self, image, mask):
        data = {
            'image': image.astype('uint8'),
            'mask': mask.astype('uint8')
        }
        aug = self.get_augs()
        res = aug(**data)
        return res['image'], res['mask']


if __name__ == '__main__':
    prc_path = {
        "a": "test-seg-fault/a",
        "b": "test-seg-fault/b"
    }

    sid = 'test1'

    with open(f'{prc_path["a"]}/{sid}', 'wb') as f:
        pickle.dump(np.zeros((1, 512, 512, 3)), f)
    with open(f'{prc_path["b"]}/{sid}', 'wb') as f:
        pickle.dump(np.zeros((1, 512, 512, 1)), f)

    with open(f'{prc_path["a"]}/{sid}', 'rb') as f:
        input_a = pickle.load(f)
    with open(f'{prc_path["b"]}/{sid}', 'rb') as f:
        input_b = pickle.load(f)

    image, mask = DataAug().apply(image=input_a[0, :, :, :], mask=input_b[0, :, :, 0])

    print(image.shape, mask.shape)

The backtrace of the Minimal Reproducible Example:

Thread 31 "python" received signal SIGSEGV, Segmentation fault.
[Switching to Thread 0x7fffa8dd3700 (LWP 28584)]
__GI___libc_free (mem=0x10035) at malloc.c:3103
3103    malloc.c: No such file or directory.
(gdb) backtrace
#0  __GI___libc_free (mem=0x10035) at malloc.c:3103
#1  0x00007ffff5214a39 in ?? () from /<miniconda env path>/lib/python3.6/site-packages/cv2/cv2.cpython-36m-x86_64-linux-gnu.so
#2  0x00007ffff4e8bef5 in ?? () from /<miniconda env path>/lib/python3.6/site-packages/cv2/cv2.cpython-36m-x86_64-linux-gnu.so
#3  0x00007ffff558c6c5 in ?? () from /<miniconda env path>/lib/python3.6/site-packages/cv2/cv2.cpython-36m-x86_64-linux-gnu.so
#4  0x00007ffff52d463e in ?? () from /<miniconda env path>/lib/python3.6/site-packages/cv2/cv2.cpython-36m-x86_64-linux-gnu.so
#5  0x00007ffff52d504a in ?? () from /<miniconda env path>/lib/python3.6/site-packages/cv2/cv2.cpython-36m-x86_64-linux-gnu.so
#6  0x00007ffff52d5d1d in ?? () from /<miniconda env path>/lib/python3.6/site-packages/cv2/cv2.cpython-36m-x86_64-linux-gnu.so
#7  0x00007ffff52d5e89 in ?? () from /<miniconda env path>/lib/python3.6/site-packages/cv2/cv2.cpython-36m-x86_64-linux-gnu.so
#8  0x00007ffff7bbd6db in start_thread (arg=0x7fffa8dd3700) at pthread_create.c:463
#9  0x00007ffff78e688f in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:95

EDIT: Removing the dump and pickle read and declaring the variables input_a = np.zeros((1, 512, 512, 3)), input_b = np.zeros((1, 512, 512, 1)) segmentations fault still happens.

Seg faults don't really happen in Python, so if you encounter one most of the time it's not your fault---it's an issue with the library / its bindings. In particular, segfaults are not entirely uncommon in OpenCV python code, but usually they can be avoided after encountering them. Can you please edit down your code to the absolute minimal amount of code necessary to reproduce the error, and include the calling code (i.e. whatever creates the class and so on)? Also it's no surprise the edit doesn't change anything, that code is almost identical :) — alkasm, Jan 06 '20 at 19:43
@alkasm the code I am working with is pretty big, it would be difficult to edit down to the absolute minimal amount. Another thing I notice is the error only happens when I am loading pickle files instead of creating the objects in code. To avoid applying this process everytime I save a bunch of numpy arrays using pickle to save time the next execution. — Gustavo, Jan 06 '20 at 19:49
That seems highly relevant, can you at least include your code that unpickles an image and calls this function? Also, this code seems pretty close to the minimal amount needed to repro the error. Maybe try getting an image, loading it, pickle it, unpickle it, and see if you can repro the error from there. In general, distilling your problem down to a [mcve] might even solve your problem, so it's highly worth doing. You'll likely at least find out *exactly* where the problem is. — alkasm, Jan 06 '20 at 20:18
I created a minimal example that executes the same part of the code after loading the pickle files, but no seg fault is happening. — Gustavo, Jan 07 '20 at 14:44
I've managed to create a minimal reproducible example, I will update the question with it. — Gustavo, Jan 07 '20 at 16:14
I encounter the same problem. Did you manage to make it work ? Ubuntu 18.04 with python3.6, OpenCV 4.2.0.34 — secavfr, Apr 15 '20 at 07:07
No, I could not find any solution, I just didn't use the library. I posted a bug issue in their github https://github.com/albumentations-team/albumentations/issues/509 maybe you should reply there, there's still no solution — Gustavo, Apr 15 '20 at 19:40

How to better identify what causes segmentation fault in python?

0 Answers0