1

I try to make an extension in C for Python. I've never done this before. Regardless of what the function in C will do, the problem is as follows. I use malloc to allocate memory. But when I use the free() function

setup.py:

from distutils.core import setup, Extension
setup(name='rpd', version='1.0',  \
      ext_modules=[Extension('rpd', ['RPB.c'])])

I install the module with: python setup.py install. But then after I type:

import rpd
rpd.rpd("No reason to give string")

the IDLE GUI freezes and restarts. And I don't see not calling free() on allocated memory as an option. Any help would be appreciated.

EDIT:

So I managed to run the previous version of code, although I didn't make any dramatic changes. And I can't still figure out why yesterday the same code wouldn't work. Then I wanted to make a step towards the full C code I have. So I added a function in RPB.c code and some minor changes. You can see the updated code below.. but now, I got the same problem again :/

RPB.c:

 #include "Python.h"

 char** tokenize(const char* input)
 {
    char* str = strdup(input);
    int count = 0;
    int capacity = 8;
    char** result = malloc(capacity*sizeof(*result));

    char* tok=strtok(str," ");

    while(1)
    {
        if (count >= capacity)
            result = realloc(result, (capacity*=2)*sizeof(*result));

        result[count++] = tok? strdup(tok) : tok;

        if (!tok) break;

        tok=strtok(NULL," ");
    }

    // When this line is executed... b00m!
    free(str);
    return result;
}

static PyObject* rpd(PyObject *self, PyObject *args){

    char *filename, **test;
    if (!PyArg_ParseTuple(args, "s", &filename)) {
        return Py_BuildValue("s", "Error during reading passed parameters!");
    }

    // test = malloc(strlen(filename)+1);

    test = tokenize(filename);

    // This is never returned due to free in tokenize - placed here for debug purposes
    return Py_BuildValue("s", "What again?!");

    // When this line is executed... b00m!
    //free(test);
   
    // This is never returned, because program crashes
    //return Py_BuildValue("s", "Everything ok");
   
}

static char rpd_docs[] ="rpd(c): Give a filename!\n";

static PyMethodDef rpd_funcs[] = {
        {"rpd", (PyCFunction)rpd,
         METH_VARARGS, rpd_docs},
        {NULL, NULL, 0, NULL}
};

void initrpd(void)
{
      Py_InitModule3("rpd", rpd_funcs, "Give a filename!!");
}
Community
  • 1
  • 1
user3674296
  • 69
  • 1
  • 7
  • 2
    You should probably be allocating one more than `strlen(filename)` (although that depends on the mystery code). Are you saying that the exact code you've posted above blows up? Or does the code that blows up have something between the `malloc` and the `free`? – ooga May 26 '14 at 00:14
  • Are you sure that your `malloc` isn't failing? Because if it does, you will be calling `free` on `NULL` (0), causing a Segmentation Fault. As per [the documentation](http://linux.die.net/man/3/malloc): *"The malloc() and calloc() functions return a pointer to the allocated memory that is suitably aligned for any kind of variable. On error, these functions return NULL. NULL may also be returned by a successful call to malloc() with a size of zero, or by a successful call to calloc() with nmemb or size equal to zero."* – 7heo.tk May 26 '14 at 00:17
  • @ooga No, there is nothing between malloc and free. I know strlen should have one more. I wrote this code for demonstration purposes. The exact above code blows up. I tried without using free() and it is just fine. I don't get it. The documentation for PyArg_ParseTuple states: "Also, you won’t have to release any memory yourself, except with the es, es#, et and et# formats." But that's malloc()'ed memory by me!!! – user3674296 May 26 '14 at 00:19
  • 1
    @7heo.tk read reference of `free`. – BLUEPIXY May 26 '14 at 00:22
  • @7heo.tk I am pretty sure. Because if I put a string in it and change the return statement to return Py_BuildValue("s", test); the string is returned. (With the free() commented out.. which creates a memory leak ?!? ) – user3674296 May 26 '14 at 00:22
  • 1
    @7heo.tk There is no problem to `free(NULL)`. – BLUEPIXY May 26 '14 at 00:24
  • @BLUEPIXY Alright, my bad; It was a wild guess, but it would have explained the behavior very well. Thanks for pointing my mistake out. – 7heo.tk May 26 '14 at 00:25
  • @user3674296 Do you have some more information on the nature of the crash? (maybe using gdb?) – 7heo.tk May 26 '14 at 00:26
  • @7heo.tk No, I haven't tried this yet. I will. But.. wtf why is this happening ?! I read all those google examples and this is how it is used... – user3674296 May 26 '14 at 00:36
  • @user3674296 It's really strange indeed, from the code you pasted I see no reason why this would happen; other than the `free(NULL)` guess I had; but that have been ruled out by BLUEPIXY... I will try to compile something here and see if it bugs too. – 7heo.tk May 26 '14 at 00:39
  • 1
    @user3674296 Okay... Here's what I did: I copied your `c` code (all 29 lines) into a file named `RPB.c`. I copied your distutils `python` code into a `build.py` file. I executed it; got the `build` directory, symlinked `build/lib.linux-x86_64-2.7/rpd.so` in the current directory; and executed your two lines of python in python 2.7. *"Everything ok"* was printed out; no problem whatsoever. And no, I didn't comment the `free` out. So there probably is a problem on your side. I'm running Debian 7, and using python 2.7. – 7heo.tk May 26 '14 at 01:11
  • `#include ` - that doesn't look right. `Python.h` isn't a standard library header. Shouldn't it be `"Python.h"`? You may have done something very weird with your installation. – user2357112 May 26 '14 at 01:29
  • @user2357112 please read http://stackoverflow.com/questions/21593 and https://docs.python.org/2/extending/extending.html – 7heo.tk May 26 '14 at 01:53
  • @user2357112 What makes you thing `<>` only applies to standard header files? – glglgl May 26 '14 at 06:53
  • @glglgl: My understanding may be incomplete, and I may be mixing up convention and actual compiler behavior. I believe it is conventional to restrict usage of `<>` to system headers. Some searching online seems to suggest that `""` looks in a superset of the places that `<>` does, and that `Python.h` would in fact likely be found in one of the places `<>` looks. – user2357112 May 26 '14 at 07:07
  • @user2357112 Right. IMO `""` should be used for all header files which belong to the current project, and `<>` for all header files which are extern to it. That includes system headers as well as external libraries. – glglgl May 26 '14 at 07:44
  • Not quite answering your question, but for dealing with the C API you can use Cython. It will auto generate the C code for you. Having it installed is only a requirement at build time. – Davidmh May 26 '14 at 10:31

1 Answers1

0

A plausible cause would be a buffer overrun (or other memory-related error) before this code is called. This would not be detected until the free(), because this is when the memory allocator attempts to coalesce blocks.

The code looks fine, so I suggest you look for the problem before this function is even called.

david.pfx
  • 10,520
  • 3
  • 30
  • 63