3

If it makes a difference, I am interested in an answer regarding Python 3.

The docs state (here and here) that PyBuffer_Release() should be called after PyArg_Parse*() with s*, y*.

Nothing of the sort is written about Py_BuildValue(). Is it an oversight, or in case of Py_BuildValue() a simple Py_DECREF() is enough?

Here is my specific case:

uint8_t buf = (uint8_t *)malloc(bufSize);
PyObject *pyBuf = Py_BuildValue("y#", (char *)buf, bufSize);
free(buf);

// do something with pyBuf

// maybe a PyBuffer_Release(get_underlying_buffer(pyBuf)) here?
Py_DECREF(pyBuf);
Xantium
  • 11,201
  • 10
  • 62
  • 89
Zoltan K.
  • 1,036
  • 9
  • 19

1 Answers1

2

I think no:

  1. In both the PyArg_Parse* and Py_BuildValue functions, y# refers to a string and length, rather than a buffer, and therefore there is no underlying buffer object to be released.

  2. The documentation for Py_BuildValue says:

    When memory buffers are passed as parameters to supply data to build objects, as for the s and s# formats, the required data is copied. Buffers provided by the caller are never referenced by the objects created by Py_BuildValue().

    The purpose of holding a lock on the buffer used by PyArg_Parse* is that you have got a reference to some data out of Python to C, and you want to process it in C without any chance of it being modified by Python. In this case you have copied some data from C into Python and so there's no need to protect the original data from modification.

DavidW
  • 29,336
  • 6
  • 55
  • 86
  • **"In this case you have copied some data from C into Python and so there's no need to protect the original data from modification."** - I think at this point Python might decide that there is a new object that has a reference to that copied buffer, therefore during the construction of `Py_buffer` some internal reference counter is increased that later must be released by `PyBuffer_Release()`. At least, this is what I fear. Maybe the newly created object is itself a `Py_buffer`. Maybe later I will create some testcode to ascertain what objects are created and what resources they hold. – Zoltan K. Jul 07 '18 at 03:29
  • When I test this the newly created object is a `bytes` object (the only other realistic option was a `bytearray`). This does expose the buffer interface, and so It would be possible to get a `Py_buffer`. However, even if `Py_BuildValue` produced one requiring the buffer to be released, you could do nothing about it since you don't have access to whatever `Py_buffer` it had created – DavidW Jul 07 '18 at 07:45
  • So basically, whatever resources are automatically held within `bytes`, those should be released by the `bytes` object automatically. That makes sense. I think for absolute certainty, the python code would have to be looked up. For now it is good enough for me. – Zoltan K. Jul 07 '18 at 22:12