python: ctypes, read POINTER(c_char) in python

Question

I have a ctypes field that is a POINTER(c_char) (it had to be, per the documentation, c_char_p didn't work for my application: https://docs.python.org/3.7/library/ctypes.html#ctypes.c_char_p)

For a general character pointer that may also point to binary data, POINTER(c_char) must be used.

However, this usage recommended by ctypes itself seems to have the downside that, it claims to be a pointer to a single character, however, it isn't, it's a pointer to an array of bytes.

How can I read the array returned by the ctypes fucntion (I know the length) in Python? Trying to index it like foo[0:len] where foo is a POINTER(c_char) blows up with TypeError: 'c_char' object is not subscriptable

I can print the first character of the bytestring using either print(foo) or print(foo[0])

I was thinking that ctypes.cast might work, however I don't know how to pass it the length of the cast (as in interpret the first N bytes from address foo as a bytes object)

EDIT: some code.

So I have a structure:

class foo(Structure):
_fields_ = [("state", c_int),
            ("type", c_int),
            ("len", c_int),
            ("payload", POINTER(c_char))]  # according to th following the python bytes are already unsinged https://bytes.com/topic/python/answers/695078-ctypes-unsigned-char

And I have another function that returns a POINTER(foo)

lib3 = CDLL(....so)
f = lib3.f
f.restype = POINTER(foo)

I call f, which returns a POINTER(foo):

ptrf = f(....)

And then I was trying to access ptrf.payload. The following code works:

def get_payload(ptr_to_foo):

    val = cast(ptr_to_foo.contents.payload, c_char_p).value
    return val[:ptr_to_foo.contents.len]

So I do

 ptrf = f(....)
 get_payload(ptrf)

I was wondering whether the get_payload function would be written more easily.

Answer 1

If you truly have a POINTER(c_char) type, it is subscriptable. In the future provide code that reproduces your issue:

>>> p = cast(create_string_buffer(b'Hello, world!'),POINTER(c_char))
>>> p
<ctypes.LP_c_char object at 0x000001C2F6B58848>
>>> p[0]
b'H'
>>> p[:14]
b'Hello, world!\x00'
>>> cast(p,c_char_p).value  # only if known to be nul-terminated
b'Hello, world!'

Answer 2

As [Python.Docs]: ctypes - A foreign function library for Python states, you must not use c_char_p with binary data.
Of course that can be ignored, but then surprises (string silently truncated) may occur.

Although it could be exemplified in ~5 lines of code, pasting the whole thing:

dll.c:

#include <stdlib.h>

#if defined(_WIN32)
#  define DLL_EXPORT __declspec(dllexport)
#else
#  define DLL_EXPORT
#endif

#define LEN 5


typedef struct CharPtrWrapperTag {
    int len;
    char *data;
} CharPtrWrapper;


DLL_EXPORT CharPtrWrapper *get() {
    CharPtrWrapper *ret = malloc(sizeof(CharPtrWrapper));
    ret->len = LEN;
    ret->data = malloc(LEN * sizeof(char));
    ret->data[0] = 'A';
    ret->data[1] = 'B';
    ret->data[2] = 0;
    ret->data[3] = 'C';
    ret->data[4] = 'D';
    return ret;
}


DLL_EXPORT void release(CharPtrWrapper *pWrap) {
    if (pWrap) {
        free(pWrap->data);
        pWrap->data = NULL;
        pWrap->len = 0;
        free(pWrap);
    }
}

code.py:

#!/usr/bin/env python3

import sys
import ctypes


DLL_NAME = "./dll.dll"
CharPtr = ctypes.POINTER(ctypes.c_char)

class CharPtrWrapper(ctypes.Structure):
    _fields_ = [
        ("len", ctypes.c_int),
        ("data", CharPtr),
    ]


CharPtrWrapperPtr = ctypes.POINTER(CharPtrWrapper)


def main():
    dll = ctypes.CDLL(DLL_NAME)
    get = dll.get
    get.restype = CharPtrWrapperPtr
    release = dll.release
    release.argtypes = [CharPtrWrapperPtr]
    wrap_ptr = get()
    wrap = wrap_ptr.contents
    print("{:}\n    Len: {:d}".format(wrap, wrap.len))
    for idx in range(wrap.len):
        print("        {:d}: {:}".format(idx, wrap.data[idx]))

    s = ctypes.cast(wrap.data, ctypes.c_char_p).value[:wrap.len]
    print("\nctypes.c_char_p cast: {:}".format(s))

    CharArr = ctypes.c_char * wrap.len
    char_arr = CharArr(*wrap.data[:wrap.len])
    print("CharArr: {:}".format(char_arr.raw))
    release(wrap_ptr)
    print("\nDone.")


if __name__ == "__main__":
    print("Python {:s} on {:s}\n".format(sys.version, sys.platform))
    main()

Output:

[cfati@CFATI-5510-0:e:\Work\Dev\StackOverflow\q055103298]> sopr.bat
*** Set shorter prompt to better fit when pasted in StackOverflow (or other) pages ***

[prompt]> "c:\Install\x86\Microsoft\Visual Studio Community\2015\vc\vcvarsall.bat" x64

[prompt]> dir /b
code.py
dll.c

[prompt]> cl /nologo /DDLL /MD dll.c  /link /NOLOGO /DLL /OUT:dll.dll
dll.c
   Creating library dll.lib and object dll.exp

[prompt]> dir /b
code.py
dll.c
dll.dll
dll.exp
dll.lib
dll.obj

[prompt]> "e:\Work\Dev\VEnvs\py_064_03.06.08_test0\Scripts\python.exe" code.py
Python 3.6.8 (tags/v3.6.8:3c6b436a57, Dec 24 2018, 00:16:47) [MSC v.1916 64 bit (AMD64)] on win32

<__main__.CharPtrWrapper object at 0x000001279250D248>
    Len: 5
        0: b'A'
        1: b'B'
        2: b'\x00'
        3: b'C'
        4: b'D'

ctypes.c_char_p cast: b'AB'
CharArr: b'AB\x00CD'

Done.