Struggling to call the method from native DLL using C#

Question

Here is the method signature I am trying to call.

EXTERN_C
HRESULT
QueryData(
    _Outptr_opt_result_bytebuffer_(*SizeOfData)  PBYTE * Data,
    _Out_opt_                                   UINT32* SizeOfData,
    _In_                                        BOOL    IsDataType
)

The above method is not my code it's vendor code and unfortunately I don't have enough knowledge how to call this method. All I know is it's suppose to get me a blob of data.

Here is what I have done so far.

 [DllImport("DataGetter.dll")]
        internal static extern int QueryData(IntPtr data, UIntPtr sizeOfData, bool isDataType);

IntPtr data= new IntPtr();
            UIntPtr sizeOfData= new UIntPtr();
            bool isDataType= true;
            int hresult = QueryData(data, sizeOfData, isDataType);

My method doesn't fail but it doesn't return any thing in the data. Any idea how to call this weird method from C#?

Did you try calling any simpler functions? Or explicitrly set calling convention ? `[DllImport("DataGetter.dll", CallingConvention = CallingConvention.Cdecl)]` — kocica, Aug 17 '17 at 18:42
I have used P/Invoke in the past with simple functions but, regarding this DLL this is the only method It exposes. Interesting when I add calling convention the calls fails with following error. "QueryData has unbalanced the stack. This is likely because the managed PInvoke signature doesn't match the unmanaged target signature. Check the calling convention and parameters of the PInvoke signature match the target unmagnaged signature." — user1144852, Aug 17 '17 at 18:43
It's been a long time since I've done any p/invoke, but I think your problem is the 'Data' member. The function is accepting a BYTE** (not a BYTE*). Evidently it allocates a buffer internally and returns you both a pointer to the buffer and its size. Unless you do something to clean up after the call, you will likely end up with a memory leak. — Peter Ruderman, Aug 17 '17 at 18:53
Yeah, how are you going to deallocate the memory? Is it allocated on the COM heap? — David Heffernan, Aug 17 '17 at 19:24
Not sure about how can I deallocate the memory? Do you know a way to deallocate from C# code itself? Is this sufficient enough Marshal.DestroyStructure(data, typeof(IntPtr));? — user1144852, Aug 17 '17 at 19:55

R2RT · Accepted Answer · 2017-08-18T05:57:20.397

2

You have two problems here: first is to get values set by QueryData into Data and sizeOfData, which get pointers to local variables. You can do it with ref or out keyword, so UINT32* SizeOfData in C++ becomes ref System.UInt32 SizeOfData. Key difference between them is that out arguments do not have to be initialized before function call. Second is to transfer unmanaged array defined in C++ into C#. You can do it with Marshall.Copy.

One thing remains unclear, but should be stated in documentation - whenever array returned from C++ is allocated dynamically and needs to be freed in C# or not. If it does you will have memory leak that will increase memory usage with every call of function. The easiest way to test for it is to call function 1000000 times and check memory usage.

Full code:

    [DllImport("DataGetter.dll"]
    internal static extern int QueryData(out IntPtr data, out System.UInt32 sizeOfData, bool isDataType);

    void example()
    {
        IntPtr dataPtr;
        System.UInt32 sizeOfData;
        bool isDataType = false;
        int hresult = QueryData(out dataPtr, out sizeOfData, isDataType);
        var data = new byte[sizeOfData];
        Marshal.Copy(dataPtr, data, 0, (int)sizeOfData);
        // data now contains retreived bytes
    }

OLD POST: Try with.

    [DllImport("DataGetter.dll")]
    internal static extern int QueryData(ref IntPtr data, ref System.UInt32 sizeOfData, bool isDataType);

I'm not sure what is PBYTE but I suppose it is pointer to BYTE. Function should change data and sizeOfData variables.

edited Aug 18 '17 at 05:57

answered Aug 17 '17 at 18:53

R2RT

2,061
15
25

This returns something but, do you know how to get byte array from IntPtr? – user1144852 Aug 17 '17 at 19:02
@user1144852 https://msdn.microsoft.com/en-us/library/ms146631(v=vs.110).aspx This should work – R2RT Aug 17 '17 at 19:13
I did something like follow and seems to be working not sure if that correct though. IntPtr data= new IntPtr(); UIntPtr sizeOfData= new UIntPtr(); bool isDataType= false; int hresult = QueryData(ref data, ref sizeOfData, isDataType); List dataBits = new List(); for (int i = 0; i < sizeOfData.ToUInt32(); i++) { dataBits.Add(Marshal.ReadByte(data, i)); } string hex = BitConverter.ToString(dataBits.ToArray()); – user1144852 Aug 17 '17 at 19:48
@user1144852 I've updated my answer with code snipped and some explanation. I don't know how could I help you with veryfying whenever data is correct. Can you make some test? Do you know what to expect? – R2RT Aug 17 '17 at 20:50
@DavidHeffernan I had to make a little reserach ahd I am not convinced. As far as I found it is "__cdecl is the default for C and C++ programs according to the Visual C++ compiler and the WinAPI functions use the __stdcall convention." (https://stackoverflow.com/a/3404412/5945883, point 2). Acording to winnt.h stdcall is defined with STDAPI, not EXTERN_C. So there is no stdcall keyword, thus uses default, that is cdecl, right? – R2RT Aug 17 '17 at 21:55
Your solution works but I can't use cdecl. It fails when calling convention is set to cdecl. If you can update your answer without cdecl I would mark it as answer. – user1144852 Aug 17 '17 at 22:43
Should really be out params rather than ref. And I guess that something else, somewhere in the unmanaged code, specified stdcall. – David Heffernan Aug 18 '17 at 04:41
@DavidHeffernan https://stackoverflow.com/q/33815276/5945883 It's long answer, but one of comments is short: "using ref is not optional and cannot be substituted with [In, Out]. You must generate a pointer if the native code requires one and that requires using ref. The next consideration is then the attributes to describe the data flow." – R2RT Aug 18 '17 at 05:16
I said out not [Out] – David Heffernan Aug 18 '17 at 05:17

Struggling to call the method from native DLL using C#

1 Answers1