C++ dll and C# call

Question

I have a function made in C++ that calls a COM interface's function Its signature:

BOOL func(LPWSTR strIn, __out LPWSTR strOut)
{
  //initcom
  //do something
  // release pointers
}

In C#:

[DllImport("funcdll.dll")]
static extern bool func(String strIn, ref String strOut);

// use it

for(int i=0;i<10;i++)
{
   if(func(strin, strout))
   {
      //do something with strout
   }
}

I have tested my dll in a C++ console application, it works, but in C# it crashes with an unknown error.

Answer 1

You've got three problems that I can see.

1. The calling conventions don't match. Your C++ code is cdecl and your C# code is stdcall.
2. The C++ code uses wide strings, but the C# code marshals ANSI strings.
3. The second parameter doesn't match. Your C# code assumes that the C++ code returns a new pointer to a C string which the C# code then deallocates with the COM allocator. Your C++ code doesn't do this.

Now, dealing with these in more detail.

Calling conventions

This is pretty easy to fix. Simple change the C++ code to stdcall, or the C# code to cdecl. But don't do both. I'd change the C# code:

[DllImport("funcdll.dll"), CallingConvention=CallingConvention.Cdecl]

Unicode/ANSI strings

I presume you are wanting to use Unicode strings since you have explicitly selected them in the C++ code. But P/invoke defaults to marshalling ANSI strings. You can change this again in the DllImport like so:

[DllImport("funcdll.dll"), CallingConvention=CallingConvention.Cdecl, 
    CharSet=CharSet.Unicode]

Returning a string from C++ to C#

Your current C++ function declaration is so:

BOOL func(LPWSTR strIn, __out LPWSTR strOut)

The __out decorator has no real effect, other than documenting that you want to modify the buffer pointed to by strOut and have those modifications returned to the caller.

Your C# declaration is:

static extern bool func(String strIn, ref String strOut);

Now, ref String strOut simply does not match. A ref string parameter matches this in C++:

BOOL func(LPWSTR strIn, LPWSTR *strOut)

In other words the C# code is expecting you to return a new pointer. In fact it will then proceed to deallocate the buffer you returned in strOut by calling CoTaskMemFree. I'm confident that's not what you want.

Your original C++ code can only return a string to the C# code by modifying the buffer that was passed to it. That code would look like this:

BOOL func(LPWSTR strIn, __out LPWSTR strOut)
{
    ...
    wcscpy(strOut, L"the returned string");
    ...
}

If this is what you want then you should allocate a sufficient buffer in C# in a StringBuilder object.

[DllImport("funcdll.dll"), CallingConvention=CallingConvention.Cdecl, 
    CharSet=CharSet.Unicode]
static extern bool func(string strIn, StringBuilder strOut);
...
StringBuilder strOutBuffer = new StringBuilder(128);
bool res = func("input string", strOutBuffer);
string strOut = StringBuilder.ToString();

If you simply cannot decide in the C# code how big a buffer you need then your best bet is to use a BSTR to marshal strOut. See this answer for details.

Answer 2

It's hard for me to tell without seeing the details of your C++ method... but, I've never had much luck using String with P/Invoke.

Try using IntPtr instead of String, and use Marshal.PtrToStringUni for the outgoing string, and marshal your managed string into unmanaged land with Marshal.StringToHGlobalUni and, after the function call, make sure to free the unmanaged string with Marshal.FreeHGlobal.

Also, coming from C99 land, my bools don't work with .NET bools... I don't know why. I have to use byte and check == 1. Don't know if you'll run into this with C++... but, if you want my advice on a place to start which seems least-breakable in my experience, here ya go:

[DllImport("funcdll.dll")]
static extern byte func(IntPtr strIn, out IntPtr strOut);

// use it
string myString = "Testing";

IntPtr stringOut;
IntPtr stringIn = Marshal.StringToHGlobalUni(myString);

   if(func(stringIn, out stringOut) == 1)
   {
      //do something with strout
      string stringOutValue = Marshal.PtrToStringUni(stringOut);
      // Depending on how you dealt with stringOut in your
      // unmanaged code, you may have to: Marshal.FreeCoTaskMem(stringOut);
      // or Marshal.FreeHGlobal(stringOut) if you're returning
      // an owning reference to a copied string.
   }

Marshal.FreeHGlobal(stringIn);

Answer 3

I beleive you're having a calling convention mismatch. The default calling convention in C++ is cdecl while for .NET it's stdcall. try

[DllImport("funcdll.dll", CallingConvention = CallingConvention.Cdecl)] 
static extern bool func(String strIn, ref String strOut); 

Also you might have to specifically tell the marshallar that you want to marshal the strings as LPWSTR using [MarshalAs(UnmanagedType.LPWStr)] attributes.

[DllImport("funcdll.dll", CallingConvention = CallingConvention.Cdecl)] 
static extern bool func([MarshalAs(UnmanagedType.LPWStr)]String strIn
                       ,[MarshalAs(UnmanagedType.LPWStr)]ref String strOut);

have a look at http://msdn.microsoft.com/en-us/library/s9ts558h.aspx

Answer 4

In C++ make sure you have something like this (at least for the second argument)

extern "C" BOOL __stdcall func( BSTR * pBstr )
{
    *pBstr = SysAllocString( L"Foobar" );
    return 0;
}

In C# write something like this (for the second argument) :

static extern bool func( [MarshalAs(UnmanagedType.BStr)] ref String strOut); 

Answer 5

Sorry I do not have a big answer but just remember something from my experience.. did you try to use StringBuilder eg changing your c# import function signature as

   [System.Runtime.InteropServices.DllImport("funcdll.dll")]
static extern bool func(String strIn, System.Text.StringBuilder strOut);