Convert System::String to std::string in UTF8, which later converted to char* as c_str

Question

I have a System::String^ variable in C++ code. This variable should be converted to std::string which is later converted to const char* via c_str.

// original string
System::String^ path = ...;

// convert to std::string
msclr::interop::marshal_context context;
std::string filename(context.marshal_as<std::string>(path));

// call API function that internally connects to sqlite3 using sqlite3_open as
//   sqlite3_open(filename.c_str())
// https://www.sqlite.org/c3ref/open.html - 
//  const char *filename,   /* Database filename (UTF-8) */
doCalculation(filename)

It works well with the ASCII paths, but fails if the path contains non-latin characters.

So Somehow I need to convert marshalled std::string from current implementation (ASCII?) to UTF8.

I've tried

    std::wstring dbPath(context.marshal_as<std::wstring>(path));
    std::wstring_convert<std::codecvt_utf8_utf16<wchar_t>, wchar_t> convert;
    std::string dbPathU8 = convert.to_bytes(dbPath);

but it does not work.

That's not C++, it's a weird Microsoft-ism called C++/CLI - I've fixed your tags for you. — Paul R, Dec 17 '14 at 12:58
Why not use C++/CLI to retrieve a UTF-8-byte array for the string, and convert *that* to a `std::string`? — Konrad Rudolph, Dec 17 '14 at 12:59
related/duplicate: https://stackoverflow.com/questions/1271022/convert-from-c-cli-pointer-to-native-c-pointer — stijn, Oct 07 '22 at 18:18
Use std::codecvt_utf8 instead of std::codecvt_utf8_utf16. It works for me. — PavelS, Oct 24 '22 at 13:14

score 7 · Accepted Answer · answered Dec 17 '14 at 14:08

What you want to do is use the .Net methods to convert directly to UTF-8.

The available methods in the Encoding class aren't exactly what you're looking for (direct from managed String to unmanaged string or byte array), so we'll need an intermediary and some manual copying.

String^ path = ...;

// First, convert to a managed array of the bytes you want.
array<Byte>^ bytes = Encoding::UTF8->GetBytes(path);

// Then, copy those bytes from the managed byte array to an unmanaged string.
std::string str;
str.resize(bytes->Length);
Marshal::Copy(bytes, 0, IntPtr(str.data()), bytes->Length);

// OR, copy directly to the char* you want eventually.
char* chars = new char[bytes->Length + 1]; // or malloc(), or whatever.
Marshal::Copy(bytes, 0, IntPtr(chars), bytes->Length);
chars[bytes->Length] = '\0'; // null terminate.
// don't forget to free the buffer when you're done with it!

There are several GetBytes variants available, but the parameters to them seem to either be both managed, or both unmanaged. (String^ and array^, or char* and byte*, but not String^ and byte*.) Therefore, we have the Encoding class create a managed byte array, then we use the Marshal::Copy method to copy those bytes either to the unmanaged string object, or directly to a char*.

For the first call ``Marshal.Copy`` I receive error C2440: "": "const char *" cannot be converted to "System::IntPtr" The second option works. Afterwards I copy the array into a new string with ``std::string(chars)`` Thanks for posting this! — Florian Straub, Jan 19 '21 at 10:59

Convert System::String to std::string in UTF8, which later converted to char* as c_str

1 Answers1