how can C++ wcout utf-16 encoded char array?

Question

I am reading the well-known answer about string and wstring and come up some confusion.

source charset and execution charset are all set as utf-8, Windows x64, VC++ compiler, git bash console (can print unicode characters), system default codepage 936(GB2312).

My expertiment code:

#include <cstring>
#include <iostream>
using namespace std;
int main(int argc, char* argv[])
{
    wchar_t c[] = L"olé";
    wchar_t d[] = L"abc";
    wcout << c << endl;
    wcout << d << endl;

    return 0;
}

Can print "abc" but can't print "é".

I understand that wchar_t is used along with L prefix string literal. And under Windows wchar_t is encoded with UTF-16(It's hard coded right? No matter what source charset or execution charset I choose, L"abc" would always have the same UTF-16 code units).

The question is：How can it wcout a UTF-16 encoded string("abc"), while my source file is utf-8 and execution charset is utf-8. The program should not be able to recognize UTF-16 encoded stuff unless I set everything to utf-16.

And if it can print UTF-16 in some way, then why can't it print é?

Answer 1

You need a non-standard Windows system call to enable UTF-16 output.

#include <iostream>
#include <io.h>
#include <fcntl.h>
#include <stdio.h>

int main()
{
    _setmode(_fileno(stdout), _O_U16TEXT); // <=== Windows madness
    std::wcout << L"olé\n";
}

Note you cannot use cout after doing this, only wcout.

Also note your source code file must have BOM, otherwise the compiler will not recognise it as Unicode.

Answer 2

The Windows Console does not support UTF-16 output. It only supports 8-bit output, and it has partial support for 8-bit MBCS, such as Big5 or UTF-8.

To display Unicode characters on the console you will need to do conversion to UTF-8 or another MBCS in your code, and also put the console into UTF-8 mode (which requires an undocumented system call).

See also this answer