C++ Builder TRestRequest with Unicode parameter

Question

I am using TRestRequest to get data from a server. I need to fill a parameter with a Unicode string value: "ôpen". However, I get a crash calling Execute with this Unicode string as a query parameter.

My code :

    RESTRequest->ResetToDefaults();
    RESTRequest->AddParameter("Content-Type", "application/x-www-form-urlencoded; charset=UTF-8 ", TRESTRequestParameterKind::pkHTTPHEADER);
    //  Get Indexing Status
    RESTRequest->Resource = "XXX/collection1"+ Form1->teReuqestHandler->Text+"?";
    RESTRequest->Method = rmGET;
    // replace all space in name field with '\ '
    UnicodeString lcQuery = Form1->teQuery->Text; // this value should be support french language or ...
    // Body
    RESTRequest->AddParameter("q",lcQuery, TRESTRequestParameterKind::pkGETorPOST);
    // Run
    String str1 = RESTRequest->GetFullRequestURL();
    RESTRequest->Execute(); // here when pass "ôpen" to lcQuery, it crash

How do I correctly add "ôpen" to my URL?

You should encode your URL parameters with UTF-8: http://stackoverflow.com/a/306934/937125 — kobik, Feb 26 '15 at 09:30
use `UrlEncode(Utf8Encode(Text))`. you also did not specify which version of c++builder you use. is the Text unicode (WideString)? — kobik, Feb 26 '15 at 11:11
the environement is embarcadero C++ XE5... the texte is UnicodeString.. — , Feb 26 '15 at 12:09
Dest = UTF8Encode(Form1->teQuery->Text); RESTRequest->AddParameter("q", TIdURI::URLEncode(Dest), TRESTRequestParameterKind::pkGETorPOST); // it crash in URLEncode — , Feb 26 '15 at 12:16
@DevDev Why can't you use UTF8 as kobik suggests? It's Unicode, and URL-encodeable... — David, Feb 26 '15 at 15:42
What is "a crash"? What is the exact error / exception etc? Does it give you a stack trace, and if so what is it? — David, Feb 26 '15 at 15:45
@DavidM when i use this : `RESTRequest->AddParameter("q", TIdURI::URLEncode(UTF8Encode(lcQuery)), TRESTRequestParameterKind::pkGETorPOST);` I get this exception : ** EIdURIException with message 'Protocol field is empty** ! — , Feb 26 '15 at 16:44
@DevDev Are you sure the problem is the text you are passing? I mean, if you pass something without any special characters, say, "open", does it work? For the exception you are getting it would seem the problem is somewhere else — Rodrigo Gómez, Feb 26 '15 at 16:57
Thx @RodrigoGómez when i pass "open", i get some error exception `Protocol field is empty` — , Feb 26 '15 at 17:22
with debug mode it raise exception in the file IdUri.pas : `if FProtocol = '' then begin raise EIdURIException.Create(RSURINoProto); end;` — , Feb 26 '15 at 17:27
This is telling you that you need to specify that property. It doesn't have anything to do with the params or encoding. Are you sure the Resource is complete? I haven't used the TRestRequest object, but for the code it seems that you are missing the URL itself to query, which would include the protocol (HTTP, etc) — Rodrigo Gómez, Feb 26 '15 at 17:32
@DevDev: Your `TIdURI` examples are wrong. `TIdURI` handles the UTF-8 encoding for you (in fact, UTF-8 is its default encoding), so *DO NOT* manually encode the input strings to UTF-8 beforehand. Also, `TIdURI::URLEncode()` is the wrong method to use in this situation. It expects a full URL, which `teQuery` does not contain. Use `TIdURI::ParamsEncode()` instead: `RESTRequest->AddParameter(L"q", TIdURI::ParamsEncode(Form1->teQuery->Text), TRESTRequestParameterKind::pkGETorPOST);` — Remy Lebeau, Feb 26 '15 at 18:56

score 0 · Accepted Answer · answered Feb 26 '15 at 19:48

The Content-Type: application/x-www-form-urlencoded; charset=UTF-8 HTTP header has no meaning in a GET request, only in a POST request.

Embarcadero's REST framework does not handle non-ASCII charsets correctly. Even though it uses Indy internally, it does not utilize Indy's charset handling. As such, in order to send UTF-8 encoded data, you have to either:

encode a UnicodeString to UTF-8 manually and then put the UTF-8 octets back into a UnicodeString so TRESTRequest can send them:

UnicodeString EncodeAsUtf8(const UnicodeString &s)
{
    UTF8String utf8 = s;
    UnicodeString ret;
    ret.SetLength(utf8.Length());
    for (int x = 1; x <= utf8.Length(); ++x)
        ret[x] = (WideChar) utf8[x];
    return ret;
}

...

RESTRequest->ResetToDefaults();
RESTRequest->Method = rmGET;
RESTRequest->Resource = L"XXX/collection1" + Form1->teReuqestHandler->Text;
RESTRequest->AddParameter(L"q", EncodeAsUtf8(Form1->teQuery->Text), TRESTRequestParameterKind::pkGETorPOST);
String str1 = RESTRequest->GetFullRequestURL();
RESTRequest->Execute();

encode the parameter data yourself:

#include <IdGlobal.hpp>
#include <IdURI.hpp>

RESTRequest->ResetToDefaults();
RESTRequest->Method = rmGET;
RESTRequest->Resource = L"XXX/collection1" + Form1->teReuqestHandler->Text;
RESTRequest->AddParameter(L"q", TIdURI::ParamsEncode(Form1->teQuery->Text, IndyTextEncoding_UTF8), TRESTRequestParameterKind::pkGETorPOST, TRESTRequestParameterOptions() << TRESTRequestParameterOption::poDoNotEncode);
String str1 = RESTRequest->GetFullRequestURL();
RESTRequest->Execute();

Or:

#include <IdGlobal.hpp>
#include <IdURI.hpp>

RESTRequest->ResetToDefaults();
RESTRequest->Method = rmGET;
RESTRequest->Resource = L"XXX/collection1" + Form1->teReuqestHandler->Text + L"?q={q}";
RESTRequest->AddParameter(L"q", TIdURI::ParamsEncode(Form1->teQuery->Text, IndyTextEncoding_UTF8), TRESTRequestParameterKind::pkURLSEGMENT, TRESTRequestParameterOptions() << TRESTRequestParameterOption::poDoNotEncode);
String str1 = RESTRequest->GetFullRequestURL();
RESTRequest->Execute();

Otherwise, switch to Indy's TIdHTTP component:

UnicodeString Response = IdHTTP1->Get(L"http://server/XXX/collection1" + Form1->teReuqestHandler->Text + L"?q=" + TIdURI::ParamsEncode(Form1->teQuery->Text, IndyTextEncoding_UTF8));

hard to believe that in 2015 Embarcadero are still coming out with non-Unicode-enabled APIs — M.M, Feb 26 '15 at 22:10

C++ Builder TRestRequest with Unicode parameter

1 Answers1