To avoid discarding response body by TIdHTTP component in case of non-2xx status, you'd also need to use hoWantProtocolErrorContent among HttpOptions. Unfortunatelly these options are poorly documented even in help file shipped with RAD Studio help (Help→Third-Party Help→Indy Library Help).
To get the response body as a string use Get/Put/Post invariant that returns string, not TIdHTTP.Response.ResponseText or TIdHTTP.ResponseText for short which contains status line of HTTP response; e.g.: HTTP/1.1 400 Bad Request.
Sample code:
var
  IdHTTP: TIdHTTP;
begin
  IdHTTP := TIdHTTP.Create();
  try
    IdHTTP.HTTPOptions := IdHTTP.HTTPOptions + [hoNoProtocolErrorException, hoWantProtocolErrorContent];
    Writeln(IdHTTP.ResponseText);
    Writeln;
    Writeln(IdHTTP.Get('https://postman-echo.com/status/400'));
  finally
    IdHTTP.Free;
  end;
end;
Sample output:
HTTP/1.1 400 Bad Request  
{"status":400}