How to avoid getting an error 10053 (WSAECONNABORTED) if a HttpGet operation in Android is lasting too long?

Question

I have an Android application communicating with a Delphi 2006 web service application using Indy 10 TIdHttpServer (coming with Delphi 2006). The Delphi application generates a big XML file and serves this. The XML generation may last more than 5 minutes.

If the duration of GenerateXml() is more than about 5 minutes (*), I detect an error 10053 in TIdHTTPResponseInfo.WriteContent if running in the Delphi IDE:

Socket Error # 10053 Software caused connection abort.

However, on the android side nothing is detected and the HttpGet-call lasts forever.

My questions are:

1.) Why do I get the error 10053 and how can I avoid it? It seems like android times out the connection, but http.socket.timeout is set to infinite.

and

2.) What can I do to detect such an error on the client side (other than setting timeout, which would have to be too big to be useful)? Can I do something in TIdHttpServer.OnException?

Here is my code. Android - download function, which is run inside an AsyncTask:

protected static HttpEntity downloadEntity(String url) throws IOException {
    HttpClient client = new DefaultHttpClient();  

    //Check because of Error 10053: but timeout is null -> infinite
    Log.d("TAG", "http.socket.timeout: " + client.getParams().getParameter("http.socket.timeout"));

    HttpGet get = new HttpGet(url);
    HttpResponse response;
    try {
        //in case of Error 10053 the following call seems to last forever (in PlainSocketImpl.read)
        response = client.execute(get);
    } catch (ClientProtocolException e) {
        //...
    }

    //...

    return response.getEntity();  
}   

Delphi implementation of TIdHttpServer.OnCommandGet:

procedure ServeXmlDoc(XmlDoc: IXMLDocument; ResponseInfo: TIdHTTPResponseInfo);
var
    TempStream: TMemoryStream;
begin
    ResponseInfo.ContentType := 'text/xml';
    TempStream := TMemoryStream.Create;
    XMLDoc.SaveToStream(TempStream);
    ResponseInfo.FreeContentStream := True; 
    ResponseInfo.ContentStream := TempStream;
end;

procedure TMyService.HTTPServerCommandGet(AContext: TIdContext; RequestInfo: TIdHTTPRequestInfo;
  ResponseInfo: TIdHTTPResponseInfo);
begin
    Coinitialize(nil); 
    try
        //...
        ServeXmlDoc(GenerateXml(), ResponseInfo);
    finally
        CoUninitialize;
    end;
end;

Edit: (*) I have done further testing and experienced the error even in cases where the whole process had a duration of under 2 minutes.

Answer 1

Something between Android and your server, such as a firewall/router, is likely cutting the connection after it is idle for too long. You should try enabling TCP keep-alives to avoid that.

On the other hand, this is the kind of situation that HTTP 1.1's chunked transfer encoding was designed to handle (assuming you are using HTTP 1.1 to begin with). Instead of waiting 5 minutes for the entire XML to be generated in full before then sending it to the client, you should send the XML in pieces as they are being generated. Not only does that keep the connection active, but it also reduces the server's memory footprint since it doesn't have to store the entire XML in memory at one time.

TIdHTTPServer does not (yet) natively support sending chunked responses (but TIdHTTP does support receiving chunked responses), however it would not be very difficult to implement manually. Write a custom TStream derived class and overwrite its virtual Write() method (or use Indy's TIdEventStream class) to write data to the HTTP client using the format outlined in RFC 2616 Section 3.6.1. With that, you can have ServeXmlDoc() set the ResponseInfo.TransferEncoding property to 'chunked' and call the ResponseInfo.WriteHeader() method without setting either the ResponseInfo.ContentText or ResponseInfo.ContentStream properties, then pass your custom stream to IXMLDocument.SaveToStream() so it will finish writing the response data after the headers. For example:

type
  TMyChunkedStream = class(TStream)
  private
    fIO: TIdIOHandler;
  public
    constructor Create(AIO: TIdIOHandler);
    function Write(const Buffer; Count: Longint): Longint; override;
    procedure Finished;
    ...
  end;

constructor TMyChunkedStream.Create(AIO: TIdIOHandler);
begin
  inherited Create;
  fIO := AIO;
end;

function TMyChunkedStream.Write(const Buffer; Count: Longint): Longint; override;
begin
  if Count > 0 then
  begin
    fIO.WriteLn(IntToHex(Count, 1));
    fIO.Write(RawToBytes(Buffer, Count));
    fIO.WriteLn;
  end;
  Result := Count;
end;

procedure TMyChunkedStream.Finished;
begin
  fIO.WriteLn('0');
  fIO.WriteLn;
end;

procedure ServeXmlDoc(XmlDoc: IXMLDocument; ResponseInfo: TIdHTTPResponseInfo);
var
  TempStream: TMyChunkedStream;
begin
  ResponseInfo.ContentType := 'text/xml';
  ResponseInfo.TransferEncoding := 'chunked';
  ResponseInfo.WriteHeader;

  TempStream := TMyChunkedStream.Create(ResponseInfo.Connection.IOHandler);
  try
    XMLDoc.SaveToStream(TempStream);
    TempStream.Finished;
  finally
    TempStream.Free;
  end;
end;

If, on the other hand, the bulk of your waiting is inside of GenerateXml() and not in XmlDoc.SaveToStream(), then you need to rethink your server design, and figure out a way to speed up GenerateXml(), or just get rid of IXMLDocument and create the XML manually so you can send it using the ResponseInfo.Connection.IOHandler as you are creating the XML content.