Upload a file through an HTTP form, via MultipartEntityBuilder, with a progress bar

Question

The short version - org.apache...MultipartEntity is deprecated, and its upgrade, MultipartEntityBuilder, appears under-represented in our online forums. Let's fix that. How does one register a callback, so my (Android) app can display a progress bar as it uploads a file?

The long version - Here's the "missing dirt-simple example" of MultipartEntityBuilder:

public static void postFile(String fileName) throws Exception {
    // Based on: https://stackoverflow.com/questions/2017414/post-multipart-request-with-android-sdk

    HttpClient client = new DefaultHttpClient();
    HttpPost post = new HttpPost(SERVER + "uploadFile");
    MultipartEntityBuilder builder = MultipartEntityBuilder.create();        
    builder.setMode(HttpMultipartMode.BROWSER_COMPATIBLE);
    builder.addPart("file", new FileBody(new File(fileName)));
    builder.addTextBody("userName", userName);
    builder.addTextBody("password", password);
    builder.addTextBody("macAddress", macAddress);
    post.setEntity(builder.build());
    HttpResponse response = client.execute(post);
    HttpEntity entity = response.getEntity();

    // response.getStatusLine();  // CONSIDER  Detect server complaints

    entity.consumeContent();
    client.getConnectionManager().shutdown(); 

}  // FIXME  Hook up a progress bar!

We need to fix that FIXME. (An added benefit would be interruptible uploads.) But (please correct me whether or not I'm wrong), all the online examples seem to fall short.

This one, http://pastebin.com/M0uNZ6SB, for example, uploads a file as a "binary/octet-stream"; not a "multipart/form-data". I require real fields.

This example, File Upload with Java (with progress bar), shows how to override the *Entity or the *Stream. So maybe I can tell the MultipartEntityBuilder to .create() an overridden entity that meters its upload progress?

So if I want to override something, and replace the built-in stream with a counting stream that sends a signal for every 1000 bytes, maybe I can extend the FileBody part, and override its getInputStream and/or writeTo.

But when I try class ProgressiveFileBody extends FileBody {...}, I get the infamous java.lang.NoClassDefFoundError.

So while I go spelunking around my .jar files, looking for the missing Def, can someone check my math, and maybe point out a simpler fix I have overlooked?

Answer 1

The winning code (in spectacular Java-Heresy(tm) style) is:

public static String postFile(String fileName, String userName, String password, String macAddress) throws Exception {

    HttpClient client = new DefaultHttpClient();
    HttpPost post = new HttpPost(SERVER + "uploadFile");
    MultipartEntityBuilder builder = MultipartEntityBuilder.create();        
    builder.setMode(HttpMultipartMode.BROWSER_COMPATIBLE);

    final File file = new File(fileName);
    FileBody fb = new FileBody(file);

    builder.addPart("file", fb);  
    builder.addTextBody("userName", userName);
    builder.addTextBody("password", password);
    builder.addTextBody("macAddress",  macAddress);
    final HttpEntity yourEntity = builder.build();

    class ProgressiveEntity implements HttpEntity {
        @Override
        public void consumeContent() throws IOException {
            yourEntity.consumeContent();                
        }
        @Override
        public InputStream getContent() throws IOException,
                IllegalStateException {
            return yourEntity.getContent();
        }
        @Override
        public Header getContentEncoding() {             
            return yourEntity.getContentEncoding();
        }
        @Override
        public long getContentLength() {
            return yourEntity.getContentLength();
        }
        @Override
        public Header getContentType() {
            return yourEntity.getContentType();
        }
        @Override
        public boolean isChunked() {             
            return yourEntity.isChunked();
        }
        @Override
        public boolean isRepeatable() {
            return yourEntity.isRepeatable();
        }
        @Override
        public boolean isStreaming() {             
            return yourEntity.isStreaming();
        } // CONSIDER put a _real_ delegator into here!

        @Override
        public void writeTo(OutputStream outstream) throws IOException {

            class ProxyOutputStream extends FilterOutputStream {
                /**
                 * @author Stephen Colebourne
                 */

                public ProxyOutputStream(OutputStream proxy) {
                    super(proxy);    
                }
                public void write(int idx) throws IOException {
                    out.write(idx);
                }
                public void write(byte[] bts) throws IOException {
                    out.write(bts);
                }
                public void write(byte[] bts, int st, int end) throws IOException {
                    out.write(bts, st, end);
                }
                public void flush() throws IOException {
                    out.flush();
                }
                public void close() throws IOException {
                    out.close();
                }
            } // CONSIDER import this class (and risk more Jar File Hell)

            class ProgressiveOutputStream extends ProxyOutputStream {
                public ProgressiveOutputStream(OutputStream proxy) {
                    super(proxy);
                }
                public void write(byte[] bts, int st, int end) throws IOException {

                    // FIXME  Put your progress bar stuff here!

                    out.write(bts, st, end);
                }
            }

            yourEntity.writeTo(new ProgressiveOutputStream(outstream));
        }

    };
    ProgressiveEntity myEntity = new ProgressiveEntity();

    post.setEntity(myEntity);
    HttpResponse response = client.execute(post);        

    return getContent(response);

} 

public static String getContent(HttpResponse response) throws IOException {
    BufferedReader rd = new BufferedReader(new InputStreamReader(response.getEntity().getContent()));
    String body = "";
    String content = "";

    while ((body = rd.readLine()) != null) 
    {
        content += body + "\n";
    }
    return content.trim();
}

#  NOTE ADDED LATER: as this blasterpiece gets copied into various code lineages, 
#  The management reminds the peanut gallery that "Java-Heresy" crack was there
#  for a reason, and (as commented) most of that stuff can be farmed out to off-
#  the-shelf jar files and what-not. That's for the java lifers to tool up. This
#  pristine hack shall remain obviousized for education, and for use in a pinch.

#  What are the odds??

Answer 2

Cant thank Phlip enough for that solution. Here are the final touches for adding your progressbar support. I ran it inside an AsyncTask - progress below enables you to post the progress back to a method in the AsyncTask that invokes AsyncTask.publishProgress() for your class running in the AsyncTask. The progress bar isn't exactly smooth but at least it moves. On a Samsung S4 uploading a 4MB imagefile after the preamble it was moving 4K chunks.

     class ProgressiveOutputStream extends ProxyOutputStream {
            long totalSent;
            public ProgressiveOutputStream(OutputStream proxy) {
                   super(proxy);
                   totalSent = 0;
            }

            public void write(byte[] bts, int st, int end) throws IOException {

            // FIXME  Put your progress bar stuff here!
            // end is the amount being sent this time
            // st is always zero and end=bts.length()

                 totalSent += end;
                 progress.publish((int) ((totalSent / (float) totalSize) * 100));
                 out.write(bts, st, end);
            }

Answer 3

first of all: huge thanks for the original question/answer. Since HttpPost is now deprecated, I reworked it a bit though, using additional input from this article and made a micro library of it: https://github.com/licryle/HTTPPoster

It wraps the whole in an ASync task; uses the MultipartEntityBuilder & HttpURLConnection and let's you listen for callbacks.

To use:

1. Download & extract
2. In your build.gradle Module file, add the dependency:

dependencies 
{    
     compile project(':libs:HTTPPoster') 
}

3. You need a class to implement the HttpListener interface so you can listen to the callbacks. It has four callbacks in HTTPListener:

   • onStartTransfer
   • onProgress
   • onFailure
   • onResponse

4. Configure the ASyncTask & start it. Here's a quick usage:

HashMap<String, String> mArgs = new HashMap<>();
mArgs.put("lat", "40.712784");
mArgs.put("lon", "-74.005941");

ArrayList<File> aFileList = getMyImageFiles();

HttpConfiguration mConf = new HttpConfiguration(
    "http://example.org/HttpPostEndPoint",
    mArgs,
    aFileList,
    this, // If this class implements HttpListener
    null,  // Boundary for Entities - Optional
    15000  // Timeout in ms for the connection operation
    10000, // Timeout in ms for the reading operation
);

new HttpPoster().execute(mConf);

hope that can help :) Feel also free to suggest improvements! It's very recent, and I extend it as I need it.

Cheers