Servlet 3.0 using @MultipartConfig throws exception when using getPart()

Question

I am getting wierd behaviour with server-api 3.0 when using the @MultipartConfig. When I am calling the servlet from a jsp page it works 100% , but when I make a call to the servlet from my own java client (using the java.net api) I get an exception. below is my source code and the output I get in both scenarios.

I am using Java 1.6.0 , and running the servlet on apache-tomcat-7.0.11.

Servet :

import java.io.IOException;
import java.io.InputStream;
import java.util.Collection;
import java.util.Enumeration;

import javax.servlet.ServletException;
import javax.servlet.annotation.MultipartConfig;
import javax.servlet.annotation.WebServlet;
import javax.servlet.http.HttpServlet;
import javax.servlet.http.HttpServletRequest;
import javax.servlet.http.HttpServletResponse;
import javax.servlet.http.Part;

@WebServlet(urlPatterns="/MultipartUploadServlet" , name="MultipartUploadServlet")
@MultipartConfig(location="/tmp", maxFileSize = 10485760L)
public class MultipartUploadServlet extends HttpServlet{

    protected void doPost(HttpServletRequest req, HttpServletResponse resp) throws ServletException, IOException {
        System.out.println("MultipartUploadServlet.doPost");
        try {

            System.out.println("Print out the request header");
            Enumeration<String> hn = req.getHeaderNames();
            while(hn.hasMoreElements()) {
                String n = hn.nextElement();
                System.out.println(n + " [" + req.getHeader(n) + "]");
            }

            Collection<Part> requestParts = req.getParts();
            System.out.println("there are [" + requestParts.size() +"] multiparts");


            System.out.println("printing out request inputstream");
            InputStream is = req.getInputStream();
            int charRead = 0;
            System.out.println("[");
            while((charRead = is.read()) != -1){
                System.out.print((char)charRead);
            }
            is.close();

            System.out.println("]");

        } catch (Exception excp) {
            excp.printStackTrace();
        } 
    }

}

Jsp client

<html>
    <head></head>
    <body>
        <p>Commons File Upload Example</p>
        <form action="MultipartUploadServlet" enctype="multipart/form-data" method="post">
            <input type="file" name="file1"><br>
            <input type="Submit" value="Upload File"><br>s
        </form>
    </body>
</html>

output on tomcat using jsp client

host [localhost:8080]
user-agent [Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10.5; en-US; rv:1.9.2.13) Gecko/20101203 Firefox/3.6.13]
accept [text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8]
accept-language [en-us,en;q=0.5]
accept-encoding [gzip,deflate]
accept-charset [ISO-8859-1,utf-8;q=0.7,*;q=0.7]
keep-alive [115]
connection [keep-alive]
referer [http://localhost:8080/scrappyWeb/test.jsp]
cookie [JSESSIONID=06A3E14F91D4B8E558B7438B1D9C7E99]
content-type [multipart/form-data; boundary=---------------------------1137522503144128232716531729]
content-length [223]
there are [1] multiparts

java client

package com.scrappy.web.client.compass.verification;

import java.io.BufferedReader;
import java.io.File;
import java.io.FileInputStream;
import java.io.IOException;
import java.io.InputStreamReader;
import java.io.OutputStream;
import java.io.OutputStreamWriter;
import java.io.PrintWriter;
import java.net.HttpURLConnection;
import java.net.MalformedURLException;
import java.net.URL;
import java.net.URLConnection;

public class TestClient {

    public static void main(String ... aaa)  {
        System.setProperty("http.keepAlive", "false");

        String boundary = "---------------------------" + Long.toHexString(System.currentTimeMillis());
        try {
            URLConnection connection = new URL("http://localhost:8080/scrappyWeb/MultipartUploadServlet").openConnection();
            ((HttpURLConnection)connection).setRequestMethod("POST");
            ((HttpURLConnection)connection).setConnectTimeout(60000);
            connection.setRequestProperty("User-Agent", "Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10.5; en-US; rv:1.9.2.13) Gecko/20101203 Firefox/3.6.13"); // Do as if you're using Firefox 3.6.3.
            connection.setDoOutput(true);
            connection.setRequestProperty("Content-Type", "multipart/form-data; boundary=" + boundary);
            connection.setRequestProperty("keep-alive", "115");
            connection.setRequestProperty("accept-encoding", "gzip,deflate");
            connection.setRequestProperty("connection", "keep-alive");
            connection.setRequestProperty("accept-charset","ISO-8859-1,utf-8;q=0.7,*;q=0.7");
            connection.setRequestProperty("accept", "text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8");
            PrintWriter writer = null;
            OutputStream output = connection.getOutputStream();

            String charset = "UTF-8";

            File textFile = new File("/Users/christiaan/test.txt");

            writer = new PrintWriter(new OutputStreamWriter(output, charset), true);
            writer.println("--" + boundary);
            writer.println("Content-Disposition: form-data; name=\"file1\"; filename=\"" + textFile.getName() + "\"");
            writer.println("Content-Type: text/plain");
            writer.println();
            BufferedReader reader = null;
            try {
                reader = new BufferedReader(new InputStreamReader(new FileInputStream(textFile), charset));
                for (String line; (line = reader.readLine()) != null;) {
                    writer.println(line);
                }
            } finally {
                if (reader != null) try { reader.close(); } catch (IOException logOrIgnore) {}
            }
            writer.println(); // Important! Indicates end of binary boundary.

            // End of multipart/form-data.
            writer.println("--" + boundary + "--");
            writer.close();

         ((HttpURLConnection)connection).getResponseCode();

        } catch (MalformedURLException e) {
            e.printStackTrace();
        } catch (IOException e) {
            e.printStackTrace();
        }
    }

}

output on tomcat using java client

user-agent [Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10.5; en-US; rv:1.9.2.13) Gecko/20101203 Firefox/3.6.13]
content-type [multipart/form-data; boundary=---------------------------12f24455f6f]
accept-encoding [gzip,deflate]
accept-charset [ISO-8859-1,utf-8;q=0.7,*;q=0.7]
accept [text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8]
host [localhost:8080]
connection [close]
content-length [183]
java.io.IOException: org.apache.tomcat.util.http.fileupload.FileUploadException: Stream ended unexpectedly
    at org.apache.catalina.connector.Request.parseParts(Request.java:2639)
    at org.apache.catalina.connector.Request.getParts(Request.java:2539)
    at org.apache.catalina.connector.RequestFacade.getParts(RequestFacade.java:1077)
    at com.scrappy.web.servlet.compass.verification.MultipartUploadServlet.doPost(MultipartUploadServlet.java:31)
    at javax.servlet.http.HttpServlet.service(HttpServlet.java:641)
    at javax.servlet.http.HttpServlet.service(HttpServlet.java:722)
    at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:304)
    at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:210)
    at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:240)
    at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:164)
    at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:498)
    at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:164)
    at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:100)
    at org.apache.catalina.valves.AccessLogValve.invoke(AccessLogValve.java:562)
    at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:118)
    at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:394)
    at org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:243)
    at org.apache.coyote.http11.Http11Protocol$Http11ConnectionHandler.process(Http11Protocol.java:188)
    at org.apache.coyote.http11.Http11Protocol$Http11ConnectionHandler.process(Http11Protocol.java:166)
    at org.apache.tomcat.util.net.JIoEndpoint$SocketProcessor.run(JIoEndpoint.java:302)
    at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
    at java.lang.Thread.run(Thread.java:680)
Caused by: org.apache.tomcat.util.http.fileupload.FileUploadException: Stream ended unexpectedly
    at org.apache.tomcat.util.http.fileupload.FileUploadBase.parseRequest(FileUploadBase.java:336)
    at org.apache.tomcat.util.http.fileupload.servlet.ServletFileUpload.parseRequest(ServletFileUpload.java:129)
    at org.apache.catalina.connector.Request.parseParts(Request.java:2609)
    ... 22 more
Caused by: org.apache.tomcat.util.http.fileupload.MultipartStream$MalformedStreamException: Stream ended unexpectedly
    at org.apache.tomcat.util.http.fileupload.MultipartStream.readHeaders(MultipartStream.java:488)
    at org.apache.tomcat.util.http.fileupload.FileUploadBase$FileItemIteratorImpl.findNextItem(FileUploadBase.java:861)
    at org.apache.tomcat.util.http.fileupload.FileUploadBase$FileItemIteratorImpl.<init>(FileUploadBase.java:827)
    at org.apache.tomcat.util.http.fileupload.FileUploadBase.getItemIterator(FileUploadBase.java:282)
    at org.apache.tomcat.util.http.fileupload.FileUploadBase.parseRequest(FileUploadBase.java:302)
    ... 24 more

The only real difference I see in the headers is the content length. But what gets me , is that if I get the inputStream from the HttpServletRequest and I print that , it prints everyting in the test.txt file as it should be.

I have followed BalusC post

I must be missing something or not understading something , hope someone can help !

Is there any reason why the content-length in the Java client is hard coded to 225 ? In any case, whether that matter or not, I think it would be better to take a look at the traffic flowing on the wire, using Wireshark, as the MalformedStreamException is thrown when the structure of the HTTP request does not obey certain rules. — Vineet Reynolds, Apr 05 '11 at 06:42
I have edided my most to remove the content-lenght - that was just me trying desperate things. it gets overridden anyway. Thanks for mentioning Wireshark - will check it out. — ChristiaanP, Apr 05 '11 at 06:47
I have checked the packets with wireshark , and on each packet send I get the same error : Header checksum: 0x0000 [incorrect, should be 0x49a5] - how do one fix this ? — ChristiaanP, Apr 05 '11 at 07:19
@ChristiaanP, I suspect that is a different issue altogether and completely unrelated to this one. Are you able to view the contents of the HTTP payload sent by the client? If so, you could paste the contents of the payload as well. From the logs posted, it appears that the one of the headers in the request has malformed data. If it isnt a problem (from the point of view of privacy), you might as well share the Wireshark dump. — Vineet Reynolds, Apr 05 '11 at 08:15

score 6 · Accepted Answer · answered Apr 05 '11 at 13:49

6

Mac/Linux uses \n as default line separator on println() while HTTP mandates \r\n (and is the default on Windows). Sorry, that was a pretty silly mistake on the URLConnection tutorial. I'll fix it asap.

answered Apr 05 '11 at 13:49

BalusC

1,082,665
372
3,610
3,555

score 3 · Answer 2 · edited May 23 '17 at 11:47

Using wireshark to interogte the packets sent I saw that the MIME Multipart packet was malformed. I basically changed the client to write the content-disposition stuff to use the outputstream straight and not a PrintWriter. It seems to work , but I am not sure why , as both so essentialy the same thing.

Thanks for Vineet Reynolds for pointing me to WireShark

Here is the new client :

package com.scrappy.web.client.compass.verification;

import java.io.BufferedReader;
import java.io.File;
import java.io.FileInputStream;
import java.io.IOException;
import java.io.InputStreamReader;
import java.io.OutputStream;
import java.io.OutputStreamWriter;
import java.io.PrintWriter;
import java.net.HttpURLConnection;
import java.net.MalformedURLException;
import java.net.URL;
import java.net.URLConnection;

public class TestClient {

    public static void main(String ... aaa)  {
        System.setProperty("http.keepAlive", "false");

        String boundary = "---------------------------" +  Long.toHexString(System.currentTimeMillis());
        try {
            URLConnection connection = new URL("http://localhost:8080/scrappyWeb/MultipartUploadServlet").openConnection();
            ((HttpURLConnection)connection).setRequestMethod("POST");
            ((HttpURLConnection)connection).setConnectTimeout(6000000);
            connection.setRequestProperty("User-Agent", "Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10.5; en-US; rv:1.9.2.13) Gecko/20101203 Firefox/3.6.13"); // Do as if you're using Firefox 3.6.3.
            connection.setDoOutput(true);
            connection.setRequestProperty("Content-Type", "multipart/form-data; boundary=" + boundary);
            connection.setRequestProperty("accept-encoding", "gzip,deflate");
            connection.setRequestProperty("connection", "keep-alive");
            connection.setRequestProperty("accept-charset","ISO-8859-1,utf-8;q=0.7,*;q=0.7");
            connection.setRequestProperty("accept", "text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8");
            connection.setRequestProperty("accept-language", "en-us,en;q=0.5");
            OutputStream output = connection.getOutputStream();

            String charset = "UTF-8";

            File textFile = new File("/Users/christiaan/test.txt");
            output.write(("\r\n\r\n--" + boundary +"\r\n").getBytes());
            output.write(("Content-Disposition: form-data; name=\"file1\"; filename=\"" + textFile.getName() + "\"\r\n").getBytes());
            output.write(("Content-Type: text/plain\r\n").getBytes());
            output.write(("\r\n").getBytes());


            PrintWriter writer = null;
            writer = new PrintWriter(new OutputStreamWriter(output, charset), true);
            BufferedReader reader = null;
            try {
                reader = new BufferedReader(new InputStreamReader(new FileInputStream(textFile), charset));
                for (String line; (line = reader.readLine()) != null;) {
                    writer.println(line);
                }
            } finally {
                if (reader != null) try { reader.close(); } catch (IOException logOrIgnore) {}
            }

            // End of multipart/form-data.
            output.write(("\r\n").getBytes());
            output.write(("--" + boundary +"--\r\n").getBytes());
            output.flush();
            writer.close();



         ((HttpURLConnection)connection).getResponseCode();



        } catch (MalformedURLException e) {
            e.printStackTrace();
        } catch (IOException e) {
            e.printStackTrace();
        }
    }

}

score 1 · Answer 3 · edited Jan 01 '12 at 16:03

1

Two things come to mind.

Writers deal with characters and streams deal with bytes. You would need to know what default character set is being used by the writer, and it complies with what you're passing through.
With multipart mime types I'm not sure that the contents of the file you're uploading is encoded. Perhaps there is something in that file thats malformed.

Cheers

edited Jan 01 '12 at 16:03

answered Apr 07 '11 at 07:40

Creigh

11
1

Thanks @Creigh , the problem was the CRLF that the HTTP protocal mandates. So the packet that contained the multipart data was actually malformed. – ChristiaanP Apr 07 '11 at 10:32
The charset is already specified in `InputStreamReader` and `OutputStreamWriter`. – BalusC Apr 07 '11 at 11:11

Servlet 3.0 using @MultipartConfig throws exception when using getPart()

3 Answers3