SpringBoot: Large Streaming File Upload Using Apache Commons FileUpload

Question

Am trying to upload a large file using the 'streaming' Apache Commons File Upload API.

The reason I am using the Apache Commons File Uploader and not the default Spring Multipart uploader is that it fails when we upload very large file sizes (~2GB). I working on a GIS application where such file uploads are pretty common.

The full code for my file upload controller is as follows:

@Controller
public class FileUploadController {

    @RequestMapping(value="/upload", method=RequestMethod.POST)
    public void upload(HttpServletRequest request) {
        boolean isMultipart = ServletFileUpload.isMultipartContent(request);
        if (!isMultipart) {
            // Inform user about invalid request
            return;
        }

        //String filename = request.getParameter("name");

        // Create a new file upload handler
        ServletFileUpload upload = new ServletFileUpload();

        // Parse the request
        try {
            FileItemIterator iter = upload.getItemIterator(request);
            while (iter.hasNext()) {
                FileItemStream item = iter.next();
                String name = item.getFieldName();
                InputStream stream = item.openStream();
                if (item.isFormField()) {
                    System.out.println("Form field " + name + " with value " + Streams.asString(stream) + " detected.");
                } else {
                    System.out.println("File field " + name + " with file name " + item.getName() + " detected.");
                    // Process the input stream
                    OutputStream out = new FileOutputStream("incoming.gz");
                    IOUtils.copy(stream, out);
                    stream.close();
                    out.close();

                }
            }
        }catch (FileUploadException e){
            e.printStackTrace();
        }catch (IOException e){
            e.printStackTrace();
        }
    }

    @RequestMapping(value = "/uploader", method = RequestMethod.GET)
    public ModelAndView uploaderPage() {
        ModelAndView model = new ModelAndView();
        model.setViewName("uploader");
        return model;
    }

}

The trouble is that the getItemIterator(request) always returns an iterator that does not have any items (i.e. iter.hasNext() ) always returns false.

My application.properties file is as follows:

spring.datasource.driverClassName=org.postgresql.Driver
spring.datasource.url=jdbc:postgresql://localhost:19095/authdb
spring.datasource.username=georbis
spring.datasource.password=asdf123

logging.level.org.springframework.web=DEBUG

spring.jpa.hibernate.ddl-auto=update

multipart.maxFileSize: 128000MB
multipart.maxRequestSize: 128000MB

server.port=19091

The JSP view for the /uploader is as follows:

<html>
<body>
<form method="POST" enctype="multipart/form-data" action="/upload">
    File to upload: <input type="file" name="file"><br />
    Name: <input type="text" name="name"><br /> <br />
    Press here to upload the file!<input type="submit" value="Upload">
    <input type="hidden" name="${_csrf.parameterName}" value="${_csrf.token}" />
</form>
</body>
</html>

What might I be doing wrong?

Have you disabled springs multipart support else your solution won't work and Spring will already have parsed the requests. Replace all the `multipart` properties with a single `multipart.enabled=false` to disable the default handling. — M. Deinum, Sep 25 '15 at 12:36
I hadn't done anything specific for disabling spring multipart support. I tried adding `multipart.enabled=false` in my `application.properties` file. However, once I do that, I just get a `405: Request method 'POST' not supported` error every time I make an upload. — balajeerc, Sep 25 '15 at 13:12
Which would indicate a wrong mapping or post to wrong url... Enable debug logging and see which URL you are posting to and to which URL your controller method is matched. — M. Deinum, Sep 25 '15 at 13:30
Its definitely not a case of posting to the wrong URL since when I remove the `multipart.enabled=false` then my controller does get invoked (and I once again have the problem described in my post above). — balajeerc, Sep 25 '15 at 13:44
@M.Deinum I think I figured out why I was getting the 404. It was because I was removing the default resolver by setting `multipart.enabled=false` but not adding any other resolver in its place. I have updated my post above with code I added to set the CommonsMultipartResolver. That solves the 404. However, I still have my original problem. Can you please take a look? I have enabled debug logging and I don't see any errors in it. — balajeerc, Sep 25 '15 at 13:58
No you don't want any `MultipartResolver` as that will parse the incoming requests and put the file(s) in memory. You want to handle it yourself so you don't want any thing else messing up your multipart request. — M. Deinum, Sep 25 '15 at 18:43
This post is helpful example of what is described at [Why Streaming?](https://commons.apache.org/proper/commons-fileupload/streaming.html), applied to Spring Boot. — Brent Bradburn, Jul 24 '18 at 19:53

score 46 · Accepted Answer · answered Sep 25 '15 at 20:39

Thanks to some very helpful comments by M.Deinum, I managed to solve the problem. I have cleaned up some of my original post and am posting this as a complete answer for future reference.

The first mistake I was making was not disabling the default MultipartResolver that Spring provides. This ended up in the resolver processing the HttpServeletRequest and thus consuming it before my controller could act on it.

The way to disable it, thanks to M. Deinum was as follows:

multipart.enabled=false

However, there was still another hidden pitfall waiting for me after this. As soon as I disabled default multipart resolver, I started getting the following error when trying to make an upload:

Fri Sep 25 20:23:47 IST 2015
There was an unexpected error (type=Method Not Allowed, status=405).
Request method 'POST' not supported

In my security configuration, I had enabled CSRF protection. That necessitated that I send my POST request in the following manner:

<html>
<body>
<form method="POST" enctype="multipart/form-data" action="/upload?${_csrf.parameterName}=${_csrf.token}">
    <input type="file" name="file"><br>
    <input type="submit" value="Upload">
</form>
</body>
</html>

I also modified my controller a bit:

@Controller
public class FileUploadController {
    @RequestMapping(value="/upload", method=RequestMethod.POST)
    public @ResponseBody Response<String> upload(HttpServletRequest request) {
        try {
            boolean isMultipart = ServletFileUpload.isMultipartContent(request);
            if (!isMultipart) {
                // Inform user about invalid request
                Response<String> responseObject = new Response<String>(false, "Not a multipart request.", "");
                return responseObject;
            }

            // Create a new file upload handler
            ServletFileUpload upload = new ServletFileUpload();

            // Parse the request
            FileItemIterator iter = upload.getItemIterator(request);
            while (iter.hasNext()) {
                FileItemStream item = iter.next();
                String name = item.getFieldName();
                InputStream stream = item.openStream();
                if (!item.isFormField()) {
                    String filename = item.getName();
                    // Process the input stream
                    OutputStream out = new FileOutputStream(filename);
                    IOUtils.copy(stream, out);
                    stream.close();
                    out.close();
                }
            }
        } catch (FileUploadException e) {
            return new Response<String>(false, "File upload error", e.toString());
        } catch (IOException e) {
            return new Response<String>(false, "Internal server IO error", e.toString());
        }

        return new Response<String>(true, "Success", "");
    }

    @RequestMapping(value = "/uploader", method = RequestMethod.GET)
    public ModelAndView uploaderPage() {
        ModelAndView model = new ModelAndView();
        model.setViewName("uploader");
        return model;
    }
}

where Response is just a simple generic response type I use:

public class Response<T> {
    /** Boolean indicating if request succeeded **/
    private boolean status;

    /** Message indicating error if any **/
    private String message;

    /** Additional data that is part of this response **/
    private T data;

    public Response(boolean status, String message, T data) {
        this.status = status;
        this.message = message;
        this.data = data;
    }

    // Setters and getters
    ...
}

When multipart.enabled=false, MockMultipartFile is not work in unit test case.Is there any unit test case example for upload file with MockMvc — free斩, Dec 18 '18 at 07:22
I tried setting spring.servlet.multipart.enabled=false, still the Iterator dont return me any value. Why would be this ? — Suvojit, May 06 '20 at 15:09
In spring-boot-2 the correct property name is `spring.servlet.multipart.enabled=false` — tchudyk, Jun 06 '22 at 13:23

score 18 · Answer 2 · edited Jun 20 '20 at 09:12

If you're using a recent version of spring boot (I'm using 2.0.0.M7) then the property names have changed. Spring started using technology specific names

spring.servlet.multipart.maxFileSize=-1

spring.servlet.multipart.maxRequestSize=-1

spring.servlet.multipart.enabled=false

If you're getting StreamClosed exceptions caused by multiple implementations being active, then the last option allows you to disable the default spring implementation

score 3 · Answer 3 · edited Aug 18 '17 at 14:44

3

Please try to add spring.http.multipart.enabled=false in application.properties file.

edited Aug 18 '17 at 14:44

user2771704

5,994
6
37
38

answered Aug 18 '17 at 14:21

HopeBing

49
3

score 0 · Answer 4 · edited Jun 29 '18 at 02:05

I use kindeditor + springboot. When I use (MultipartHttpServletRequest) request. I could get the file, but I use appeche-common-io:upload.parse(request) the return value is null.

public BaseResult uploadImg(HttpServletRequest request,String type){
                MultipartHttpServletRequest multipartRequest = (MultipartHttpServletRequest) request;
                MultiValueMap<String, MultipartFile> multiFileMap = multipartRequest.getMultiFileMap();

score 0 · Answer 5 · edited Aug 20 '19 at 16:45

0

You Can simply add spring properties:

spring.servlet.multipart.max-file-size=20000KB
spring.servlet.multipart.max-request-size=20000KB

here my maximum file size is 20000KB, you can change if required.

edited Aug 20 '19 at 16:45

Jeevan Patil

6,029
3
33
50

answered Aug 20 '19 at 16:01

pranav patil

21
3

SpringBoot: Large Streaming File Upload Using Apache Commons FileUpload

5 Answers5

Linked

Related