How to prevent node.js from hanging when a big file upload is initiated?

Question

I'm trying to protect the app I'm working on from payload DOS attacks (when someone tries to upload a very big file). My middlewares look like this (I'm using express-fileupload and body-parser:

import fileUpload from "express-fileupload";
import { json, urlencoded } from "body-parser";

// ...

app.use(
    fileUpload({
        limits: { fileSize: 2 * 1024 * 1024 }
    })
);
app.use(json({ limit: "2mb" }));
app.use(urlencoded({ limit: "2mb", extended: true }));

My problem is that despite the fact that I configured my middlewares to limit request size to 2mb when I try to upload a 7gb file the process halts.

How do I prevent this properly?

Which exact NPM modules are you using for your file upload middleware? You show three different ones in your question. Which exact modules are those? — jfriend00, Aug 15 '20 at 14:36
Usually it is suggested to use temp file for buffering upload data instead of memory — Vinay, Aug 15 '20 at 15:11

score 1 · Accepted Answer · answered Aug 15 '20 at 14:51

1

If you want to stop the process if the size is over, use abortOnLimit option. (Otherwise, it will truncate automatically)

Like this

app.use(
    fileUpload({
        limits: { fileSize: 2 * 1024 * 1024 }
    }),
    abortOnLimit: true
);

answered Aug 15 '20 at 14:51

Gavin Kwon

126
4

I tried this and the proccess still hangs when I try to upload a 7 gig file :( – Adam Arold Aug 15 '20 at 17:50
Ok, I tried the debug option and it turns out that node indeed stops the upload, it is just the browser that hangs. If I reload the page all is well. – Adam Arold Aug 15 '20 at 18:09

lukaszkups · Answer 2 · 2020-08-15T14:25:08.753

-3

Have you considered using web workers ? As far as I know you can run specific scripts thanks to them in the background/separate thread so the main one won't be blocked.

Here's the good tutorial how to create one: link

EDIT:

Ok, since the above solution is not something that OP asked for here's another one - maybe try to validate file size on the front-end side? Code snippet below (source)

function ValidateSize(file) {
        var FileSize = file.files[0].size / 1024 / 1024; // in MB
        if (FileSize > 2) {
            alert('File size exceeds 2 MB');
           // $(file).val(''); //for clearing with Jquery
        } else {

        }
    }

 <input onchange="ValidateSize(this)" type="file">

edited Aug 15 '20 at 14:25

answered Aug 15 '20 at 14:11

lukaszkups

5,790
9
47
85

I want to prevent my users from uploading gigabytes of data instead of offloading the unnecessary operations to workers. – Adam Arold Aug 15 '20 at 14:14
1

Web Workers are browsers-side so are no good for helping with Node.js performance. It might be possible to use Worker Threads, but I haven't a clue how to (or if they can be) hook them into Express to unburden the system from a large file upload. A good answer would explain how to do that. – Quentin Aug 15 '20 at 14:16
I've updated my answer with different approach. Unfortunately my experience with Workers is not sufficient to deliver a solid answer example, sorry. – lukaszkups Aug 15 '20 at 14:26

How to prevent node.js from hanging when a big file upload is initiated?

2 Answers2