How to check size of uploaded file safely in bottlepy?

Question

I'm really afraid of that read() operation because it uses memory. For instance, anybody could DDoS my server by uploading a 1gb file, correct?

name = request.forms.get('name')
data = request.files.get('data')
if name and data.file:
    raw = data.file.read() # This is dangerous for big files
    filename = data.filename
    return "Hello %s! You uploaded %s (%d bytes)." % (name, filename, len(raw))

Is there any safe solution to get the uploaded file size? One guess would be to get file size from the file system; request.files.get('data') is probably stored somewhere in temp file right?

If you run bottle via wsgi under something like apache, apache can limit upload sizes. — jordanm, Jun 27 '12 at 02:48
Bottle stores request.body and all uploaded files in temporary files (or ByteIO buffers if small enough) as soon as you access any of them. If you want to restrict upload size before the server does all the work, check for request.content_length. If you just want to be sure everything fits into memory, read/copy the uploaded files in smaller chunks as described by pyfunc in his answer. — defnull, Jun 28 '12 at 14:23

pyfunc · Accepted Answer · 2012-07-10T06:37:59.720

2

Can you check if you can read chunks of data, one at a time.

If this is possible then:

name = request.forms.get('name')
data = request.files.get('data')
raw = ""
if name and data.file:
    while True:
        datachunk = data.file.read(1024)
        if not datachunk:
            break
        raw = raw + datachunk

    filename = data.filename
    return "Hello %s! You uploaded %s (%d bytes)." % (name, filename, len(raw))

If this is possible, then you should be able to also add a tracking mechanism on how large a file you want to read and if exceeded abort this operation.

How ever this solves only one of the possible ways of DDOS.

edited Jul 10 '12 at 06:37

answered Jun 27 '12 at 02:29

pyfunc

65,343
15
148
136

could you fix your code? size= must be removed. this produces TypeError: read() takes no keyword arguments, if only 1024 specified it works – holms Jul 10 '12 at 05:50
@holms: Thanks for the correction. I assumed there would be a named argument to pass. Try looking at the code, it may be named some thing else. – pyfunc Jul 10 '12 at 06:39

score -3 · Answer 2 · edited May 23 '17 at 10:26

-3

This is an interesting question. Normally you would either get the statistics on the file object or use os.path to get the size. This question has been covered before.

But you are asking about how to tell the client side file size from the server before we spend time uploading it. I think that the best way to do this is with JavaScript. This question can get you started on how to get some JavaScript code into your BottlePy application.

Use the JavaScript to validate your input so that you can assume in your server code that the file is within your limits. Once you figure this out, I suggest that you ask the BottlePy people to add support for this directly to the BaseRequest.files support.

edited May 23 '17 at 10:26

Community

1
1

answered Jun 27 '12 at 02:55

ChipJust

1,376
12
20

`But you are asking about how to tell the client side file size from the server before we spend time uploading it.` <-- really? I don't remember writing that. you can change that field to size that pass validation, and pass 1gb file so what..? – holms Jun 27 '12 at 02:56
Oh, maybe I misunderstood then. I thought you wanted to get the file size without exposing yourself to DDOS...maybe I was assuming something there. The only way to do that is to check the file size before you actually upload it, otherwise what is the point. The big file is already uploaded, so you didn't prevent DDOS. The other option is like pyfunc suggests, but you add a check to break out at some limit. The last option is as you suggest and get the stats on the temp file. – ChipJust Jun 27 '12 at 03:00
@holms "you can change that field to size that pass validation, and pass 1gb file so what..?" I don't get your point. You control the client side validation code, so you can allow whatever size you want... – ChipJust Jun 27 '12 at 03:05
since when "you" control CLIENT code?? open firebug and modify anything you want. – holms Jun 27 '12 at 03:07
DDOS is not about file size you got uploaded, i doubt that somebody will try to upload 500gb file in there *lol*. it's dangerous when you check file size itself in python with data.file.read(), because you need to put whole file to RAM first and `pyfunc` method with chunks seems legit, after chunk size gets to the limit I can abort operation and return error. even so it's possible to DDOS with multiple uploads, but then I can just reduce connection limit. – holms Jun 27 '12 at 03:08
minus 1 for suggesting useless client-side validation. – Jonathon Reinhart Apr 27 '15 at 11:17

How to check size of uploaded file safely in bottlepy?

2 Answers2