Flask+cURL concatenates multi-line CSV input into a single string

Question

I have a route in my flask app that starts off like this:

@app.route('/invocations', methods=['POST'])
def predict():
    """
    Do an inference on a single batch of data.
    """
    if flask.request.content_type == 'text/csv':
        X_train = flask.request.data.decode('utf-8')
        X_train = pd.read_csv(StringIO(X_train), header=None).values

To test this path I am sending a POST request to the server from a csv formatted file with multiple lines in it: curl -X "POST" -H "Content-Type: text/csv" -d @health-check-data.csv http://localhost:5000/invocations.

However, to my surprise, when I execute X_train = flask.request.data.decode('utf-8'), I get the contents of the csv concatenated into a single string with newlines removed.

Why is flask (or curl?) doing this, and how do I get around this behavior?

Elliot B. · Answer 1 · 2018-12-18T22:01:49.093

2

That's just the expected behavior for flask.request.data.decode('utf-8') -- it returns a string.

You can convert that string to a buffer that read_csv can read:

upload = flask.request.data.decode('utf-8')
buffer = StringIO()
buffer.write(upload)
buffer.seek(0)
X_train = pd.read_csv(buffer), header=None).values

However, you can bypass that process by reading the csv file via Flask's request.files.get which returns a buffer:

X_train = pd.read_csv(request.files.get('name'),  header=None).values

edited Dec 18 '18 at 22:01

answered Dec 18 '18 at 20:36

Elliot B.

17,060
10
80
101

As I understand this answer is actually incorrect. For the second part: I am running `POST` from the command line with the `-d` argument, and that data is not being passed through a form, so `request.files` is empty. For the first part: `cURL` will strip newlines when passed data with the `-d` flag ([ref](https://stackoverflow.com/questions/6408904/send-post-request-with-data-specified-in-file-via-curl#comment70445817_6409028)). You can avoid this behavior by using the `--data-binary` flag, which is what I did, and that worked. So the data is already mangled by the time it hits `flask`. – Aleksey Bilogur Dec 18 '18 at 22:31
It's unfortunate that the [`man` entry](https://curl.haxx.se/docs/manpage.html#-d) for the `curl -d` doesn't specify _how_ the encoding is handled. I wonder how else the `-d` flag mutates input... – Aleksey Bilogur Dec 18 '18 at 22:35
1

@AlekseyBilogur Ahh, I see that you edited your question after I posted my answer to now say `string with newlines removed` instead of the original `string with no newlines removed` -- that is a very different meaning for this question :) – Elliot B. Dec 18 '18 at 22:41
Yes, that was a terrible, no good, very bad typo, my apologies! – Aleksey Bilogur Dec 19 '18 at 00:32

Flask+cURL concatenates multi-line CSV input into a single string

1 Answers1