ValueError: Data must not be a string

Question

I am trying to do the following with requests:

data = {'hello', 'goodbye'}
json_data = json.dumps(data)
headers = {
        'Access-Key': self.api_key,
        'Access-Signature': signature,
        'Access-Nonce': nonce,
        'Content-Type': 'application/json',
        'Accept': 'text/plain'
    }
r = requests.post(url, headers=headers, data=json_data, 
                 files={'file': open('/Users/david/Desktop/a.png', 'rb')})

However, I get the following error:

ValueError: Data must not be a string.

Note that if I remove the files parameter, it works as needed. Why won't requests allow me to send a json-encoded string for data if files is included?

Note that if I change data to be just the normal python dictionary (and not a json-encoded string), the above works. So it seems that the issue is that if files is not json-encoded, then data cannot be json-encoded. However, I need to have my data encoded to match a hash signature that's being created by the API.

Yes, but then it screws up the API signature (which every other method depends on). — David542, Dec 18 '14 at 18:16
You need to specify the header so that the module knows it's a JSON string. See [this question](http://stackoverflow.com/questions/9733638/post-json-using-python-request) for details. — tyteen4a03, Dec 18 '14 at 18:18
@tyteen4a03 -- no, that's not it. I already have those headers. It has to do with the data being passed is a string but the files being passed is not. — David542, Dec 18 '14 at 18:21
Just saw that you have specified a file; I don't think you can upload a file and use a JSON body at the same time. — tyteen4a03, Dec 18 '14 at 18:24
@tyteen4a03 -- ok, thanks for that insight. How would I upload this file then? Send its data as a string? — David542, Dec 18 '14 at 18:25
@David542 Encapsulate your JSON string in a normal POST key-value pair (i.e "json" => json.dumps(data)). — tyteen4a03, Dec 18 '14 at 18:27
Ok -- if I try putting that in my dict and doing `json` encoding I get a `UnicodeDecodeError` from the decoder though. It seems I can't encode this type of data in json? — David542, Dec 18 '14 at 18:30

score 9 · Accepted Answer · edited May 23 '17 at 12:33

9

When you specify your body to a JSON string, you can no longer attach a file since file uploading requires the MIME type multipart/form-data.

You have two options:

Encapsulate your JSON string as part as the form data (something like json => json.dumps(data))
Encode your file in Base64 and transmit it in the JSON request body. This looks like a lot of work though.

edited May 23 '17 at 12:33

Community

1
1

answered Dec 18 '14 at 18:30

tyteen4a03

1,812
24
45

Thanks, however when I try doing json.dumps with the image data, it returns a `UnicodeDecodeError`. – David542 Dec 18 '14 at 18:31
5

Can someone please show example of such encapsulation suggested above by tyteen4a03? Encapsulate your JSON string as part as the form data (something like json => json.dumps(data)) – Itai Malek Dec 13 '17 at 08:50
1

example needed pls – illuminato Oct 07 '21 at 11:37

score 4 · Answer 2 · answered Aug 23 '21 at 10:22

4

1.Just remove the line json_data = json.dumps(data) and change in request as data=data.

2.Remove 'Content-Type': 'application/json' inside headers.

This worked for me.

answered Aug 23 '21 at 10:22

Akash Raj

41
2

score 1 · Answer 3 · answered Jul 01 '19 at 01:25

1

removing the following helped me in my case:

'Content-Type': 'application/json'

then the data should be passed as dictionary

answered Jul 01 '19 at 01:25

Nabat Farsi

840
1
9
17

score 0 · Answer 4 · answered Jul 10 '15 at 11:24

Alternative solution to this problem is to post data as file.

You can post strings as files. Read more here: http://docs.python-requests.org/en/latest/user/quickstart/#post-a-multipart-encoded-file
Here is explained how to post multiple files: http://docs.python-requests.org/en/latest/user/advanced/#post-multiple-multipart-encoded-files

Houman · Answer 5 · 2022-05-26T19:05:42.923

If your files are small, you could simply convert the binary (image or anything) to base64 string and send that as JSON to the API. That is much simpler and more straight forward than the suggested solutions. The currently accepted answer claims that is a lot of work, but it's really simple.

Client:

with open('/Users/houmie/Downloads/log.zip','rb') as f:
   bytes = f.read()
   tb = b64encode(bytes)
   tb_str = tb.decode('utf-8')
   body = {'logfile': tb_str}
   r = requests.post('https://prod/feedback', data=json.dumps(body), headers=headers)

API:

def create(event, context):
    data = json.loads(event["body"])
    if "logfile" in data:
        tb_back = data["logfile"].encode('utf-8')
        zip_data = base64.b64decode(tb_back)

ValueError: Data must not be a string

5 Answers5

Linked

Related