1

Developers on my project moved from apllication/json to multipart/form-data. Because of that i wanted to rebuild my api tests but i have a problem with posting json object with multipart/form-data

for example simple sign in request:

json that sends from app for sign in (while application/json):

{
  "session": {
    "email": "john_doe@example.com",
    "password": "examplepassword"
  }
}

With posting through multipart/form-data transforms into

--Boundary+.....
Content-Disposition: form-data; name="session[email]"

john_doe@example.com
--Boundary+.....
Content-Disposition: form-data; name="session[password]"

examplepassword
--Boundary+.....

If I'm understand correctly and as developers approves they still sends same json objects but app automatically forms it like this when its is posted as multipart/form-data. They using AFnetworking

I created next function for multipart posting

def post_request_multipart_with_status_and_response(self, endpoint, data):
    request_post = requests.post(self.domain + "%s" % endpoint, files=data, headers=self.header)
    status = request_post.status_code
    requests_post_json = request_post.json()
    return status, requests_post_json

FYI: i only put header with autorization token inside of requests.post. I don't try to put my boundary or Content type because i know that requests will handle it by itself.

And when i send next data into it - everything works cool:

data = {
            "session[email]": (None, "john_doe@example.com"),
            "session[password]": (None, "examplepassword")
        }

but if i try to send a json into it:

data = {
      "session": {
        "email": "john_doe@example.com",
        "password": "examplepassword"
      }
    }
data = {(None, json.dumps(data))
    }

Then it sends in another manner 

Content-Type: multipart/form-data; boundary=AAAbbbCCCd

'
send: b'--AAAbbbCCCd
Content-Disposition: form-data; 

{"session": {"password": "examplepassword", "email": "john_doe@example.com"}}
--8a78b52e98d34fecae72d054e577c8ad--
'

So i want requests to split what it would otherwise send as normal form-data and add it as a separate multipart field for each node. When i'm passing data = { "session": { "email": "john_doe@example.com", "password": "examplepassword" } }. Requests will solve this nested structures by itself. Like this session[email] and this session[password] automatically

Unbrok3n
  • 27
  • 1
  • 9
  • I can't imagine why you would think that. The result of `json.dumps()` is a string. How would requests be expected to reformat your string in the way that you want? – Daniel Roseman May 30 '17 at 11:26
  • The `requests.post` function has keyword parameter `json` which can take a simple Python dict. This is usually my way of posting json... – user2390182 May 30 '17 at 11:27
  • Why in the world would you switch to `multipart/form-data` if you expect to send complex/nested data to it? While, technically, URL encoding (which is what is used in form-data) supports nested structures in the form of `variable[subvariable][subvariable]=value` you'll be in a world of hurt attempting to deal with `really complicated` JSONs delivered that way. Also, how are you giving your `data` to `requests`? As a `dict` passed to its `data` argument, or are you first serializing it? – zwer May 30 '17 at 11:27
  • @Daniel Roseman thats why i am asking how to solve it because i understand that i'm doing something incorrectly. But i didn't found any info related to my question while googling it. – Unbrok3n May 30 '17 at 11:29
  • Here is the [relevant section of the `requests` documentation](http://docs.python-requests.org/en/master/user/quickstart/#more-complicated-post-requests). As others have already mentioned, form-encoding is the default and you shouldn't pass a JSON-encoded string as `data`, but a Python data structure (a dict or an iterable of 2-tuples). – das-g May 30 '17 at 11:31
  • @zwer so tried both. when i tried to send it as dict `data = { "session": { "email": "examplepassword", "password": "john_doe@example.com" } }` it throw `AttributeError: 'dict' object has no attribute 'read'`. How it sends when i also json.dumps() it, i showed in first post. – Unbrok3n May 30 '17 at 11:34
  • @Unbrok3n - you used `request_post = requests.post(self.domain + "%s" % endpoint, data=data, headers=self.header)` (notice the `data` not `files` argument - why would you try to send `files` anyway?) and the `data` you passed to the function was a `dict`? I find that hard to believe unless you're using some custom-built version of `requests`. – zwer May 30 '17 at 11:37
  • @zwer i am using `files` argument (look at the fuction from first message). – Unbrok3n May 30 '17 at 11:39
  • @zwer the main question here, that i believed that request can form this nested structures to keys automatically. So when i'm passing `data = { "session": { "email": "john_doe@example.com", "password": "examplepassword" } }`. Requests will solve this nested structures by itself. Like this `session[email]` and this `session[password]`. Its hard for me to imagine that now i should hardcode all this keys by myself for other complicated jsons. I thought that they can be formed automatically. – Unbrok3n May 30 '17 at 11:42
  • @zwer because sending it with `files` argument is the correct way if base on official documentation or related posts https://stackoverflow.com/questions/12385179/how-to-send-a-multipart-form-data-with-requests-in-python – Unbrok3n May 30 '17 at 11:44
  • That explains a lot... What you are asking `requests` to do in your case is to split what it would otherwise send as normal `form-data` and add it as a separate `multipart` field for each node (again, I leave it to the API developers to explain why would anyone with a modicum of sense want to do that). Based on how `requests` works, you might be able to pull it by passing an empty `list` as `files` **and** your data `dict` as `data` argument. – zwer May 30 '17 at 12:12
  • @zwer. Yep, thanks, i edited my question. I rebuilded my function to `request_post = requests.post(self.domain + "%s" % endpoint, files=[], data=data, headers=self.header)` and send my dict `data = { "session": { "email": "john_doe@example.com", "password": "examplepassword" }}` as data argument. But it posted simpe url encoded like this. `Content-Type: application/x-www-form-urlencoded session=email&session=password` – Unbrok3n May 30 '17 at 12:33
  • @Unbrok3n - dang... there's a `if files:` check in the `requests` module before it tries to build a multipart body. Is it ok by your API to send some extra data? If so, try passing an empty dummy stream as your `files` argument and keep the `data` field set to your `dict` with data. Something like: `requests.post(self.domain + "%s" % endpoint, files={'dummy': io.BytesIO()}, data=data, headers=self.header)` (you'll have to `import io`, obviously) – zwer May 30 '17 at 12:45
  • @zwer didn't help either. it sent it like send: `b'--AAAbbbCCCd Content-Disposition: form-data; name="session" password` With `password`/`email` as a key names, not as values – Unbrok3n May 30 '17 at 12:53
  • @Unbrok3n - In that case you'll have to deal with your own multipart building (via the `requests_toolbelt.multipart.encoder.MultipartEncoder` for example) - what you are asking is anything but standard (more like AFNetworking perk) and expecting for `requests` to do it for you is a bit of a stretch. Why would anyone sane want to send structured data like this is still a mystery that you should discuss with 'developers on your project' that moved from a perfectly reasonable request structure to this. – zwer May 30 '17 at 13:13
  • @zwer yep, i agree that this is really strange decission from developers side. Thanks for help anyway. – Unbrok3n May 30 '17 at 13:28

0 Answers0