Trying to send JSON payload with urllib2, getting HTTPError 400

Question

Here's a code sample:

# Not actual payload, but exact same structure.  Verified that payload is correct.
payload = {
    'object': {
        'token': '6867r474850557',
        'contact': {
            'FirstName': 'Jim',
            'LastName': 'Bob',
            'Email': 'email@fmail.com',
            'Phone': '111-111-1111'
        },
        'request': {
            'Subject': 'Hello!',
            'Message': 'Test Message'
        },
        'fields': [ "LastName", "FirstName" ]
    }
}

r = urllib2.Request("https://someawesomeurl.com", json.dumps(payload), {"Content-type": "application/json"})
f = urllib2.urlopen(r)
resp = f.read()

It fails on the call to urlopen with the error urllib2.HTTPError: HTTP Error 400: Bad Request. I'm guessing I'm not sending the JSON payload properly but I'm not sure what I have to do to correct it.

EDIT Here's the working PHP implementation I'm trying to implement in Python:

$url = 'https://someawesomeurl.com';

$body = array (
    'object' => array(
    'token' => '6867r474850557',
        'contact' => array (
            'FirstName' => "Jim",
            'LastName' => "Bob",
            'Email' => "email@fmail.com",
            'Phone' => "111-111-1111"
        ),
        'request' => array (
            'Subject' => "Hello!",
            'Message' => "Test Message"
        ),
    'fields' => array('LastName')
));

$params = json_encode($body);
$curl = curl_init($url);

curl_setopt($curl, CURLOPT_SSL_VERIFYPEER, false);
curl_setopt($curl, CURLOPT_HEADER, false);
curl_setopt($curl, CURLOPT_RETURNTRANSFER, true);
curl_setopt($curl, CURLOPT_POST, true);
curl_setopt($curl, CURLOPT_POSTFIELDS, $params);
curl_setopt($curl, CURLOPT_HTTPHEADER, array(  "Content-type: application/json"));

$response = curl_exec($curl);

EDIT #2 Is it possible that urllib2 is modifying the JSON when sending it out? I've wrapped the urlopen call in a try/except block and printed out HTTPError's exception message:

 [{"message":"Premature end of list at [line:1, column:727]","errorCode":"JSON_PARSER_ERROR"}]

I've run the JSON through lint and it comes back as valid so I don't know why the server would be complaining about bad formatting.

I would recommend taking a look at http://stackoverflow.com/questions/3290522/urllib2-and-json — Matt Seymour, Dec 12 '12 at 16:14
Tried a couple of suggestions on that page. Adding the content length to the header and urlecoding the payload instead of using json.dumps gives me the same error. — Don, Dec 12 '12 at 16:53
are you pointing the request to a valid domain which can accept what the data you are sending it. Can you setup something on localhost to better diagnose the issue. It could be more of an issue with the server. — Matt Seymour, Dec 12 '12 at 16:54
Yes. I can send the same payload to the same address through PHP (using curl) and it works fine. Need a Python implementation for App Engine. — Don, Dec 12 '12 at 16:58

score -3 · Answer 1 · answered Dec 12 '12 at 16:12

This doesn't work with urllib2, you must "requests" (http for human):

http://pypi.python.org/pypi/requests

The documentation provide examples for your usecase: http://docs.python-requests.org/en/latest/

an example from one of my project:

def query(self, path, **kwargs):
    """the token must be given as GET and args as POST JSON object"""
    encoded_args = urllib.urlencode({"token": self.token})
    url = self.url_tpl % ('/api/%s' % path, encoded_args)
    headers = {'content-type': 'application/json'}

    data = json.dumps(kwargs).encode('utf-8')
    response = requests.post(url, data=data, headers=headers)
    if response.status_code == 200:
        return response.json
    else:
        raise Exception("response status: %s" % response.status_code)

This was helpful to me & fixed my problem trying to connect to SugarCRM's API in a similar fashion. One thing that caught me out is that Sugar expects a "PUT" & urllib2 only allows "POST" or "GET" unless you tweak it (http://stackoverflow.com/questions/111945/is-there-any-way-to-do-http-put-in-python) — Danimal, Jul 29 '14 at 12:24

Trying to send JSON payload with urllib2, getting HTTPError 400

1 Answers1