In Python, are there any libraries that mock out responses for httplib?

Question

I'm currently using python_flickr_api to upload photos for my app: it uses httplib to perform a multipart POST request.

Problem: I want to verify that the upload really is issued in an integration test by intercepting the POST request and creating a pre-canned success response so that my tests can run completely offline and not depend on Flickr (I don't want to upload the same test image 100 times, either!)

To this end, I've tried using two incredible libraries: VCRPy and HTTPretty. Neither of them solves my problem because neither of them support httplib (HTTPretty comes closest, with support for httplib2 only), and I get an error that looks something like this:

 Failure/Error: [Errno 32] Broken pipe

 Traceback:
 ...
 File "/usr/local/lib/python2.7/site-packages/flickr_api/upload.py", line 92, in upload
     r = post(UPLOAD_URL,auth.AUTH_HANDLER,args,photo_file)
 File "/usr/local/lib/python2.7/site-packages/flickr_api/upload.py", line 52, in post
     r = multipart.posturl(url,fields,files)
 File "/usr/local/lib/python2.7/site-packages/flickr_api/multipart.py", line 19, in posturl
     return post_multipart(urlparts[1], urlparts[2], fields,files)
 File "/usr/local/lib/python2.7/site-packages/flickr_api/multipart.py", line 33, in post_multipart
     h.send(body)
 File "/usr/local/Cellar/python/2.7.5/Frameworks/Python.framework/Versions/2.7/lib/python2.7/httplib.py", line 805, in send
     self.sock.sendall(data)
 File "/usr/local/lib/python2.7/site-packages/httpretty/core.py", line 243, in sendall
     return self._true_sendall(data)
 File "/usr/local/lib/python2.7/site-packages/httpretty/core.py", line 216, in _true_sendall
     self.truesock.sendall(data, *args, **kw)
 File "/usr/local/Cellar/python/2.7.5/Frameworks/Python.framework/Versions/2.7/lib/python2.7/socket.py", line 224, in meth
     return getattr(self._sock,name)(*args)

So clearly httpretty is intercepting but is causing a broken pipe.

How can I fix this?

I want to keep using `flickr_api` for this functionality because it handles nasty things like OAuth for me. I want to use [FakeWeb](http://fakeweb.rubyforge.org/)-esque libraries for testing because I don't want to couple my tests to `flickr_api`. — fatuhoku, Sep 15 '13 at 16:13
Can you show up the test and the method you are testing, I can help write a Mock for you. — Mahdi Yusuf, Sep 15 '13 at 16:51
Well, the upload is one line. `flickr.upload(photo_file="/path/to/foobar.jpg", title="foobar")` But ya writing a mock for this is super simple, but as I said I don't want to couple my tests to `python_flickr_api`'s interface. — fatuhoku, Sep 15 '13 at 17:00

score 5 · Answer 1 · answered Sep 15 '13 at 21:07

5

I'm the author of VCR.py. I think I may have fixed the bug preventing VCR from working with the flickr API library you're using. Try installing the version in the fix-flickr-api branch and let me know if that works for you.

answered Sep 15 '13 at 21:07

Kevin McCarthy

51
1
2

Mahdi Yusuf · Answer 2 · 2013-09-15T16:51:29.420

1

There are tons of libraries for this type of functionality, checkout Mock

But it looks like your are looking for something similar to FakeWeb so I would checkout HTTPretty

Seeing as you have already tried HTTPretty. I would mock out the method call that flickr_api makes and have it return a stub that has the payload you are expecting.

Goodluck.

edited Sep 15 '13 at 16:51

answered Sep 15 '13 at 16:32

Mahdi Yusuf

19,931
26
72
101

Yeah, I guess I may already have exhausted the options on this. This indicates to me that I should try to perform the upload a different way. – fatuhoku Sep 15 '13 at 16:57

score 1 · Answer 3 · answered Sep 15 '13 at 16:42

1

Why don't you write a simple WSGI server which prints what you need to?

For example, the following code (using only the Python Standard Library):

def application(environ, start_response):
    print environ
    status = "200 OK"
    headers = [("Content-Type", "text/plain"),
               ("Content-Length", "0")]
    start_response(status, headers)
    return ""

if __name__ == "__main__":
    from wsgiref.simple_server import make_server
    httpd = make_server("localhost", 8080, application)
    httpd.serve_forever()

will print to the screen the environment of the request (stuff like request method, url, etc). You can also very simply print the request body, etc.

In your flickr application, simply replace the URL by localhost:8080 and make the requests to your own computer.

answered Sep 15 '13 at 16:42

Escualo

40,844
23
87
135

Hmmm, I was really hoping for something lighter weight. Besides mocking, setting up a server does seem to be necessary. – fatuhoku Sep 15 '13 at 17:04
Lighter than about 10 lines? – Escualo Sep 15 '13 at 19:19
1

If you're looking at memory cost, yeah, 10 lines of code is barely .5kB — but of course this needs to be integrated into either a bash script or an integration test file which would add a further 6-10 lines. There's an overhead in launching tests because I need to manage a resource: port 8080 (imagine two of these running at the same time...) Tests are more difficult to write because it's non-trivial to reuse the same sort of server for all test responses... The list goes on and on; I mean 'lighter weight' as in lower cost of maintenance in the longer term. – fatuhoku Sep 15 '13 at 19:27
AND it's not as readable as other potential solutions, which is the deal breaker for me, especially for testing. – fatuhoku Sep 15 '13 at 19:28
I guess we have different understandings of Python and web development. Peace. – Escualo Sep 15 '13 at 20:16

In Python, are there any libraries that mock out responses for httplib?

3 Answers3