6

Using Django (hosted by Webfaction), I have the following code 

import time
def my_function(request):
    time.sleep(10)
    return HttpResponse("Done")

This is executed via Django when I go to my url, www.mysite.com

I enter the url twice, immediately after each other. The way I see it, both of these should finish after 10 seconds. However, the second call waits for the first one and finishes after 20 seconds.

If, however, I enter some dummy GET parameter, www.mysite.com?dummy=1 and www.mysite.com?dummy=2 then they both finish after 10 seconds. So it is possible for both of them to run simultaneously.

It's as though the scope of sleep() is somehow global?? Maybe entering a parameter makes them run as different processes instead of the same???

It is hosted by Webfaction. httpd.conf has:

KeepAlive Off
Listen 30961
MaxSpareThreads 3
MinSpareThreads 1
ServerLimit 1
SetEnvIf X-Forwarded-SSL on HTTPS=1
ThreadsPerChild 5

I do need to be able to use sleep() and trust that it isn't stopping everything. So, what's up and how to fix it?

Edit: Webfaction runs this using Apache.

user984003
  • 28,050
  • 64
  • 189
  • 285
  • What is the use-case for using `sleep`? Seems like you're doing something very odd if you want to use that in a request/response cycle. Maybe you could explain what you're trying to do, and we can suggest alternatives. – Daniel Roseman Feb 26 '13 at 09:11

3 Answers3

8

As Gjordis pointed out, sleep will pause the current thread. I have looked at Webfaction and it looks like their are using WSGI for running the serving instance of Django. This means, every time a request comes in, Apache will look at how many worker processes (that are processes that each run a instance of Django) are currently running. If there are none/to view it will spawn additonally workers and hand the requests to them.

Here is what I think is happening in you situation:

  • first GET request for resource A comes in. Apache uses a running worker (or starts a new one)
  • the worker sleeps 10 seconds
  • during this, a new request for resource A comes in. Apache sees it is requesting the same resource and sends it to the same worker as for request A. I guess the assumption here is that a worker that recently processes a request for a specific resource it is more likely that the worker has some information cached/preprocessed/whatever so it can handle this request faster
  • this results in a 20 second block since there is only one worker that waits 2 times 10 seconds

This behavior makes complete sense 99% of the time so it's logical to do this by default.

However, if you change the requested resource for the second request (by adding GET parameter) Apache will assume that this is a different resource and will start another worker (since the first one is already "busy" (Apache can not know that you are not doing any hard work). Since there are now two worker, both waiting 10 seconds the total time goes down to 10 seconds.

Additionally I assume that something is **wrong** with your design. There are almost no cases which I can think of where it would be sensible to not respond to a HTTP request as fast as you can. After all, you want to serve as many requests as possible in the shortest amount of time, so sleeping 10 seconds is the most counterproductive thing you can do. I would recommend the you create a new question and state what you actual goal is that you are trying to achieve. I'm pretty sure there is a more sensible solution to this!
simplegr33n
  • 130
  • 2
  • 10
Martin Thurau
  • 7,564
  • 7
  • 43
  • 80
  • Thanks for looking into this. Even without sleep, there seems a lot of reasons for why I would NOT want calls to run in the same thread. For example, if a call is looking up info from another website. urlopen() can be slow waiting for a response and I'd sure hate for every call to be waiting in a row for the other calls to finish. Or a slow database query. I will ask another question about how to start each call in a new thread. – user984003 Feb 26 '13 at 09:40
  • Django is single-threaded, so that's always gonna be a problem. If you need asynchronous, non-blocking stuff, you'll probably want to look at something like Tornado. – Jack Shedd Feb 26 '13 at 09:49
5

Assuming you run your Django-server just with run() , by default this makes a single threaded server. If you use sleep on a single threaded process, the whole application freezes for that sleep time.

Gjordis
  • 2,540
  • 1
  • 22
  • 32
  • So I just can't use sleep() because this will affect every call?? Is this just a Django issue? I do need some way to have my functions pause without every call being paused. – user984003 Feb 26 '13 at 08:36
  • BTW, not sure if using Django automatically means that I am running the Django server. Webfaction uses Apache. – user984003 Feb 26 '13 at 08:42
  • Good point, still the effect seems the same. Only one thread handles both requests, if both of them hang – Gjordis Feb 26 '13 at 08:47
1

It may simply be that your browser is queuing the second request to be performed only after the first one completes. If you are opening your URLs in the same browser, try using the two different ones (e.g. Firefox and Chrome), or try performing requests from the command line using wget or curl instead.

dragonroot
  • 5,653
  • 3
  • 38
  • 63