How to limit request execution time?

Question

I am supposed to give reply in 3 seconds.

My handler talks with 3rd party server, which could reply in 3 seconds or could not.

I think about the following code -

class MainReply(webapp2.RequestHandler):
    def get(self):
        # do something
        # start task to talk with 3rd server
        for i in range(300): # wait 3 seconds
            # check task status
            # if finished, then break
            time.sleep(0.01)
        # if not finished, inform user

Is it correct approach? Or is there better solution?

Upd. I am working on Voice Assistant bot (something similar to Google Assistant), where bot must reply within 3 seconds. And bot can not initiate answer itself, i.e. I can not give another answer, once request is completed. And since this is voice assistant, I can not give link. I was thinking about the following approach - if I can give a normal answer in 3 seconds, then give it. If not - ask user to ask again with simple word like "Status".

In terms of programming fundamentals I'd say that would never be the approach. You should search for event suscription, and observer patterns and if the language does not provide it by default check available [options](https://stackoverflow.com/questions/1092531/event-system-in-python). — MikeMajara, Oct 07 '19 at 06:45
@MikeMajara, thanks, but I don't think it will work in my case. Let's say I subscribe for task completion event. But it may happen before 3 seconds or after. How will it help? I can answer user *only* in 3 seconds. — LA_, Oct 07 '19 at 06:54
What do you mean by answer the user only in 3 seconds? If the answer is there in < 3s: give it back; else return error? — MikeMajara, Oct 07 '19 at 07:27

score 0 · Answer 1 · answered Oct 07 '19 at 11:04

Ground rule is you should never do a pool as described in your post because (1) you "manually" block more resources than needed (2) you mess up your code simulating asynchronous code that really is not, and (3) languages generally provide tools to deal with concurrent programming, which are specially meant for this type of tasks, and this do the job for you far better than you could implement it (in most cases).

Following your requirements. I wrote a simple program you can test. Check here for asyncio.wait_for documentation. Find here also some useful guide into async.

Copy code into a file and run $ python [filename]

import asyncio
import random
from concurrent.futures._base import TimeoutError

async def safe_get():
    result = "I'm currently out of my office."
    try:
        result = await asyncio.wait_for(get(), 3)
    except TimeoutError as e:
        print("Responder thought too much about his answer")
    return result

async def get(a=1,b=6):
    random_time = random.randint(a,b)
    await asyncio.sleep(random_time)
    return "I'm doing good thanks! Sorry it took me " + str(random_time) + "s to answer."

async def main():
    # Toggle comment on the two following lines to see (1) unwanted behaviour and (2) wanted behaviour
    # response = await get()
    response = await safe_get()
    print(response)

if __name__ == "__main__":
    asyncio.run(main())

Thanks. But I am afraid it will not work with Google App Engine... But here is the option - https://cloud.google.com/appengine/docs/standard/python/issue-requests?csw=1#issuing_an_asynchronous_request — LA_, Oct 07 '19 at 13:34
So it will not work, why? I must be missing out something. The link you send states "To issue an asynchronous request, your application must: ..." And you also have instructions there that tell you how to set a timeout in your request. I cannot run the google-api code, but it seems to me quite similar. Can you point out for me what makes it impossible to do? — MikeMajara, Oct 07 '19 at 15:05
Sorry, I didn't mention initially that I use Python 2.7. And here is the confirmation that it doesn't work - https://stackoverflow.com/questions/53719006/how-to-do-async-api-requests-in-a-gae-application. — LA_, Oct 08 '19 at 20:08
And what is keeping you from using the code snippet in the [answer](https://stackoverflow.com/a/53723814/3157057) to that post, together with the `deadline` parameter from `urlfetch.create_rpc()` stated in the [docs](https://cloud.google.com/appengine/docs/standard/python/issue-requests?csw=1#issuing_an_asynchronous_request)? — MikeMajara, Oct 09 '19 at 17:42

score 0 · Answer 2 · answered Oct 07 '19 at 11:10

Apart from a questionable user experience - slow response and potentially not the response your users are waiting for - such approach raises a risk for your server's integrity: for every pending request your server will use resources and in the event of a high incoming requests peak it may run either run over its available pool or resources (if limited) or may drive your costs up (for example by launching additional instances to keep up with the other incoming requests if it is configured for automatic scaling).

In general for requests that can take a long time it's better to reply immediately with an ID of a pending request and a complementary URLs where the client can check the request status and eventually get the expected response, when available. Your server would take care - in the background, not on the original incoming request thread - of getting the answer from the 3rd party server. This way you can deal with any kind of request which can take a long time to obtain the answer for, either from a 3rd party or from internal processing.

Thanks, Dan. Let me give you more details - I am working on Voice Assistant bot (something similar to Google Assistant), where bot must reply within 3 seconds. And bot can not initiate answer itself, i.e. I can not give another answer, once request is completed. And since this is voice assistant, I can not give link. I was thinking about the following approach - if I can give a normal answer in 3 seconds, then give it. If not - ask user to ask again with simple word like "Status". — LA_, Oct 07 '19 at 13:05

score 0 · Answer 3 · answered Oct 08 '19 at 21:17

Do you talk to this 3rd party server via http? can you just use the timeout parameter in python-requests?

class MainReply(webapp2.RequestHandler):
    def get(self):
        # do something
        try:
            r = requests.post(... , timeout=3)
            # finished, inform user
        except requests.exceptions.Timeout:
            # not finished, inform user

How to limit request execution time?

3 Answers3