2

I wrote a helper decorator for Tornado RequestHandler:

def return_response(method):
    @wraps(method)
    async def wrapper(self, *args, **kwargs):
        response = method(self, *args, **kwargs)
        if inspect.isawaitable(response):
            response = await response
        self.set_status(response.status)
        self.write(response.body)
    return wrapper

For some reason it fails with the following error:

Traceback (most recent call last):
  File ".../.venv/lib/python3.6/site-packages/tornado/web.py", line 1511, in _execute
    result = yield result
  File ".../.venv/lib/python3.6/site-packages/tornado/gen.py", line 1055, in run
    value = future.result()
  File ".../.venv/lib/python3.6/site-packages/tornado/concurrent.py", line 238, in result
    raise_exc_info(self._exc_info)
  File "<string>", line 4, in raise_exc_info
  File ".../.venv/lib/python3.6/site-packages/tornado/gen.py", line 307, in wrapper
    yielded = next(result)
  File "<string>", line 6, in _wrap_awaitable
  File ".../src/http_handlers/decorators.py", line 106, in wrapper
    response = await response
  File ".../src/http_handlers/decorators.py", line 106, in wrapper
    response = await response
  File ".../src/http_handlers/decorators.py", line 106, in wrapper
    response = await response
  [Previous line repeated 959 more times]
  File ".../src/http_handlers/decorators.py", line 104, in wrapper
    response = method(self, *args, **kwargs)
RecursionError: maximum recursion depth exceeded in comparison

This error appeares only when the application is up for a while. Is it a bug in Python, Tornado or my code is wrong? Did somebody come across the same problem?

Update

I've managed to solve the problem. It was in completely different place. I used to apply this decorator at runtime in the __new__ method. By mistake I was applying it to methods of the cls, not the actual instance. So methods were being wrapped again each time a new instance af a class was created.

mcferden
  • 316
  • 3
  • 11
  • 2
    To make it clearer, inside `await response`, there may be another function decorated by `return_response`, and again and again. – Sraw Dec 26 '17 at 06:00
  • Why would I await it again and again? If I understand correctly, the wrapper will be called and suspended on the `response = await response` and then will be resumed right after this line. The `if` statement will not be checked twice. – mcferden Dec 26 '17 at 06:02
  • I'm sorry, the first comment is a little confusing, I will delete it. – Sraw Dec 26 '17 at 06:04

1 Answers1

2

Maybe take a look at this What is the maximum recursion depth in Python, and how to increase it? However the answer is not going to be to increase it.

When you keep calling a function from within itself, it keeps adding to the stack, which keeps filling up memory. Python has a limit set on this so you don't run into a stack overflow. After a while, it appears that you reached that limit.

Whatever is happening on line 106 of decorators.py is causing the same method to be called over and over again within itself. I'm not sure we'd be able to tell what exactly is happening without seeing the full usage, but hopefully that gives you a bit of an idea.

I'd suggest trying to find where your code is being recursively called and figuring out a way to restructure your program so it doesn't happen.

SCB
  • 5,821
  • 1
  • 34
  • 43