How can I get the values of the locals of a function after it has been executed?

Question

Suppose I have a function like f(a, b, c=None). The aim is to call the function like f(*args, **kwargs), and then construct a new set of args and kwargs such that:

1. If the function had default values, I should be able to acquire their values. For example, if I call it like f(1, 2), I should be able to get the tuple (1, 2, None) and/or the dictionary {'c': None}.
2. If the value of any of the arguments was modified inside the function, get the new value. For example, if I call it like f(1, 100000, 3) and the function does if b > 500: b = 5 modifying the local variable, I should be able to get the the tuple (1, 5, 3).

The aim here is to create a a decorator that finishes the job of a function. The original function acts as a preamble setting up the data for the actual execution, and the decorator finishes the job.

Edit: I'm adding an example of what I'm trying to do. It's a module for making proxies for other classes.

class Spam(object):
    """A fictional class that we'll make a proxy for"""
    def eggs(self, start, stop, step):
        """A fictional method"""
        return range(start, stop, step)

class ProxyForSpam(clsproxy.Proxy):
    proxy_for = Spam
    @clsproxy.signature_preamble
    def eggs(self, start, stop, step=1):
        start = max(0, start)
        stop = min(100, stop)

And then, we'll have that:

ProxyForSpam().eggs(-10, 200) -> Spam().eggs(0, 100, 1)

ProxyForSpam().eggs(3, 4) -> Spam().eggs(3, 4, 1)

Answer 1

There are two recipes available here, one which requires an external library and another that uses only the standard library. They don't quite do what you want, in that they actually modify the function being executed to obtain its locals() rather than obtain the locals() after function execution, which is impossible, since the local stack no longer exists after the function finishes execution.

Another option is to see what debuggers, such as WinPDB or even the pdb module do. I suspect they use the inspect module (possibly along with others), to get the frame inside which a function is executing and retrieve locals() that way.

EDIT: After reading some code in the standard library, the file you want to look at is probably bdb.py, which should be wherever the rest of your Python standard library is. Specifically, look at set_trace() and related functions. This will give you an idea of how the Python debugger breaks into the class. You might even be able to use it directly. To get the frame to pass to set_trace() look at the inspect module.

Answer 2

I've stumbled upon this very need today and wanted to share my solution.

import sys

def call_function_get_frame(func, *args, **kwargs):
  """
  Calls the function *func* with the specified arguments and keyword
  arguments and snatches its local frame before it actually executes.
  """

  frame = None
  trace = sys.gettrace()
  def snatch_locals(_frame, name, arg):
    nonlocal frame
    if frame is None and name == 'call':
      frame = _frame
      sys.settrace(trace)
    return trace
  sys.settrace(snatch_locals)
  try:
    result = func(*args, **kwargs)
  finally:
    sys.settrace(trace)
  return frame, result

The idea is to use sys.trace() to catch the frame of the next 'call'. Tested on CPython 3.6.

Example usage

import types

def namespace_decorator(func):
  frame, result = call_function_get_frame(func)
  try:
    module = types.ModuleType(func.__name__)
    module.__dict__.update(frame.f_locals)
    return module
  finally:
    del frame

@namespace_decorator
def mynamespace():
  eggs = 'spam'
  class Bar:
    def hello(self):
      print("Hello, World!")

assert mynamespace.eggs == 'spam'
mynamespace.Bar().hello()

Answer 3

I don't see how you could do this non-intrusively -- after the function is done executing, it doesn't exist any more -- there's no way you can reach inside something that doesn't exist.

If you can control the functions that are being used, you can do an intrusive approach like

def fn(x, y, z, vars):
   ''' 
      vars is an empty dict that we use to pass things back to the caller
   '''
   x += 1
   y -= 1
   z *= 2
   vars.update(locals())

>>> updated = {}
>>> fn(1, 2, 3, updated)
>>> print updated
{'y': 1, 'x': 2, 'z': 6, 'vars': {...}}
>>> 

...or you can just require that those functions return locals() -- as @Thomas K asks above, what are you really trying to do here?

Answer 4

Witchcraft below read on your OWN danger(!)

I have no clue what you want to do with this, it's possible but it's an awful hack...

Anyways, I HAVE WARNED YOU(!), be lucky if such things don't work in your favorite language...

from inspect import getargspec, ismethod
import inspect


def main():

    @get_modified_values
    def foo(a, f, b):
        print a, f, b

        a = 10
        if a == 2:
            return a

        f = 'Hello World'
        b = 1223

    e = 1
    c = 2
    foo(e, 1000, b = c)


# intercept a function and retrieve the modifed values
def get_modified_values(target):
    def wrapper(*args, **kwargs):

        # get the applied args
        kargs = getcallargs(target, *args, **kwargs)

        # get the source code
        src = inspect.getsource(target)
        lines = src.split('\n')


        # oh noes string patching of the function
        unindent = len(lines[0]) - len(lines[0].lstrip())
        indent = lines[0][:len(lines[0]) - len(lines[0].lstrip())]

        lines[0] = ''
        lines[1] = indent + 'def _temp(_args, ' + lines[1].split('(')[1]
        setter = []
        for k in kargs.keys():
            setter.append('_args["%s"] = %s' % (k, k))

        i = 0
        while i < len(lines):
            indent = lines[i][:len(lines[i]) - len(lines[i].lstrip())]
            if lines[i].find('return ') != -1 or lines[i].find('return\n') != -1:
                for e in setter:
                    lines.insert(i, indent + e)

                i += len(setter)

            elif i == len(lines) - 2:
                for e in setter:
                    lines.insert(i + 1, indent + e)

                break

            i += 1

        for i in range(0, len(lines)):
            lines[i] = lines[i][unindent:]

        data = '\n'.join(lines) + "\n"

        # setup variables
        frame = inspect.currentframe()
        loc = inspect.getouterframes(frame)[1][0].f_locals
        glob = inspect.getouterframes(frame)[1][0].f_globals
        loc['_temp'] = None


        # compile patched function and call it
        func = compile(data, '<witchstuff>', 'exec')
        eval(func, glob, loc)
        loc['_temp'](kargs, *args, **kwargs)

        # there you go....
        print kargs
        # >> {'a': 10, 'b': 1223, 'f': 'Hello World'}

    return wrapper



# from python 2.7 inspect module
def getcallargs(func, *positional, **named):
    """Get the mapping of arguments to values.

    A dict is returned, with keys the function argument names (including the
    names of the * and ** arguments, if any), and values the respective bound
    values from 'positional' and 'named'."""
    args, varargs, varkw, defaults = getargspec(func)
    f_name = func.__name__
    arg2value = {}

    # The following closures are basically because of tuple parameter unpacking.
    assigned_tuple_params = []
    def assign(arg, value):
        if isinstance(arg, str):
            arg2value[arg] = value
        else:
            assigned_tuple_params.append(arg)
            value = iter(value)
            for i, subarg in enumerate(arg):
                try:
                    subvalue = next(value)
                except StopIteration:
                    raise ValueError('need more than %d %s to unpack' %
                                     (i, 'values' if i > 1 else 'value'))
                assign(subarg,subvalue)
            try:
                next(value)
            except StopIteration:
                pass
            else:
                raise ValueError('too many values to unpack')
    def is_assigned(arg):
        if isinstance(arg,str):
            return arg in arg2value
        return arg in assigned_tuple_params
    if ismethod(func) and func.im_self is not None:
        # implicit 'self' (or 'cls' for classmethods) argument
        positional = (func.im_self,) + positional
    num_pos = len(positional)
    num_total = num_pos + len(named)
    num_args = len(args)
    num_defaults = len(defaults) if defaults else 0
    for arg, value in zip(args, positional):
        assign(arg, value)
    if varargs:
        if num_pos > num_args:
            assign(varargs, positional[-(num_pos-num_args):])
        else:
            assign(varargs, ())
    elif 0 < num_args < num_pos:
        raise TypeError('%s() takes %s %d %s (%d given)' % (
            f_name, 'at most' if defaults else 'exactly', num_args,
            'arguments' if num_args > 1 else 'argument', num_total))
    elif num_args == 0 and num_total:
        raise TypeError('%s() takes no arguments (%d given)' %
                        (f_name, num_total))
    for arg in args:
        if isinstance(arg, str) and arg in named:
            if is_assigned(arg):
                raise TypeError("%s() got multiple values for keyword "
                                "argument '%s'" % (f_name, arg))
            else:
                assign(arg, named.pop(arg))
    if defaults:    # fill in any missing values with the defaults
        for arg, value in zip(args[-num_defaults:], defaults):
            if not is_assigned(arg):
                assign(arg, value)
    if varkw:
        assign(varkw, named)
    elif named:
        unexpected = next(iter(named))
        if isinstance(unexpected, unicode):
            unexpected = unexpected.encode(sys.getdefaultencoding(), 'replace')
        raise TypeError("%s() got an unexpected keyword argument '%s'" %
                        (f_name, unexpected))
    unassigned = num_args - len([arg for arg in args if is_assigned(arg)])
    if unassigned:
        num_required = num_args - num_defaults
        raise TypeError('%s() takes %s %d %s (%d given)' % (
            f_name, 'at least' if defaults else 'exactly', num_required,
            'arguments' if num_required > 1 else 'argument', num_total))
    return arg2value

main()

Output:

1 1000 2
{'a': 10, 'b': 1223, 'f': 'Hello World'}

There you go... I'm not responsible for any small children that get eaten by demons or something the like (or if it breaks on complicated functions).

PS: The inspect module is the pure EVIL.

Answer 5

Since you are trying to manipulate variables in one function, and do some job based on those variables on another function, the cleanest way to do it is having these variables to be an object's attributes.

It could be a dictionary - that could be defined inside the decorator - therefore access to it inside the decorated function would be as a "nonlocal" variable. That cleans up the default parameter tuple of this dictionary, that @bgporter proposed.:

def eggs(self, a, b, c=None):
   # nonlocal parms ## uncomment in Python 3
   parms["a"] = a
   ...

To be even more clean, you probably should have all these parameters as attributes of the instance (self) - so that no "magical" variable has to be used inside the decorated function.

As for doing it "magically" without having the parameters set as attributes of certain object explicitly, nor having the decorated function to return the parameters themselves (which is also an option) - that is, to have it to work transparently with any decorated function - I can't think of a way that does not involve manipulating the bytecode of the function itself. If you can think of a way to make the wrapped function raise an exception at return time, you could trap the exception and check the execution trace.

If it is so important to do it automatically that you consider altering the function bytecode an option, feel free to ask me further.