Python closures and replacing surrounding scope

Question

I know when using Groovy closures, I can change the delegate on the closure so function calls made within the closure could be defined externally.

Can I do something similar in Python?

Specifically, if you take the following code:

def configure():
  build()

def wrap(function):
  def build():
    print 'build'

  function()

wrap(configure)

I'd like it to print 'build' (only making changes to wrap()).

Some notes:

I don't want to pass functions into configure() since there may be a large number of functions that can be called by configure().

I also don't want to define those globally, because, once again, there may be a large number of functions that can be called by configure() and I don't want to pollute the global namespace.

score 5 · Accepted Answer · edited May 23 '17 at 10:34

5

Whether or not is a good way to do this is debatable, but here's a solution that doesn't modify the global namespace.

def configure():
  build()

def wrap(f):
  import new
  def build():
    print 'build'

  new.function(f.func_code, locals(), f.func_name, f.func_defaults, f.func_closure)()

wrap(configure)

I found it at How to modify the local namespace in python

edited May 23 '17 at 10:34

Community

1
1

answered Jan 11 '12 at 21:45

Reverend Gonzo

39,701
6
59
77

Indeed - this is the exact answer for the question asked. The `new.function`type is also avaliable as `types.FunctionType` - and what it does is make the local variables in `wrap` available as global variables in the newly created function. – jsbueno Jan 11 '12 at 22:30

millimoose · Answer 2 · 2012-01-11T20:45:31.113

3

This is doable without metaprogramming. Have configure take the build function as a parameter:

def default_build():
    print "default build"

def configure(build_func=None):
    build_func = build_func or default_build
    build_func()

def wrap(func):
    def build():
        print "wrap build"

    func(build)

wrap(configure)

This way makes it explicit that the behaviour of the configure function can change.

You can fiddle with the namespaces configure sees as well to do something more like what I understand Groovy does:

def build():
    print "default build"

def configure():
    build()

def wrap(func):
    def _build():
        print "wrap build"

    old_build = func.func_globals['build']
    func.func_globals['build'] = _build
    func()
    func.func_globals['build'] = old_build

edited Jan 11 '12 at 20:45

answered Jan 11 '12 at 20:40

millimoose

39,073
9
82
134

Sure, but let's say there's the potential of many functions that may be called. I'd rather not make changes to configure(). – Reverend Gonzo Jan 11 '12 at 20:41
1

@ReverendGonzo: I edited the answer to add a more direct method. I still think it will lead to confusing code though. If a function's behaviour can be parameterised, I'd just make that fact explicit even if it's laborious. – millimoose Jan 11 '12 at 20:47
@ReverendGonzo: Actually, what I'd probably do is apply OO and make `configure` and `build` methods of a class and change them by subclassing / monkeypatching instances of the class. – millimoose Jan 11 '12 at 20:51
so modifying func_globals is effectively the same as modifying the global namespace. I'm trying to go the class route because I'm trying to have a scriptable DSL that's as simple as possible. I think modifying the global namespace may be the only option, but still digging. – Reverend Gonzo Jan 11 '12 at 21:01
Maybe you can wrap that with contextmanager and use "with" syntax? But in any case I agree explicit is better than implicit. Bad design is hard to implement in Python ;) – Roman Susi Jan 11 '12 at 21:02
@ReverendGonzo Hm. I thought `func_globals` is a *copy* of the global namespace, apparently I was wrong. This would indeed make my second solution substantially similar to one in the other answers, with the exception that a) `wrap` and `configure` don't have to have the same global namespace - `wrap` should change whatever enclosing namespace is applicable for `configure`, and b) you can undo the pollution instead of clobbering the namespace. – millimoose Jan 11 '12 at 21:12

score 2 · Answer 3 · edited Jun 20 '20 at 09:12

If you're feeling crazy and awesome, have a look at this article about dynamic scoping.

Basically, the idea is to modify the bytecode of a function (using the byteplay module) and replace all references that are not strictly local-scope with ones that are. To illustrate the basic concept (in Python-pseudocode):

code = byteplay.extractcode(function)
newbytecode = []

for opcode, arg in code.code:
    if opcode in (NONLOCAL_CODES):
        opcode = LOCAL_EQUIVALENT

    newbytecode.append((opcode, arg))

code.code = newbytecode

return code.to_code()

It's slightly more complicated than that, but the article provides some great information.

He also recommends not to use it in production. :D

score 0 · Answer 4 · answered Jan 11 '12 at 20:41

0

One way to do it would be to declare build as global in wrap:

def configure():
  build()

def wrap(function):
  global build
  def build():
    print 'build'

  function()

wrap(configure)

However, I don't really recommend this since it will pollute the namespace.

answered Jan 11 '12 at 20:41

jcollado

39,419
8
102
133

Right, I don't want to pollute the namespace. I just want to modify the scope when configure is called. – Reverend Gonzo Jan 11 '12 at 20:42

score 0 · Answer 5 · answered Jan 11 '12 at 20:41

0

You need to use the global statement so that build() is defined in the global scope, try the following:

def wrap(function):
  global build
  def build():
    print 'build'

  function()

answered Jan 11 '12 at 20:41

Andrew Clark

202,379
35
273
306

I don't want build() to be in global scope, as that would defeat the purpose of wrapping configure()'s scope. – Reverend Gonzo Jan 11 '12 at 20:42
You could add `del build` after the `function()` line to remove `build()` from the global namespace. This is the only way I am aware of to get your code to work without modifying `configure()`. – Andrew Clark Jan 11 '12 at 20:46

Python closures and replacing surrounding scope

5 Answers5