Is it possible to call exec so that it's compatible with both Python 3 and Python 2?

Question

I'm using the exec statement in some Python 2 code, and I'm trying to make that code compatible with both Python 2 and Python 3, but in Python 3, exec has changed from a statement into a function. Is it possible to write code that is compatible with both Python 2 and 3? I've read about Python 2 and Python 3 dual development, but I'm interested in specific solutions to the exec statement/function changes.

I realize that exec is generally discouraged, but I'm building an Eclipse plugin that implements live coding on top of PyDev. See the project page for more details.

Answer 1

Some Python porting guides get the exec wrong:

If you need to pass in the global or local dictionaries you will need to define a custom function with two different implementations, one for Python 2 and one for Python 3. As usual six includes an excellent implementation of this called exec_().

No such custom function is needed to port Python 2 code into Python 3 (*). You can do exec(code), exec(code, globs) and exec(code, globs, locs) in Python 2, and it works.

Python has always accepted Python 3 compatible "syntax" for exec for as long that exec existed. The reason for this is that Python 2 and Python 1 (?!) have a hack to stay backwards-compatible with Python 0.9.8 in which exec was a function. Now, if exec is passed a 2-tuple, it is interpreted as (code, globals) and in case of a 3-tuple, it is interpreted as (code, globals, locals). Yes, the exec_ in six is unnecessarily complicated.

Thus,

exec(source, global_vars, local_vars)

is guaranteed to work the same way in CPython 0.9.9, 1.x, 2.x, 3.x; and I have also verified that it works in Jython 2.5.2, PyPy 2.3.1 (Python 2.7.6) and IronPython 2.6.1:

Jython 2.5.2 (Release_2_5_2:7206, Mar 2 2011, 23:12:06) 
[Java HotSpot(TM) 64-Bit Server VM (Oracle Corporation)] on java1.8.0_25
Type "help", "copyright", "credits" or "license" for more information.
>>> exec('print a', globals(), {'a':42})
42

---

*) There are subtle differences so that not all Python 3 code works in Python 2, namely

• foo = exec is valid in Python 3 but not in Python 2, and so is map(exec, ['print(a + a)', 'print(b + b)']), but I really don't know any reason why anyone would want to use these constructs in real code.

• As found out by Paul Hounshell, in Python 2, the following code will raise SyntaxError: unqualified exec is not allowed in function 'print_arg' because it contains a nested function with free variables:

  def print_arg(arg):
      def do_print():
          print(arg)
      exec('do_print()')
  

  The following construct works without exception.

  def print_arg(arg):
      def do_print():
          print(arg)
      exec 'do_print()' in {}
  

  Before Python 2.7.9, if one used exec('do_print()', {}) for the latter instead, the same SyntaxError would have been thrown; but since Python 2.7.9 the parser/compiler would allow this alternate tuple syntax too.

Again, the solution in edge cases might be to forgo the use of exec and use eval instead (eval can be used to execute bytecode that is compiled with compile in exec mode):

def print_arg(arg):
    def do_print():
        print(arg)

    eval(compile('do_print(); print("it really works")', '<string>', 'exec'))

---

I have written a more detailed answer on internals of exec, eval and compile on What's the difference between eval, exec, and compile in Python?

Answer 2

I found several options for doing this, before Antti posted his answer that Python 2 supports the Python 3 exec function syntax.

The first expression may also be a tuple of length 2 or 3. In this case, the optional parts must be omitted. The form exec(expr, globals) is equivalent to exec expr in globals, while the form exec(expr, globals, locals) is equivalent to exec expr in globals, locals. The tuple form of exec provides compatibility with Python 3, where exec is a function rather than a statement.

If you don't want to use that for some reason, here are all the other options I found.

Import Stubs

You can declare two different import stubs and import whichever one works with the current interpreter. This is based on what I saw in the PyDev source code.

Here's what you put in the main module:

try:
    from exec_python2 import exec_code #@UnusedImport
except:
    from exec_python3 import exec_code #@Reimport

Here's what you put in exec_python2.py:

def exec_code(source, global_vars, local_vars):
    exec source in global_vars, local_vars

Here's what you put in exec_python3.py:

def exec_code(source, global_vars, local_vars):
    exec(source, global_vars, local_vars)

Exec in Eval

Ned Batchelder posted a technique that wraps the exec statement in a call to eval so it won't cause a syntax error in Python 3. It's clever, but not clear.

# Exec is a statement in Py2, a function in Py3

if sys.hexversion > 0x03000000:
    def exec_function(source, filename, global_map):
        """A wrapper around exec()."""
        exec(compile(source, filename, "exec"), global_map)
else:
    # OK, this is pretty gross.  In Py2, exec was a statement, but that will
    # be a syntax error if we try to put it in a Py3 file, even if it isn't
    # executed.  So hide it inside an evaluated string literal instead.
    eval(compile("""\
def exec_function(source, filename, global_map):
    exec compile(source, filename, "exec") in global_map
""",
    "<exec_function>", "exec"
    ))

Six package

The six package is a compatibility library for writing code that will run under both Python 2 and Python 3. It has an exec_() function that translates to both versions. I haven't tried it.

Answer 3

I needed to do this, I couldn't use six, and my version of Python doesn't support @Antti's method because I used it in a nested function with free variables. I didn't want unnecessary imports either. Here's what I came up with. This probably needs to be in the module, not in a method:

try:
  # Try Python2.
  _exec_impls = {
    0: compile('exec code', '<python2>', 'exec'),
    1: compile('exec code in _vars[0]', '<python2>', 'exec'),
    2: compile('exec code in _vars[0], _vars[1]', '<python2>', 'exec'),
  }

  def _exec(code, *vars):
    impl = _exec_impls.get(len(vars))
    if not impl:
      raise TypeError('_exec expected at most 3 arguments, got %s' % (len(vars) + 1))
    return eval(impl, { 'code': code, '_vars': vars })

except Exception as e:
  # Wrap Python 3.
  _exec = eval('exec')

Afterwards, _exec works like the Python3 version. You can either hand it a string, or run it through compile(). It won't get the globals or locals you probably want, so pass them in:

def print_arg(arg):
  def do_print():
    print(arg)
  _exec('do_print(); do_print(); do_print()', globals(), locals())

print_arg(7)  # Prints '7'

Or not. I'm a StackOverflow post, not a cop.

Updates:

Why don't you just use eval()? eval() expects an expression, while exec() expects statements. If you've just got an expression it really doesn't matter what you use because all valid expressions are valid statements, but the converse is not true. Just executing a method is an expression, even if it doesn't return anything; there's an implied None returned.

This is demonstrated by trying to eval pass, which is a statement:

>>> exec('pass')
>>> eval('pass')
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "<string>", line 1
    pass
       ^
SyntaxError: unexpected EOF while parsing