Python3 -m run configuration in Eclipse

Question

Update 2021: Solution is built into PyDev/Eclipse

See accepted answer for details

Original Question (and old answers) for below

As many comments/questions/rants on SO and other places will tell you, Python3 packages using relative imports want to be run from a central __main__.py file. In the case where a module, say "modA" within a package, say "packA", that uses relative imports needs to be run (for instance because a test package is run if __name__ == '__main__'), we are told to run instead run python3 -m modA.packA from the directory above modA if sys.path() does not contain the directory above modA. I may dislike this paradigm, but I can work around it.

When trying to run modA from Eclipse/PyDev, however, I can't figure out how to specify a run configuration that will execute the module properly with the -m flag. Has anyone figured out how to set up a run configuration that will do this properly?

References: Relative imports for the billionth time ; Relative import in Python 3 is not working ; Multilevel relative import

Answer 1

Nowadays (since PyDev 5.4.0 (2016-11-28)) you can go to the Settings > PyDev > Run and select Launch modules with python -m mod.name instead of python filename.py ;)

See: https://www.pydev.org/history_pydev.html

---

For older versions of PyDev (old answer)

Unfortunately, right now, it's not automatic running with -m in PyDev, so, I'll present 3 choices to work in PyDev with relative imports which are preceded by a dot (in PyDev version 4.3.0):

1. Don't use relative imports, only absolute imports in your __main__ modules.

2. Create a separate module for the __main__ which will do an absolute import for the module you want to run and run that module instead (if you're distributing your application, this is probably needed anyways as the usual way for people to launch your code in Python is by passing the script as an argument to Python and not using the -m switch).

3. Add the -m module to the vm arguments in your run configuration by doing:

• Make the run (which will fail because of the relative import)
• Right-click the editor > Copy Context Qualified Name
• Open the run configuration: Alt, R, N (i.e.: Toolbar > Run > Run Configuration)
• Open arguments tab and add the '-m Ctrl+V' (to add the -m and the module name you copied previously).

Although this is definitely not ideal: you'll now receive an argument with the filename (as PyDev will always pass that to run the file) and the whole process is a nuisance.

As a note, I do hope to provide a way to make runs within PyDev using the -m soon (hopefully for PyDev 4.4.0)... although this may not be possible if the file being run is not under the PYTHONPATH (i.e.: to run an external file it still has to support the option without the -m).

Answer 2

There's a bit nasty trick possible here to work around this issue. I'm using PyDev 9.2.0

Put your venv right in the workspace, say under the dir "venv".

Refresh your eclipse workspace and ensure that it uses this venv (through your interpreter setup).

After the refresh, go to the run configuration and edit the "Main Module" by clicking the Browse button. The venv will now appear.

Browse into the venv/lib/python3.8/site-packages

There you will find the pip-installed module source codes and you can select the module you want to run.

Answer 3

Update 2021: This answer is no longer needed. See accepted answer for details.

Here's what I was able to do after Fabio's great suggestion.

Create a program called /usr/local/bin/runPy3M with world read/execute permissions, with the following code:

#!/usr/local/bin/python3 -u
'''
Run submodules inside packages (with relative imports) given
a base path and a path (relative or absolute) to the submodule
inside the package.

Either specify the package root with -b, or setenv ECLIPSE_PROJECT_LOC.
'''

import argparse
import os
import re
import subprocess
import sys

def baseAndFileToModule(basePath, pyFile):
    '''
    Takes a base path referring to the root of a package
    and a (relative or absolute) path to a python submodule
    and returns a string of a module name to be called with
    python -m MODULE, if the current working directory is
    changed to basePath.

    Here the CWD is '/Users/cuthbert/git/t/server/tornadoHandlers/'.

    >>> baseAndFileToModule('/Users/cuthbert/git/t/', 'bankHandler.py')
    'server.tornadoHandlers.bankHandler'
    '''
    absPyFilePath = os.path.abspath(pyFile)
    absBasePath = None
    if basePath is not None:
        absBasePath = os.path.abspath(basePath)
        commonPrefix = os.path.commonprefix([absBasePath, absPyFilePath])
        uncommonPyFile = absPyFilePath[len(commonPrefix):]
    else:
        commonPrefix = ""
        uncommonPyFile = absPyFilePath 

    if commonPrefix not in sys.path:
        sys.path.append(commonPrefix)

    moduleize = uncommonPyFile.replace(os.path.sep, ".")
    moduleize = re.sub("\.py.?$", "", moduleize)
    moduleize = re.sub("^\.", "", moduleize)
    return moduleize    

def exitIfPyDevSetup():
    '''
    If PyDev is trying to see if this program is a valid
    Python Interpreter, it expects to function just like Python.
    This is a little module that does this if the last argument
    is 'interpreterInfo.py' and then exits.
    '''
    if 'interpreterInfo.py' in sys.argv[-1]:
        interarg = " ".join([sys.executable] + sys.argv[1:])
        subprocess.call(interarg.split())
        exit()
    return

exitIfPyDevSetup()

parser = argparse.ArgumentParser(description='Run a python file or files as a module.')
parser.add_argument('file', metavar='pyfile', type=str, nargs=1,
                   help='filename to run, with .py')
parser.add_argument('-b', '--basepath', type=str, default=None, metavar='path',
                   help='path to directory to consider the root above the package')
parser.add_argument('-u', action='store_true', help='unbuffered binary stdout and stderr (assumed)')
args = parser.parse_args()
pyFile = args.file[0]
basePath = args.basepath

if basePath is None and 'ECLIPSE_PROJECT_LOC' in os.environ:
    basePath = os.environ['ECLIPSE_PROJECT_LOC']

modName = baseAndFileToModule(basePath, pyFile)

allPath = ""
if basePath:
   allPath += "cd '" + basePath + "'; "

allPath += sys.executable
allPath += " -m " + modName 

subprocess.call(allPath, shell=True)  # maybe possible with runpy instead...

Then add a new interpreter to PyDev -- call it what you'd like (e.g., runPy3M) and point the interpreter to /usr/local/bin/runPy3M. Click okay. Then move it up the list if need be. Then add to environment for the interpreter, Variable: ECLIPSE_PROJECT_LOC (name here DOES matter) with value ${project_loc}.

Now submodules inside packages that choose this interpreter will run as modules relative to the sub package.

I'd like to see baseAndFileToModule(basePath, pyFile) added to runpy eventually as another run option, but this will work for now.

EDIT: Unfortunately, after setting all this up, it appears that this configuration seems to prevent Eclipse/PyDev from recognizing builtins such as "None", "True", "False", "isinstnance," etc. So not a perfect solution.