How to import Django models from script started with Popen() from views.py?

Question

I want to start a python script in a different process when a certain request arrives at my django server. I can successfully start it, but its imports fail.

In my django app, I have a views.py. In this I have a ViewSet with an update() method that looks like this (reduced snippet to relevant code):

from . import models # notice this import, which works

def update(self, request, pk=None):
    dirname = os.path.dirname(__file__)
    filename = os.path.join(dirname, 'Process.py')
    p = subprocess.Popen(["python", filename],
                         stdout=sys.stdout, 
                         stdin=subprocess.PIPE)

This works as expected: When performing the appropriate request that causes update() to be called, it executes Process.py, which is a file in the same directory as views.py.

In Process.py, I want to access the models of the django backend, so I do as I did before:

from . import models

# more code here

But I get the error:

ImportError: cannot import name 'models'

Why does this import fail in Process.py even though it works for views.py?

I tried

• a suggestion on the internet that the environment should be setup to be the same for the child process, like so:

  p = subprocess.Popen(["python", filename],
                       env = {'PYTHONPATH': os.pathsep.join(sys.path)}, # this is new
                       stdout=sys.stdout, 
                       stdin=subprocess.PIPE)
  

  But that failed to start python at all:

  Fatal Python error: failed to get random numbers to initialize Python
  

• to write the import statement in Process.py in a different way:

  from models import stuff
  

  Oddly enough, this does import models.py (yay!) but fails deeper down the rabbit hole with:

  ModuleNotFoundError: No module named '<django project name>'
  

  Said rabbit hole starts at the first model class in models.py and then dives into django code, ending up at some bootstrapper:

  Traceback (most recent call last):
  File "<path to django app>\Process.py", line 1, in <module>
    from models import stuff
  File "<path to django app>\models.py", line 3, in <module>
    class <some model class>(models.Model):
  File "<path to conda environment>\lib\site-packages\django\db\models\base.py", line 108, in __new__
    app_config = apps.get_containing_app_config(module)
  File "<path to conda environment>\lib\site-packages\django\apps\registry.py", line 253, in get_containing_app_config
    self.check_apps_ready()
  File "<path to conda environment>\lib\site-packages\django\apps\registry.py", line 135, in check_apps_ready
    settings.INSTALLED_APPS
  File "<path to conda environment>\lib\site-packages\django\conf\__init__.py", line 82, in __getattr__
    self._setup(name)
  File "<path to conda environment>\lib\site-packages\django\conf\__init__.py", line 69, in _setup
    self._wrapped = Settings(settings_module)
  File "<path to conda environment>\lib\site-packages\django\conf\__init__.py", line 170, in __init__
    mod = importlib.import_module(self.SETTINGS_MODULE)
  File "<path to conda environment>\lib\importlib\__init__.py", line 126, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
  

  It looks like django interferes with my imports somehow, but I'm not sure how and how to correct them.

How do I successfully import (and use) models.py of a django app from a python script that's started with subprocess.Popen(...) from views.py of django-rest-framework?

---

from the comments

Is Process.py a regular Python script, or is it written as a Django Management Command?

It's a regular script. I modified it for debugging purposes like so:

import sys
for p in sys.path:
    print(p)

from . import models

What happens if you run the script manually from the shell? Same error?

from shell	django views.py via Popen()
windows cmd: django-project>python manage.py shell then In [1]: from django-app import Process	see above	call detail
success	ImportError: cannot import name 'models'	result
includes django-project path and empty path	includes django-project/django-app	differences of sys.path

Answer 1

Process.py should be a Django Management Command, so that the Django Framework is loaded and model.py used by Django.

The PYTHONPATH is still required so that Django knows where the base directory is.

Instad of ["python", filename], the process can be started with ["python", "manage.py", filename].

Answer 2

Probably you are missing the reference to your Django app, to import all models from it, you have to do in the python script:

from <django-app-name>.models import *