Installing dependencies of a local dependency with pipenv

Question

Background

We have project with the following high-level directory structure*

./datascience/
├── core
│   └── setup.py
├── notebooks
│   └── Pipfile
└── web
    └── Pipfile

^{*Excluded all the irrelevant files and directories for brevity.}

The core package is a library. It's a dependency of both the notebooks and web applications.

The core package, being a library, has its dependencies specified in setup.py

import setuptools

setuptools.setup(
    install_requires=[
        'some-dependency',
        'another-dependency'
    ]
)

The web and notebooks applications are using pipenv for dependency management. Their dependencies are specified in a Pipfile.

For example, here's how the web dependencies are specified in web/Pipfile:

[packages]
datascience-core = {path = "./../core"}
flask = "~= 1.0"

Notice how the core dependency is a local dependency, hence the relative path.

Problem

Doing a pipenv install from inside the the web or notebooks directory, does not install the dependencies of the core library as I expected!

I also tried using a Pipfile for core, hoping that pipenv would pick it up in its graph and download all the nested dependencies. But it doesn't.

How can dependencies of the core app be installed automatically when pipenv is installing dependencies for the web or notebooks app?

Answer 1

Found a solution thanks to this comment in a pipenv issue thread: https://github.com/pypa/pipenv/issues/209#issuecomment-337409290

I've continued listing the core's dependencies in setup.py.

I've changed the web and notebook apps to use an editable installation of the core package.
This was done by running the following in both the web and notebooks directory:

pipenv install --editable ../core

It produced this diff

[packages]
- datascience-core = {path = "./../core"}
+ datascience-core = {editable = true,path = "./../core"}

Now running pipenv install from the web and notebooks directory results in the installation of the core package and its dependencies!

It also solved another very annoying problem, which was having to pipenv install every time there was a change in core. Now it picks up development changes without having to re-install the local package!