Write setup.py using git repo in install_requires to detect already installed

Question

I have a package with a git+ssh dependency, like this:

setup(
    name="setuprequires",
    version="0.1",
    packages=["setuprequires"],
    install_requires=[
        r"hello-world @ git+ssh://git@github.com/hello/hello-world.git#egg=hello-world"
    ],  
)

(I'm following the most up-to-date answer here regarding git repo dependencies.)

However, I'd like to skip installing hello-world if it's already installed in my current environment. Yet, if I run pip install -e . twice, the second time will still clone hello-world. In contrast, if I had just used install_requires=[r"hello-world"], it would've correctly detected that this requirement was already satisfied.

It seems to me that git+ssh doesn't play nicely with detecting already-satisfied requirements. I would think that having #egg=hello-world would've fixed this problem. Am I missing something?

Peter Badida · Accepted Answer · 2021-08-26T18:34:48.990

To detect a repository is up to date (or a certain commit is used) you need to clone it first, therefore to have a cache present even if it attempted to fetch it from cache you'd still first need to have it cloned.

After the first clone, it should compare it with the existing, cloned repository.

As you haven't specified any commit, I'm guessing it's still pulling because of that. See Git._should_fetch() class method.

Instead, try to specify the commit/tag you want to use, as mentioned in the docs, it should start caching:

git+https://git.example.com/MyProject.git@<hash>#egg=MyProject
git+https://git.example.com/MyProject.git@<tag>#egg=MyProject
git+ssh://git.example.com/MyProject.git@<hash>#egg=MyProject
git+ssh://git.example.com/MyProject.git@<tag>#egg=MyProject

Edit:

What I've noticed is that if you use a branch, the caching is broken. It's not refs/..., so it's not an explicit remote branch pulling, nor it's a hash. However for commit hashes it works flawlessly and stores the created wheel in the --cache-dir destination which is then picked on the 2nd+ execution of the pip install command. (bug created)

Example:

from setuptools import setup

setup(
    name="testing-git",
    install_requires=[
        # cached properly
        "requests @ git+ssh://git@github.com/psf/requests.git@cd4762d5a3b56d8933d1d9c1dff365fc5db4c768#egg=requests-3.0"
        # not cached
        # "requests @ git+ssh://git@github.com/psf/requests.git@3.0#egg=requests-3.0"
    ]
)

Write setup.py using git repo in install_requires to detect already installed

1 Answers1