34

I'm trying to write the setup.py install file for a private project, which has both public and private dependencies. The public ones are hosted on PyPI, whereas the private ones are hosted on a server running simplepypi.

I would like both public and private dependencies to be resolved and fetched during installation.

I thus added my dependencies to setup.py:

setup(
    ...
    install_requires = [
        # public dependencies
        'argparse==1.2.1',
        'beautifulsoup4==4.1.3',
        'lxml==3.1.0',
        'mongoengine==0.8.2',
        'pymongo==2.5.2',
        'requests==1.1.0',
        'Cython==0.18',
        # private dependencies
        'myprivatepackage1',
        'myprivatepackage2'
    ],
    dependency_links=['http://pypi.myserver.com/packages'],
    ...
)

I build the package tarball using the command python setup.py sdist and install it in an activated virtualenv using pip install --verbose path/to/tarball.tar.gz.

However, the pip log lines do not mention my private PyPI server anywhere, and https://pypi.python.org/simple/ seems to have been queried twice.

Running setup.py egg_info for package from file:///home/b/code/mapado/mypackage/dist/mypackage-0.5.1.tar.gz
    running egg_info
    creating pip-egg-info/mypackage.egg-info
    writing requirements to pip-egg-info/mypackage.egg-info/requires.txt
    writing pip-egg-info/mypackage.egg-info/PKG-INFO
    writing top-level names to pip-egg-info/mypackage.egg-info/top_level.txt
    writing dependency_links to pip-egg-info/mypackage.egg-info/dependency_links.txt
    writing manifest file 'pip-egg-info/mypackage.egg-info/SOURCES.txt'
    warning: manifest_maker: standard file '-c' not found
    
    reading manifest file 'pip-egg-info/mypackage.egg-info/SOURCES.txt'
    reading manifest template 'MANIFEST.in'
    writing manifest file 'pip-egg-info/mypackage.egg-info/SOURCES.txt'
Downloading/unpacking myprivatepackage (from mypackage==0.5.1)
  Could not fetch URL https://pypi.python.org/simple/myprivatepackage/: HTTP Error 404: Not Found (myprivatepackage does not have any releases)
  Will skip URL https://pypi.python.org/simple/myprivatepackage/ when looking for download links for myprivatepackage (from mypackage==0.5.1)
  Could not fetch URL https://pypi.python.org/simple/myprivatepackage/: HTTP Error 404: Not Found (myprivatepackage does not have any releases)
  Will skip URL https://pypi.python.org/simple/myprivatepackage/ when looking for download links for myprivatepackage (from mypackage==0.5.1)
  Could not find any downloads that satisfy the requirement myprivatepackage (from mypackage==0.5.1)
Cleaning up...

What am I missing?

mkrieger1
  • 19,194
  • 5
  • 54
  • 65
Balthazar Rouberol
  • 6,822
  • 2
  • 35
  • 41

3 Answers3

22

it looks like you didnt specify your host like the doc of simplepy said you need to setup your ~/.pypirc with the good hostname like

To use it run "simplepypi". You can upload packages by:

[...]

Not using twine yet? Here is the legacy way of uploading Python packages (not recommended):

Modify your ~/.pypirc so it looks like:

[distutils]
index-servers =
    pypi
    local

[local]
username: <whatever>
password: <doesn't matter, see above>
repository: http://127.0.0.1:8000

[pypi]
...

then you'll upload your package on it

python setup.py sdist upload -r local

and could install it from there

pip install -i http://127.0.0.1:8000/pypi <your favorite package>

Hope this will help.

Neuron
  • 5,141
  • 5
  • 38
  • 59
  • 3
    I fixed it! My mistake was that, as you pointed out, I needed to add the ``"http://127.0.0.1:8000/pypi/"`` URL to ``dependency_links``. **Warning**: it needs to be ``"http://127.0.0.1:8000/pypi/"`` and not ``"http://127.0.0.1:8000/pypi"``, as the latter returns a 404. – Balthazar Rouberol Sep 16 '13 at 13:43
  • 3
    Can I do this _without_ modifying `~/.pypirc`? That is, just purely through `setup.py`? – Thomas Ruble Nov 18 '20 at 16:46
  • Downvoted because the question is about `setup.py`. Modifying `~/.pypirc` is not an option for me. – bfontaine Aug 04 '23 at 14:37
0

dependency_links is ignored by default (at least in pip 9.0.1)

In order for it to reach out to your sever you need to add --process-dependency-links

I believe pip 10 will bring a new mechanism, but for now this has got it working for me

I also had to update dependency_links to include the package name, for example:

dependency_links=[
    "http://internal-pyp:5678/simple/your_package_name"
]
r89m
  • 365
  • 3
  • 6
0

You could make your package as a normal pip package and publish it to the private repo. To install it, you can specify global option --extra-index-url in the config file:

$ cat ~/.pip/pip.conf
[global]
extra-index-url = https://...
yartem
  • 41
  • 3