17

My Python project uses various kind of libraries. What is the protocol if the end-user didn't have one or any of them?

Should a window pop up and notify him/her which package and version to download in his/her environment? Or I should include the libraries within my project?

What's the proper action?

Kepler 186
  • 369
  • 2
  • 11

5 Answers5

15

Whatever your project is, you could try making it into a python package that the end-user would install. The way this works is

In the root directory of your package you would include a setup.py. You could include in this file a list of requirements/dependencies (the install_requires key) that would be installed along with your package when the end-user installs it.

The end user could then use pip to install your package eg 

pip install YourPackage

and all dependencies listed in setup.py would be installed first.

Additionally, as @Devesh Kumar Singh pointed out in his comment, you could also include a requirements.txt file. The user could then install using this file with 

pip install -r requirements.txt YourPackage

See this guide for building a python package, setuptools documentation

Mooseman
  • 18,763
  • 14
  • 70
  • 93
RSHAP
  • 2,337
  • 3
  • 28
  • 39
  • Interesting. Also, I could just include os.system("pip install -r requirements.txt") in the setup.py right? Or the requirements.txt meant to be read by the user? – Kepler 186 May 02 '19 at 13:54
  • I believe the general practice would be to read in the file and list that as `install_requires`. EG something *like* `with open('requirements.txt', 'r') as f: deps=f.readlines()` and then in setup() do `install_requires=deps` – RSHAP May 02 '19 at 13:56
  • 1
    @RSHAP No, that would conflate the environment with the dependencies – Arne May 02 '19 at 13:59
  • what do you mean? – RSHAP May 02 '19 at 13:59
  • I mean, it's what pbr does, but it's a very bad practice. You want to keep those two separate – Arne May 02 '19 at 14:00
  • 2
    @RSHAP dstufft on the matter: https://caremad.io/posts/2013/07/setup-vs-requirement/ – Arne May 02 '19 at 14:00
  • Oh, so the setuptools.setup() would take care of the required packages automatically – Kepler 186 May 02 '19 at 14:01
  • 2
    yes, if you build your code into a package and have set up setup.py correctly by giving it all the dependencies in its `setup('requires'=[...])` field, they will be fetched automatically if you try to install your package – Arne May 02 '19 at 14:05
  • @Kepler186 *No.* That is an incorrect approach. You need to list them as the answer describes. You won't even be distributing `setup.py` if you actually create a package and distribute it that way (which most of your end users will want). Only people doing source installations would use `setup.py` at all, and you certainly don't want to be invoking that command every time someone imports your module. – jpmc26 May 02 '19 at 22:21
  • 1
    @Kepler186 And if the end user uses `pip3` instead? Or if they use `python -m pip` instead (because `pip` isn't in the environment variables)? There are many flaws with your os approach. – MilkyWay90 May 02 '19 at 23:49
  • @MilkyWay90 Well, I could create a file called **Libs** in the root directory and include the wanted pip version only. After that, I would include the **project/Libs/** as a place in which Python should searches for modules by writing`sys.path + [f'{os.getcwd()}/Libs/']`. Wouldn't that fix the pip versioning problem? – Kepler 186 May 03 '19 at 00:12
  • @Kepler186 Many versions of Python do not have an environment variable at all, and how would you make Python search for modules in `Project/Libs`? Even if you did, it's more trouble than the other solutions to install the dependencies. – MilkyWay90 May 03 '19 at 00:45
  • `setup.py` has dependency specification (direct deps only). `requirements.txt` has all the deps and *pinned versions*, it's more like a "deployment configuration". Do not attempt to read, parse, or use one directly from the other. – wim May 03 '19 at 03:31
5

To show other users, what libraries are needed for your project, you have multiple options. All options are some kind of files, that say which libraries are needed for this project.

Files that I am aware of

Martijn Pieters
  • 1,048,767
  • 296
  • 4,058
  • 3,343
Uli Sotschok
  • 1,206
  • 1
  • 9
  • 19
5

Another option: You can use PyInstaller to freeze (packages) Python applications into stand-alone executables, under Windows, GNU/Linux, Mac OS X, FreeBSD, Solaris and AIX.

PyInstaller Quickstart

This has worked very well for me. Indeed, you do not have to worry about whether the final user has Python installed.

user11293039
  • 51
  • 2
1

Here is where packaging a python project into a module comes handy modules We include a requirements.txt file which contains all python module requirements needed for that python library, and installs them automatically when the module is setup.

A good primer on how to setup your module to be distributable is Structuring your project

Devesh Kumar Singh
  • 20,259
  • 5
  • 21
  • 40
  • Just a `requirements.txt` is not enough right? I think you also need an `__init__.py`, `setup.py` and maybe more. https://docs.python-guide.org/writing/structure/ – 3UqU57GnaX May 02 '19 at 13:24
  • Yes, the question was about how to installed required libraries, which are listed in `requirements.txt`, and on top of it, we have these things you mentioned! – Devesh Kumar Singh May 02 '19 at 13:25
1

So you made a package. Now you will want to share it. What next?

Developer

Goal - make a distribution (also called a "package") to share

Preamble

You are now packaging your package and wish to distribute it. There are two main kinds of packages:

  • an application: deploy a source to a server, github, website e.g. CLI
  • a library: publish a source distribution (sdist) or binary (e.g. wheel) usually to PyPI via twine

Traditional Ways

Several files may be included in a distribution, but here are the main ones:

  • source: your code
  • setup.py: specify metadata, and dependencies required to make an sdist.
  • requirements.txt: a list of dependencies

Contemporary Ways

Use pyproject.toml to specify which tool to use in creating your sdist or binary:

Modern tools to create + deploy/publish a package include:

  1. pipenv: makes a package and substitutes requirements.txt (recommended by PyPA)
    • develop: > pipenv install <dependency>, > pipenv install
    • publish: > pipenv -e . + twine
  2. poetry: makes a package and publishes to PyPI
    • develop > poetry add <dependency>, > poetry install
    • publish: > poetry publish
  3. flit: makes a package and publishes to PyPI
    • develop: > flit install
    • publish: > flit publish

The first two options have features to make clean virtual environments and safely install dependencies using lock files. I would encourage exploring these newer options later on as they clear up a lot of packaging headaches by obviating setup.py and setuptools.

User

Goal - get a package and install it with dependencies

Traditional Ways

Applications have a variety of deployment methods, e.g. uploading your app to a hosting service e.g. heroku, DigitalOcean, etc. The user may indirectly interface with your app through a website, CLI or more.

Libraries are often uploaded to PyPI. From here the user can usually install a package using pip independent from how they are made:

  • > pip install <package> (recommended)
  • > pip install <packatge> -r requirements (explicit, optional)

These commandline invocations will fetch the distribution from PyPI, install the package and specified dependencies.

Contemporary Ways

Here are some alternatives to pip:

  1. pipx: "safely" install packages in isolated environments
    • > pipx install <package>

See Also

  • Official docs on publishing packages by PyPA
  • Tutorial on How To Package Your Python Code
  • Podcast interview with B. Cannon on pyproject.toml and modern packaging tools
Dodge
  • 3,219
  • 3
  • 19
  • 38
pylang
  • 40,867
  • 14
  • 129
  • 121