Using environment variables in `pyproject.toml` for versioning

Question

I am trying to migrate my package from setup.py to pyproject.toml and I am not sure how to do the dynamic versioning in the same way as before. Currently I can pass the development version using environment variables when the build is for development.

The setup.py file looks similar to this:

import os

from setuptools import setup

import my_package


if __name__ == "__main__":
    dev_version = os.environ.get("DEV_VERSION")
    version = dev_version if dev_version else f"{my_package.__version__}"
    
    setup(
        name="my_package",
        version=version,
        ...
    )

Is there a way to do similar thing when using pyproject.toml file?

Answer 1

At the time that I write this answer, the most genuinely valid approach on how to do that is to go ahead and write your pyproject.toml as you normally would, but instead of having a version = "..." field you add a dynamic = ["version"].

Then you can keep a very minimal setup.py file which only resolves the dynamic value of version. For example:

# pyproject.toml
[project]
name = "hello-world"
readme = "README.md"
authors = [...]
dependencies = [...]
dynamic = ["version"]

[build-system]
requires = ["setuptools"]
build-backend = "setuptools.build_meta:__legacy__"
# The difference between `__legacy__` and the regular `build_meta`
# is that `__legacy__` does the equivalent of
# `sys.path.insert(0, os.path.dirname(__file__))`.
# This allows you to `import` your modules from the `CWD`.
# If you don't like using `__legacy__` you can
# manually add `CWD` to `sys.path` inside your `setup.py`.

# setup.py
import os
from setuptools import setup

import my_package

dev_version = os.environ.get("DEV_VERSION")

setup(
  version=dev_version if dev_version else f"{my_package.__version__}"
)

These are my remarks about this approach:

• Setuptools allows you to keep all your static metadata/configuration parameters in pyproject.toml and at the same time use logic to compute all the dynamic parts in setup.py. Setuptools docs explicitly say:

  Metadata and configuration supplied via setup() is complementary to ... the information present in ... pyproject.toml.

  It is important however to remember to write dynamic = [...] in accordance with the spec to allow setuptools to overwrite any value that can be specified via the [project] table.

• setup.py is only deprecated as a "runnable CLI tool" that can be invoked directly with python setup.py .... But it is still a perfectly valid configuration file in the same way that conftest.py is a valid configuration file for pytest and noxfile.py is a configuration file for nox (you usually don't call python conftest.py or python noxfile.py...). One evidence for this is article https://blog.ganssle.io/articles/2021/10/setup-py-deprecated.html written by one of the authors of PEP 621 (which stablishes the [project] table in pyproject.toml) and maintainer of setuptools.

• Note that I used in the example build-backend = "setuptools.build_meta:__legacy__" instead of build-backend = "setuptools.build_meta". This happens because __legacy__ tries to emulate the "side-effect" behaviour of running python setup.py ..., i.e. to add CWD to sys.path. If you don't like that, you can manipulate sys.path explicitly in your setup.py file before the relevant imports.

Answer 2

I wanted something similar.

Unfortunately, toml does not support environment variables and I don't think it wants to, ever.

I've experimented a bit with custom build backend wrappers, and sifting through the layers of setuptools (and setuputils3).

My conclusion is that there is no simpler solution than to write a short build script to dynamically edit the pyproject.toml file...

This kind of thing:

#!/bin/bash

#fix version - as toml doesn't support environment vars...
sed -i -e "s/.*version.*/version=\"${DEV_VERSION}\"/g" pyproject.toml

#build python source and binary packages
python3 -m build

For your other case (non-development) you can "dynamically" pull in the version as described by someone else in this answer: https://stackoverflow.com/a/74514742/4126317