0

For one python project, I want to ship a data file with the package. Following various (and partially contradictory) advice on the mess that is Python package data, I ended up trying different things and got it to work locally on my machine with the following setup.

My setup.cfg contains, among other things that shouldn't matter here,

[options]
include_package_data = True

and no package_data or other data related keys. My MANIFEST.in states

recursive-include lexedata clics3-network.gml.zip

My setup.py is pretty bare, essentially

from setuptools import setup
readline = "readline"
setup(extras_require={"formatguesser": [readline]})

To load the file, I use

pkg_resources.resource_stream("lexedata", "data/clics3-network.gml.zip")

I test this using tox, configured with

[tox]
isolated_build = True
envlist = general
[testenv]
passenv = CI
deps =
  codecov
  pytest
  pytest-cov
commands =
  pytest --doctest-modules --cov=lexedata {envsitepackagesdir}/lexedata
  pytest --cov=lexedata --cov-append test/
  codecov

On my local machine, when I run pip install ., the data file lexedata/data/clics2-network.gml.zip is properly deposited inside the site-packages/lexeadata/data directory of the corresponding virtual environment, and tox packages it inside .tox/dist/lexedata-1.0.0b3.tar.gz as well as in its venv site packages directory .tox/general/lib/python3.8/site-packages/lexedata/data/.

However, continuous integration using Github actions fails on all Python 3 versions I'm testing with

UNEXPECTED EXCEPTION: FileNotFoundError(2, 'No such file or directory')
FileNotFoundError: [Errno 2] No such file or directory: '/home/runner/work/lexedata/lexedata/.tox/general/lib/python3.10/site-packages/lexedata/data/clics3-network.gml.zip'

at the equivalent of that same tox venv path.

What could be going wrong here?

Arne
  • 17,706
  • 5
  • 83
  • 99
Anaphory
  • 6,045
  • 4
  • 37
  • 68
  • This was [my take on package data](https://sinoroc.gitlab.io/kb/python/package_data.html), if it helps. Not sure what is going on in your case. Anyway, I would recommend building the distributions (sdist and wheel) and look what is in there, if your data file is at the correct location. Between attempts, make sure to clean everything. – sinoroc Nov 24 '21 at 14:01

1 Answers1

1

You almost did it right, try slightly update your MANIFEST.in to any of the following examples:

  • include src/lexedata/data/*.zip
  • recursive-include src/* *.zip
  • recursive-include **/data clics3-network.gml.zip

As you can find in docs include command defines files as paths relative to the root of the project (that's why first example starts from src folder)

recursive-include expect first argument being as dir-pattern (glob-style), so it is better include asterisks

ujlbu4
  • 1,058
  • 8
  • 8