0

Is there a pythonic way to access data files both in local testing of a module and in the distributed package (e.g. with wheel)?

Specifically, I have the same question as the comment linked below.

Python Access Data in Package Subdirectory

Californian
  • 808
  • 1
  • 8
  • 15
  • The [documentation](https://docs.python.org/3/distutils/setupscript.html#installing-additional-files) should give you the answer you need. – MCBama Dec 06 '17 at 20:54
  • Essentially setup your `setup.py` file to include a `data_files` list and python should package them up with the distributed package (such as when you turn it into a `wheel`. It'll work locally as well assuming the directory structure doesn't change any during the distribution. – MCBama Dec 06 '17 at 20:55
  • The problem is that `data_files` *does* change the directory structure -- it puts the files at the root under the `data` directory. For files that are in different folders and subfolders, this would require a manual re-creation of those folders and subfolders, which is tedious and error-prone (any changes to directory structure would have to be re-implemented there). The documentation linked is also for `distutils`, not for `setuptools` (sorry I didn't specify in the question, but I already have the file in the `wheel`, I just need access); does that just happen to have the same specification? – Californian Dec 06 '17 at 21:24
  • from the documentation: `Note that you can specify the directory names where the data files will be installed, but you cannot rename the data files themselves.` which means you should be able to move where the data files get deployed to. `data_files=[('path/to/data/files',['data_file1.dat','data_file2.dat']`. I think `setuptools` handles it the same but not 100% on that – MCBama Dec 06 '17 at 21:33
  • 1
    after double checking on how `setuptools` does their data files it looks to me like it's supposed to copy them in-place. So wherever they are in your local folder should be where they end up in your deployment. What's your `setup.py` look like? – MCBama Dec 06 '17 at 21:38
  • `setuptools` also comes with a [ResourceManager](http://setuptools.readthedocs.io/en/latest/pkg_resources.html#resourcemanager-api) that they suggest you use to access your data files. Having never made use of that myself the best I can do is point you at it. Perhaps someone has more experience in the matter. – MCBama Dec 06 '17 at 21:42
  • Thanks, they do in fact copy in-place. The output from `python setup.py bdist_wheel` is just extremely misleading -- it says `copying into /data`, but doesn't actually do that. Python really needs to fix its package creation system. – Californian Dec 07 '17 at 01:48

1 Answers1

0

Putting packages in setup.py in the argument setup(..., data_files=[('', YOUR_DATA_FILES)],...) (the '' is the significant part) copies them in-place, and they can be accessed as they would otherwise be, both in local testing and when distributed.

Californian
  • 808
  • 1
  • 8
  • 15