Installing a full directory of data files

Question

I'm trying to build a package for my project, using setup.py and setuptools. A general search has suggested that data files should be placed under my python package.
For instance, project/src/package/data.

But my situation is more in line with this question.

My project consists on a Python console application and a HTML interactive application. Right now (before setup.py), I have the console application copy the HTML files to a specific directory with a special command. I'm following this directory structure in source control.

project
|-+ console
|   |-+ package
|       |-- __init__.py, etc.
|-+ viz
    |-+ css
    |-+ js
    |-- index.html

I'm not sure how this behaviour can be replicated with setup.py and egg files.

1. How can I make sure that viz will be packaged? The package_data option seems to be relative to a package (implying that viz should be placed under a package).

   So, package_data = {'': ['viz/*']} is not what I am looking for.

2. How is the extraction done?
   Should I use pkg_resources.resource_filename to get viz into the cache, and then shutil to copy the files to the intended location? Is there a more practical alternative?

Answer 1

I considered changing the whole structure and turning viz into a package, moving the code that installs it there as well. The result would be like so.

project
|-- setup.py
|-+ top-level-package
    |-- __init__.py
    |-+ console
    |   |-- __init__.py, etc.
    |-+ viz
        |-+ css
        |-+ js
        |-- index.html
        |-- __init__.py

While this would have worked, I found an answer that actually works for my original purpose. setup.py has to recursively copy the files as package_data (and the relative paths work with ..).

def package_files(directory):
    paths = []
    for (path, directories, filenames) in os.walk(directory):
        for filename in filenames:
            paths.append(os.path.join("..", path, filename))
    return paths

extra_files = package_files("viz") # this is relative to the project root

setup(
    ...,
    package_data    = {"mypackage": extra_files}
)

I just collapsed the original console and package into one, so the Python package sits directly under the project root. This is my final structure.

project
|-- setup.py
|-+ package
|   |-- __init__.py, etc.
|   |-+ private
|       |-- ...
|-+ viz
    |-+ css
    |-+ js
    |-- index.html