Configure conda build to exclude some folders from the conda package output

Question

tl;dr:

How do you exclude folders from being packaged by conda build ?

Context

I am building my first packages with conda build.

My meta.yaml file looks like this:

package:
    name: 'some_name'
    version: {{ load_setup_py_data().get('version') }}

source:
    path: ./

build:
    script: python setup.py install --single-version-externally-managed --record=record.txt

requirements:
    run:
        - python >=3.6
        - pandas >=0.2
        - numpy >=1.12
        # Packages that must be installed
        # in the user's conda environment
        # to run this package.

    build:
        # Packages used by setup.py
        # to install this package.
        # May also install compilers
        # for non-python code.
        - python
        - setuptools

And my root directory (where lies the setup.py & meta.yaml files) looks like this:

$ ls
README.md   __pycache__      input       isi_classif meta.yaml   
notebooks   output      scripts     setup.py    version.py

Some folders are only there because they were useful during the prototyping but I don't want to delete them from the repo.

How do I exclude a folder (like input or notebooks here) and its content from the package that conda builds ?

For info, I build with the following command:

$ conda build some_folder_name

Answer 1

I came across this thread while trying to address a similar issue. Since my meta.yaml is in a subfolder of the project root and I wanted to exclude only files that were not part of the repo (e.g. .tox folder), I ended up using the repository itself to specify the files to be included as follows:

...  
source:
    git_url: ..
...

Note:

• the .tox folder wasn't actually being included in the final conda package - I just wanted to avoid having conda-build copy a large folder for no good reason.
• it may be possible to achieve what you want by having your additional items live only in a specific branch of the repo, and using a different branch without those items for your build - see the docs for details.

Answer 2

I think the best way would be to create a directory recipe/ and move the recipe related files there. Then change the meta.yaml to

source:
    path: ./..

So only the contents of the recipe directory would be copied over by conda into the package. Secondly, the folders notebooks and input would only be included if they are specified in the setup.py to be included. Otherwise they are ignored. So they are not installed as part of the setup.py install and would not be included in the package. So your source directory structure would look like:

some_folder_name
|--README.md
|--__pycache__
|--input
|--isi_classif
|--recipe
   |--meta.yaml   
|--notebooks
|--output
|--scripts
|--setup.py
|--version.py

then you could still build the package using conda build some_folder_name

Answer 3

I am looking for an answer and everything I found were already in the comment of the question. Since those solutions seems working (not as intended but still works), I will explain it here. I try the second solution on a test project and it works. I still not used to conda.

First: Explicite output

Explicit file lists:

Explicit file lists are relative paths from the root of the build prefix. Explicit file lists support glob expressions. Directory names are also supported, and they recursively include contents.

outputs:
  - name: subpackage-name
    files:
      - a-file
      - a-folder
      - *.some-extension
      - somefolder/*.some-extension

Scripts that move files into the build prefix:

Scripts that create or move files into the build prefix can be any kind of script. Known script types need only specify the script name. Currently the list of recognized extensions is py, bat, ps1 and sh.

outputs:
  - name: subpackage-name
    script: move-files.py

Specifying files to include in output

Second: Ignore prefix files

To specify individual filenames, use:

build:   ignore_prefix_files:
- file1

Ignore prefix files

I didn't try the Michael Sarahan answer but it should work if correctly done.

"You need to alter your build.sh/bld.bat to remove files from $PREFIX before the end of those scripts."