3

I have a conda environment with packages installed via conda install. I also have two local development packages that were each installed with pip install -e .. Now, conda env export shows everything, including both local development packages. However, I don't want conda to include them when creating the same environment on other machines - I want to keep doing it via pip install -e ..

So how can I exclude both local packages when creating the environment.yml file? Do I need to manually remove them or is there a way to this from the command line?

sinoroc
  • 18,409
  • 2
  • 39
  • 70
olamarre
  • 145
  • 3
  • 1
    Remove the offending lines from the YAML. Though, recognize that you can run `pip install -e` commands *through* the YAML (e.g., https://stackoverflow.com/a/59864744/570918) – merv Sep 04 '22 at 16:16
  • Oh I didn't know that! Either way requires manually editing the environment.yml file, but at least with the latter I won't need to run `pip install -e .`. Thanks! – olamarre Sep 04 '22 at 16:30

1 Answers1

2

While there are some alternative flags for conda env export that change output behavior (e.g., --from-history is most notable), there really isn't anything as specific as OP describes. Instead, manually remove the offending lines.

Note that YAMLs do support all pip install commands, so the editable installs can also be included. For example, https://stackoverflow.com/a/59864744/570918.

Consider Prioritizing the YAML Specification

In a software engineering setting, I would expect that users should not even be hitting development environments with conda install or pip install commands. Instead, the team should have a maunally-written, version-controlled YAML to begin with and all installations/changes to the environment are managed through editing the YAML file and using conda env update to propagate changes in the YAML to the environment.

That is, conda env export should not be necessary because the environment already has a well-defined means of creation.

merv
  • 67,214
  • 13
  • 180
  • 245
  • 1
    Thanks a lot for the detailed answer! I have a follow-up question: how do you find out which package version works (is compatible) with the rest of the environment? For example, let's say that I want to add numpy to an existing environment. Would you simply add `- numpy` to your yaml file (without version number), propagate the changes to your environment, see which numpy version conda ended up choosing (say, `numpy=1.19.2`) and then updating your entry in the yaml file with this version? – olamarre Sep 04 '22 at 17:47
  • @olamarre yeah, that's almost exactly what I do. I'll add a new package to the YAML first without any constraint, solve the whole environment (using `mamba`), inspect the solved version, and then specify [up-through minor version](https://stackoverflow.com/a/64594513/570918) (so `numpy=1.19` in your example). However, it depends on your purpose - I'm mostly developing Snakemake components these days where I want reproducibility. Someone developing a library for reuse might be more concerned with identifying a minimal version to support and use `>=` specifications. – merv Sep 06 '22 at 19:23