0

As far as I know, I believe what conda do about python pkgs is not more than

  1. manage the python pkg installing path and which path to find python pkgs. ~/miniconda/envs/$(env-name)/lib//$(python-version)/site-packages/...
  2. Automatically install dependencies of python pkgs in these inter-env-seperate pathes

And I believe:

  1. If you use ~/miniconda/bin/pipor conda install to install python pkgs, all the dependencies will be install into ~/miniconda/envs/<env-name>/lib/<python-version>, and ~/miniconda/bin/python can find them
  2. If a python pkg pyfoo depends on a c++ libary libfoo already installed by apt install into /usr/lib/<python-version>, conda install pyfoo will install libfoo again into ~/miniconda/envs/<env-name>/lib/<python-version>
  3. If you use apt to install python pkgs, the install path will be /usr/lib/<python-version> and conda can't find them.

My question is:

  1. Am i right?
  2. Does apt install c++ libs conflicts with any of c++ libs installed into ~/miniconda/envs/<env-name>/lib/<python-version> and verse vice?
  3. Can i just use conda to manage python pkgs in different env and apt to manage c++ pkgs without worrying about any conflicts? Which means when i build and run a hybird(c++ and python) project, i just need activate the conda env and conda install all the python dependencies and apt install all desired c++ deps, everything will work fine?
zhixin
  • 113
  • 7

1 Answers1

0

What you say is correct: Conda will almost always bring its own dependencies. The exception to this is using a shell package, but I only know this to exist for mpich.

Otherwise, the answer to the main question, "Can I bring my own shared libs?", is practically no. This is because Conda packages precompile as much as possible and this often leads to specific symbol references to the exact builds of dynamic libraries. Even getting dynamic libraries from different channels can lead to errors of missing symbols.

However, you could just use Conda to manage environments and still use Pip to install libraries. That is, use Conda to create Python environments, then use Pip to install Python packages. Something like:

conda create -n my-env python=3.9 pip
conda activate my-env
pip install pkg1 pkg2 ...

Note, however, that this loses all the redundancy reduction that Conda does under the hood. It also means that many of your packages will still be compiled locally, whereas Conda precompiles everything, which is why installs are so fast. While more Pip packages are on wheels, those are essentially static builds, meaning even more unnecessary duplication of what should be shared libraries.

merv
  • 67,214
  • 13
  • 180
  • 245
  • Thank you very much! So with conda installed, 1)if I want to develop a pure python project in a seperate virtual env, I should just use the conda env and `conda install` or `pip install` to manage py libs and `conda` has a extra bonus of the redundancy reduction. Although installing py libs will lead to installing some shared c++ libs, conda will handle it and no worry of conflicts with the system-wide c++ libs system (installed by `apt`) 2) If to develop a c++ project or hybird one, do not use conda env, just deactivate the conda env and install shared c++ libs. – zhixin Dec 18 '21 at 02:44
  • By the way, what do you mean "specific symbol references to the exact builds of dynamic libraries"? is it for example the env variable $LD_LIBRARY_PATH? – zhixin Dec 18 '21 at 02:47