Not able to install packages that rely on Tensorflow on Mac M1

Question

I successfully installed Tensorflow 2.7.0 on my MacBook with an M1 chip following this guide by Apple: https://developer.apple.com/metal/tensorflow-plugin/

I now want to install a package (ethnicolr) in a project that relies on Tensorflow >=1.15.2. This should not be an issue, but it sadly is.

requirements.txt of my project

pandas==1.3.4
ethnicolr==0.4.0

requirements.txt of ethnicolr:

tensorflow>=1.15.2

Running pip install -r requirements.txt yields

ERROR: Could not find a version that satisfies the requirement tensorflow>=1.15.2 (from ethnicolr) (from versions: none) ERROR: No matching distribution found for tensorflow>=1.15.2

Running pip list shows, that Tensorflow was installed. But it's not called tensorflow, but tensorflow-macos or tensorflow-metal.

tensorboard             2.7.0
tensorboard-data-server 0.6.1
tensorboard-plugin-wit  1.8.0
tensorflow-estimator    2.7.0
tensorflow-macos        2.7.0
tensorflow-metal        0.3.0

What is a solution here? There must be more packages out there with the requirement of Tensorflow...

I do not know if tensorflow officially support M1... Have you tried with a guide like [that](https://mobiarch.wordpress.com/2021/09/24/installing-tensorflow-in-macos-m1-chip/) ? — Nicola Landro, Dec 25 '21 at 16:37
Yes, I have tried multiple ways. And I am able to install Tensorflow. The thing is that the dependency is called "tensorflow-macos", but ethicolr depends on "tensorflow" and this I think throws an error. I'm not sure what to do here. — Ole Spaarmann, Dec 25 '21 at 17:04
It returns `/Users/olespaarmann/mambaforge/envs/diversity_scraper/bin/python`. So to clarify: I am able to install TensorFlow, but not the M1 optimized version. Or, to be even more precise: I am also able to install the M1 optimized version, but other packages don't recognize that I have TensorFlow installed. My guess is because the package is called `tensorflow-macos`. — Ole Spaarmann, Dec 25 '21 at 22:01

score 0 · Answer 1 · answered Dec 26 '21 at 13:36

So I got it to run. I'm not sure if this is ideal but I'm sharing my solution to maybe help anyone running into the same issue. I installed tensorflow 2.6.0 in a virtual environment using conda / mambaforge. I opted for 2.6.0 because 2.5.2 is not available for M1 and 2.5.0 wasn't working. You can read about installing Mamaforge here. After that, I installed Tensorflow with

conda install tensorflow==2.6.0

I also set the pandas version to pandas>1.2.3 in the requirements.txt of my own project (same as in ethnicolr). This resolved to pandas 1.3.3.

Next, I had to solve the dependency issue with ethnicolr, since ethnicolr requires tensorflow 2.5.2. I did that by forking the ethnicolr repo and creating a branch where I pin the tensorflow version to 2.6.0 in the requirements.txt and setup.py. You find this branch over here. To use this github branch, I changed the line in my requirements.txt to:

git+https://github.com/ospaarmann/ethnicolr.git@apple_m1_support_tensorflow_2_6_0#egg=ethnicolr

Now I had an issue with a dependency mismatch with numpy. It is described in this StackOverflow thread. What happened was that importing pandas or ethnicolr would throw this error:

>>> from ethnicolr import census_ln, pred_census_ln, pred_wiki_name
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/Users/olespaarmann/mambaforge/envs/diversity_scraper/lib/python3.8/site-packages/ethnicolr/__init__.py", line 2, in <module>
    from ethnicolr.census_ln import census_ln
  File "/Users/olespaarmann/mambaforge/envs/diversity_scraper/lib/python3.8/site-packages/ethnicolr/census_ln.py", line 6, in <module>
    import pandas as pd
  File "/Users/olespaarmann/mambaforge/envs/diversity_scraper/lib/python3.8/site-packages/pandas/__init__.py", line 22, in <module>
    from pandas.compat import (
  File "/Users/olespaarmann/mambaforge/envs/diversity_scraper/lib/python3.8/site-packages/pandas/compat/__init__.py", line 15, in <module>
    from pandas.compat.numpy import (
  File "/Users/olespaarmann/mambaforge/envs/diversity_scraper/lib/python3.8/site-packages/pandas/compat/numpy/__init__.py", line 7, in <module>
    from pandas.util.version import Version
  File "/Users/olespaarmann/mambaforge/envs/diversity_scraper/lib/python3.8/site-packages/pandas/util/__init__.py", line 1, in <module>
    from pandas.util._decorators import (  # noqa
  File "/Users/olespaarmann/mambaforge/envs/diversity_scraper/lib/python3.8/site-packages/pandas/util/_decorators.py", line 14, in <module>
    from pandas._libs.properties import cache_readonly  # noqa
  File "/Users/olespaarmann/mambaforge/envs/diversity_scraper/lib/python3.8/site-packages/pandas/_libs/__init__.py", line 13, in <module>
    from pandas._libs.interval import Interval
  File "pandas/_libs/interval.pyx", line 1, in init pandas._libs.interval
ValueError: numpy.ndarray size changed, may indicate binary incompatibility. Expected 88 from C header, got 80 from PyObject

The solution here is to just ignore the dependency issues and manually install a newer version of numpy. It doesn't work when I set the numpy version in my requirements.txt because this throws a dependency error:

ERROR: Cannot install numpy>=1.20.0 and tensorflow==2.6.0 because these package versions have conflicting dependencies.

The conflict is caused by:
    The user requested numpy>=1.20.0
    tensorflow 2.6.0 depends on numpy~=1.19.2

So I just installed it with python -m pip install numpy==1.20.0. And now everything seems to work.

Not able to install packages that rely on Tensorflow on Mac M1

1 Answers1