Is Pandas for Mac M1 Silicon available?

Question

I recently got the miniconda version installed which is released in Feb'22 for M1 Mac (Apple Silicon).

https://repo.anaconda.com/miniconda/Miniconda3-latest-MacOSX-arm64.sh

Then I created and env and did conda install pandas.

It shows me the following packages available.

Does the path here mean these are M1 Native packages?

The following NEW packages will be INSTALLED:

  blas               pkgs/main/osx-arm64::blas-1.0-openblas
  bottleneck         pkgs/main/osx-arm64::bottleneck-1.3.4-py39heec5a64_0
  libgfortran        pkgs/main/osx-arm64::libgfortran-5.0.0-11_1_0_h6a59814_26
  libgfortran5       pkgs/main/osx-arm64::libgfortran5-11.1.0-h6a59814_26
  libopenblas        pkgs/main/osx-arm64::libopenblas-0.3.18-hea475bc_0
  llvm-openmp        pkgs/main/osx-arm64::llvm-openmp-12.0.0-haf9daa7_1
  numexpr            pkgs/main/osx-arm64::numexpr-2.8.1-py39h144ceef_0
  numpy              pkgs/main/osx-arm64::numpy-1.21.5-py39h25ab29e_2
  numpy-base         pkgs/main/osx-arm64::numpy-base-1.21.5-py39h974a1f5_2
  packaging          pkgs/main/noarch::packaging-21.3-pyhd3eb1b0_0
  pandas             pkgs/main/osx-arm64::pandas-1.4.2-py39hc377ac9_0
  pyparsing          pkgs/main/noarch::pyparsing-3.0.4-pyhd3eb1b0_0
  python-dateutil    pkgs/main/noarch::python-dateutil-2.8.2-pyhd3eb1b0_0
  pytz               pkgs/main/noarch::pytz-2021.3-pyhd3eb1b0_0
  six                pkgs/main/noarch::six-1.16.0-pyhd3eb1b0_1

How can one confirm if a Python package is M1 Silicon optimized?

I've had best luck with the miniforge version of conda, it was the only one that installed the M1 version of Python automatically, not to mention packages. `brew install --cask miniforge` — BeRT2me, May 04 '22 at 04:10
You don't need miniforge now. Anaconda has released their own M1 optimized Miniconda. That should be well tested over mini forge — Baktaawar, May 04 '22 at 05:02

score 1 · Answer 1 · answered May 04 '22 at 05:59

Python code itself is compiled to platform-independent bytecode and run on a virtual machine. This is why purely Python packages are designated as noarch::foo. Some packages are wrappers around native code or depend on external native libraries and as such as OS- and architecture-dependent. They are designated as <os>-<architecture>::bar.

As evident from osx-arm64::, the listed packages are indeed native for M1. Whether those are optimised or not is an entirely different story. My understanding is that they are just as optimised as they are on x86. There is llvm-openmp among the dependencies, which means that some packages (likely OpenBLAS) are compiled with OpenMP enabled and can take full advantage of M1 as a CPU. Do not expect support for GPU computing or for the neural engine though.

If you still need tangible proof that a package is native, you need to find it in the environment's pkgs directory, then look for .so or .dyld files inside it, e.g., index.cpython-39-darwin.so, and finally use file to confirm the presence of ARM64 binary objects:

$ file index.cpython-39-darwin.so
index.cpython-39-darwin.so: Mach-O 64-bit bundle arm64

If it only shows x86_64 but not arm64 (fat binaries could have both), then the package is x86-native.

Is Pandas for Mac M1 Silicon available?

1 Answers1