Missing numpy/arrayobject.h while compiling .pyx file

Question

Installed canopy from enthought. While building my .pyx file, I get this error (followed by more)

Do I need to easy_install additional packages to get the "development" version so I get the .h files?

gcc -fno-strict-aliasing -fno-common -dynamic -arch x86_64 -DNDEBUG -g -O3 -arch x86_64 -I/Applications/Canopy.app/appdata/canopy-1.1.0.1371.macosx-x86_64/Canopy.app/Contents/include/python2.7 -c tsBinner.c -o build/temp.macosx-10.6-x86_64-2.7/tsBinner.o
tsBinner.c:314:31: error: numpy/arrayobject.h: No such file or directory
tsBinner.c:315:31: error: numpy/ufuncobject.h: No such file or directory

More Context

This compiles and runs under several Linux installations, but does not work with my recently-installed Canopy distribution python

here is the content of setup.py

from distutils.core import setup
from distutils.extension import Extension
from Cython.Distutils import build_ext

ext_modules = [Extension("tsBinner",["tsBinner.pyx"])]

setup(
  name ='time stamp binner',
  cmdclass = {'build_ext': build_ext},
  ext_modules = ext_modules 
)

Here is the content of tsBinner.py

from __future__ import division
import numpy as np
cimport numpy as np

#cimport cython
#@cython.boundscheck(False)
def tsBinner(np.ndarray[np.uint64_t, ndim=1] tstamps, \
    np.ndarray[np.int_t, ndim=1] bins):
    """
    bin the values of the tstamps array into the bins array.

    tstamps array is of type np.uint64

    bins array is of type int
    """
    cdef int nTStamps = tstamps.shape[0]
    cdef int iTStamp
    for iTStamp in range(nTStamps):
        bins[tstamps[iTStamp]] += 1
    return

Here are the versions of python and gcc

mac-119562:cosmic stoughto$ which python
/Users/stoughto/Library/Enthought/Canopy_64bit/User/bin/python
mac-119562:cosmic stoughto$ which gcc
/usr/bin/gcc
mac-119562:cosmic stoughto$ gcc --version
i686-apple-darwin11-llvm-gcc-4.2 (GCC) 4.2.1 (Based on Apple Inc. build 5658) (LLVM build     2336.11.00)
Copyright (C) 2007 Free Software Foundation, Inc.
This is free software; see the source for copying conditions.  There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.

mac-119562:cosmic stoughto$ python --version
Python 2.7.3 --  64-bit

Running on a MacBook Pro Mac OS X Version 10.7.5

I had the same problem, you have to include also `-I.../site-packages/numpy/core/include` in your gcc command... — Saullo G. P. Castro, Sep 27 '13 at 19:46
You can try to import from `numpy.distutils.core` instead of `distutils.core`, not sure if it will make any difference. — Jaime, Sep 27 '13 at 19:48
Saullo, thanks, that helps. I did it by setting the environment variable CPATH to /Users/.../site-packages/numpy/core before running the command "python setup.py buils_ext --inplace" command. Is there a way to include logic (in setup.py?) to make this more portable? — user2824549, Sep 27 '13 at 20:37
Possible duplicate: http://stackoverflow.com/questions/14657375/cython-fatal-error-numpy-arrayobject-h-no-such-file-or-directory Did you try using the include_dirs argument as suggested there? — IanH, Sep 28 '13 at 03:44
@user2824549 the portable solution I use here is to create a shell script that includes all the `-I...` when calling `gcc` or `cython`. [Here is an example of such script for Windows...](https://gist.github.com/saullocastro/6741739) — Saullo G. P. Castro, Sep 28 '13 at 12:46
@user2824549 I think you can set the environment variable `INCLUDE` and `LIB` to tell the compiler where to look for headers and libraries, respectively... — Saullo G. P. Castro, Sep 28 '13 at 12:47

score 5 · Answer 1 · edited May 23 '17 at 12:09

This is a fairly old question, but it is a frequent problem and the other answers are very bad. Hard-wiring numpy's include directory completely breaks virtual environments and would make the package very hard to distribute and install automatically. The correct way to solve this is using numpy.get_include() to obtain the directory associated with the currently loaded numpy version. This is shown in this answer to a similar problem. The setup.py would be like this:

import numpy
setup(
  name ='time stamp binner',
  cmdclass = {'build_ext': build_ext},
  ext_modules = ext_modules,
  include_dirs = [numpy.get_include()] #Include directory not hard-wired
)

score 1 · Answer 2 · answered Oct 02 '13 at 16:02

You can tell the compiler to include the directory where your header files reside using the following parameter in your setup.py:

ext_modules = [Extension("tsBinner",["tsBinner.pyx"],
              include_dirs = ["/full/path/python2.7/site-packages/numpy/core/include/"])];

score 1 · Answer 3 · answered Oct 12 '13 at 15:46

This is a problem found in all Canopy distributions. The answer below is for a Mac. It should be fairly obvious what to do in Linux and Windows; the folders are just slightly different.

After making the .c file, the setup.py file automatically runs the following for you

gcc -fno-strict-aliasing -fno-common -dynamic -arch x86_64 -DNDEBUG -g -O3 -arch x86_64 -I/Applications/Canopy.app/appdata/canopy-1.1.0.1371.macosx-x86_64/Canopy.app/Contents/include/python2.7 -c MyCythonCode.c -o build/temp.macosx-10.6-x86_64-2.7/MyCythonCode.o

Then you get the errors about the missing .h files: arrayobject.h and ufuncobject.h

Looking at the code above, in particular the -I option, the problem lies in that gcc is looking for these files in:

/Applications/Canopy.app/appdata/canopy-1.1.0.1371.macosx-x86_64/Canopy.app/Contents/include/python2.7

Type the following into a terminal window, to find where these files are:

find /Applications/ -name "ufuncobject.h"

For the Canopy installation on my Mac, they appeared in a couple of place. The location I'm interested in is the folder of the latest Canopy update (1.1.0.1371 for me). So the files ere located, for me, here:

/Applications//Canopy.app/appdata/canopy-1.1.0.1371.macosx-x86_64/Canopy.app/Contents/lib/python2.7/site-packages/numpy/core/include/numpy/ufuncobject.h

Note that the error messages ask for two files within a folder called numpy. If you just copy the two files in question, then you'll hit more errors. Copy the whole of the numpy folder, in this case the folder located in (one folder up from before):

/Applications//Canopy.app/appdata/canopy-1.1.0.1371.macosx-x86_64/Canopy.app/Contents/lib/python2.7/site-packages/numpy/core/include/

Paste this folder into the folder the compiler is looking in (i.e. the one listed as the -I option above). In my case, this is:

/Applications//Canopy.app/appdata/canopy-1.1.0.1371.macosx-x86_64/Canopy.app/Contents/lib/python2.7/

Now try running the setup file again. It should be okay.

I have always found this to be an issue with Enthought distributions. As great as EPD was and Canopy is, it's a real pain that this has to be manually fixed each time.

Hope this helps.

Missing numpy/arrayobject.h while compiling .pyx file

More Context

3 Answers3