element wise sum of structured arrays in numpy

Question

I wonder if it is possible at all to perform element-wise sum (or other operations) of two structured numpy arrays of an identical shape.

arr1 = np.array([[1,2,3],[2,3,4]], dtype=[("x", "f8"),("y", "f8")])
arr2 = np.array([[5,4,3],[9,6,4]], dtype=[("x", "f8"),("y", "f8")])
arr3 = np.sum(arr1, arr2)

says "ufunc 'add' did not contain a loop with signature matching types dtype([('x', '

If it is not possible, it will be great to understand why that is impossible or impractical to implement in numpy.

Have to do the sum, or other math field by field. In general fields may include strings and/or other dtypes that don't implement math. — hpaulj, Feb 04 '19 at 14:10
if the dtype is uniform... arr.view((arr.dtype[0], len(arr.dtype.names))).sum(axis=0) … but separate it out into components so you can see what is going on — NaN, Feb 04 '19 at 14:24
@hpaulj That's very true. I never store strings in ndarrays, so I overlooked that simple fact...! If you want, I will take it as the answer. It may look too short, but your comment clearly solved the problem :) — Hoseung Choi, Feb 04 '19 at 14:40
This works for you? `arr.view(np.float32).reshape(arr.shape + (-1,))` I think it is similar to @NaN solution — LocoGris, Feb 04 '19 at 14:51
@NaN - I think my example was confusing. I've updated the example. Your previous suggestion is only applicable to a single array and shrinks the array along an axis. So, I'm afraid they don't work. — Hoseung Choi, Feb 04 '19 at 15:03
@JonnyCrunch - I've updated the example. Like NaN's suggestion, your previous suggestion is only applicable to a single array and it duplicates the array. I must have confused you. — Hoseung Choi, Feb 04 '19 at 15:05
related : https://stackoverflow.com/questions/50931421/numpy-structured-array-fails-basic-numpy-operations — Tarifazo, Feb 04 '19 at 16:41
Normally structured arrays are initialized with list(s) of tuples. Since you don't have tuples it creates a (2,2) array with the `x` and `y` values repeated. @JonnyCrunch s point is that in some cases a structured array can be `viewed` as a uniform simple dtype, but you have to watch the shape. — hpaulj, Feb 04 '19 at 17:20
Expanding on the `view` idea, it is possible to define a `dtype` with overlapping fields. In this case a dtype that defines both `x` and `y`, and an `xy` field that occupies the same slots. IN that case you could do math on `arr['xy']` and see the results in the `x`,`y` fields. But we'd have to study the `dtype` documentation to do that right. — hpaulj, Feb 04 '19 at 17:24

score -1 · Accepted Answer · answered Feb 04 '19 at 19:21

With your array:

In [236]: arr1 = np.array([[1,2,3],[2,3,4]], dtype=[("x", "f8"),("y", "f8")])
In [237]: arr1
Out[237]: 
array([[(1., 1.), (2., 2.), (3., 3.)],
       [(2., 2.), (3., 3.), (4., 4.)]], dtype=[('x', '<f8'), ('y', '<f8')])
In [238]: arr1['x']
Out[238]: 
array([[1., 2., 3.],
       [2., 3., 4.]])

Normally the data for a structured array is provided in the form a list(s) of tuples., same as displayed in Out[237]. Without the tuples np.array assigns the same value to both fields.

You have to do math on each field separately:

In [239]: arr1['y'] *= 10
In [240]: arr1
Out[240]: 
array([[(1., 10.), (2., 20.), (3., 30.)],
       [(2., 20.), (3., 30.), (4., 40.)]],
      dtype=[('x', '<f8'), ('y', '<f8')])

Math operations are defined for simple dtypes like int and float, and uses compiled code where possible.

This error means that the add ufunc has not been defined for this compound dtype. And I think that's true for all compound dtypes.

In [242]: arr1 + arr1
---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
<ipython-input-242-345397c600ce> in <module>()
----> 1 arr1 + arr1

TypeError: ufunc 'add' did not contain a loop with signature matching types dtype([('x', '<f8'), ('y', '<f8')]) dtype([('x', '<f8'), ('y', '<f8')]) dtype([('x', '<f8'), ('y', '<f8')])

Since the fields in this case have the same base dtype, we can define another compound dtype that can 'view' it:

In [243]: dt2 = np.dtype([('xy', 'f8', 2)])
In [244]: arr2 = arr1.view(dt2)
In [245]: arr2
Out[245]: 
array([[([ 1., 10.],), ([ 2., 20.],), ([ 3., 30.],)],
       [([ 2., 20.],), ([ 3., 30.],), ([ 4., 40.],)]],
      dtype=[('xy', '<f8', (2,))])
In [246]: arr2['xy']
Out[246]: 
array([[[ 1., 10.],
        [ 2., 20.],
        [ 3., 30.]],

       [[ 2., 20.],
        [ 3., 30.],
        [ 4., 40.]]])

Math on that field will be seen in the original array:

In [247]: arr2['xy'] += .1
In [248]: arr2
Out[248]: 
array([[([ 1.1, 10.1],), ([ 2.1, 20.1],), ([ 3.1, 30.1],)],
       [([ 2.1, 20.1],), ([ 3.1, 30.1],), ([ 4.1, 40.1],)]],
      dtype=[('xy', '<f8', (2,))])
In [249]: arr1
Out[249]: 
array([[(1.1, 10.1), (2.1, 20.1), (3.1, 30.1)],
       [(2.1, 20.1), (3.1, 30.1), (4.1, 40.1)]],
      dtype=[('x', '<f8'), ('y', '<f8')])

We can also view it as a simple dtype, but will have to adjust the shape:

In [250]: arr3 = arr1.view('f8')
In [251]: arr3
Out[251]: 
array([[ 1.1, 10.1,  2.1, 20.1,  3.1, 30.1],
       [ 2.1, 20.1,  3.1, 30.1,  4.1, 40.1]])
In [252]: arr3.reshape(2,3,2)
Out[252]: 
array([[[ 1.1, 10.1],
        [ 2.1, 20.1],
        [ 3.1, 30.1]],

       [[ 2.1, 20.1],
        [ 3.1, 30.1],
        [ 4.1, 40.1]]])

I like the trick using a view to get a "pure" number array! I think this is overkill to the present problem, but will be useful at some point. — Hoseung Choi, Feb 07 '19 at 16:08

element wise sum of structured arrays in numpy

1 Answers1

Linked