flatten arrays in a list in python

Question

I have multiple numpy masked arrays arr0, arr1, ..., arrn.

I put them in a list arrs = [arr0, ..., arrn].

I want to flatten these arrays et put a mask on them. I did something like:

for arr in arrs:
    arr = np.ravel(arr)
    arr[mask] = ma.masked

I do not understand when Python make copies and when it is just a pointer. This for loop does not flatten the arr0, ..., arrn, (whereas ravel outputs a view and not a copy) it just flattens the variable arr, although it does change their mask !

As I understand it, arr is a view of the elements in the list arrs, so when I change elements of arr it changes the elements of the corresponding array in the list. But when I assign a new value to arr it does not change the original array, even if the assignement is supposed to be a view of this array. Why ?

Edit with an example:

Arrays to flatten:

arr0 = masked_array(data=[[1,2],[3,4]], mask=False)
arr1 = masked_array(data=[[5,6],[7,8]], mask=False)
mask = [[False,True],[True,False]]

Expected output:

arr0 = masked_array(data=[[1,--],[--,4]], mask=[[False,True],[True,False]])
arr1 = masked_array(data=[[5,--],[--,8]], mask=[[False,True],[True,False]])

I'd like to do this in a loop because I have a lot of arrays (15 more or less), and I want to use the arrays name in the code. Is there no other way than do to:

arr0 = np.ravel(arr0)
...
arrn = np.ravel(arrn)

Could you add a numerical example of what you're trying to achieve? — Oliver W., Jun 21 '16 at 10:11
the problem is with list iterator. You can't assign it within the loop. — hpaulj, Jun 21 '16 at 10:13
`arr = np.ravel(arr)` creates a new (flattened) view to `arr` and reassigns the name *arr* to point to that created view. Plain assignment just [reassigns names to point to something](http://stackoverflow.com/questions/2438938/python-copy-through-assignment). As you've noted `ravel()` creates a view and so `arr[mask] = ma.masked` does modify the original as well. — Ilja Everilä, Jun 21 '16 at 10:16
Assigning and mutating are different operations. Variables are always just references. Assigning a name to a new value always just changes what that reference refers to, not the original value. — Daniel Roseman, Jun 21 '16 at 10:17

hpaulj · Answer 1 · 2016-06-21T16:44:10.153

In [1032]: arr0 = np.ma.masked_array(data=[[1,2],[3,4]], mask=False)    
In [1033]: arr1 = np.ma.masked_array(data=[[5,6],[7,8]], mask=False)

This is the basic way of iterating over a list, applying some action to each element, and collecting the results in another list:

In [1037]: ll=[arr0,arr1]    
In [1038]: ll1=[]
In [1047]: for a in ll:
    a1=a.flatten()   # makes a copy
    a1.mask=mask
    ll1.append(a1)

In [1049]: ll1
Out[1049]: 
[masked_array(data = [1 -- -- 4], mask = [False  True  True False],
        fill_value = 999999), 
        masked_array(data = [5 -- -- 8], mask = [False  True  True False],
        fill_value = 999999)]

Often that can be writen a list comprehension

 [foo(a) for a in alist]

but the action here isn't a neat function

If I use ravel instead, a1 is a view (not a copy), and applying mask to it changes the mask of a as well - the result is changed masks for arr0, but no change in shape:

In [1051]: for a in ll:
   ......:     a1=a.ravel()
   ......:     a1.mask=mask

(the same happens with your a=a.ravel(). The a= assigns a new value to a, breaking the link to the iteration value. That's true for any Python iteration. It's best to use new variable names inside the iteration like a1 so you don't confuse yourself.)

Essentially the same as

In [1054]: for a in ll:
   ......:     a.mask=mask

I can change the shape in the same in-place way

In [1055]: for a in ll:
   ......:     a.shape=[-1]   # short hand for inplace ravel
   ......:     a.mask=mask   

In [1056]: arr0
Out[1056]: 
masked_array(data = [1 -- -- 4],
             mask = [False  True  True False],
       fill_value = 999999)

Here's a functional way of creating new arrays with new shape and mask, and using it in a list comprehension (and no change to arr0)

[np.ma.masked_array(a,mask=mask).ravel() for a in [arr0,arr1]]

Understanding these alternatives does require understanding how Python assigns iterative variables, and how numpy makes copies and views.

just wanted to point out that the **none np way** can be found here: https://stackoverflow.com/a/953097/2139007. for those who wonder :) — luQ, Jun 26 '23 at 15:00

flatten arrays in a list in python

1 Answers1