Simpler code to combine multiple lists into one array? I actually wish "lists" are still "lists" in that array

Question

I have a question similar to this and that, but the solutions to those is that all lists are "united" with lack of "differentiation".

My Python code is like following:

y = np.empty([1,len(test)]) 
count = -1 
for feature in test :
    N = engine.neighbours(np.asarray(feature))
        if len(N) != 4096:
            print "error"
            continue        
        count = count + 1
    y [count] = engine.neighbours(np.asarray(feature))

I am just wondering if there is any simplified code to do the job?

sorry for the poor code writing, here is the actually screen printout -- http://screencloud.net/v/jhyA — user381509, Jun 20 '15 at 09:28
Your question seems like it has nothing to do with the other questions. Those questions are about flattening lists into single lists. You don't seem to want that. Have you tried `append`? — nneonneo, Jun 20 '15 at 09:31
thanks for the quick reply. yes i tried, does not looks good, though i do not know why the outcome is none, i hope it is going to generate one array with two lists (elements) inside the array. http://screencloud.net/v/hxHT , another example is here http://screencloud.net/v/lRnA thanks :) — user381509, Jun 20 '15 at 09:54
You may want to correct the indentation of the code for making it readable. Just use spaces, and not tab. — KalEl, Jun 20 '15 at 12:55
Don't recommend `np.append`. It is just a form of `concatenate`, and is often used erroneously. — hpaulj, Jun 20 '15 at 15:34

hpaulj · Accepted Answer · 2015-06-20T16:10:23.603

The linked questions have to do with flattening a list of lists. In your code you are processing a list of lists (I'm guessing), filtering out some, and collecting the rest in a 2d array.

Oops! y is initialized with 1 row, yet you try to place values in y[count], where count could increase to the size of test.

y = np.empty([1,len(test)]) 
count = -1    
# I would prefer to start with 0, but thats a style issue
for feature in test :
    N = engine.neighbours(np.asarray(feature))
    # so engine.neighbors requires an array; and returns an array?
        if len(N) != 4096:
            print "error"
            continue        
        count = count + 1
    y [count] = engine.neighbours(np.asarray(feature))
    # why call neighbors again?  why not reuse N?

A common way of creating an array incrementally is:

alist = []
for feature in tests:
     N = engine.neighbors(np.array(feature)
     # test N length
     alist.append(N)
y = np.array(alist)

Since by the length test, all N have the same length, the resulting array will be 2d, shape (n,4096), where n is the number of tests with the correct length.

Initializing y to np.empty((length(tests),4096)], and inserting y[count,:] = N may be faster. You may end up with unfilled rows if some fail the length test, but you can remove those.

Initializing y to np.empty((1,4096)], and inserting y=np.append(y,N) should also work. Note that this append is different from the list append. And slower. I prefer that people use concatenate directly:

y = np.concatenate([y, N[None,:]], axis=0)

The concatenation is explicit, and the required dimension manipulation clear.

To make a 1d array of arrays, you have to do something like this:

y=np.empty((4,),dtype=object)
for i in range(len(y)):
    y[i]=np.arange(i,2*i)

producing:

array([array([], dtype=int32), array([1]), array([2, 3]), array([3, 4, 5])], dtype=object)

which is arguably little more than the y list produced by

y=[]
for i in range(4):
    y.append(np.arange(i,2*i))

In all this I'm assuming that engine.neighbors() takes a 1d array, and returns a 1d array. If if took/returned multiple feature we could 'vectorize' things. But as long as we can only give it one feature at a time we are stuck with some form of iteration.

Simpler code to combine multiple lists into one array? I actually wish "lists" are still "lists" in that array

1 Answers1