From list of lists to list of lists of lists

Question

i have a list like this:

first_list = [[ 1.        , 45.4,  9.1],
              [ 2.        , 45.5,  9.1],
              [ 2.        , 45.4,  9.2],
              [ 2.        , 45.4,  9.2],
              [ 3.        , 45.4,  9.1],
              [ 3.        , 45.4,  9.1],
              [ 3.        , 45.4,  9.1] ]

I want to use the folio function HeatMapWithTime, and to do that i need to group the data above according to the first item of each sublist (1., 2., 3. ecc):

new_list = [ [ [45.4, 9.1] ],                               # All coords for 1.
             [ [45.5, 9.1], [45.4, 9.2], [45.4, 9.2] ],     # All coords for 2.
             [ [45.4, 9.1], [45.4, 9.1], [45.4, 9.2] ] ]    # All coords for 3.

How can i do that?

yatu · Answer 1 · 2020-01-29T14:23:11.063

3

Assuming the list is sorted by the first elements, as it seems, you can use itertools.groupby:

from itertools import groupby
from operator import itemgetter
[[i[1:] for i in v] for k,v in groupby(first_list, itemgetter(0))]

#[[[45.4, 9.1]],
# [[45.5, 9.1], [45.4, 9.2], [45.4, 9.2]],
# [[45.4, 9.1], [45.4, 9.1], [45.4, 9.1]]]

edited Jan 29 '20 at 14:23

answered Jan 29 '20 at 14:17

yatu

86,083
12
84
139

Mike Müller · Answer 2 · 2020-01-29T14:51:30.317

2

You can collect all coordinates in a dictionary:

res = {}
for entry in first_list:
    res.setdefault(entry[0], []).append(entry[1:])

This gives you:

>>> res
{1.0: [[45.4, 9.1]],
 2.0: [[45.5, 9.1], [45.4, 9.2], [45.4, 9.2]],
 3.0: [[45.4, 9.1], [45.4, 9.1], [45.4, 9.1]]}

If your list was already sorted, convert the values into a list (Python 3.6+ only):

>>> list(res.values())
[[[45.4, 9.1]],
 [[45.5, 9.1], [45.4, 9.2], [45.4, 9.2]],
 [[45.4, 9.1], [45.4, 9.1], [45.4, 9.1]]]

Otherwise, you need to sort them first:

>>> [res[key] for key in sorted(res.keys())]
[[[45.4, 9.1]],
 [[45.5, 9.1], [45.4, 9.2], [45.4, 9.2]],
 [[45.4, 9.1], [45.4, 9.1], [45.4, 9.1]]]

edited Jan 29 '20 at 14:51

answered Jan 29 '20 at 14:34

Mike Müller

82,630
20
166
161

'If your list was already sorted, convert he values into a list' Are you sure about that? I thought dictionaries don't preserve order – Nathan Jan 29 '20 at 14:46
1

Starting from Python 3.6 the do preserve insertion order. https://stackoverflow.com/questions/39980323/are-dictionaries-ordered-in-python-3-6 – Mike Müller Jan 29 '20 at 14:50

Nathan · Answer 3 · 2020-01-29T14:49:04.713

1

One way to do this is to first sort your list:

lst_data = sorted(first_list)

And then to loop over it, creating a new ljst when the fist index changes:

first_index = None
final_lst = []
for i in lst_data:
    if i[0] != first_index:
        final_lst.append([])
        first_index = i[0]
    final_lst[-1].append(i[1:])

edited Jan 29 '20 at 14:49

answered Jan 29 '20 at 14:22

Nathan

3,558
1
18
38

score 1 · Answer 4 · answered Jan 29 '20 at 14:26

I would use a dict for that, you might want to bring it back to a list, if you need it as a list, but using a dict for grouping is usually helpful:

first_list = [[ 1.        , 45.4,  9.1],
              [ 2.        , 45.5,  9.1],
              [ 2.        , 45.4,  9.2],
              [ 2.        , 45.4,  9.2],
              [ 3.        , 45.4,  9.1],
              [ 3.        , 45.4,  9.1],
              [ 3.        , 45.4,  9.1] ]
result = dict()
for group, *values in first_list:
    if group not in result:
        result[group] = [values]
    else:
        result[group].append(values)
print(result)
### if you want it back as a list:
result_list = [v for k,v in result.items()]
print(result_list)

Output:

#dict:
{1.0: [[45.4, 9.1]], 2.0: [[45.5, 9.1], [45.4, 9.2], [45.4, 9.2]], 3.0: [[45.4, 9.1], [45.4, 9.1], [45.4, 9.1]]}
#list:
[[[45.4, 9.1]], [[45.5, 9.1], [45.4, 9.2], [45.4, 9.2]], [[45.4, 9.1], [45.4, 9.1], [45.4, 9.1]]]

score 0 · Accepted Answer · answered Jan 29 '20 at 14:38

0

One solution using pandas, a wise choice when dealing with complex data formats:

import pandas as pd    
pd.DataFrame(first_list).set_index(0).groupby(df.index).apply(lambda x: x.values.tolist()).tolist()
#-> 
[[[45.4, 9.1]],
 [[45.5, 9.1], [45.4, 9.2], [45.4, 9.2]],
 [[45.4, 9.1], [45.4, 9.1], [45.4, 9.1]]]

answered Jan 29 '20 at 14:38

ibarrond

6,617
4
26
45

Great, i'm using pandas, so i came back to the original df and ran this command: ```heat_list = df.groupby(df.day).apply(lambda x: x.values.tolist()).tolist()```. The output return a sub-sublist like this: ```[1.0, 45.44430542, 9.16762733]```, but i don't want the first element, only lat and lon. How can i do that? – alcor Jan 29 '20 at 14:47
@Ibarrond pandas can be really useful, but I think this solution is very complex (difficult to debug in the future) and I suspect it could be quite slow – Nathan Jan 29 '20 at 14:48
I agree, but I use Pandas very often, that's why i'd like to solve the problem in this way. – alcor Jan 29 '20 at 14:51
@Nathan if you execute each operation step by step, you can debug it incredibly easy. WRT performance, unless the question asks for it, there is no point on providing an ultrafast solution, but rather one that can be understood and reused. – ibarrond Jan 29 '20 at 14:57
1

@alcor I would try something like this: `heat_list = df.groupby(df.day).apply(lambda x: x.values[1:].tolist()).tolist()`. (Note the [1:] to remove the first element) – ibarrond Jan 29 '20 at 14:59
@ibarrond I've been programming in python for several years now and it took me a second to understand what your function does. Having had to work on legacy code containing many examples of such solutions I politely dissagree about how easy it is to find (and solve) such errors – Nathan Jan 29 '20 at 15:04

From list of lists to list of lists of lists

5 Answers5