How to read lists of list from a file and make individual lists?

Question

I have a file here https://www.dropbox.com/s/mwz8s2kap2mnwo0/data.dat?dl=0 where each line is a list of list of varying number of elements. I would like to create two separate lists say A and B out of the data given in the file above where A contains the elements such as [3, 3, 1], [3, 3, 1], [3, 3, 3, 1], [3, 3, 3, 1] and B contains the elements [5.55, 4.786504465682655, 4.786504465682655], [4.801464620300768, 4.786504465682655, 4.801464620300768] and so on.

I first tried to read the data from the file as following:

with open('data.dat', 'r') as f:
     x = f.read().splitlines()
x

That gives me the following output:

['[[3, 3, 1], [5.55, 4.786504465682655, 4.786504465682655]]',
 '[[3, 3, 1], [4.801464620300768, 4.786504465682655, 4.801464620300768]]',
 '[[3, 3, 3, 1], [2.7663717212261982, 2.7663717212261982, 2.7663717212261982, 5.5327434424523965, 2.775, 4.786504465682655]]',
 '[[3, 3, 3, 1], [2.775, 2.775, 4.801464620300768, 3.9244426355853386, 4.801464620300768, 4.801464620300768]]',
 '']

From here I am not sure how to move ahead and separate two lists and append them in two different lists. Any help is much appreciated!

EDIT: If I just add another following line after importing ast

x = ast.literal_eval(x)

gives me following output:

/anaconda3/lib/python3.6/ast.py in _convert(node)
     82                 else:
     83                     return left - right
---> 84         raise ValueError('malformed node or string: ' + repr(node))
     85     return _convert(node_or_string)
     86 

ValueError: malformed node or string: ['[[3, 3, 1], [5.55, 4.786504465682655, 4.786504465682655]]', '[[3, 3, 1], [4.801464620300768, 4.786504465682655, 4.801464620300768]]', '[[3, 3, 3, 1], [2.7663717212261982, 2.7663717212261982, 2.7663717212261982, 5.5327434424523965, 2.775, 4.786504465682655]]', '[[3, 3, 3, 1], [2.775, 2.775, 4.801464620300768, 3.9244426355853386, 4.801464620300768, 4.801464620300768]]', '']

Possible duplicate of [Convert string representation of list to list](https://stackoverflow.com/questions/1894269/convert-string-representation-of-list-to-list) — Boris Verkhovskiy, Apr 08 '19 at 19:14
You just have to run `eval` (or some equivalent) on each string/line. — hpaulj, Apr 08 '19 at 19:16
`ast.literal_eval` would be the preferred equivalent; `eval` should be avoided as much as possible. — chepner, Apr 08 '19 at 19:28
Please have a look my edit. Seems there are still some problems even after using ast. — baban, Apr 08 '19 at 19:48
you want to run `ast.literal_eval` on just the string part, so `ast.literal_eval(x[0])` in this case — ddg, Apr 08 '19 at 19:51

score 1 · Accepted Answer · answered Apr 08 '19 at 20:08

Try this:

A, B = [], []

for i in x:
    if not i: # for ''
        continue
    tmp_list = ast.literal_eval(i)
    A.append(tmp_list[0])
    B.append(tmp_list[1])

print A, B

A: [[3, 3, 1], [3, 3, 1], [3, 3, 3, 1], [3, 3, 3, 1]]
B: [[5.55, 4.786504465682655, 4.786504465682655], [4.801464620300768, 4.786504465682655, 4.801464620300768], [2.7663717212261982, 2.7663717212261982, 2.7663717212261982, 5.5327434424523965, 2.775, 4.786504465682655], [2.775, 2.775, 4.801464620300768, 3.9244426355853386, 4.801464620300768, 4.801464620300768]]

How to read lists of list from a file and make individual lists?

1 Answers1