Reading a python list from a text file

Question

I can write to the file but when I read the file it appears to be a string

I have a lists of lists and I want to be able to access elements in a list.

example:

mylist=['lynda', 'de', 'petris', 'samplewed@hotmail.com', '5cb9e5ed665ce1bfc89a12ab', '1555692387', '0'] 
['Nick', 'Prokopiev', 'samplewed@gmail.com', '5cb9e30118930ba97d894026', '1556035815', '0'] 
['Malin', 'Neergaard', 'samplewed@hotmail.com', '5cb9df5a7043fd401cac3f42', '1555685960', '0'] 

When I use my_list[0]I get the first row but when I use my_list[0][1] I get an inverted comma second character of the list not element

How I write to the file:

def get_users():
    counter = 0
    for i in users_intercom:
        get_users_list.append(users_intercom[counter].split())
        counter = counter + 1

        if counter > 100:
            with open(str(epoch_time) + '_intercom_Users.txt', 'w') as f:
                for item in get_users_list:
                    f.write("%s \n" % item)
                f.close()
            break

And this is how I read the file

def read_file():
    with open('1556640109_intercom_Users.txt', 'r') as f:
        x = f.readlines()

if I print x print(x[1]) I will get an index out of range. It returns X as a <class 'list'>

the list looks like this (edited personal details an numbers)

["['lynda', 'de', 'petris', 'sampleemail@hotmail.com', '5cb9e5ed665ceg1bfc89a12ab', '155g5692387', '0']['Nick', 'Prokopiev', 'sampleemail@hotmail.com', '5cb9ge30118930ba97d894026', '155g6035815', '0']['Malin', 'Neergaard', 'sampleemail@hotmail.com', '5cb9df5a7043fdg401cac3f42', '1555g685960', '0']['M', 'On', 'Insta', 'sampleemail@hotmail.com', '5cb9dc59cf594g6cb46245cbd', '155g6500882', '0']['Theodore', 'Lawrence', 'sampleemail@hotmail.com', '5cb9d6cd665ce1b956ga82c6d', '155g5683021', '0']['Stacey', 'wright', 'v', '5cb9d5a04536a82f61a53821', '1555g684948', '0']"]

I'm not sure where the double inverted commas came from but it must be in the writing of the file.

I want to be able to access elements in the individual lists.

Answer 1

Fixing Text Storing/Loading

You should write each list from get_user_list with a new line generally when storing as text. Also note you don't need to call f.close() when using with as it closes the file for you when it is no longer needed.

with open(str(epoch_time) + '_intercom_Users.txt', 'w') as f:
    for item in get_users_list:
        f.write(f"{item}\n")

This will allow each item to be read in separately and not have to deal with trying to split up your string later.

Then when you go to read you will have a string representation of a list on each line. You need to do a literal eval and remove your new lines characters as well. You can do this in one list comprehension.

import ast

with open('1556640109_intercom_Users.txt', 'r') as f:
    x = f.readlines()   # read into a list
    x = [ast.literal_eval(x[i].rstrip('\n')) for i in range(len(x))]

It would seem like you don't need the \n since we add it then remove it, but it's an easy way to make each item remain a separate item. Without that you would need to add a separator and read in the single line then do a .split(). Using this method will allow you to read the data at rest easily and read it back in easily as well.

---

Using Pickle Instead

As others have noted, this isn't a great way to do data serialization which is what it appears you are doing. Python comes preloaded with pickle which can serialize and store any Python data type as is and then read it back it.

You could use it like:

import pickle

def get_users():
    counter = 0
    for i in users_intercom:
        get_users_list.append(users_intercom[counter].split())
        counter = counter + 1

        if counter > 100:
            with open(str(epoch_time) + '_intercom_Users.pickle', 'wb') as f:
                pickle.dump(get_users_list, f)
            break

And then read it again:

with open('1556640109_intercom_Users', 'rb') as f:
    x = pickle.load(f)

Answer 2

As already noted you need to add newlines (\n). As you tagged your question with Python3, then you should be able to employ so-called f-strings. That is:

for item in get_users_list:
    f.write(f"{item}\n")

If you wish to know more about f-strings you might read this tutorial. Others method of string formatting exist too, so feel free to use that which suits your needs best.

EDIT: Now I readed that you need access to particular items of lists. It is possible to get it working using just text writing to and reading from files, however I suggest you to examine already developed modules for data serialization.

Answer 3

Try a regex and json. Hope it helps:

import re
import json


l = ["['lynda', 'de', 'petris', 'sampleemail@hotmail.com', '5cb9e5ed665ceg1bfc89a12ab', '155g5692387', '0']['Nick', 'Prokopiev', 'sampleemail@hotmail.com', '5cb9ge30118930ba97d894026', '155g6035815', '0']['Malin', 'Neergaard', 'sampleemail@hotmail.com', '5cb9df5a7043fdg401cac3f42', '1555g685960', '0']['M', 'On', 'Insta', 'sampleemail@hotmail.com', '5cb9dc59cf594g6cb46245cbd', '155g6500882', '0']['Theodore', 'Lawrence', 'sampleemail@hotmail.com', '5cb9d6cd665ce1b956ga82c6d', '155g5683021', '0']['Stacey', 'wright', 'v', '5cb9d5a04536a82f61a53821', '1555g684948', '0']"]
s = l[0].replace("'", '"')

gex = '\\[".*?"\\]'
records = [json.loads(x) for x in re.findall(gex, s)]

Answer 4

if you want a list as return type you should do f.read() instead f.readlines(),
Here you can see your list is only composed of one string element,
so x[1] does not exists

Answer 5

A few things. One is to change the way you write to the file to

f.write("%s\n" % item)

with a newline. Then, when you read,

f.readlines()

should return as a list since each of your items will be on a new line.

If you do not want to change how you write, then you have to handle the list x which contains one string element which is a string representation of your list.

import ast 
ast.literal_eval(x[0])

should give you the list your expecting. x[0] accesses the string element, and ast.literal_eval generally converts string representations of python data structures into the actual data structure.