0

I'm reading, updating and writing the following single line of data to a separate fruit.txt file, in this specific structure and format:

[["Peach",0,0,0], ["Banana",0,0,0], ["Apple",0,0,0], ["Pear",0,0,0], ["Mango",0,0,0], ["Orange",0,0,0], ["Apricot",0,0,0]]

When I do this at the start of the Python file

with open("fruit.txt") as file: fruit = file.read()

It correctly assigns the data in fruit.txt to the fruit var. When I print(fruit), it even looks like the intended nested list.

However, when I come to query fruit var e.g. dynamically changing the 0's via other functions, or when I try to sort the nested list, or do anything to alter the data, it always returns various errors. It's as though when it sets the fruit var to the data from the file, it doesn't recognise it as a nested list, but more a string. So therefore I can't operate on that list.

If however I do:

with open("fruit.txt") as file: fruit_count = file.read() fruit = eval(fruit_count)

Then the fruit var behaves fine - values update and it writes back to the file ready for further use etc.

I understand eval is theoretically evil (or at least should be avoided under most circumstances) unless you trust the data source (which I do in this instance). But I'm just wondering if there's a different solution?

Ideally I don't want to use any additional modules e.g. ast.literal_eval

user9099702
  • 109
  • 8
  • 2
    If you don't want to use library modules and you don't want to use `eval`, you're left with writing your own parser. I'd imagine it's easier to just use `ast.literal_eval` though if you insist on that format. – merlin2011 Oct 06 '18 at 07:38
  • If you are willing to use a different storage format, you could probably make manually writing parsing code easier by structuring your text file with two different delimiters for the two layers of nesting instead of the Python list format, or some kind of binary format. – merlin2011 Oct 06 '18 at 07:39
  • OK thanks - so is my assessment right that without using eval(), it's simply interpreting the fruit var as a string? – user9099702 Oct 06 '18 at 07:40
  • Even though you don't want to use additional modules, everything would be a lot easier if you saved the initial list-of-lists vwith Python's [`pickle`](https://docs.python.org/3/library/pickle.html#module-pickle) module. See my answer to the question [Saving an Object (Data persistence)](https://stackoverflow.com/questions/4529815/saving-an-object-data-persistence) for more information and example code. – martineau Oct 06 '18 at 07:54
  • @martineau Can we please not recommend pickle - which comes with a risk of arbitrary code execution - when JSON would be absolutely sufficient? – Aran-Fey Oct 06 '18 at 07:56
  • @Aran-Fey: Using `pickle` is fine here since the data would not be from an untrusted source. – martineau Oct 06 '18 at 07:58
  • I had difficulties when initially saving and returning this data as a JSON list as I couldn't correctly get the numbers to return as integers and the fruit names to return as string. From what I remember everything was being treated as strings (wrapped in quotes)? – user9099702 Oct 06 '18 at 08:03

1 Answers1

1

It may not be ideal, but here's how to do it with the json module (even though you don't want to use an additional module):

import json
from pprint import pprint

lists = [["Peach",0,0,0], ["Banana",0,0,0], ["Apple",0,0,0], ["Pear",0,0,0],
         ["Mango",0,0,0], ["Orange",0,0,0], ["Apricot",0,0,0]]

with open("fruits.json", "w") as fp:
    json.dump(lists, fp)

with open("fruits.json", "r") as fp:
    fruits = json.load(fp)

pprint(fruits)

Output:

[['Peach', 0, 0, 0],
 ['Banana', 0, 0, 0],
 ['Apple', 0, 0, 0],
 ['Pear', 0, 0, 0],
 ['Mango', 0, 0, 0],
 ['Orange', 0, 0, 0],
 ['Apricot', 0, 0, 0]]

Note that the data read back does indeed have the proper data-types in it.

martineau
  • 119,623
  • 25
  • 170
  • 301
  • Thanks for showing me with JSON. Is there any benefit to using JSON other than avoiding eval()? Although if I'm gonna use JSON then I guess I could just import ast and swap eval() for ast.literal_eval instead? – user9099702 Oct 06 '18 at 08:22
  • @user9099702: The main advantage is that JSON format is an externally defined standard format supported by many computer languages. Data stored via `ast` is inherently Python-specific and not supported by many (any?) other languages. In both cases, the data in the file would be human-readable, so they're even in that regard (if you even care). Using the `pickle` module would also be very Python-centeric. – martineau Oct 06 '18 at 08:29
  • json also tends to be faster then `ast.literal_eval` and `eval`. – sup Oct 06 '18 at 08:39
  • Nice, thanks avram. Although in this instance I guess speed isn't so much an issue as the external file is only ever gonna be 1 line. – user9099702 Oct 06 '18 at 08:41
  • One advantage that `pickle` has over JSON and `ast` is that it can save most user-defined class instances (and many Python built-in data-types) automatically, as well as "primitives" like dictionaries, lists, and numbers. – martineau Oct 06 '18 at 08:41