`r+`, `a+`, `w+` for doing both reading and writing file in python

Question

I have a file named dict_file.json in current folder with empty dict content {}

I want to open it such that I can do both read and modify the content. That is I will read the json file as dict, modify that dict and write it back to json file.

I tried r and r+ below:

import json
f = open('dict_file.json', 'r')  # same output for r+
json.loads(list_file.read())

This prints

{}

When I tried w+:

f = open('dict_file.json', 'r')

this first clears the file. Then,

json.loads(list_file.read())

gives error:

Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "F:\ProgramFiles\Python37\lib\json\__init__.py", line 296, in load
    parse_constant=parse_constant, object_pairs_hook=object_pairs_hook, **kw)
  File "F:\ProgramFiles\Python37\lib\json\__init__.py", line 348, in loads
    return _default_decoder.decode(s)
  File "F:\ProgramFiles\Python37\lib\json\decoder.py", line 337, in decode
    obj, end = self.raw_decode(s, idx=_w(s, 0).end())
  File "F:\ProgramFiles\Python37\lib\json\decoder.py", line 355, in raw_decode
    raise JSONDecodeError("Expecting value", s, err.value) from None
json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0)

When I tried a+:

f = open('dict_file.json', 'a+')
json.loads(list_file.read())

gives error:

Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "F:\ProgramFiles\Python37\lib\json\__init__.py", line 296, in load
    parse_constant=parse_constant, object_pairs_hook=object_pairs_hook, **kw)
  File "F:\ProgramFiles\Python37\lib\json\__init__.py", line 348, in loads
    return _default_decoder.decode(s)
  File "F:\ProgramFiles\Python37\lib\json\decoder.py", line 337, in decode
    obj, end = self.raw_decode(s, idx=_w(s, 0).end())
  File "F:\ProgramFiles\Python37\lib\json\decoder.py", line 355, in raw_decode
    raise JSONDecodeError("Expecting value", s, err.value) from None
json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0)

Though, it does not clear the file.

So, I guess I should be using r+ for my usecase scenario. Also I tried reading writing both :

f = open('dict_file.json', 'r+')
a = json.loads(list_file.read())
a = {'key':'value'}  # this involves complex logic instead of plain assignment
json.dump(a,f)
f.flush()

Now when I open the file, its contents are weird:

{}{'key':'value'}

whereas I want it to be:

{'key':'value'}

So I have few questions:

Q1. Why w+ and a+ gives error?
Q2. Why file contained {}{'key':'value'} instead of {'key':'value'}?
Q3. How to correctly do reading and writing (with or without with block)?

PS: I am reading file, then running some loop which computes new dict and write it to file. Then loop sleeps for some time and repeats the same. Thats why I felt flush() will be correct here. That is I open file once outside loop and only flush inside the loop. No need to open file in each iteration.

https://stackoverflow.com/questions/6648493/how-to-open-a-file-for-both-reading-and-writing — Cheery, Dec 04 '21 at 16:15

score 3 · Answer 1 · answered Dec 04 '21 at 16:15

Why w+ and a+ gives error?

Because the file reader is pointed at the end of the file, where there's no json object to read

its contents are weird

Seems like you're not resetting the file to an actual valid JSON object throughout your tests, and so you've managed to clear the file, read nothing, maybe write some other object, clear it again, then append one or two objects to it, etc etc.

Keep it simple. Start over

filename = "data.json"
with open(filename) as f:
    data = json.load(f)

data["foo"] = "bar"

with open(filename, "w") as f:
    json.dump(data, f)

score 0 · Answer 2 · answered Dec 04 '21 at 17:42

You have to think in terms of what the operations do to a file when they open it:

r and r+ open a file for reading. This means that the contents of the file is intact, and the file pointer is the beginning of the file. The entire file is visible to the reader. After reading, the file pointer will be at the end, meaning that you're effectively appending at that point.
w and w+ open a file for writing. That means that the contents of the file is truncated. There is nothing to be read in, since the original contents is destroyed.
a and a+ open the file for appending. The contents of the file are unchanged. However, the file pointer is at the end, so a reader will see no data and raise an error.

The correct way to do this is to open the file twice, and use with blocks to do it:

with open('dict_file.json', 'r') as f:
    a = json.load(f)

# you don't need the file to be open here
a = {'key': 'value'}

with open('dict_file.json', 'w') as f:
    json.dump(a, f)

You might be tempted to try to open the file in read-write mode (r+), read it, then rewind it, so the pointer returns to the beginning, and then overwrite. There are two reasons not to do this:

Not every stream is rewindable
If the contents of the file decreases in size, you will end up with trash at the end. You would need to truncate to the current pointer after writing, which is an unnecessary layer of complexity

`r+`, `a+`, `w+` for doing both reading and writing file in python

2 Answers2