How to read from a file and put the contents into a json method in Python?

Question

I am trying to read the data from a file that has content that looks like this

{"name": "Host1","type": "ipmask","subnet": ["0.0.0.0","255.255.255.255"],"dynamic_mapping": None},
{"name": "Host2","type": "ipmask","subnet": ["0.0.0.0","255.255.255.255"],"dynamic_mapping": None},

into a variable in python so that I can use it in the code below for a post request using JSON.

with open('data.txt', 'r') as file:
    dat2 = file.read()

post2 = {
        "id": 5,
        "method": "set",
        "params": [
            {
                "data": [
                     dat2
                    ],
                "url": "/config/url"
            },
        "session": sessionkey,
        "verbose": 1
        }

I'm stuck because when I copy and past the data into the JSON where dat2 is, the request goes through sucessfully. I really don't understand what I am doing wrong, I have even tried stripping newlines and whitepaces. If anyone could help I would greatly appreciate it.

Update: I found that the difference in the requests I was making has to do with single quotes. When I copy the contents into the `dat2` variable, it adds singles quotes to the data so the content starts with`'params': [{'data': ['{"name":` instead of `'params': [{'data': [{"name":` and the same thing at the end of the data. I tried to used the replace method, but that doesn't work on dict objects. Do you have any idea on how to strip those single quotes? — Talon Stromgren, Jun 22 '20 at 21:16

score 0 · Answer 1 · answered Jun 19 '20 at 21:53

0

This stack overflow question has a lot of good answers: How to read a file line-by-line into a list?. You can probably find one that works.

answered Jun 19 '20 at 21:53

Strat5

13
1
6

score 0 · Answer 2 · answered Jun 19 '20 at 22:29

I guess you would like to send request for every line in the data.txt? In this case simple loop will help to read line, strip it of ',' and send request:

with open('data.txt', 'r') as file:
    for line in file.readlines():
    # strip whilespace with .strip() and then delete last character in a string (,)
    dat2 = line.strip()[:-1] 

    post2 = {
        "id": 5,
        "method": "set",
        "params": [
            {
                "data": [
                     dat2
                    ],
                "url": "/config/url"
            },
        "session": sessionkey,
        "verbose": 1
        }

score 0 · Answer 3 · answered Jun 20 '20 at 15:27

There are a couple of things that I can see that might cause you some issues, so will mention them first, then suggest a few tweaks...

The two main things are that first, strict JSON parsers will have an issue with None in the data file - I'm wondering if there's a mixing of JSON vs the string output of a Python data-structure going on there. The other one is that the post2 structure you show is missing the closing square bracket of the 'params' element, so it's clearly not running as is...

I'd suggest you'd want to set the 'post2' structure up once, then just update the 'data' array contents for each line, e.g.

post = { ... }
data = post2["params"][0]["data"]

then loop through the file using

with open(...) as file:
    for line in file:
      line = line.strip()[:-1] # strip while-space and the trailing comma
      # NB: here I'm assuming it's well-formed JSON, which it's not at the moment
      d = json.loads(line)
      data[0] = d
      # TODO - POST call here...

How to read from a file and put the contents into a json method in Python?

3 Answers3