3

I have a data.txt file which I want to convert to a data.json file and print a nice first 2 entries (data.txt contains 3 unique IDs).

The data.txt can oublicly found here (this is a sample - original file contains 10000 unique "linkedin_internal_id).

I tried the following:

with open("data.txt", "r") as f:
    content = f.read()

data = json.dumps(content, indent=3)

This code doesn't print the appropriate JSON format of data.txt (it also includes \\). Also, my jupyter notebook gets stacked because of the large file size, for this, I want to nicely print only the first 2 entries.

Joe
  • 575
  • 6
  • 24

2 Answers2

7

It is called new line delimited json where each line is a valid JSON value and the line separator is '\n', you can read it like this line by line and push it to a list, so later it will be easy for you to iterate/process it further. See: ldjson

import json

with open("data.txt", "r") as f:
    contents = f.read()

data = [json.loads(item) for item in contents.strip().split('\n')]
print(data[0:2])
A l w a y s S u n n y
  • 36,497
  • 8
  • 60
  • 103
  • no need to str(item) here, just item should word – funnydman Aug 21 '22 at 15:53
  • Thanks! I tried running your code but I have this traceback: `JSONDecodeError: Extra data: line 1 column 3575 (char 3574)` – Joe Aug 21 '22 at 16:09
  • seems issue with your large file: https://stackoverflow.com/a/51830719/1138192 but the file you shared earlier works fine for me while testing – A l w a y s S u n n y Aug 21 '22 at 16:13
  • 1
    Now it's working. I saved `data.txt` file as `data.json` first and then used your solution. – Joe Aug 21 '22 at 16:20
  • 1
    I added `print(json.dumps(data[:2], indent=3))` instead of `print(data[0:2])` to nicely print output. – Joe Aug 21 '22 at 16:22
0

Something like this?

import json
with open('data.txt', 'r') as f:
   data = [json.loads(f.readline()) for i in range(2)]
print(json.dumps(data))

This only reads and parses the first two lines of the data file, instead of loading the whole thing and then extracting the first two items.

Mark Reed
  • 91,912
  • 16
  • 138
  • 175