Python: json response to csv?

Question

I am calling an API (https://min-api.cryptocompare.com/data/histoday?fsym=BTC&tsym=USD&limit=10) that gives the following response:

{"Response":"Success","Type":100,"Aggregated":false,"Data":[{"time":1561852800,"close":10769.05,"high":12200.02,"low":10677.83,"open":11884.1,"volumefrom":80893.36,"volumeto":917158052.37},{"time":1561939200,"close":10591.87,"high":11207,"low":10006.43,"open":10769.05,"volumefrom":115739.97,"volumeto":1225129699.57},{"time":1562025600,"close":10844.13,"high":10927.6,"low":9678.1,"open":10591.87,"volumefrom":120994.95,"volumeto":1239524970.43},{"time":1562112000,"close":11981.61,"high":12009.59,"low":10841.91,"open":10844.13,"volumefrom":115565.16,"volumeto":1313585829.89},{"time":1562198400,"close":11156.52,"high":12055.11,"low":11067.68,"open":11981.61,"volumefrom":71141.03,"volumeto":831236841.56},{"time":1562284800,"close":10993.25,"high":11435.38,"low":10787.94,"open":11156.52,"volumefrom":66066.75,"volumeto":734424868.07},{"time":1562371200,"close":11248.94,"high":11709.27,"low":10985.4,"open":10993.25,"volumefrom":48172.2,"volumeto":549769169.13},{"time":1562457600,"close":11474.28,"high":11605.43,"low":11109.42,"open":11248.94,"volumefrom":36847.21,"volumeto":418161890.29},{"time":1562544000,"close":12296.16,"high":12386.28,"low":11339.02,"open":11474.28,"volumefrom":63847.27,"volumeto":762033323.29},{"time":1562630400,"close":12537.38,"high":12808.06,"low":12117.31,"open":12296.16,"volumefrom":79366.56,"volumeto":990863142.59},{"time":1562716800,"close":12855.54,"high":12855.54,"low":12537.38,"open":12537.38,"volumefrom":0,"volumeto":0}],"TimeTo":1562716800,"TimeFrom":1561852800,"FirstValueInArray":true,"ConversionType":{"type":"direct","conversionSymbol":""},"RateLimit":{},"HasWarning":false}

I need to convert this to a csv with columns for time, close, high, low, open, volumefrom and volumeto per the above. How should this be done? I've tried some existing answers for converting json to csv but they don't work in this case.

Answer 1

It's quite easy to do with pandas:

import pandas as pd
import json

# Parse the json string to a python dictionary
data = json.loads(json_data)

# The desired data is in the `Data` field, use pandas to construct the data frame
df = pd.DataFrame(data["Data"])

# Save to a csv file
df.to_csv("result.csv")

Answer 2

Since you did not mention the use of a file, I simply generated the comma seperated values without the csv library. However, this could also have been done with it.

import requests
import json

r = requests.get("https://min-api.cryptocompare.com/data/histoday?fsym=BTC&tsym=USD&limit=10")

# this is assuming the response has no errors and is status code 200
data = json.loads(r.text)

# in csv the headers are conventionally the first row
csv_data = "time, close, high, low, open"

for row in data["Data"]:

    # since the order is specific, select the values from the row in that order
    row_data = [row["time"], row["close"], row["high"], row["low"], row["open"]]

    # here I am using list comprehension to convert all the data from the row to a string
    # then I use the .join method to concatonate all these values as one comma seperated list
    csv_row = ", ".join([str(j) for j in row_data])

    csv_data += "\n" + csv_row

# here is your data
print(csv_data)

Documentation Links

• requests - A commonly used, friendly wrapper around python's request library.

• List Comprehension - A more compact way of generating lists in python, and also slightly more efficient.

Answer 3

Here you go! This should work for you! Feel free to ask any additional questions, I've tried to keep everything commented and reasonably neat. c:

#!/usr/bin/env python3
import json, csv

# Here is our input string/response. You can replace this!
responseString = '{"Response":"Success","Type":100,"Aggregated":false,"Data":[{"time":1561852800,"close":10769.05,"high":12200.02,"low":10677.83,"open":11884.1,"volumefrom":80893.36,"volumeto":917158052.37},{"time":1561939200,"close":10591.87,"high":11207,"low":10006.43,"open":10769.05,"volumefrom":115739.97,"volumeto":1225129699.57},{"time":1562025600,"close":10844.13,"high":10927.6,"low":9678.1,"open":10591.87,"volumefrom":120994.95,"volumeto":1239524970.43},{"time":1562112000,"close":11981.61,"high":12009.59,"low":10841.91,"open":10844.13,"volumefrom":115565.16,"volumeto":1313585829.89},{"time":1562198400,"close":11156.52,"high":12055.11,"low":11067.68,"open":11981.61,"volumefrom":71141.03,"volumeto":831236841.56},{"time":1562284800,"close":10993.25,"high":11435.38,"low":10787.94,"open":11156.52,"volumefrom":66066.75,"volumeto":734424868.07},{"time":1562371200,"close":11248.94,"high":11709.27,"low":10985.4,"open":10993.25,"volumefrom":48172.2,"volumeto":549769169.13},{"time":1562457600,"close":11474.28,"high":11605.43,"low":11109.42,"open":11248.94,"volumefrom":36847.21,"volumeto":418161890.29},{"time":1562544000,"close":12296.16,"high":12386.28,"low":11339.02,"open":11474.28,"volumefrom":63847.27,"volumeto":762033323.29},{"time":1562630400,"close":12537.38,"high":12808.06,"low":12117.31,"open":12296.16,"volumefrom":79366.56,"volumeto":990863142.59},{"time":1562716800,"close":12855.54,"high":12855.54,"low":12537.38,"open":12537.38,"volumefrom":0,"volumeto":0}],"TimeTo":1562716800,"TimeFrom":1561852800,"FirstValueInArray":true,"ConversionType":{"type":"direct","conversionSymbol":""},"RateLimit":{},"HasWarning":false}'

# Turn the response into JSON and only grab the 'Data' list.
responseString = json.loads(responseString)['Data']

# Open us a new file!
with open('output.csv', 'w') as output_csv_file:
    # Create us a new csv_object using the keys of the data as fieldnames.
    csv_object = csv.DictWriter(output_csv_file, fieldnames=responseString[0].keys())

    # For each row of data in the JSON, print it and write it to the CSV.
    for row in responseString:
        print(row)
        csv_object.writerow(row)

# Automatically close CSV file/object and print "Done!"
print("Done!")