What is the best way to save and load mixture data in python?

Question

I have mixture data in python, which consist of

1. some list of dictionary and
2. some numpy array.

I can save first part as json, and second part as binary npz. But which format should i prefer for save both of them in one file?

Make it clear.

I have something like this:

import numpy as np
params = {
    "param1": 1,
    "param2": "abcd",
    "param3": [1, 2, 3],
    "param4": {
        "subparam1": "e",
        "subparam2": 3
    }
}
data = np.random.uniform(0, 10, [3, 3])

and try to save params and data to file(s)

def save(params, params_path, data, data_path):
    with open(params_path, 'w') as f:
        json.dump(params, f, indent=4)
    np.savez(data_path, data=data)

and I have two files now, but i want only one.

I can do something like this:

def save_union(params, data, out_path):
    data_npz = io.BytesIO()
    np.savez_compressed(data_npz, data=data)
    union_data = {
        "params": params,
        "data": base64.b64encode(data_npz.getvalue()).decode("utf-8")
    }
    with open(out_path, 'w') as f:
        json.dump(union_data, f, indent=4)

but now i have big file because data coded in base64.

Answer 1

Do you need the saved data to be human-readable? If not, you can try Pickle https://docs.python.org/3/library/pickle.html

For examples please see the related question: How can I use pickle to save a dict?

(Example code from @Blender in above link)

import pickle

a = {'hello': 'world'}

with open('filename.pickle', 'wb') as handle:
    pickle.dump(a, handle, protocol=pickle.HIGHEST_PROTOCOL)

with open('filename.pickle', 'rb') as handle:
    b = pickle.load(handle)

print(a == b)