Pandas DataFrame to List of Dictionaries

Question

I have the following DataFrame:

customer    item1      item2    item3
1           apple      milk     tomato
2           water      orange   potato
3           juice      mango    chips

which I want to translate it to list of dictionaries per row

rows = [
    {
        'customer': 1,
        'item1': 'apple',
        'item2': 'milk',
        'item3': 'tomato'
    }, {
        'customer': 2,
        'item1':
        'water',
        'item2': 'orange',
        'item3': 'potato'
    }, {
        'customer': 3,
        'item1': 'juice',
        'item2': 'mango',
        'item3': 'chips'
    }
]

Please consider changing your accepted answer. The currently accepted answer, by ComputerFellow, was later updated to incorporate John Galt (Zero)'s answer - essentially stealing it. For some reason, hundreds of people have upvoted ComputerFellow's REPRODUCTION of John Galt (Zero)'s answer, possibly because it is the accepted answer. Perhaps you did not have opportunity to review the edit history of each answer and see what ComputerFellow did. — cssyphus, Dec 03 '21 at 21:14
Thanks @cssyphus for the comment. Yes, I agree. It's changed as suggested — Mohamad Ibrahim, Dec 04 '21 at 22:30

score 517 · Accepted Answer · edited Dec 03 '15 at 21:38

517

Use df.to_dict('records') -- gives the output without having to transpose externally.

In [2]: df.to_dict('records')
Out[2]:
[{'customer': 1L, 'item1': 'apple', 'item2': 'milk', 'item3': 'tomato'},
 {'customer': 2L, 'item1': 'water', 'item2': 'orange', 'item3': 'potato'},
 {'customer': 3L, 'item1': 'juice', 'item2': 'mango', 'item3': 'chips'}]

edited Dec 03 '15 at 21:38

mirosval

6,671
3
32
46

answered Apr 23 '15 at 07:08

Zero

74,117
18
147
154

5

How would I change it to include the index value into each entry of the resulting list? – Gabriel L. Oliveira Aug 16 '16 at 18:08
11

@GabrielL.Oliveira you can do df.reset_index().to_dict('records') – Wei Ma Feb 02 '17 at 22:28
Is the order of the columns reserved in each case i.e. is the nth entry in the resulting list always also the nth column? – Cleb May 23 '18 at 12:03
@Cleb are ```i.e. is the nth entry in the resulting list always also the nth column?``` nth column or nth row? – Nauman Naeem Nov 20 '19 at 13:03
3

Is `records` some kind of magic column name? Are there other magic columns? – WestCoastProjects Sep 07 '20 at 16:18
1

@StephenBoesch See the documentation: https://pandas.pydata.org/pandas-docs/stable/reference/api/pandas.DataFrame.to_dict.html 'Records' is a str arg to orientation of the result. – Cain Dec 29 '20 at 21:21
@GabrielL.Oliveira You can also do df.todict('index') to get a nested {index -> {col->value}} set of dicts – Cain Dec 29 '20 at 21:22
@Cain How do you personally go about finding that documention? I use `PyCharm` that is close to worthless in that regard (though invaluable as a debugger and in other IDE functions) – WestCoastProjects Dec 29 '20 at 23:07
I agree that it's a bit "magical" in a less than intuitive way... but it works! – openwonk Oct 20 '22 at 23:57

score 330 · Answer 2 · edited Dec 03 '21 at 22:49

330

Use df.T.to_dict().values(), like below:

In [1]: df
Out[1]:
   customer  item1   item2   item3
0         1  apple    milk  tomato
1         2  water  orange  potato
2         3  juice   mango   chips

In [2]: df.T.to_dict().values()
Out[2]:
[{'customer': 1.0, 'item1': 'apple', 'item2': 'milk', 'item3': 'tomato'},
 {'customer': 2.0, 'item1': 'water', 'item2': 'orange', 'item3': 'potato'},
 {'customer': 3.0, 'item1': 'juice', 'item2': 'mango', 'item3': 'chips'}]

edited Dec 03 '21 at 22:49

Zoe

27,060
21
118
148

answered Apr 23 '15 at 06:36

ComputerFellow

11,710
12
50
61

2

What would be the solution in the case of a dataframe containing for each Customer many rows? – Aziz Dec 03 '16 at 12:21
3

When I use `df.T.to_dict().values()`, I loose the sort order also – Hussain May 02 '17 at 13:03
When opening a csv file to list of dicts, I'm getting twice the speed with `unicodecsv.DictReader` – radtek Mar 09 '18 at 16:01
7

Check the answer below, of @mirosval and @Zero. Best way is to use `df.to_dict('records')` – Corina Roca Mar 11 '21 at 10:23

score 25 · Answer 3 · edited May 23 '17 at 11:47

As an extension to John Galt's answer -

For the following DataFrame,

   customer  item1   item2   item3
0         1  apple    milk  tomato
1         2  water  orange  potato
2         3  juice   mango   chips

If you want to get a list of dictionaries including the index values, you can do something like,

df.to_dict('index')

Which outputs a dictionary of dictionaries where keys of the parent dictionary are index values. In this particular case,

{0: {'customer': 1, 'item1': 'apple', 'item2': 'milk', 'item3': 'tomato'},
 1: {'customer': 2, 'item1': 'water', 'item2': 'orange', 'item3': 'potato'},
 2: {'customer': 3, 'item1': 'juice', 'item2': 'mango', 'item3': 'chips'}}

score 7 · Answer 4 · answered Apr 30 '20 at 16:02

If you are interested in only selecting one column this will work.

df[["item1"]].to_dict("records")

The below will NOT work and produces a TypeError: unsupported type: . I believe this is because it is trying to convert a series to a dict and not a Data Frame to a dict.

df["item1"].to_dict("records")

I had a requirement to only select one column and convert it to a list of dicts with the column name as the key and was stuck on this for a bit so figured I'd share.

score 0 · Answer 5 · answered Aug 03 '21 at 01:08

Also you can iterate over rows:

rows = []
for index, row in df[['customer', 'item1', 'item2', 'item3']].iterrows():
    rows.append({
            'customer': row['customer'],
            'item1': row['item1'],
            'item2': row['item2'],
            'item3': row['item3'],
            })

Pandas DataFrame to List of Dictionaries

5 Answers5

Linked

Related