1

I have a pandas dataframe called results that contain 2 columns, one called years containing lists of years and another called line_items containing lists of integers. I am trying to sort the list values in a copy of the original dataframe, called dfcopy but so that it DOESNT CHANGE THE ORIGINAL DATAFRAME. The two reproducible dfs are as such:

results = {'years': {0: ['2020', '2021'], 1: ['2019', '2020'], 2: ['2020', '2021'], 3: ['2019', '2020'], 4: ['2019', '2020']}, 'line_items': {0: ['3818', '4121'], 1: ['105', '154'], 2: ['1030', '942'], 3: ['4681', '4849'], 4: ['3439', '3656']}}

dfcopy = results[['years', 'line_items']].copy()

I am using an applymap on dfcopy to sort the lists which IS changing results. This to me makes no sense at all as both my understanding and the docs for .copy() suggest that this does not change the original dataframe. Here is my code:

dfcopy.applymap(lambda x: x.sort())

Can anyone help me to understand why this is happening. (PS I know I could just do results.copy() but this should not change the functionality of the .copy() function)

geds133
  • 1,503
  • 5
  • 20
  • 52

1 Answers1

0

Because lambda x: x.sort() mutates the list objects in-place.

And .copy does not copy the python objects that happen to be in the dataframe, this won't even happen if you pass deep=True to df.copy.

In general, pandas is not designed around having lists inside your dataframe.

juanpa.arrivillaga
  • 88,713
  • 10
  • 131
  • 172
  • I'm confused as to what `.copy()` is for is that case? – geds133 Sep 01 '21 at 21:42
  • @geds133 it *copies the dataframe*. `copy(deep=False)` will not copy the underlying buffers, but `copy(deep=True)` does. However, even if you copy the underlying buffers, when you put a list in a dataframe, the dtype *must be `object`*. Which basically just means pointers to python objects. Really, you probably don't want to be using lists inside dataframes – juanpa.arrivillaga Sep 01 '21 at 22:02
  • @geds133 anyway, you could just do `dfcopy.applymap(sorted)` – juanpa.arrivillaga Sep 01 '21 at 22:02