Subtracting value from data frame column: SettingWithCopyWarning

Question

I have a data-frame df:

                    A           B           C
Date    
24/03/2014  -0.114726   -0.076779   -0.012105
25/03/2014  -0.118673   -0.078756   -0.008158
26/03/2014  -0.132919   -0.078067    0.006088
27/03/2014  -0.153581   -0.068223    0.026750
28/03/2014  -0.167744   -0.063045    0.040913
31/03/2014  -0.167399   -0.067346   -0.040568
01/04/2014  -0.166249   -0.068801    0.039418
02/04/2014  -0.160876   -0.077259    0.034045
03/04/2014  -0.156089   -0.090062    0.029258
04/04/2014  -0.161735   -0.079317   -0.034904
07/04/2014  -0.148305   -0.080767    0.021474
08/04/2014  -0.150812   -0.074792    0.023981
09/04/2014  -0.135339   -0.079736    0.008508
10/04/2014  -0.156345   -0.083574    0.029514

I am selecting the last value of column B using:

testValue=df['B'].iloc[-1]

in this case testValue equals -0.083574

Then I am trying to create a new column 'D' into which I subtract testValue from all elements of column 'C'. So my desired output would look like:

                   A            B           C           D
    Date    
24/03/2014  -0.114726   -0.076779   -0.012105   -0.071469
25/03/2014  -0.118673   -0.078756   -0.008158   -0.075416
26/03/2014  -0.132919   -0.078067    0.006088   -0.089662
27/03/2014  -0.153581   -0.068223    0.026750   -0.110324
28/03/2014  -0.167744   -0.063045    0.040913   -0.124487
31/03/2014  -0.167399   -0.067346   -0.040568   -0.043006
01/04/2014  -0.166249   -0.068801    0.039418   -0.122992
02/04/2014  -0.160876   -0.077259    0.034045   -0.117619
03/04/2014  -0.156089   -0.090062    0.029258   -0.112832
04/04/2014  -0.161735   -0.079317   -0.034904   -0.048670
07/04/2014  -0.148305   -0.080767    0.021474   -0.105048
08/04/2014  -0.150812   -0.074792    0.023981   -0.107555
09/04/2014  -0.135339   -0.079736    0.008508   -0.092082
10/04/2014  -0.156345   -0.083574    0.029514   -0.113088

to subtract the testValue from column 'C' I am using:

df['D']=testValue-df['C']

I however get the following copy warning:

SettingWithCopyWarning: 
A value is trying to be set on a copy of a slice from a DataFrame.
Try using .loc[row_indexer,col_indexer] = value instead

See the caveats in the documentation: http://pandas.pydata.org/pandas-docs/stable/indexing.html#indexing-view-versus-copy
  df['D']=testValue-df['C']

Why do I get the copy warning, and how can I fix it?

curiously, I did not have this warning. May I ask you about your pandas version? mine is `0.25.1`. — antoine, Jan 19 '21 at 08:12
see [post](https://stackoverflow.com/questions/20625582/how-to-deal-with-settingwithcopywarning-in-pandas) — Ehsan Tabatabaei, Jan 19 '21 at 09:05

jezrael · Accepted Answer · 2021-01-19T08:06:58.793

You can change selecting last values by DataFrame.loc:

df['D']=df.loc[df.index[-1], 'B']-df['C']

Or by DataFrame.iloc with position by B column by Index.get_loc:

df['D']=df.iloc[-1, df.columns.get_loc('B')]-df['C']

If still error, you can check how is created df, if there is some filtration need DataFrame.copy:

df = df1[df1['C'] > -0.3].copy()

df['D']=df.loc[df.index[-1], 'B']-df['C']

If you modify values in df later you will find that the modifications do not propagate back to the original data (df1), and that Pandas does warning.

score 1 · Answer 2 · answered Jan 19 '21 at 08:48

The warning that you receive suggests that you are setting on a copy of a certain original data. As far as your data showed, there is no such a copy operation, especially in the line that you doubted: df['D']=testValue-df['C'].

I would bet that your df is a subset of something else. If that is the case, you can fix this by explicityly add .copy()at the end of the filter operation.

As an example, if we create a dffrom scratch, e.g. df = pd.DataFrame(data), we will not have your warning.

Now, if another dataframe is filter from df, and we do the same operation, we will have this warning.

Finally, by adding .copy(), after filter, we will not have the warning anymore.

Hope this helps. Kr

score -1 · Answer 3 · answered Jan 19 '21 at 08:04

-1

If other solutions didn't work, this warning can be ignored using:

import pandas as pd
pd.options.mode.chained_assignment = None

Run above spinnet once and no longer will receive SettingWithCopyWarning

answered Jan 19 '21 at 08:04

Ehsan Tabatabaei

143
7

Subtracting value from data frame column: SettingWithCopyWarning

3 Answers3