Pandas replace works on some values of a column does not on others

Question

I need to replace some column of a dataframe with NA's. I am able to do so partially however, some values in a column are not converted to NA's.

Here is an example dataframe you can work on:


df = pd.DataFrame({'away_score': {672: 2.0,
  673: 1.0,
  674: 2.0,
  675: 2.0,
  676: 1.0,
  677: 1.0,
  678: 2.0,
  679: 1.0,
  680: 1.0,
  681: 2.0},
 'home_score': {672: 2.0,
  673: 2.0,
  674: 3.0,
  675: 0.0,
  676: 0.0,
  677: 2.0,
  678: 2.0,
  679: 1.0,
  680: 2.0,
  681: 2.0},
 'match_id': {672: 273236,
  673: 273234,
  674: 273239,
  675: 273231,
  676: 273232,
  677: 273238,
  678: 273237,
  679: 273240,
  680: 273233,
  681: 273235},
 'match_status': {672: 'Finished',
  673: 'Finished',
  674: 'Finished',
  675: 'Finished',
  676: 'Finished',
  677: 'Finished',
  678: 'Finished',
  679: 'Finished',
  680: 'Finished',
  681: 'Finished'}})

My code so far:


columns_to_fillna = ["match_status","home_score","away_score"]

for column in columns_to_fillna:
    df[column] = df[column].apply(lambda row: df[column].replace(row,np.nan))

This somehow changes some rows for home_score and away_score column and some rows remain the same. What could be the issue here?

there is no NA in the example provided, also how about `df.fillna()` ? — anky, Dec 26 '19 at 11:42
@anky_91 I'm trying to convert the values to NA's not the other way around — Mine, Dec 26 '19 at 11:43
so `df.loc[:,["match_status","home_score","away_score"]]=np.nan` — anky, Dec 26 '19 at 11:44
@anky_91 This worked like a charm thank you! But why doesn't it work with replace I'm very curious ? — Mine, Dec 26 '19 at 11:46

Dishin H Goyani · Accepted Answer · 2019-12-26T12:38:38.393

>>> columns_to_fillna = ["match_status","home_score","away_score"]

>>> df.replace(df[columns_to_fillna], np.nan)

     away_score  home_score  match_id  match_status
672         NaN         NaN    273236           NaN
673         NaN         NaN    273234           NaN
674         NaN         NaN    273239           NaN
675         NaN         NaN    273231           NaN
676         NaN         NaN    273232           NaN
677         NaN         NaN    273238           NaN
678         NaN         NaN    273237           NaN
679         NaN         NaN    273240           NaN
680         NaN         NaN    273233           NaN
681         NaN         NaN    273235           NaN

Pandas replace works on some values of a column does not on others

1 Answers1