`dropdf[df.columnName != 'Value']` Not Working?

Question

Why, when running the subsequent code, on the following table, do Unknown values still show in the EVENT_TIME column?

Table:

EVENT ID  EVENT_DATE EVENT_TIME 
85        2014/06/19    Unknown
86        2014/07/09   16.54.00
87        2014/07/10   19.24.00
88        2014/07/13   19.45.00 
89        2014/07/15   16.15.00
91        2015/04/20   16.55.00 
92        2015/06/25   12.48.00
93        2015/07/13    Unknown 
94        2015/07/14   17.30.00 
95        2015/07/16   18.05.00 
96        2015/07/29   17.57.00

df.dropna(inplace=True) #get rid of blanks

df[df.EVENT_TIME != 'Unknown'] #get rid of "Unknown" values

print(df)

Results:

122     3:00:00 PM
123     4:00:00 PM
124     3:45:00 PM
125     4:45:00 PM
126        Unknown
127     4:55:00 PM
128     2:05:00 PM
129     4:15:00 PM
130    11:04:09 PM
131     9:05:00 PM
132     3:20:00 PM
133    11:20:00 PM
134     2:00:00 PM
135     5:44:24 PM
136        Unknown
137     2:35:00 PM
138     3:30:00 PM

Thank you

edesz · Accepted Answer · 2019-04-22T02:55:59.600

1

In your step to get rid of Unknown values, you need to assign the resulting filtered DataFrame to a new variable

df = df[df.EVENT_TIME != 'Unknown'] # alternative: df = df.query("EVENT_TIME!='Unknown'")

You can also do this with df.drop, using inplace=True (see this SO post)

df.drop(df.loc[df['EVENT_TIME']=='Unknown'].index, inplace=True)

edited Apr 22 '19 at 02:55

answered Apr 22 '19 at 02:50

edesz

11,756
22
75
123

Awesome. Thank you so much for the solution and the additional insights. You too @Bugbeeb! – SeaDude Apr 24 '19 at 02:24

score 1 · Answer 2 · answered Apr 22 '19 at 02:55

1

As @edesz says, you are creating a copy of the dataframe by slicing, so you must reassign the object. If you are going to keep manipulating this new copy, call .copy() after the slice.

answered Apr 22 '19 at 02:55

Bugbeeb

2,021
1
9
26

`dropdf[df.columnName != 'Value']` Not Working?

2 Answers2