Given the following pandas DataFrame where some indices are NaN, how to drop the third and eight row since their index is NaN? Thanks
import pandas as pd
import numpy as np
data = list('abcdefghil')
indices = [0, 1, np.nan, 3, 4, 5, 6, np.nan, 8, 9]
df = pd.DataFrame(data, index=indices, columns=['data'])
You can call dropna on the index:
In[68]:
df.loc[df.index.dropna()]
Out[68]:
data
0.0 a
1.0 b
3.0 d
4.0 e
5.0 f
6.0 g
8.0 i
9.0 l
Note that the presence of NaN makes the index dtype float, to change it to int cast the type:
In[70]:
df = df.loc[df.index.dropna()]
df.index = df.index.astype(int)
df
Out[70]:
data
0 a
1 b
3 d
4 e
5 f
6 g
8 i
9 l
You can also call notnull on the index would also work (somehow undocumented)
In[71]:
df = df.loc[df.index.notnull()]
df.index = df.index.astype(int)
df
Out[71]:
data
0 a
1 b
3 d
4 e
5 f
6 g
8 i
9 l
there is also isna:
In[78]:
df.loc[~df.index.isna()]
Out[78]:
data
0.0 a
1.0 b
3.0 d
4.0 e
5.0 f
6.0 g
8.0 i
9.0 l
and the more readable inverse notna:
In[79]:
df.loc[df.index.notna()]
Out[79]:
data
0.0 a
1.0 b
3.0 d
4.0 e
5.0 f
6.0 g
8.0 i
9.0 l
As commented by @jpp you can use the top-level notnull also:
In[80]:
df.loc[pd.notnull(df.index)]
Out[80]:
data
0.0 a
1.0 b
3.0 d
4.0 e
5.0 f
6.0 g
8.0 i
9.0 l
There is also top-level isna, notna, and isnull but I'm not going to display those, you can check the docs
You can use the following:
df = df[df.index.isnull() == False]
You might want to reset the index after
Using np.isnan and taking the negative:
res = df[~np.isnan(df.index)]
print(res)
data
0.0 a
1.0 b
3.0 d
4.0 e
5.0 f
6.0 g
8.0 i
9.0 l
