How to specify blank space in pandas

Question

I have dataframe as below with blank space highlighted by blue line.
What I want to do is that if there is blank place like this, I want to fill in using fillna method. However, this method does'nt work because blank space is not recognized as na.
I tried to convert blank to na value while the data is imported using pd.read_csv(csv,na_values=""), but didn't work.

Does anyone know how I can apply fillna method to this dataframe?

Answer 1

I think you need replace one or more whitespaces with ^ for match start of string and $ for end of strings, because empty strings in read_csv are converted to NaNs:

df = pd.DataFrame({
    'A': ['a','a','a','  ','b','d  ','b','c','  dd'],
    'B': list(range(9))
})


df1 = df.replace('^\s+$', np.nan, regex=True)
print (df1)
      A  B
0     a  0
1     a  1
2     a  2
3   NaN  3
4     b  4
5   d    5
6     b  6
7     c  7
8    dd  8

If omit ^ and & then are replaced all whitepaces:

df2 = df.replace(r'\s+', np.nan, regex=True)
print (df2)
     A  B
0    a  0
1    a  1
2    a  2
3  NaN  3
4    b  4
5  NaN  5
6    b  6
7    c  7
8  NaN  8