Pandas select date_range with empty rows does not work

Question

in a pandas data frame I have a column with dates and empty values like that

15    2018-04-13 13:26:54 UTC
16                           
    ...
28                           
29    2018-05-15 00:00:00 UTC
30                           
    ...
40                           
41                           
42    2018-03-24 20:32:36 UTC
    ...
46    2018-04-10 20:41:39 UTC
47                           
48                           
49    2018-01-26 20:30:22 UTC
    ....
58   2017-05-30 09:26:04 UTC
59   2010-09-09 14:09:03 UTC

and I am searching for values empty and in a date range. Unfortunately nothing like that worked

df[df['date_column'].loc['2017-01-01':'2018-01-01']]
df['date_column']isin(pd.date_range('two_months', periods=2, freq='M'))
df[df['date_column'].str.contains(regex_filters_date)]

How would I correctly select dates within a given range ?

BENY · Accepted Answer · 2018-05-17T14:24:06.317

1

For example you have following data frame

df=pd.DataFrame({'Date':['2018-03-24 20:32:36 UTC','','2018-01-26 20:30:22 UTC','']})
s=pd.to_datetime(df.Date)
df[(s>pd.to_datetime('2018-02-01'))&(s<pd.to_datetime('2018-04-01'))]
                      Date
0  2018-03-24 20:32:36 UTC

If you want empty selected

df[((s > pd.to_datetime('2018-02-01')) & (s < pd.to_datetime('2018-04-01')))|s.isnull()]
Out[831]: 
                      Date
0  2018-03-24 20:32:36 UTC
1                         
3

edited May 17 '18 at 14:24

answered May 17 '18 at 14:10

BENY

317,841
20
164
234

that does purge the empty values but works fine with the date range – user3732793 May 17 '18 at 14:22
@user3732793 yw : -) happy coding – BENY May 17 '18 at 14:33

score 0 · Answer 2 · answered May 17 '18 at 14:22

0

My preferred method of specifying a date range in pandas is to use a Boolean Mask, however there are other methods using tools such as the DatetimeIndex class.

Here is some documentation from an earlier thread I think you would find useful!

Using a boolean mask, your solution would look something like:

mask = (df['date_column'] > '2017-01-01') & (df['date_column'] <= '2018-01-01')
df = df.loc[[mask]]

answered May 17 '18 at 14:22

RobertMonaco

81
5

1

looks promising but gives me back "too many values to unpack" for the column and nothing for the whole df, thanks for the link – user3732793 May 17 '18 at 14:26

Pandas select date_range with empty rows does not work

2 Answers2