Convert Pandas Series to DateTime in a DataFrame

Question

I have a Pandas DataFrame as below

        ReviewID       ID      Type               TimeReviewed
205     76032930  51936827  ReportID 2015-01-15 00:05:27.513000
232     76032930  51936854  ReportID 2015-01-15 00:06:46.703000
233     76032930  51936855  ReportID 2015-01-15 00:06:56.707000
413     76032930  51937035  ReportID 2015-01-15 00:14:24.957000
565     76032930  51937188  ReportID 2015-01-15 00:23:07.220000

>>> type(df)
<class 'pandas.core.frame.DataFrame'>

TimeReviewed is a series type

>>> type(df.TimeReviewed)
<class 'pandas.core.series.Series'>

I've tried below, but it still doesn't change the Series type

import pandas as pd
review = pd.to_datetime(pd.Series(df.TimeReviewed))
>>> type(review)
<class 'pandas.core.series.Series'>

How can I change the df.TimeReviewed to DateTime type and pull out year, month, day, hour, min, sec separately? I'm kinda new to python, thanks for your help.

score 80 · Accepted Answer · edited Feb 18 '21 at 17:23

You can't: DataFrame columns are Series, by definition. That said, if you make the dtype (the type of all the elements) datetime-like, then you can access the quantities you want via the .dt accessor (docs):

>>> df["TimeReviewed"] = pd.to_datetime(df["TimeReviewed"])
>>> df["TimeReviewed"]
205  76032930   2015-01-24 00:05:27.513000
232  76032930   2015-01-24 00:06:46.703000
233  76032930   2015-01-24 00:06:56.707000
413  76032930   2015-01-24 00:14:24.957000
565  76032930   2015-01-24 00:23:07.220000
Name: TimeReviewed, dtype: datetime64[ns]
>>> df["TimeReviewed"].dt
<pandas.tseries.common.DatetimeProperties object at 0xb10da60c>
>>> df["TimeReviewed"].dt.year
205  76032930    2015
232  76032930    2015
233  76032930    2015
413  76032930    2015
565  76032930    2015
dtype: int64
>>> df["TimeReviewed"].dt.month
205  76032930    1
232  76032930    1
233  76032930    1
413  76032930    1
565  76032930    1
dtype: int64
>>> df["TimeReviewed"].dt.minute
205  76032930     5
232  76032930     6
233  76032930     6
413  76032930    14
565  76032930    23
dtype: int64

If you're stuck using an older version of pandas, you can always access the various elements manually (again, after converting it to a datetime-dtyped Series). It'll be slower, but sometimes that isn't an issue:

>>> df["TimeReviewed"].apply(lambda x: x.year)
205  76032930    2015
232  76032930    2015
233  76032930    2015
413  76032930    2015
565  76032930    2015
Name: TimeReviewed, dtype: int64

I couldn't use `.dt` it give me an error: `AttributeError: 'Series' object has no attribute 'dt'` — 1EnemyLeft, Jan 25 '15 at 03:59
@user3596895: you're probably using an older version of pandas. What does `print(pd.version.version)` give? — DSM, Jan 25 '15 at 04:02
I see, I'm using visual studio python tool import pandas, will look up how to update my pandas version, thanks! — 1EnemyLeft, Jan 25 '15 at 04:10

score 4 · Answer 2 · edited Jun 19 '19 at 06:00

4

df=pd.read_csv("filename.csv" , parse_dates=["<column name>"])

type(df.<column name>)

example: if you want to convert day which is initially a string to a Timestamp in Pandas

df=pd.read_csv("weather_data2.csv" , parse_dates=["day"])

type(df.day)

The output will be pandas.tslib.Timestamp

edited Jun 19 '19 at 06:00

NOhs

2,780
3
25
59

answered Jun 19 '19 at 05:38

Shashwat Yadav

41
2

Hi, what if I have to parse the dates for two columns? Nothing I tried seems to work. – Arti Agarwal Nov 18 '20 at 14:36

score 2 · Answer 3 · answered May 04 '16 at 06:43

2

Some handy script:

hour = df['assess_time'].dt.hour.values[0]

answered May 04 '16 at 06:43

CodeFarmer

2,644
1
23
32

Convert Pandas Series to DateTime in a DataFrame

3 Answers3

Linked

Related