0

I have bunch of csv files with different time/date stamps of different formats (I do not know ahead of time the format).

Few of the formats are:

09/30/2013 12:00:00 AM
12:00:00 AM 09/30/2013
09/30/2013
12:00:00 AM
12AM 2013

etc

When I read csv file using pandas it simply assigns "object" time to all of these columns of different date time formats.

Is there any way to identify datetime format for various formats and convert those into pandas-friendly format so I can set pandas column type to datetime format instead of object?

YohanRoth
  • 3,153
  • 4
  • 31
  • 58
  • Possible duplicate of [datetime dtypes in pandas read\_csv](https://stackoverflow.com/questions/21269399/datetime-dtypes-in-pandas-read-csv) – roganjosh Sep 06 '19 at 16:58
  • 2
    Essentially, you can set the parser up yourself. However, `12:00:00 AM` cannot ever be a sane datetime from what I can see, nor can `12AM 2013` – roganjosh Sep 06 '19 at 16:59
  • I think it's a little bit more complicated than that. `pandas` can only automatically parse a select few specific formats. Outside of that you need to supply an exact format, and it will only parse when it's an exact match. With multiple formats, you need to loop over the formats and `combine_first`. There's a considerable performance hit when you let it try to infer the format too. – ALollz Sep 06 '19 at 17:00
  • I.e. see something like https://stackoverflow.com/questions/55463652/how-to-detect-dates-and-convert-to-datetime64-data-type – ALollz Sep 06 '19 at 17:02
  • 1
    @ALollz sure, that's why I used "sane" :) I cannot really think of how that would be desirable unless the interest was solely on the time of day, and just disregards the rest of datetime. – roganjosh Sep 06 '19 at 17:09
  • @ALollz although, actually, I didn't know that the date would default to today for orphaned timestamps. This is a break with the standard `datetime` behaviour of Python, which will default to year 1900. Do you know if there's a specific reason for that split in defaults? – roganjosh Sep 06 '19 at 17:16
  • nm, silly question, I guess it's because the year portion is supplied – roganjosh Sep 06 '19 at 17:18

0 Answers0