avro schema timestamp format

Question

I am looking to get the timestamp in this format: MMDDYYYYHHMMSS

For avro schema format, can I use:

{
 "logicalType": "timestamp-millis"
 "type": "long",
 "date-format": "MMDDYYYYHHMMSS"
}

Or is there a better way to do this?

Answer 1

Based on your tag, it looks like you are using fastavro. By default, fastavro will decode a timestamp-millis into a datetime object. Given a datetime object, you can use the strftime function to convert it to the format you want.

If instead you wanted it to automatically decode to a string with the format you specified, you would need to patch the current decoder to do what you want. Here's an example script that would do that:

import io
from datetime import datetime

import fastavro
from fastavro import writer, reader


original_decoder = fastavro.read.LOGICAL_READERS["long-timestamp-millis"]

def decode_as_custom_string(data, *args):
    dt = original_decoder(data, *args)
    return dt.strftime("%m%d%Y%H%M%S")

fastavro.read.LOGICAL_READERS["long-timestamp-millis"] = decode_as_custom_string


writer_schema = fastavro.parse_schema({
    "type": "record",
    "name": "root",
    "fields": [
        {
            "name": "some_date",
            "type": [
                "null",
                {
                    "type": "long",
                    "logicalType": "timestamp-millis",
                },
            ],
        },
    ]
})

records = [
    {"some_date": datetime.now()}
]

bio = io.BytesIO()

writer(bio, writer_schema, records)

bio.seek(0)

for record in reader(bio):
    print(record)