Azure Synapse Pipeline Get Metadata Last Modified Date on file instead of on folder

Question

I am trying to get the Last Modified date on the files (CSV linked service) in my repository using Get Metadata activity.

Field List

However, I am getting the same date/time for all of the files... the LastModified date on the folder instead of the individual files inside.

same LastModifiedDate output example

My design pattern is to Get the list of files and then iterate in a For Each over each item.

Synapse Pipeline design pattern

I am able to get the Filenames with

@activity('Get Metadata').output.childItems

I just don't see a LastModified date in the attributes of the Jason.

This code needs to go a level deeper:

@activity('Get Metadata').output.lastModified

What is the syntax I am missing to access each of the file properties?

Answer 1

To Get Metadata Last Modified Date on file instead of on folder Follow below steps:

• First you need to get the list of files. using Get metadata activity and field list as Child items.

• Then pass this child items to the Foreach activity to loop on that.

• Under this foreach activity take another get metadata activity to get Last modified date for each file. For this activity use parameterized data set with filename. -- Create parameter in Dataset -- Use this parameter in the dataset.

• Then append your required values in Append Variable with expression:

@json(Concat('{"name": "',item().name,'",','"lastModified": "',activity('Get Metadata2').output.lastModified,'"}'))

This will create an array with Json of filename and last modified date of each file the use it further.

Output: