Download files from FTP server in C# to local folder with modification date greater than specified

Question

I work on C# web application and need to download files using FTP to local folder. Those images need to have modification date greater than date I specify.

Code:

public static List<FTPLineResult> GetFilesListSortedByDate(string ftpPath, Regex nameRegex, DateTime cutoff, System.Security.Cryptography.X509Certificates.X509Certificate cert)
{
    List<FTPLineResult> output = new List<FTPLineResult>();

    if (cert != null)
    {
        FtpWebRequest request = FtpWebRequest.Create(ftpPath) as FtpWebRequest;
        request.Credentials = new NetworkCredential("unm", "pwd");
        request.ClientCertificates.Add(cert);

        ConfigureProxy(request);
        request.Method = WebRequestMethods.Ftp.ListDirectoryDetails;
        FtpWebResponse response = request.GetResponse() as FtpWebResponse;
        StreamReader directoryReader = new StreamReader(response.GetResponseStream(), System.Text.Encoding.ASCII);
        var parser = new FTPLineParser();
        while (!directoryReader.EndOfStream)
        {
            var result = parser.Parse(directoryReader.ReadLine());
            if (!result.IsDirectory && result.DateTime > cutoff && nameRegex.IsMatch(result.Name))
            {
                output.Add(result);
            }
        }
        // need to ensure the files are sorted in ascending date order
        output.Sort(
            new Comparison<FTPLineResult>(
                delegate(FTPLineResult res1, FTPLineResult res2)
                {
                    return res1.DateTime.CompareTo(res2.DateTime);
                }
            )
        );
    }

    return output;
}

I have to use certificate (.p12).
How can I do this?

Answer 1

You have to retrieve timestamps of remote files to select those you want.

Unfortunately, there's no really reliable and efficient way to retrieve timestamps using features offered by .NET framework as it does not support FTP MLSD command. The MLSD command provides listing of remote directory in a standardized machine-readable format. The command and the format is standardized by the RFC 3659.

Alternatives you can use, that are supported by the .NET framework:

• the ListDirectoryDetails method (FTP LIST command) to retrieve details of all files in a directory and then you deal with FTP server specific format of the details

  *nix format: Parsing FtpWebRequest ListDirectoryDetails line DOS/Windows format: C# class to parse WebRequestMethods.Ftp.ListDirectoryDetails FTP response

• the GetDateTimestamp method (FTP MDTM command) to individually retrieve timestamps for each file. Advantage is that the response is standardized by RFC 3659 to YYYYMMDDHHMMSS[.sss]. Disadvantage is that you have to send a separate request for each file, what can be quite inefficient.

  const string uri = "ftp://example.com/remote/path/file.txt";
  FtpWebRequest request = (FtpWebRequest)WebRequest.Create(uri);
  request.Method = WebRequestMethods.Ftp.GetDateTimestamp;
  FtpWebResponse response = (FtpWebResponse)request.GetResponse();
  Console.WriteLine("{0} {1}", uri, response.LastModified);
  

---

Alternatively you can use a 3rd party FTP client implementation that supports the modern MLSD command or that can directly download files given time constraint.

For example WinSCP .NET assembly supports both MLSD and time constraints.

There's even an example for your specific task: How do I transfer new/modified files only?
The example is for PowerShell, but translates to C# easily:

// Setup session options
SessionOptions sessionOptions = new SessionOptions
{
    Protocol = Protocol.Ftp,
    HostName = "ftp.example.com",
    UserName = "username",
    Password = "password",
};

using (Session session = new Session())
{
    // Connect
    session.Open(sessionOptions);

    // Download files created in 2017-06-15 and later
    TransferOptions transferOptions = new TransferOptions();
    transferOptions.FileMask = "*>=2017-06-15";
    session.GetFiles(
        "/remote/path/*", @"C:\local\path\", false, transferOptions).Check();
}

---

Though for web application, WinSCP is probably not the best solution. You maybe able to find another 3rd party library with similar functionality.

---

WinSCP also supports authentication with a client certificate. See SessionOptions.TlsClientCertificatePath. But that's really for a separate question.

(I'm the author of WinSCP)