How can you catch a RequestFailedException if making multiple DownloadToAsync calls in parallel?

Question

I've got some code like this:

public async Task<IEnumerable<Response>> SaveFiles(IEnumerable<string> filePaths)
{
    var baseFolder = "C:\\Temp";
    var fullTempPath = $"{baseFolder}\\{Guid.NewGuid()}";
    var fileDirectory = Directory.CreateDirectory(fullTempPath);

    Task<Response>[] tasks = new Task<Response>[filePaths.Count()];

    for (int i = 0; i < filePaths.Count(); i++)
    {
        var blobClientDetails = GetBlobClientDetails(filePaths.ElementAt(i));
        var credentials = new AzureSasCredential(blobClientDetails.Value);
        var blob = new BlobClient(blobClientDetails.Key, credentials);
        tasks[i] = blob.DownloadToAsync(fileDirectory.FullName);
    }

    return await Task.WhenAll(tasks);
}

private KeyValuePair<Uri, string> GetBlobClientDetails(string filePath)
{
    var filePathExcludingSAS = filePath.Substring(0, filePath.IndexOf('?'));
    var sasToken = filePath.Substring(filePath.IndexOf('?') + 1);

    return new KeyValuePair<Uri, string>(new Uri(filePathExcludingSAS), sasToken);
}

The theory is that this will trigger the download of each file before waiting for the download of the previous one to complete, hence the Task.WhenAll(tasks) at the end. However, I want to be able to accurately catch an exception and be able to specify which file failed to download. How can I do that?

Is this helpful at all? [I want await to throw AggregateException, not just the first Exception](https://stackoverflow.com/questions/18314961/i-want-await-to-throw-aggregateexception-not-just-the-first-exception) — Theodor Zoulias, Jun 23 '21 at 15:58
This seems to suggest that AggregateException would catch all and then return just the first error. I was hoping there would be some way to wrap Task.WhenAll() in a try catch and the RequestFailedException would be caught and I could somehow identify the file from the exception, but it looks like the exception only returns a error code that might be of use. — sr28, Jun 23 '21 at 16:16
Would you be interested in a solution that wraps the `RequestFailedException` in some custom `Exception`-derived type, that has a `FilePath` property, and exposes the original `RequestFailedException` as its `InnerException` property? — Theodor Zoulias, Jun 23 '21 at 16:25

Theodor Zoulias · Answer 1 · 2021-06-24T09:55:06.870

The LINQ Select operator makes it easy to project native tasks to custom tasks, that have some extra functionality. In this case the extra functionality is wrapping the RequestFailedExceptions in a custom exception type:

public async Task<Response[]> SaveFiles(IEnumerable<string> filePaths)
{
    var baseFolder = "C:\\Temp";
    var fullTempPath = $"{baseFolder}\\{Guid.NewGuid()}";
    var fileDirectory = Directory.CreateDirectory(fullTempPath);

    Task<Response>[] tasks = filePaths.Select(async filePath =>
    {
        var blobClientDetails = GetBlobClientDetails(filePath);
        var credentials = new AzureSasCredential(blobClientDetails.Value);
        var blob = new BlobClient(blobClientDetails.Key, credentials);
        try
        {
            return await blob.DownloadToAsync(fileDirectory.FullName);
        }
        catch (RequestFailedException ex)
        {
            throw new FileDownloadException(ex.Message, ex) { FilePath = filePath };
        }
    }).ToArray();

    Task<Response[]> whenAll = Task.WhenAll(tasks);
    try { return await whenAll; }
    catch { whenAll.Wait(); throw; } // Propagate AggregateException
}

The FileDownloadException class:

public class FileDownloadException : Exception
{
    public string FilePath { get; init; }
    public FileDownloadException(string message, Exception innerException)
        : base(message, innerException) { }
}

You could then consume the SaveFiles method like this:

try
{
    Response[] responses = await SaveFiles(filePaths);
}
catch (AggregateException aex)
{
    foreach (var ex in aex.InnerExceptions)
    {
        if (ex is FileDownloadException fileEx)
        {
            // Our special exception wrapper
            Console.WriteLine($"{fileEx.FilePath} => {fileEx.InnerException.Message}");
        }
        else
        {
            // Something else happened (not a RequestFailedException)
            Console.WriteLine($"{ex.Message}");
        }
    }
}

I was just working on something similar, however I opted for a DownloadResult object that returned a success / failure, with content and error if there was one. I'll post what I have as it may not work as I expect. — sr28, Jun 23 '21 at 16:58
@sr28 you could also store the `FilePath` information directly in the the `RequestFailedException` instance, using the [`Exception.Data`](https://learn.microsoft.com/en-us/dotnet/api/system.exception.data) property. This way it would not be as structured and type-safe, but you would write less code. — Theodor Zoulias, Jun 23 '21 at 17:06
Yes, that might be an option. I've taken the idea of using LINQ Select but splitting the actions. Take a look at my answer and see what you think. — sr28, Jun 23 '21 at 17:07
@sr28 it's OK. Whether you use anonymous lambdas or named functions is a matter of personal taste. — Theodor Zoulias, Jun 23 '21 at 17:10

score 1 · Accepted Answer · answered Jun 23 '21 at 17:06

Thanks to Theodor for your help thus far. I've created something like this, which is currently untested and I wonder if it will be truly async.

public async Task<IEnumerable<DownloadResult>> SaveFiles(IEnumerable<string> filePaths)
{
    var baseFolder = ConfigurationManager.AppSettings["tempFolderLocation"];
    var fullTempPath = $"{baseFolder}\\{Guid.NewGuid()}";
    var fileDirectory = Directory.CreateDirectory(fullTempPath);
    var tasks = filePaths.Select(fp => DownloadFile(fp, fileDirectory.FullName));
    return await Task.WhenAll(tasks);
}

I've added a 'DownloadFile' method that now does it for each file and returns a task:

private async Task<DownloadResult> DownloadFile(string filePath, string directory)
{
    var blobClientDetails = GetBlobClientDetails(filePath);
    var credentials = new AzureSasCredential(blobClientDetails.Value);
    var blob = new BlobClient(blobClientDetails.Key, credentials);

    try
    {
        var response = await blob.DownloadToAsync(directory);
        return new DownloadResult(true, response);
    }
    catch (RequestFailedException ex)
    {
        // do some additional logging of the exception.
        return new DownloadResult(false, null, ex.Message);
    }
}

Here's DownloadResult:

public class DownloadResult
{
    public DownloadResult(bool isSuccessful, Response response, string errorMessage = "")
    {
        IsSuccessful = isSuccessful;
        Response = response;
        ErrorMessage = errorMessage;
    }

    public bool IsSuccessful { get; set; }
    public Response Response { get; set; }
    public string ErrorMessage { get; set; }
}

@TheodorZoulias My only concern is would this still download the files in parallel or would the await in the DownloadFile method mean it would only download 1 file at a time. — sr28, Jun 23 '21 at 17:10
All the files will be downloaded concurrently. That's the magic of async/await. It is composable. — Theodor Zoulias, Jun 23 '21 at 17:12

How can you catch a RequestFailedException if making multiple DownloadToAsync calls in parallel?

2 Answers2

Linked