42

Does the .net framework has an async built-in library/assembly which allows to work with the file system (e.g. File.ReadAllBytes, File.WriteAllBytes)?

Or do I have to write my own library using the async Methods of StreamReader and StreamWriter?

Something like this would be pretty nice:

var bytes = await File.ReadAllBytes("my-file.whatever");
i3arnon
  • 113,022
  • 33
  • 324
  • 344
BendEg
  • 20,098
  • 17
  • 57
  • 131

6 Answers6

51

Does the .net framework has an async built-in library/assembly which allows to work with the file system

Yes. There are async methods for working with the file system but not for the helper methods on the static File type. They are on FileStream.

So, there's no File.ReadAllBytesAsync but there's FileStream.ReadAsync, etc. For example:

byte[] result;
using (FileStream stream = File.Open(@"C:\file.txt", FileMode.Open))
{
    result = new byte[stream.Length];
    await stream.ReadAsync(result, 0, (int)stream.Length);
}
i3arnon
  • 113,022
  • 33
  • 324
  • 344
  • If the file you were serializing into a byte stream was large (3gb+) is there a possibilty that the stream length would infact be larger than an int type can store and casue it to overflow? – Dave Jan 16 '18 at 15:33
  • 3
    @Dave probably. In such cases you probably need to read in chunks. – i3arnon Jan 16 '18 at 15:50
  • 3
    You should have logic to read in chunks regardless. According to [Microsoft documentation](https://learn.microsoft.com/en-us/dotnet/api/system.io.filestream.readasync?view=netframework-4.7.1#System_IO_FileStream_ReadAsync_System_Byte___System_Int32_System_Int32_System_Threading_CancellationToken_), ReadAsync is not guaranteed to fill the whole array. You need to check the return value to see how much it _actually_ read. – Phil Feb 22 '18 at 10:58
  • 2
    You may consider changing the code to [something like this](https://pastebin.com/68PrKYJg) (for small files - don't do it like this for 3 GB+ files). – Phil Feb 22 '18 at 11:10
  • 1
    You should really be using `System.IO.StreamReader` to read instead of grabbing raw bytes. It has `ReadLineAsync` and `ReadToEndAsync` methods. – RubberDuck Apr 30 '18 at 16:45
  • There is a race which risks overflow if the file size increases. I suggest reading b.Length bytes in the last line. Also, from here https://stackoverflow.com/questions/2338778/what-is-the-maximum-length-of-an-array-in-net-on-64-bit-windows .NET objects have a limit of 2GB, so you can't read more than that at a time. This helps as b.Length < 2^21 . – vaza Jun 28 '18 at 13:42
21

It already does it. See for example Using Async for File Access MSDN article.

private async Task WriteTextAsync(string filePath, string text)
{
    byte[] encodedText = Encoding.Unicode.GetBytes(text);

    using (FileStream sourceStream = new FileStream(filePath,
        FileMode.Append, FileAccess.Write, FileShare.None,
        bufferSize: 4096, useAsync: true))
    {
        await sourceStream.WriteAsync(encodedText, 0, encodedText.Length);
    };
}
Matías Fidemraizer
  • 63,804
  • 18
  • 124
  • 206
13

Does the .net framework has an async built-in library/assembly which allows to work with the file system (e.g. File.ReadAllBytes, File.WriteAllBytes)?

Unfortunately, the desktop APIs are a bit spotty when it comes to asynchronous file operations. As you noted, a number of the nice convenience methods do not have asynchronous equivalents. Also missing are asynchronous opening of files (which is especially useful when opening files over a network share).

I'm hoping these APIs will be added as the world moves to .NET Core.

Or do I have to write my own library using the async Methods of StreamReader and StreamWriter?

That's the best approach right now.

Note that when using ReadAsync/WriteAsync and friends, you must explicitly open the file for asynchronous access. The only way to do this (currently) is to use a FileStream constructor overload that takes a bool isAsync parameter (passing true) or a FileOptions parameter (passing FileOptions.Asynchronous). So you can't use the convenience open methods like File.Open.

Stephen Cleary
  • 437,863
  • 77
  • 675
  • 810
  • 1
    Thanks for that information and especially: `FileOptions.Asynchronous` thank you! `isAsync` seems to be depricated. +1 – BendEg Mar 03 '16 at 13:39
  • 1
    @BendEg If you saw the deprecation warning near a bunch of [constructor overloads for `FileStream`](https://msdn.microsoft.com/en-us/library/system.io.filestream(v=vs.110).aspx?f=255&mspperror=-2147217396#Anchor_2), those obsoletes appear to be for the `IntPtr` variations of `FileStream()` due to the better/safer `SafeHandle` alternative. You can see that, for example, [this filename overload with `useAsync`](https://msdn.microsoft.com/en-us/library/7db28s3c%28v=vs.110%29.aspx?f=255&MSPPError=-2147217396) is not marked obsoleted. – binki Mar 21 '18 at 17:52
7

In .NET core (since version 2.0) there are now all the async flavors of corresponding ReadAll/WriteAll/AppendAll methods like:

File.(Read|Write|Append)All(Text|Lines|Bytes)Async

https://learn.microsoft.com/en-us/dotnet/api/system.io.file.readallbytesasync?view=netcore-2.1

Unfortunately, they are still missing from .NET standard 2.0.

adospace
  • 1,841
  • 1
  • 17
  • 15
  • 1
    It's now available in .net standard 2.1 https://learn.microsoft.com/en-us/dotnet/api/system.io.file.readallbytesasync?view=netstandard-2.1 – Robert Pouleijn May 08 '20 at 11:48
0

No, but you can use the FileStream to create the same behavior.

Here are the helper methods I've created for a NetStandart 2.0 class library, that was used both in NetCore 3.1 and NetFramework 4.7.2 projects.

These implementations have matched exactly the names and signatures of the net core 3.1 / net standard 2.1 File class methods, so you only need to put them in any public class. (FileHelper for example...):

Also, this should be most efficient and similar to the source code of .net implementation.

private const int DefaultBufferSize = 4096;

    // File accessed asynchronous reading and sequentially from beginning to end.
    private const FileOptions DefaultOptions = FileOptions.Asynchronous | FileOptions.SequentialScan;

    public static async Task WriteAllTextAsync(string filePath, string text)
    {
        byte[] encodedText = Encoding.Unicode.GetBytes(text);

        using FileStream sourceStream = new FileStream(filePath, FileMode.Append, FileAccess.Write, FileShare.None,
            DefaultBufferSize, true);
        await sourceStream.WriteAsync(encodedText, 0, encodedText.Length);
    }

    public static async Task<IEnumerable<string>> ReadAllLinesAsync(string filePath)
    {
        var lines = new List<string>();

        using var sourceStream = new FileStream(filePath, FileMode.Open, FileAccess.Read, FileShare.Read,
            DefaultBufferSize, DefaultOptions);
        using var reader = new StreamReader(sourceStream, Encoding.Unicode);
        string line;
        while ((line = await reader.ReadLineAsync()) != null) lines.Add(line);

        return lines;
    }

    public static async Task<string> ReadAllTextAsync(string filePath)
    {
        using var sourceStream = new FileStream(filePath, FileMode.Open, FileAccess.Read, FileShare.Read,
            DefaultBufferSize, DefaultOptions);
        using var reader = new StreamReader(sourceStream, Encoding.Unicode);
        return await reader.ReadToEndAsync();
    }
Ester Kaufman
  • 708
  • 10
  • 20
0

An answer from the future :)...

Now you can invoke File.ReadAllBytesAsync() directly without using streams starting with .Net Core 2.0.

Definition:

public static System.Threading.Tasks.Task<byte[]> ReadAllBytesAsync (string path, System.Threading.CancellationToken cancellationToken = default);

Returns:

Task<Byte[]>

A task that represents the asynchronous read operation, which wraps the byte array containing the contents of the file.

Applies to:

.NET Core 2.0, Core 2.1, Core 2.2, Core 3.0, Core 3.1, 5, 6, 7 Preview 7

This new API is available here

Ahmed Suror
  • 439
  • 6
  • 17