15

In my Azure role code I download a 400 megabytes file that is splitted into 10-megabyte chunks and stored in Blob Storage. I use CloudBlob.DownloadToStream() for the download.

I tried two options. One is using a FileStream - I create a "write" FileStream and download chunks one by one into the same stream without rewinding and so I end up with an original file. The other option is creating a MemoryStream object by passing a number slightly larger than the original file size as the stream size (to avoid reallocations) and downloading the chunks into that MemoryStream - this way I end up with a MemoryStream holding the original file data.

Here's some pseudocode:

var writeStream = new StreamOfChoice( params );
foreach( uri in urisToDownload ) {
    blobContainer.GetBlobReference( uri ).DownloadToStream( writeStream );
}

Now the only difference is that it's a FileStream in one case and a MemoryStream in the other, all the rest is the same. It turns out that it takes about 20 seconds with a FileStream and about 30 seconds with a MemoryStream - yes, the FileStream turns out to be faster. According to \Memory\Available Bytes performance counter the virtual machine has about 1 gigabyte memory available at the moment before MemoryStream is created, so it's not due to paging.

Why would writing to a file be faster than to a MemoryStream?

sharptooth
  • 167,383
  • 100
  • 513
  • 979
  • 4
    Are you sure your memory stream is not swapping? – Oded Aug 17 '12 at 13:26
  • Interesting question, if there's no paging going on MemoryStream should be much faster. What instance size are you using? And could you post some code (even if this probably doesn't matter since you're simply calling the Storage Client library). – Sandrino Di Mattia Aug 17 '12 at 13:28
  • 2
    Do you have 1 GB of *physical* memory or 1 GB of *virtual* memory? – Servy Aug 17 '12 at 13:29
  • @Oded: No, but I believe it shouldn't with one gigabyte available. – sharptooth Aug 17 '12 at 13:29
  • 3
    @Servy Even if it's what the VM calls "physical" how would we know what's going on in the real machine the VM is running in? – Jon Hanna Aug 17 '12 at 13:30
  • @Servy: It's `\Memory\Available Bytes` counter, claimed to be physical memory. – sharptooth Aug 17 '12 at 13:31
  • @Sandrino Di Mattia: I posted some pseudocode. The instance is "small" - claimed to have 1,7 gigabyte physical memory. – sharptooth Aug 17 '12 at 13:35
  • @sharptooth - I would expect swapping with that amount of physical memory . The OS as well as the framework need to load and they would already take a good chunk of that memory. Allocate another 400Mb over that, and... – Oded Aug 17 '12 at 13:38
  • 2
    @sharptooth, while troubleshooting this performance issue you should also take a look at the performance counters that report paging activity: http://technet.microsoft.com/en-us/library/cc958290.aspx (just to be 100% sure about the paging) – Sandrino Di Mattia Aug 17 '12 at 13:41
  • 1
    I would try Available Memory and Paging counters in the loop. And maybe try on a local machine. On a local machine you could configure for no page file. Not sure that would help your problem on Azure. But might verify if the symptom is consistent with swapping. – paparazzo Aug 17 '12 at 13:49
  • +1 to Blam. Running this without paging is probably not a good idea in production anyway, but it could be very telling. – Jon Hanna Aug 17 '12 at 14:31

2 Answers2

3

Jon is probably on the ball there. The most likely explanation is,

  1. The memory is actually paged out by the hypervisor to disk.
  2. The hypervisor swap file is on a lower speed disk (say local disk).
  3. The FileSystem of the VM is on a fast enterprise disk (say SAN).

Regardless of whether memory is quicker or not, you really shouldn't allocate out such large blocks of memory. Have a read about LOH vs SOH here.

Community
  • 1
  • 1
M Afifi
  • 4,645
  • 2
  • 28
  • 48
  • Well, if the disks involved were the same speed (or the same disk) we'd still expect file-write to beat page-file-write, because the file-write would be sequential which is the case any spinning-platter based disk (whether simple or RAID, but not including SSDs) is better at the sequential case. Paging tends not to hurt that much most of the time because of smarts going into when it is done, so paged memory beats file access for most cases, but not this one. – Jon Hanna Aug 17 '12 at 14:26
  • I do not agree. The paging I am referring to is Hypervisor paging rather than the OS paging. They are not one and the same thing. Hypervisor swapping is substantially more expensive as it cannot know which pages are most suitable to be paged out. Second is Hypervisor paging is frequently in large scale deployments placed on the local disk of the server rather than on the much faster shared storage. – M Afifi Aug 17 '12 at 14:49
  • Neither of those disagrees with the other. 1: SANS storage > HV paging. 2: Sequential on disk-based > random on disk-based. They can totally both be true. – Jon Hanna Aug 17 '12 at 14:52
  • Its more the latter that I can't agree with :) Sequential on disk > randon on disk, is true except in virtualisation. There are a large number of possibilities here. Random writes on system like NetApp will appear sequential because it is a COW file system. Random writes on say an HDS array will be equal to sequential write because of large caches in the controller that commit the writes in battery backed cache before writting to disk. Once you involve high end storage systems used by virtualisation, a lot of "standard" statements are just simply not true anymore. – M Afifi Aug 17 '12 at 15:09
  • That is very true. Yet another reason why I don't understand how this is my highest-voted answer in quite some time, and you've got the single +1 I gave you. Should totally be the other way around. – Jon Hanna Aug 17 '12 at 15:33
1

When using MemoryStream in debug mode (VS) the speed is very slow, even with tiny amounts of data. Running without debugger attached it is comparable or even faster than FileStream.

First I got confused by this and ended up here. Now I am fine with MemoryStream.

user2746672
  • 11
  • 2