1

I have an application that generates a stream of data as a single file. The amount of generated data is variable, but in average wouldn't nearly saturate my network or NAS speeds. However, there can be peaks for a few seconds that totally go above what's possible.

I don't have control over the application's behavior, which is just dropping data if the transfer can't keep up. If I generate the data stream onto my local SSD or even HDD, which is way slower than my NAS, everything is fine. I guess the latter one results from Windows' behavior of internally caching write operations in RAM. Is there some way to get this enabled for write operations on my NAS, too? Or are there other solutions available that would be able to solve this problem?

Also the whole data stream wouldn't fit on my SSD or even HDD, so just dumping it there and copying or syncing it later/asynchronous isn't an option for me. Ideally, if at all, I'd just want the non-synced portion of the file to reside somewhere on my local machine.

I am using Windows 10 Pro on my local machine. The NAS is a DS220+ and mounted via SMB.

Num Lock
  • 79

1 Answers1

0

Windows uses cache memory for all file writes, which also pertains to files residing on network shares. Cache memory is important - if it is not large enough for containing the data generated during these few seconds of peak activity, the best solution would be to increase the amount of RAM in the computer until it can contain it all, catching up with writing it out when activity calms down.

You could also increase the size of the pagefile, so the RAM could swap out more. Insure that the pagefile is on the SSD rather than the HDD.

You could try to improve the network speed with the aim of helping Windows to empty the RAM faster toward the NAS, therefore postponing the time that the RAM will fill up.

The following operations might help (or not). You would need to try them and see. A good test might be measuring the speed of a large file-copy to the NAS (larger than the RAM).

  • Invest in a faster network adapter, if the NAS supports it

  • Configure in Windows and in the NAS the network adapters jumbo frames, receive buffers, and interrupt moderation rate.
    On Windows, this is done in Device Management, right-click the network adapter, Properties, Advanced tab. The available settings depend on the adapter and the driver.
    Receive buffers might be more important on the NAS side, and this is done via modprobe or ethtool, depending on the adapter.

  • Turn off the Windows firewall and disable the anti-virus.

  • Use NFS for better performance. See the article
    Network share: Performance differences between NFS & SMB.

You may find more details in the articles:

harrymc
  • 498,455