1

I have 3 different 10Gbe devices on my network and 2 switches. Switch 1 has a MacMini M2 with a 10Gbe ethernet connected via RJ45. Switch 2 have a Custom NAS and a Ubiquiti NVR connected via SFP+. Switch 1 is connected to Switch 2 via SFP+. Testing using iperf3 I get:

  • MacMini to NVR: ~9.30 Gb as expected
  • NVR to NAS: ~9.30 Gb as expected
  • NAS to NVR: ~9.30 Gb as expected
  • MacMini to NAS: ~6 Gb but
  • MacMini to NAS using -R (reverse): ~9.30 Gb as expected!

Any ideas what could be limiting my connection to the NAS if it is clear I can achieve expected speeds to other devices and the NAS can also achieve expected speeds? Also why is the issue only one way? Thanks

Journeyman Geek
  • 133,878
GreenLantern22
  • 81

1 Answers1

2

I'd guess its a settings issue. 10G support is a little finicky - and I got very similar speeds to a ubuntu box set up for 10G since the TCP windows are not optimised for 10G speeds.

They're sensible defaults for a decade or more ago, but you are going to be a little more generous with resources.

I'd guess since the NVR and MacMini both have 'native' 10G (and windows drivers are provided by the OEM), they've got things appropriately set up.

On ubuntu - and likely most other linux distributions you need to add the following lines to /etc/sysctl.conf

net.core.wmem_max=4194304
net.core.rmem_max=12582912
net.ipv4.tcp_rmem = 4096 87380 4194304
net.ipv4.tcp_wmem = 4096 87380 4194304

In theory running sysctl -p should load these settings immediately but I rebooted. There's a good explanation of why it works, and my own research here

This will theoretically take up more system memory, but modern machines have a ton of memory anyway.

You will also want to turn on jumbo frames everywhere to get a little more performance. I've benchmarked up to 9.9 with these settings.

Journeyman Geek
  • 133,878