10 Gb Ethernet anyone?

Audio Gear Talk
21

Background: I have ~20 years network engineering experience.

Regarding your two questions:

  1. I have 10GB Ethernet in my home network. My main NAS (where all my local music files are stored) is connected via 2x10GB in an LACP bond to my 10GB switch. My Roon Core (running on a CentOS 7 system) is connected to the same switch via a 2x1GB LACP team. All my Roon endpoints are connected to access switches around the house that each have a 1GB link back to the core switch. Works fine.

  2. As pointed out above in this thread, 10GB between the switches will provide benefit if you have multiple devices transferring concurrently between the switches. In practical home terms, however, this is unlikely to get used much. All streaming (be it audio or video) is relatively low bandwidth – could even be done over a 100Mbit link, never mind 1GB or 10GB. File transfers can peg a link, but are generally bursty in nature and the link mostly sits idle. Backups can use a lot of bandwidth, but are typically a background task running in the middle of the night, so not really something to be worried about.

Where 10GB comes into play are

  1. you have LOTS (100s) of devices – this is uncommon for home networks, but normal in office/enterprise environments
  2. you have some “heavy hitter” workloads, such as video editors working directly off a NAS instead of using local storage. This is also a fairly niche use case in a home/SOHO environment.

I have 10GB at home mainly because a) I’m a network engineer and that’s just the way we roll, b) it was not a financial hardship for me to do so, and c) it allows me to manage and edit my photography library directly off the NAS (I use Capture One, FWIW) – not nearly as demanding as editing video, but you can still feel the difference between 1GB and 10GB doing this.

The cost of 10GB has come down a lot in the last couple years. You can get older used cards for cheap (but be aware they tend to have much higher wattage draw than the newer cards), and chips like the ANQ10x have made new client-oriented cards (so none of the fancy offload stuff that the more expensive server cards offer) available for under $100 new. There are also some “affordable” switches now from the likes of MikroTik, Netgear and Ubiquiti. (Though they are still a factor of ten more expensive than a same-port-density 1GB switch.) But for most home use cases, there is no need for 10GB today, or even likely for the next 5+ years. Unless you have one of the niche workloads that requires it, and then you will already know you need it…

15 Likes
22

Not really future-proofing if you don’t need it now. With IT the idea of future proofing is usually pointless as whatever you buy–not the standard–will be out of date soon since IT tends to age in dog-years.

23

Good read. All true…

It is kinda like I know I want a new DAC or speaker, lately a Tube Amp.
The 10GB is gonna come sooner or later.
–MD

24

Work in computing and was going to write something but I think @cwichura has already given a great explanation.

I will expand on their point about even a 100Mbit link being good enough for media consumption at the endpoint level. I checked my 4k bluray remuxes (no additional compression) and the highest bitrate file I have is 86Mbps (3:10 to Yuma). Audio needs considerably less throughput so it’s safe to say that it’ll probably be quite a while and even more advancements in resolution will be needed before a 1GB link is pushed to the limit by media.

Infrastructure/storage maybe depending on your usage but in terms of pure media endpoints I wouldn’t worry about it.

25

100% agreed… my 2-channel system is connected w/ 100Mb Airport Express to my Roon endpoint and it works flawlessly.

26

A “cheap” 10GB switch is going to perform worse than a proper 1GB switch.

Speed isn’t everything. It’s cheap to throw a 10GB transceiver (the thing that converts voltage to bits) onto a switch and call it “10 gigabit”. The harder part is the ability to move the additional frames, at line rate, that can come from that increased pipe size. Ultimately, for Roon, you don’t need the additional bandwidth but if something else on your network does then Roon may see a benefit because that other thing will spend less time actively utilizing the capacity of your network. If you take something, or a few things, that can saturate 1G between switches then Roon will be competing for the 1G between switches. You’ll see this as increased latency until the point the switch starts to drop frames; then Roon will simply throw a fit. With 10G between switches then you should not see any difference in Roon performance when the 2 devices are saturating their ports at 1G and those same devices doing nothing. This, however, only holds true if the switch can truly forward at a frames per second rate beyond what your network is actually generating. If it’s a cheap 10G switch it may not have much more switching capacity than a 1G switch and you’ll see dropped frames well before you get anywhere near 10G (moving files might deal with this fine but streaming will not). That’s not only a waste of your 10G investment its harder to troubleshoot.

Anyway, no reason not to invest in 10G if its within your budget to buy good gear. If you cannot afford a good 10G solution then its better to buy a really good 1G solution. You’ll have a more stable and reliable backbone for everything.

It also helps if you can model your existing traffic but this requires managed switches or taps to collect and analyze traffic patterns. Knowing frame size, frames per second rates, backplane capacity, etc. will help you match your traffic patterns to a properly performing switch. Without this data just buy the most robust thing you can find and assume if they are not publishing data on frames per second it’s a poorly performing switch.

*note, some of the benefit (probably most in a home scenario) of the increased inter-switch capacity can be done by bonding multiple 1G together which is a feature of most managed switches.

2 Likes
27

Thanks all for the knowledge!

28

Just to make sure I understand, is that the same as the “packet forwarding rate” (which should be ≥ 1.49Mpps ✕ (#of ports) for a gigabit switch and 10 times that for a 10G switch)?

29

Due to a wiring problem, I can only do 10mbit/half duplex in 1 room. Everything (Streaming video, Audio, Gaming, downloading) works the same there as in my 1Gbyte rooms.

30

Yes, and no. We can use packets / frames interchangeably for now.

Note that ethernet ports can / should be full-duplex and could forward double that (1.4Mpps in each direction). So, theoretical is 2.97Mpps at 1G/port-fdx and 29.76 at 10G/port-fdx. But, actual transmission at full-duplex within the forwarding latency of the switch is slim. Again, we’re talking theoretical.

The number you quote, 1.49Mpps, is based upon a frame length of 84 bytes (minimum frame size). That’s unrealistic in the real world. Unless you had a very specific application, or just wanted to melt your ethernet switch for fun, you’d never run “line rate” at that small a frame size.

As an example, I just did a packet capture to make sure nothing had changed (from the last time I looked at this). Roon is using the maximum frame size, 1514 bytes, to move audio from core to endpoint. Roon also uses TCP so each 1514 byte frame requires a 66 byte ACK in the other direction (although the distribution across the network is not 50/50 in my capture which is either a capture problem or Roon is very aggressive in retransmitting). Average frames per second for various bitrates:
16/44 avg. 208 fps
24/172 avg. 970 fps
24/192 avg. 1030 fps

So, as you can see, you can fit a whole heck of a lot of Roon streams across your ethernet switch before you get anywhere near the theoretical maximum. Largely because Roon is minimizing overhead by pushing as much data per frame as possible. This kind of traffic pattern, combined with other things on your network, is good to know so you can make sure you’re sizing things correctly. But, do you really need to know any of this if you buy good gear?

How does this compare in the real world? Let’s look at a popular Unifi switch the USW-24-PoE which is 24 ports of gigabit. It’s rated at a switching capacity of 52 Gbps, non-blocking at 26 Gbps (we didn’t talk about what this means as it gets more complicated than this forum probably needs), and forwarding rate of 38.69Mpps. So, in this example, this switch really does claim to hit above theoretical max forwarding rates for half-duplex. The Unifi 10G switches claim the same. Looking at Netgear business switches they claim right at theoretical for half-duplex. Their resi/soho switches don’t publish fps numbers (and it’s probably because their switches slow down as you turn on features like 802.1q).

Anyway… so… yes, your maths checks out. But only in a theoretical exercise. In the real world you don’t need anything near the numbers you’re calculating. But, buying a switch that does come close to those numbers will give you less headache. I always recommend the less headache option. Which goes back to the point in my previous post. If the switch your buying has published numbers its probably OK. Stay away from anything where you cannot find the numbers (common with cheaper switches).

4 Likes
31

Hi there - here I had to change my infrastructure (we moved to another place) and instead of Copper Cat6 or Cat8… I use Fiberoptic as backbone - next to that I am using Polymer Fiber just for my Audio. This really improved the sound quality! The switches I use are Netgear SG108V3 Gbit/s supporting Jumbo Frames. And direct from main switch a separate UTP/Polymer Fiber connection to my streamer.

32

Please please please let’s not let this thread go to audiophile tweaks for sound quality.

The question was technical, and objective, which was: is there a throughput increase between switches if the endpoints are not 10GBE and the answer is yes.

Then, some other good advice about a cheap 10GBE switch maybe not being better than a good 1GB switch.

All good. Let’s keep it to that topic.

33

Agreed… but the real point is this: you don’t really even need 1 GbE for consumer audio applications, let alone 10 GbE, so stop wasting your money.

34

Boy, there sure are a lot of lecturers out there. Who said that is all I do? I asked for information, and while related opinions are also welcome, with due respect, you are in no position to tell me that I would be wasting my money.

This is not a “veil lifted off the music” question. It was a simple request for information. I got the information.

35

When you ask a question, you don’t get to censor the responses. If people think you would be wasting money, they can say that. That’s their opinion. You don’t have to agree. I probably wasted some of my money by putting a 1TB SSD in my Nucleus. I have 4 albums with no plans to add any others.

2 Likes
36

I am thinking of getting the NETGEAR Nighthawk S8000 Advanced 8-Port Gigabit Ethernet Switch, this can go up to 4 gbps, but it has been sold out everywhere I look, except eBay which is listed as 3 times the MSRP.

image

image

37

I’m not censoring them, I’m censuring them. Enough said, I prefer the flow of information and banter.

Cool!

39

Just want to say that new computer motherboards are coming out with 2.5Gbps on-board Ethernet (even though the first ones have a critical flaw). People purchasing new network hardware should take this into account. Owing to Intel and motherboard manufacturers’ decision to push 2.5Gbps products, I’d argue this will become relative cheap and therefore pervasive and be more important compatibility-wise than 10Gbps.

I suppose everyone here knows the role of NUC as a Roon Core, especially running ROCK. NUC11 will have 2.5Gbps Ethernet.

As the link above shows, 2.5Gbps Ethernet adapter can have less than perfect compatibility with a 2.5Gbps switch from a different vendor. I’d not be surprised to see that some day in the future, someone with a NUC11 will find that it does not run at 2.5Gbps with his 10Gbps switch.

1 Like
40

For those who are not urgent in getting a multi Gigabit switch, I’d suggest waiting for the Intel 2.5Gbps compatibility issue to be completely settled first.

2 Likes
41

That’s excellent info, thanks Peter!