NVMe for music content

For selfish and theoretical reasons, I’d like to experiment with putting ROCK on an internal SATA SSD and storing music on the NVMe M.2 SSD (all of the things posted thus far seem to indicate that ROCK wants to do the opposite – is this a matter of sonic testing or developer convenience?). My hypothesis for trying this inversion of storage systems rests on a few suppositions:

  1. ROCK doesn’t need more bandwidth than an SSD already provides.
  2. Music files can be huge and always benefit from increased transmission bandwidth.
  3. Compared to NVMe, SATA/AHCI is an older, slower, chattier protocol for data transfer that often has to move the bits through intermediary bridge chips and before it gets to the CPU. (SATA tops out at 600Gbps, while NVMe can reach 1.6Gbps+)

By putting the music on the NVMe, we drastically decrease latency and increase bandwidth for the bits. We also eliminate the SATA protocols chatter and any intermediary bridges… it’s just PCIe straight to the CPU… entirely for more streamlined access to said bits. That should (in theory) bolster the sonic performance insofar as timing and reliable delivery of the bits plays a role in sound quality.

That said, if having the bits ‘awash’ in NUC noise is going to degrade things more than extra bandwidth helps them, this line of experimentation may be moot…

As an aside, I’m also hopeful that we’ll be able to play around with running ROCK on a Virtual Machine via Hyper-V. I’d love to ‘kick the tires’ (behaviorally, not sonically) on it before I invest in a NUC. =^)

Thanks again for all the awesome!!!

Hi Jeremy,

Your thinking regarding SATA v M.2 NVMe is interesting. A couple of thoughts:

  • Where music is being sent over Ethernet to a Roon Ready or Roon Bridge endpoint using RAAT, then latency issues won’t be as important as where there is direct connection to a DAC. If you are direct connecting then there are a lot of increasingly expensive things you can do to clean up the computer side of your system. An inexpensive network endpoint/renderer can make those things irrelevant;

  • ROCK may not be the best environment to investigate the comparative advantages/disadvantages of different storage media. You may prefer an OS that enables you to run monitors to measure timings etc;

  • Having the Core on the fastest storage minimises OS processes and analysis, search and loading times in Roon. The bandwidth required for Playback is well exceeded by both SATA and m.2 NVMe. Provided music storage is sufficient I think you would see more improvement from Core on the fastest drive than music files. Be interesting to find out though.

1 Like

This is one of those things that always comes up and sounds reasonable, but has little impact in reality.

Assume for a moment that DXD is the most sample-dense file that most uses will ever store. These files are indeed large, but the overall size has no practical impact on playback as that’s a real-time process.

The bitrate for a DXD file during playback is 24 bits * 384,000 samples/sec * 2 channels = 18,432,000 bits / second (2.3 megabytes / sec).

The crappiest average modern spinning hard drive can hit 20-30 times that without breaking a sweat. The average SATA SSD is even faster. A good NAS can easily saturate a gigabit network link (125 megabytes /sec).

In other words, even if Roon is building a memory buffer in the core that’s 30 seconds of data for every second of playback a good spinning drive can hit that no problem (and in about a second). An SSD may fill that buffer faster, but once full it’s not going to drain any faster than 2.3 megabytes / second.

An SSD / M.2 / NVMe is going to have no practical benefit on audio playback performance when used to store music files.

The Roon database is another matter as that’s a tremendous volume of tiny random reads and writes. It’s a completely different usage profile that’s perfectly suited for solid state storage.

3 Likes

No, someone somewhere will have heard a significant difference, you can be sure :sunglasses:

1 Like

I appreciate your perspective and experience, Andrew. And for the most part I completely agree with your argument. It’s very sound (no pun intended)! No contest that the the DB would benefit from the faster interface. That said…

Perhaps it would help if I unpacked the basis (or bias?) for my (as-yet-unproven) “bandwidth is nearly-always better for the music” hypothesis: My question is really, “Do faster theoretical transmission speeds result in better fidelity, even if those higher speeds are impossible to realize during playback?”

I used to think no. But then I took a closer listen to two data transports that changed my mind: USB and Ethernet. I could write a missive on each and how I once was a die-hard advocate that ‘bits are bits’ and that digital cables couldn’t possibly make one whip of difference… but I know that’s all hogwash now. I’ve proven just the opposite to myself and others with handfuls of different cables from various manufacturers.

The one trait most of these better sounding digital cables have in common is a substantially higher theoretical transmission bandwidth – usually drastically higher than the bus they are communicating on or through is capable of utilizing: 10Gb/s capable USB 2.0 cables attached to a 480Mb/s bus and 40Gb/s network cables attached to 1Gb/s NICs playing 1.4Mb/s - 24Mb/s audio files…

So that got me thinking… maybe having greater bandwidth for your data is like driving a Ferrari under a speed limit – there is a lot of enjoyment to be had with so much torque, grip, and horsepower at your fingertips, even if the sign says 60mph. =^)

Maybe a good common-ground solution that would satisfy both of our bents would be to use two internal NVMe drives. :wink:

I think you’re confusing audio playback with data transfer. They’re both moving bits, but playback is a real-time process and it requires bits to arrive absolutely on-time and in proper order. Data transfer is completely asynchronous and doesn’t care when bits arrive or in what order. The transfer protocol sorts that out behind the scenes.

In audio applications the USB connection to a DAC is a playback interface. Bits have to be shipped and received in order with very specific timing or the DAC gets pissed off. The benefit of better USB cables has little to do with their enhanced bandwidth and more to do with their ability to address the problems that are inherent with USB (noise and ground loops are two big ones).

Ethernet is a different animal and in 99% of the consumer audio applications it’s a data transfer mechanism and not a playback mechanism. Network-connected DACs buffer the incoming data stream, reorder the packets and then present that stream as audio data to the DAC itself. The audio benefits of better cabling are all about how the ethernet cable does or doesn’t conduct noise into the DAC.

In other words, cables can make a huge difference in overall system performance and the network and USB cables are just as important as any other. Their benefit isn’t a result of increased bandwidth, rather it’s a result of addressing the network/USB connection as a source of noise.

4 Likes

Noise is absolutely a key factor, and eliminating it at all costs should be done first – there are huge gains to be had by reducing and eliminating all sources of noise… so let’s talk noise. :slight_smile:

Southbound Pachyderm
With regard to bandwidth vs noise, I believe we’re really talking about two different parts of the same elephant. Consider that “less noisy” cables are also higher bandwidth cables (of their type), as (all things being equal) noise is what eventually overwhelms the signal and makes it meaningless. If you engineered a cable around a specific conductor to have greater noise rejection, lower noise generation, and superior insulation properties, you can push data more quickly down it than you can the same conductor with none of those trappings. Changing one factor (noise) changes the other (bandwidth).

To sync or not to sync?
How synchronous or asynchronous data transfer is (with regard to the playback chain) is a function of the playback software, underlying operating system, the hardware at play. How sync or async it actually is can vary at each step of the way, and it’s quite common to have synchronous paths depending on asynchronous paths and vice versa. It’s almost never purely one or the other, but async is employed to add flexibility (real or perceived) to otherwise synchronous paths such as realtime playback of music data in the DAC, and to insulate/decouple highly-responsive code paths (such as a user interface) from slower, less-than-instantaneous ones (such as I/O which takes time, can require retries, or involve queuing). Of course, one of the dirty secrets of async is that it requires threads to be suspended while they wait on other threads to perform work… which may result in more CPU context switches. More on that later.

RAM, lies, and video tape
If we peel back the covers of memory virtualization in most modern, pre-emptive multitasking operating systems, we see that the “RAM” we think our 700MB DXD file is sitting in isn’t actually physical RAM most of the time. The data is often being swapped in and out of a “paged pool” of virtual memory on the disk while the OS is happily servicing other customers, each of which may be holding a slice of physical RAM.

Come on, feel the noise!
This kind of behind-the-scenes swap behavior is in part what makes general purpose operating systems so caustic for music playback, as all of these context switches across threads, processes and page-in/page-out operations are creating noise by the score. This is where ROCK will shine, as Roon has the opportunity to negate much of this tomfoolery with a special-purpose operating system! :smiley:

Come to think of it, this may be an excellent argument for using an NVMe disk for a traditional OS’s virtual memory / swap partition, as it will satisfy virtual memory requests so much (~5x) faster than an SSD can and thus “feel” more like physical RAM… and it absolutely underscores your database argument. Databases love fast I/O!

Considerations
In ROCK’s case, how useful it would be for the operating system itself (setting the DB aside…) would depend largely on whether or not it will virtualize its memory… granted, you’ll still get performance gains with NVMe (unless the entire OS reserves space for itself in physical RAM in its entirety as soon as it boots…), but…

  • Less applications running should result in far fewer context switches, which should mean less noise generation.
  • If Roon does employ virtual memory, that becomes an argument for adding physical RAM. More RAM means less swapping. Less swapping implies less I/O and less context switching, which implies less noise…
  • Removing SATA/AHCI activity from the I/O pathway(s) should mean less overall systemic noise.

Then again, the NVMe chips might be terribly noisy in and of themselves, making the above moot. I’ve heard people debating the audible impact of installing and running Mac OS from various SD cards (and very much to your point, Andrew, the faster ones didn’t always win!).

Anecdote
I built a fanless media PC some time back based on a Computer Audiophile recommended parts list. The Intel Core i5 CPU actually introduced massive amounts of ground plane noise (whined like a banshee!) whenever it was asked to change the pixels it was drawing (such as video playback, or even just scrolling a browser or other document window). It would largely go silent as soon as the video frame buffer fill was done. Darndest thing, but it got me to realize just how dang noisy the CPU is. Intel seemingly doesn’t care if the ground plane is awash with noise as they splash about doing good work, they just care that digital instructions they are given execute without logical error. Crazy.

1 Like

An interesting post with lots of factors … I have side stepped all of that by using a network connected DAC … the LAN provides a great way to provide a good level of electrical and mechanical isolation. The only thing my core needs to be is fast enough the rest is irrelevant.

5 Likes

Somehow, I’m reminded of the scene below…

https://youtu.be/7YyBtMxZgQs

2 Likes

What’s your Core running on? =^)

If your network endpoint is isolated and asynchronous (and with ample buffer), then the core can be running on crap and it won’t affect SQ as long it puts out the same bits. In this scenario, the endpoint can bypass any issues in non-bits, since it can only be affected by the bits due to reclocking and being isolated. Now you can run a much lighter weight endpoint and get all the optimizing you were looking for, far easier and cheaper, without sacrificing any performance on the core.

4 Likes

Btw, no swap in ROCK. Totally not needed.

Also I love the section titles of your post

1 Like

Thanks, Danny!

I had a feeling swap would likely be omitted, but the DB gave me pause. RDBMSes tend to be major memory hogs, and swap can be useful when physical ram is scarce…

The network endpoint in my case is a microRendu… what’s fascinating is that nearly every upstream optimization (from the Ethernet cable to the network topology and the power supplies all around) has had an audible effect. Hence, why I’m so keen for ROCK – seems like it will be much easier to tune a much simpler, purpose built server with a tiny OS!

(That and I’d like to stop dual-purposing my primary Windows PC as a music server :slight_smile:

We also don’t use an RDBMS :slight_smile:

My home system is a NUC6i5 running ROCK. It has a 64gb SSD and a 4GB stick of RAM.

My Roon has around 120k tracks, and right now with latest RoonServer, and I’m using up a bit over 2gb RAM for RoonAppliance. The rest of the stuff running there (RoonServer wrapper, web ui, raatserver, samba, network management stuff) is taking up another 400mb or so.

I can see the paranoia of why you’d want to put 8GB of RAM in there, but 16GB is just silly. My 64GB stick is given this amount of space:

/dev/root                 1.9G     93.3M      1.7G   5% /
/dev/sda2                11.5M     36.0K     11.2M   0% /identity
/dev/sda3                 1.9G    172.7M      1.7G   9% /roon/app
/dev/sda4                55.0G      6.8G     48.2G  12% /roon/data

As you can see, it’s mostly empty. Not enough for any reasonable amount of music, but whatever. The RAM was like $30 and so was the SSD. After the BIOS, it boots in about 1.6 seconds.

If you electrically isolate your endpoint well (faraday cage + optical ethernet isolator + good power), then you shouldn’t need to clean anything else on your network.

I really should write a blog post on why bits are bits for data transmission, and bits are not bits for audio transmission. I see people spending way too much energy cleaning stuff and getting results, but failing to clean the one thing that will make all the other cleanings irrelevant.

10 Likes

I love the faraday cage suggestion!

I’ve been using a fiber media converter for the 8m run between my media PC (in my office) and my endpoint (in my listening room). I upgraded to a linear power supply both for the FMC at the endpoint and the endpoint itself. That brought nice gains, and points to the obvious problem being power noise.

… what fascinates / befuddles me still is that I get sonic improvements even when changing the short run at the PC end of the line (not as dramatic as they are at the endpoint side, but still there…) You’d think anything on the other side of the optical wouldn’t matter… and yet somehow it does.

what’s the power supply on your optical isolator look like? you arent getting network noise because its the job of the isolator (assuming your isolator is working) to kill that.

ideally you could run the isolator and the endpoint off batteries :slight_smile:

With all the paranoia in your audio world do you actually enjoy listening to anything?

I’m using an LH Labs LPS4:

…but only on the “receiving” side FMC. The “sending” side is using a fairly inexpensive, supposedly regulated Jameco supply…

Oh, absolutely!

And I love re-experiencing old favorites all the more when I make a little tweak or change that removes even more distractions and distortion between me and the music. It’s like pulling the performers deeper into my room (or transporting myself further into theirs). Amazing stuff!

I think a lot of us would be interested and appreciate if you did this.

.sjb

5 Likes