Bias works both ways. Your both so biased you can’t even think through that A) The difference is enough that you don’t need A/B switcher to recognize it and B) Someone else would be swapping things around. The listener would be blind to what they were listening to, not the experimenter. This would be a blind test not a double-blind.
I went through this recently. I spent a whole weekend trying various combinations of streamer and OS and kernel version to find the best sounding NAA endpoint with the hardware I had. Some combinations sounded the exact same. Some combinations downright awful. Other combinations noticeably better.
But, let’s not lose track of how this started.
I was trying to walk you down a path to understanding that this whole “bits are bits” thing isn’t that simple. Yes, the bits arriving from the network can be perfect and still what comes out the other side can produce something that sounds different. Streamers sound different. You want a reason why? I gave you an experiment to go test on a possible why.
Good luck.
and a note @Tim_Ashley … apologize for going so far off the rails here.
Thank you but I don’t feel we’ve left the rails at all really: as a non-technical guy with an all-in-one Streamer/DAC/Preamp/Amp who hears a significant difference between the same audio signal cooked two ways, it’s gratifying to learn that (as is always the case with complicated technology) opinions as to why this is happening vary. Because if everyone agreed as to the cause, it probably wouldn’t be happening.
I cannot speak for your particular environment, but I have had a similar discussion with the NAIM SW director a couple of years ago. I heard a SQ difference between NAIM/QOBUZ and NAIM/ROON/QOBUZ. After a long discussion, what it came down to, is that the actual signal path within the device (streamer/DAC) is different when using Qobuz directly or using your device as Roon End point. NAIM stated using their proprietary ( and superior - sic) SW stack with Qobuz direct, but “being forced” to use another SW stack when working as Roon End point.
To me this was enough to drop NAIM as a whole, and go for a solution where a new (Benchmark Media) Dac is connected via USB to the Roon Core (Rock). I did a number of tests (including twofold single blind) over a longer period, and this solution proved superior to both Naim set-ups. Buying life time Roon, a new Dac and a Nuc was completely sponsored by selling the NAIM gear second hand.
Whatever you decide to go for, whish you all the best.
Fascinating insight - and much appreciated! I have found (I mentioned this somewhere above) that using Chromecast from my Qobuz app to my EVO streamer gives identical sound to the streamer direct so I’m going to do that instead when my Roon subscription expires.
That gives us a good clue that the RAAT endpoint is using an inferior SW connection because Chromecast is only 48kHz and if sound quality is similar, then there is likely some type of re-sampling going on inside EVO. It’s too bad actually. They should have shortened the path to the DAC from the RAAT processor.
Would be interesting to test Roon via Chromecast to the Evo to see if it sounds any different to RAAT and similar to Qobuz via the Chromecast to the Evo.
Didn’t actual scientific testing on human subjects show that vast majority of people can’t even tell a decent MP3 from original 16/44.1?
I like playing the highest resolution music I can use, as a matter of principle ,but there being actual audible difference between the two seems rather unlikely.
Yes. Shame really But, yes, plenty of tests continue to show people cannot determine 128 or higher mp3 from CD. Some controlled environments where test subjects were all musicians and used the same headphones were more likely to pick the cd quality. But random samples of “regular people” you’ll rarely see scores that get better than guessing.
I can easily pick out 256k MP3 from 16/44.1. 384 is a little harder but still mostly accurate. AAC at 256 is actually pretty darn good.
Where all these things fall apart is the transients and decay. Not all recordings have good transients and decay but for recordings that do… it becomes really easy to noice how destroyed that part of the recording is when it’s compressed. There are other artifacts I listen for as well. Every CODEC has its little nuance in what it does to the sound of instruments. You start to pick those out quickly if you train your ear. And, of course, you can clearly see it in the spectrogram.
Since I know you like to look at data this site gives a good side-by-side of cd, 128kbps mp3, and 384kbps mp3.
Can we hear it? Being as I am better than guessing when taking the online tests… Yes… I believe humans can hear these differences.
The BBC published a white paper (A subjective evaluation of high bitrate coding of music) on this a few years ago.
“There is a lack of previous work in evaluating AAC at high bitrates as it has mainly been focused on lower (up to 128kbps) bitrate audio quality [3, 4] or multichannel audio quality [5]. The most relevant work was presented in [6] and compared audio quality of MP3 256kbps to WAV (44.1k, 16bit), which showed that there is no perceptible difference in quality. However, the test did not follow any standardised test method.”
“This paper presented a subjective listening test to determine whether there is likely to be a perceptible difference between lossless (FLAC) and AAC 320 kbps compression. Recommendation ITU-R ■■.1116-3 was used as guideline for the design process. A total of 18 participants took part in the test and each graded 12 test items on the ITU 5-grade impairment scale. The results were analysed using difference grades with statistical methods, such as t-test and ANOVA. The post-screening process was used to eliminate the scores of 5 participants.”
“The analysis showed that there was no statistically significant difference in quality between the uncompressed signals and AAC-LC 320 kbps compression, which means participants did not perceive difference between two formats. It also showed that there was a statistically significant difference between the uncompressed signals and HE-AAC 48 kbps compression. This means participants could perceive differences in quality between the two formats.”
“The test has shown how AAC encoders can preserve the quality of the original audio. This suggests that offering lossless audio might not have a great benefit in terms of quality increase to the consumers. However to ensure that a delivery service is transparent and original quality is always maintained, a lossless codec would be required.”
“7 Reproduction devices 7.1 General
Reference monitor loudspeakers or headphones should be chosen with the aim that all sound- programme signals or other test signals can be reproduced in an optimum way; namely, they should provide neutral sound for any type of reproduction and should be usable for monophonic assessment as well as for two- or more channel stereophonic sound systems.”
This is the BBC R&D department, not some 2-bit snake oil vendor with a hidden agenda. Fairly sure if the BBC want to setup an impartial listening test to ITU standards they have the studios, equipment and staff to do it in a professional and impartial fashion.
Oh, yeah. at 320kbps or so MP3, one possibly can tell it from the original, if it is a familiar track with the kind of sound that gets particularly badly decimated by perceptual coding. You generally have to be pretty young, too.
I prefer listening to high-res lossless files… because I can, but it is purely for the warm and fuzzy feeling. A well-mastered Red Book D (e.g. something from Pope, or Mapleshade) sounds better than most of the “high-res”
Tim, fascinating information in this thread . I am a real newbie and funnily enough have just subscribed to Roon (which I love) . I am about to purchase the Cambridge Evo 150 . Just wondered what your overall impression is of the unit ?
Hi David,
I absolutely love my EVO - it looks great, sounds wonderful with my speakers and is a real treat. It has one or two interesting little quirks (it doesn’t auto power-down when left in pause mode while streaming for example) but I thoroughly recommend it to anyone EXCEPT people who love ROON because at least in my setup it doesn’t sound as good when Qobuz is the source via the ROON core as it does when Qobuz is selected directly or sent via Chromecast. And after nearly 100 posts in this thread there is yet to be any agreement as to why that is - all I know is that it’s true to my ears, and that another contributor above had the exact same experience and when he switched to another streamer instead of the EVO, the problem disappeared so… it looks to me as if you choose either ROON or EVO but not both. I’m choosing EVO and will not subscribe to ROON when my trial ends…
I’ve went all the way with Roon since it’s serving several endpoints across my whole apartment.
Primarily my Cambridge CXNv2 in the living room and my iFi Zen DAC V2 which is hooked up to my MacBook Pro. But it also serves 3 AirPlay devices when required.
But, yes, plenty of tests continue to show people cannot determine 128 or higher mp3 from CD. Some controlled environments where test subjects were all musicians and used the same headphones were more likely to pick the cd quality. But random samples of “regular people” you’ll rarely see scores that get better than guessing.
