This post is not about the organisation of tests but about my afternoon’s attempts to nail down the issue by the way of experimentation and measurements.
Spoiler alert: I believe i have something for @brian to work on…
I felt very frustrated because of my unconclusive previous measurements. This morning again, I listened to some HD tracks, switching between “no DSP” and “DSP will Null convolution”, and finding a small but quite obvious gap.
I was also reading @Marco_de_Jonge good comments and ultimately realised that I was so convinced there was an issue, that I actually didn’t want to put some effort in organising a test that will be disputed and inconclusive…
So, taking the word of Brian that it is a math problem… back to maths then.
I started by challenging the robustness of my test platform, playing the same files several times, and checking that the recordings were always the same in a given setup. I consequently decided to select the Apowersoft loopback CoreAudio driver, who delivered consistent results, Soundflower proving to be unstable and not entirely reliable at 96kHz.
So the test-platform consists of a single MacBookAir i3 2014, 8Gb running all other software:
- Roon MacOS App, so local Roon Server and Bridge (v1.5 363 )
- HQPLayer Desktop MacOS trial version
- Audacity MacOS 2.30 (latest) for recording and export of files
- Audio DiffMaker 3.22 under Parallels/Windows 10 on the same Mac
- REW 5,20 for spectrum visualisation
- Apowersoft Device configured in Roon as fixed volume, 32bits, 192kHz max, no MQA.
Roon DSP is opened, and Null384kMono filter loaded.
HQPlayer is configured without any resampling or dithering, see below for 44.1kHz test case:
Convolution: DSP in HQPlayer is activated by ticking or not “enabled” on this screen
The same NullIR file is used in both Roon and HQPlayer.
CoreAudio device configuration is checked for gain and format, here for the 96kHz test case.
I then built with Audacity two “original” test samples of approx. 30 sec each:
- Cecile McLorin Salvant / I don’t know what time it was : 96kHz/24bit
- Pura Fe / Mohomoneh : 44.1kHz/16bit
The two files are stored in a local folder, in both HQplayer and Roon libraries.
Audio differences are computed with the oldie-but-goldie Audio DiffMaker utility:
I started tests with sample “Cecile” 96/24. Results were, in comparison with the original:
-
Roon with no DSP --> bitperfect
-
Roon with DSP button activated but convolution disabled --> bitperfect
-
HQPlayer with no DSP --> bitperfect
-
HQPlayer with DSPNullConv on --> bitperfect!
-
Roon with DSPNullconv on --> not bitperfect, correlation depth of 93dB, quite poor for a 24bit file.
The difference with the original looks like this, magnified of 15dB.
It’s mainly high frequencies, which means high frequencies are quite altered by the convolution. Definitely not looking like quantization/dithering noise.
I actually made the measurements for HQPLayer and Roon several times with the Null convolution filter in or off, to check I had made no mistake. And I had not.
Then I decided to run the same series of test for the 44.1k/16bit file "PuraFe"
-
Roon with no DSP --> bitperfect
-
Roon with DSP button activated but convolution disabled --> (not done)
-
HQPlayer with no DSP --> bitperfect
-
HQPlayer with DSPNullConv on --> not bitperfect
Correlation depth with HQPlayer was 85dB (about 14 bits). The difference file magnified of 75dB looks like:
and its spectrum is:
It’s reasonably small and could be dithering noise within HQPlayer or the CoreAudio engine (which is 32bits so there are conversions).
- Roon with DSPNullconv on --> NOT bitperfect at all , correlation depth of 45dB only! This is clearly way within audibility range.
Spectrum (in red) has strong HF content compared to HQPlayer case(in blue):
At this point I had evidenced that there really was an issue with Roon Convolution engine compared to the one of HQPlayer, which behaves as one would expect.
Then I really scratched my head hard. Yesterday, Roon has a better performance than HQ Player in terms of differences between orignal file and processed file… What the … ?
And, then this happened.
I got it. I knew what the issue was.
I double checked my settings of yesterday : I had a 96K Null IR in the Roon’s convolution engine, and I was playing a 96KHz sample. And Roon behaved well. But today, I had a 384k IR… In principle resampled by Roon automatically, as explained in the DSP online documentation.
Immediately I tried a 44.1k Null IR on my PuraFe sample and… bingo ! The correlation depth cae out as exactly the same as HQ PLayer. The spectrum was exactly the same, see HQPlayer in blue, Roon in mustard yellow,. I had to artificially offset the curves so that you see them, in practise they are exactly the same. Roon with 384k IR is in red.
Back to the 96k “Cecile” sample. I uploaded a 96K Null IR in Roon and, bingo again:
-
correlation depth of 165dB, this is more than 70dB better than with the 384k IR !
-
still it doesn’t achieve bit perfect
Amplitude magnified of 120dB and spectrum of the difference file original/processed are:
The offset in the magnitude is to be explained. The spectrum looks more like flat/quantization noise.
I have tested HQPlayer with the same 96k Null IR filter on the same file : the result is bitperfect, like with the 384k Null IR.
In conclusion, I am now convinced of two things, the first one at least requiring immediate attention from the Roon team/@brian:
-
There is a design issue / a bug in the impulse response resampling algorithm of Roon, that is provoking measurable and audible artefacts
-
A 96k Null convolution filter applied to a 96kHz/24bit stream is not bitperfect in Roon (even though it’s very close, correlation depth is >160dB). For the same IR and same file, HQPlayer is bitperfect. Mathematically and theorically, it should be bit perfect.
The only workaround I see for now is to make sure that the .zip file you are uploading comprises all possible sampling rates for your material. Roon then should automatically pick up the right IR, and not resample.
NB: my filters so far were 44.1kHz only, because I thought Roon would resample them correctly to 88kHz or 96Hz (I have a few 24/96 albums). I think I started to “feel” something different and having issues when MQA was launched by TIDAL and integrated in Roon : most of the 44.1kHz material is upsampled to 88.2kHz. My filters were good at 44.1kHz, but not at 88.2kHz due to the IR resampling bug.
NB 2: for those interested to cross-check my findings and dig deeper, all the original, recorded and Audio Diffmaker processed files are stored here. Titles are self-explicit. When no rate for NullConv is mentionned, 384k has been used. It would be good if someone could run comparable tests to consolidate my analysis.