Thanks Jussi! I do own a Sound Devices USBPre2. I may be able to send the S/PDIF out from my Devialet amp to the USBPre2 and capture the output in DiffMaker from the USB input. The USBPre2 has an AD too so I guess it would be possible to connect the speaker out on the Devialet amp via a resistor network to the line level input of the USBPre2.
Hmm… for the S/PDIF I could also use my Mutec MC-3 + USB without re-clocking…
All this might be quite time consuming so we’ll see
I have one Mutec box (1.2) that has digital in/out. But it always runs digital inputs through ASRC, and AFAIK same happens for all their other devices too. And they just got hostile at me when I suggested they would introduce a device that doesn’t do it and would have bit-perfect digital input. Oh well.
For digital I/O as well as many analog tasks I’m using my RME ADI-2 Pro’s…
I don’t think there’s much point in doing that, it has way too many potential problems and variables that in the end you don’t really know what you are measuring…
EDIT 24NOV2018: rather jump here for a better analysis
So I have made some comparisons.
I used Mac OS, Soundflower, Audacity, Roon and HQPlayer, with or without NullConvolution.
The sample is “Melody Nelson” by Serge Gainsbourg, first 20 seconds, 44.1kHz/24bit.
All in digital domain, the sound never reaches a physical device, Roon and HQPlayer are streaming to Soundflower 2ch.
I turned off volume control, dither and resampling everywhere I could in Roon, HQPlayer and Audacity. Of course I don’t know what’s going on inside Roon and HQplayer when convolution is applied, some dithering/resampling might occur.
Here are the results out of Audio Diff Maker:
Roon and HQPLayer are really bit perfect when all DSP processing is off: 300 correlation depth, the maximum.
With Null convolution activated, Roon actually gets a better score of 166 than HQPlayer (99). In both cases this is very little difference and “should be inaudible”.
I looked at the “difference” file generated by audio diff maker:
- for HQplayer, I could “hear” faintly the song in the noise, mostly high frequencies
- in the case of Roon, the difference is kind of a digital white noise, 50dB lower than the “noise of HQplayer”
For me this is an indication that the convolution process in HQPlayer is handled differently than in Roon. In both cases it is not “bit perfect” to apply a NULL convolution filter.
All files are stored here.
Here I don’t know what to think. 166dB correction depth between the original and the convoluted digital stream is huge and should not be heard.
So what else ? The thing I have not measured is the actual streaming performance of Roon Bridge to the DAC. Can it be altered by the fact Roon Server is sending data coming out of the DSP?
4 Likes
Hi @ogs! Regarding 300dB or 263dB correllation nulls : This is exactly what I meant, when I said ‘nothing in life is ever perfect’. You cannot make two exact same recordings, there are always varying factors beyond your control (clocks for instance don’t make the same timing errors, twice. Temperature varies, and with it : noise. Etc.).
Regarding the ‘167dB’ part : Correct, you must have detected a mior difference. We already KNEW, that there MUST at least be a tiny difference (after all, dithering is applied at the end of the process). And you have probably measured that.
The question was : how large is that difference, and can that be audible in any way…? I can perhaps explain those a bit, in my next answer to @alec_eiffel :
Fist off : my hat to you, @alec_eiffel. It takes real guts to post that, after stating you hear a difference. Much respect for that. You could just as easily have ‘hidden’ this.
I took your ‘Roon-difference’ WAV, and tried to analyze its spectrum.
[ EDIT : While trying to sleep, I kept thinking : this measurement cannot be right. And indeed : silly me took the wrong file, for analysis :(.
The correct measurement has exactly the same shape. But the noise is shifted 20dB, upwards. In this case, I cannot fully rule out audibility.
(But I’m not worried, too). I’m deleting a few parts of this post, just to not mis-inform people ]
The spectrum looks an awful lot like noise shaped dither. Except for the place of the rising after 4kHz; that would be unusually low. Cannot explain that, without assuming errors on my own side :).
Conclusion : as hard as I try, I still cannot find anything to worry about :(.
I really have no idea, where else can be looked…
I am really, honestly, curious to know if anyone (indeed) can detect these differences, in a controlled situation.
Would not be an easy task to setup such a test…
1 Like
Designate someone as the proctor. Record the output of Roon/HQP using whatever setup was used for the correlation measurements above (that looked very clean–it is likely to be accurate). Pass the pair of files out to the group in this thread, and see if people who could reliably pick out the difference at home can reliably pick out the difference between the two wav files by ear.
The mind is a funny thing… Perhaps your hearing is not quite as good as you perceive? And your measuring tests are not capturing everything or as robust as you believe…
Just did an unsighted test with 3 friends and they could all pick when DSP with null filters was switched on…
Thx for the suggestion @brian. I’m not sure if I’m willing to do that.
My main problem is : these would still be recordings. And not ‘the real thing’, as played ‘live’ through Roon.
As close to perfect as this approximation may be : it still leaves too much room for discussion, afterwards. 'No, we can not detect the difference between these files. But in Roon, we can. Trust me."
I do agree that it’s a practical solution, for people living so far apart. But it would not provide us (all contributors) the final answer, I’m afraid.
Now I’m writing this : I thought that you were suggesting an ABX test (as we only want to know if they sound different. Not : which one is better.)
But maybe you meant : just play both files through Roon, and let a large-ish set of people decide which file sounds ‘best’.
That could be an idea. But as of now, we do not have enough people for such a test to be meaningful…
IF we can grow this group to about 10 or more people, then I can think again. @alec_eiffel : would you be willing to supply two unrecognizable files, in such case ?
As for @Audiogeek: I am very much aware of the limitations of both my tools, and my own hearing. Thank you, those are indeed risks that need to be properly dealt with.
(You’d be amazed by the way, how resolving measuring equipment can be. If you like, I can make you some files with obvious or funny errors. Measured with the greatest ease. But I’m certain (as in : can prove) that no human car identify them)
I know this will come across more agressively than I mean…but : In uncontrolled conditions, you can make listeners believe almost anything. I’ve witnessed it more than once. I understand if you are not willing to accept that. Let’s not make this too personal; I am respectful, and I hope to receive the same from others.
I’m afraid you misunderstood my intentions a bit. Just to clarify :
I am NOT :
- Saying you are idiots, or saying you are wrong, or saying that you’re hearing a non-existant problem.
But I AM :
- Trying to identify possible areas for improvement
- Trying to determine, whether there is room for improvement at all
How else would YOU improve something, without understanding exactly what to improve ?
@Marco_de_Jonge : happy to support, let me know of a track to be used.
The more I think about it the more I believe the issue is potentially not in the DSP itself, but in the way the endpoint (Roon Bridge) is streaming to the actual DAC. Maybe the data is not arriving in the same way (I mean diffrent timing/packets…) from Roon Server when DSP is on ?
1 Like
The reason to listen to the files without running the Roon processing at the same time is to isolate the problem. That could get us a clear answer on whether the DSP (in a mathematical sense) is to blame or not. If you guys can distinguish just as reliably in that situation, then we’ve narrowed the problem substantially. Now it’s a math problem.
However if the same people can’t tell the difference…that points to some kind of out of band difference. Maybe the DAC sounds better at 16bit, or maybe the reduced network/CPU traffic, or (…). The above anecdotes in this thread are not well-controlled with respect to those variables, so getting that result would motivate more investigation in that direction.
If the math is really the wrong thing, figuring that out gets us much closer to a fix. Remember–if I measure and find nothing, this is a dead end–ad Alec’s stuff above suggests that that is exactly what will happen. It’s not like we didn’t measure this stuff before putting it into production, or like we don’t have a good idea of what kind of precision the code is capable of. But it would be good to line up his measurements with the listening results conclusively…
1 Like
I got curious because of the results and I did run the test through HQPlayer with a version that saves output that goes to DAC to a wav file. I use this for testing. Results look and sound a bit like 16-bit output without dither. Was the target output device 16-bit? Was the source in this test Roon or file played directly by HQPlayer?
For 24-bit output with TPDF dither and no upsampling:
parameters: -558,7msec, 3,000dB (L), 3,000dB ®…Corr Depth: 174,8 dB (L), 177,0 dB ®
For 32-bit output with TPDF dither and no upsampling:
parameters: -558,7msec, 3,000dB (L), 3,000dB ®…Corr Depth: 222,5 dB (L), 229,0 dB ®
Pretty much matches the output resolution that is 144 dB for 24-bit and 192 dB for 32-bit.
Looking at the difference the only difference is at the beginning and end and everything else is dead silent.
The 24-bit difference file can be found as FLAC here:
https://www.sonarnerd.net/tmp/_HQPlayer-_Original-24.flac
Edit: At 0 dBFS gain setting in HQPlayer and 32-bit output file the diff is still:
parameters: -558,7msec, 0,000dB (L), 0,000dB ®…Corr Depth: 221,8 dB (L), 220,3 dB ®
P.S. This is also good verification, because convolution engine was run on Nvidia GPU in these cases. Running the test also on CPU didn’t change the result. 
This kind of dedication to find ‘truth’ or ‘room for improvement’, is VERY hard to find in companies…
Awesome @brian, and the Roon team that you’re representing !
One thing is still unclear for me. Lets say we end up with two samples : ‘Sample_A’ and ‘Sample_B’.
An ABX test would work perfectly okay with that, but is perhaps not so feasilble for a typical user.
If instead, a user would play both files through Roon or other player. He would KNOW which file, is which. Just by its very name. Or any other file property. And thus adjust their preference or expectation, according to that file name or propery. (Cheaters will be cheaters. I know : I’m being paranoid. But such is reality…). How would we work around that ?
IF we work out that problem, than I’d first like to ask @Leong_Kin_Foong1 's permission. After all : this is HIS topic…wouldn’t want to hijack.
Then, I need a largish set of people that hear this difference.
And then, a need a statistician. I am worthless in this field… :). Does anyone feel qualfied for that ?
I just reread my post and apologize if it was condescending - was not my intention. However, it is as an often used comment - no measurable difference and I cant hear a difference, therefore it is audio perception or hearing in a group mentality.
I have an rme measuring setup, if I have the chance will try and replicate the recordings done by others.
The psychology of software, gear and hearing is intriguing. Generally someone suggests parameter x (let’s say power cable) improves sound. It cant be measured or replicated in a controlled listening test - creates a lot of skepticism.
But here we have a parameter which many hear as degrading the sound, and can be picked unsighted by some (very unusual)- yet measurements dont provide a clear answer. Then some say they don’t hear a difference.
Fascinating to see where this leads.
I’m happy to take part in a blind test and for data to be used as part of a group test. Sounds exciting and would be intrigued by the results.
Short reply for now, time for bed over here :).
Thanks @Audiogeek. No offence taken; let’s just agree to be equally superior 
. Your message is understood; that was not the message that I wanted to express as being ‘the one and only truth’. You have are nice card there, by the way…I’m jealous :).
’
Agreed : Fascinating. I would lie if I said I had no opinion yet. I AM biased already by results of measurements.
Then again : nobody is free of bias. I’ll just try to hold my opinion, until statistics have properly done their job. Hopefully…:).
To others, very shortly : @alec_eiffel : thanks to be willing to create files, if we ever get there. Thanks to @hifi_swlon for wanting to participate! Your name is noted. Let’s hope for enough participants, and for a testing protocol that suits every participant…(and me mysekf, most importantly 
! !
I’ll also participate of course! Let’s just wait for Leong’s response.
You know - you could come up with a world first here. An online, blinded audio listenening test, run through roon, with global participants. That would certainly be something. I appreciate your efforts!
As they say- " the truth always lies in the middle"
1 Like
Hello all, wow this thread has been rather busy! Sorry I have not replied any earlier cause I’m in a different time zone. Yes of course please go ahead and let’s do a listening test by all means. Really looking forward to finding out why I hear the difference.
Also I would like to say that this thread has been very civil and everyone that has contributed so far has been focused on reasons. It’s very good to see this happening using logic and not emotions. Thanks all for this.
A lot of the detailed technical discussions are beyond my understanding but may I make a request to whoever is creating the listening test to make it broad enough to capture the difference that is being heard today.
OK so let’s do it. I’ll prepare the test protocol, samples, and collect results.
I suggest we take two 30sec samples, of a different genre.
Then I will prepare 3 versions of each sample:
- without DSP
- with DSP/Null convolution
- a “secret” version, not saying what type of processing has been applied. So that nobody is influenced by any a priori.
I will then draw a letter for each of the 3 sample - let’s say F, H and P - and rename them Sample_F, Sample_H and Sample_P.
Then everybody will play the files in Roon without DSP and report to me in private, not to influence the others.
Then I’ll present all results transparently to the group. I believe if we are 8-10 at least we can have a clear answer: yes there is a difference, or no there isn’t and we have been fooled by ourselves.
Please provide me with proposals for song samples, the ones where you can clearly hear a difference between “No DSP” and “DSP with Null Filter convolution”. Preferably material you own and not TIDAL.
2 Likes
@Marco_de_Jonge, I’ll happily participate in testing and report my results. Seeing as I’m forced for the moment to use a headphone rig I may as well.
I’d suggest making 10 files and allocating F,H and P amongst them in a scheme of your choosing. That should reduce random success.
1 Like