Why not do some blind tests between the topping, holo, and mytek? Should be easy NOT to tell the difference.
2 Likes
I will post more later if your not able to understand my point, but are you ASR folks unaware of how double blind controlled tests in general work with humans? Is n of 1 a valid sample?
1 Like
What are you talking about? This is not a clinical trial comparing two forms of treatment requiring n=1000’s!
It was a controlled test to see if 1 trained individual could hear a difference. It was there for anyone to try.
Let me ask, have you ever tried such a test? Or do you rely on others heresay? Is your $5000 dac really audibly better (despite suspect engineering)? Are you actully intetested in finding out for yourself, your own ears in your own setup?
9 Likes
Considering that collectively you all are at n=0, I would say worry about your own stats before ours. We are infinitely ahead…
7 Likes
@Amir_Majidimehr So your reference for a DAC test is someone on your site is one who says:
“SWITCHING TIME: ~1 min to turn off amp and streamer, switch USB cable from Hegel to TT2, turn everything back on, switch inputs on the amp, and level match the volume.”
So your only DAC example doesn’t follow your own protocol, whereas mine did. I could switch instantly between the DACs (Lumin and Holo) from the streamer source and the output was level matched through identical Mogami balanced cabling. The switching was done blind, using the remote by someone I’d met an hour earlier. I used my system in my room with my speakers, because I never use headphones and the main issue was speaker imaging.
It is almost laughable that you seem to claim credit for the success of the Eversolo DMP-A6. Zidoo appear to have sent out test machines to just about anyone with an internet presence. The review I read was by Steve Huff as he was comparing to Lumin, which I had at the time. You would have to be living under a stone not to have heard about the DMP-A6. Most of the excitement was for the built quality and incredible screen for the price, never mind the performance.
https://www.stevehuffphoto.com/tag/eversolo/
Now that I know my test procedure with instant blind A/B switching was better than the ASR example you provided, requiring manual cable changing, and I was much closer to your own protocol, there’s nothing left to be said.
But keep digging.
2 Likes
Nothing you write here is of any help in selecting a HIFI product for consumers. You write many words, they add up to sentences, which add up to paragraphs. The result is what? Your wife selects speakers for you etc… I get that. But that will not help in my selection process.
There is expensive equipment, which measures terrible in the ASR setup. I am happy I can exclude that in my search. That is helpful for me as a consumer. I am grateful such a service exists. It can even help to make products better. If a manufacturer retreats to a position, that he has golden ears, well then the tests show that his ears are not so golden anymore, he better measures his future products.
16 Likes
Thanks for taking the time to write that. I find it very useful and plan to try it.
5 Likes
Fun, but I doubt the usefulness. If one gets 9/10 it likely is a signal, but anything else could be false negatives. Too many confounds: movement of IEMs in the ear, learning effects, frequency effects, blood pressure changes, random thoughts, number of sips of strong drink, accidental protocol slip ups, etc. A mass study might pass the sniff test. An ADC and a null test pulls the human subject out of the experimental design, but requires an exceptional ADC given the quality of these DACs/amps.
Sooooo, I suggest it’s good enough to just use excellent measuring components that lead to a high fidelity reproduction chain and have some fun otherwise.
I still haven’t ascertained what the exact complaint is about ASR. We have minor exasperation over tone, babble invoking cults, gnashing of teeth over the unfair treatment of audio vendors, and quibbles about website features (poll has a disclaimer now, BTW!). The measurements and the science is as sound as it was before this little tempest. On the other hand I read a handful of papers that enhanced my understanding of a few things, so I’ll treat sticking with it as a net win.
12 Likes
Sorry, i did not mean to offend by saying ASR folks. I am serious. Wouldn’t you need many people doing the test comparing many different devices to be able to say that the measurements provably represent what is heard? The problem is you have a measurement and some people say they heard x. But wouldn’t you need a reflexively large sample of people doing the test and saying they heard x to prove that it’s true to humans?
1 Like
You are asking the wrong question. The right question is if measurements provably, based on psychoacoustics show the device is transparent, how would you dispute that with listening tests? The answer is always given by some people like in this thread is, “well it sounds different to my ears.” To that, we say, OK, run a controlled test. Not just a “blind” test but controlled. If you refuse to take the test as we have seen in this thread, or fail it after taking it, then your claims were wrong and you need to reassess your position. In other words, if you want to counter science, your argument needs to be science based, not folklore.
If by chance you do create a statistically positive outcome while following proper protocols, then we take notice and try to duplicate to make sure there are no mistakes. After all, we weren’t there when the test was performed. If we can confirm it, then we will investigate as to the cause. Such a situation has not yet presented itself so talking about n > 1 is not material.
Now, you may be thinking of preference tests. There, you do want to sample large population to determine the distribution of such preference. We don’t get to this point if any level of audibility is not established per above explanation.
In fairness and as I explained, you also don’t want someone taking such tests that has direct interest in “not hearing.” It would be trivial for them to vote randomly in such tests. I caught a super vocal objectivist doing this very thing with foobar ABX where time stamps showed that he wasn’t even listening to the samples! I was highly critical of him and took the same times, properly listening and showing that I passed it. This is why people like Steven and Michal should be running such tests and not ask someone else to do it. They are motivated to hear the differences. Claim to hear things that we know are not audible to them, so let’s see proper proof rather than word salad posts.
18 Likes
Nice summary as we come to the end of the 3rd quarter of this game 
5 Likes
My SUMMARY of this thread:
I said in this discussion several times, you seem have missed it, this is my main point other than obnoxious culture there: Measurements are done correctly, but they are not SCIENCE! There is something called “scientific process”. Properly made measurements are the necessary, let’s say 20%, ingredient in the scientific approach here. It’s part of evidence. The missing part of evidence on ASR is the perceived sound quality of the reviewed piece. Then, entirely missing, are conclusions tying all this evidence into proving the hypothesis that isn’t there either. Correlating measurements and perceived sound quality.
ASR is useful as a measurement service. The conclusions as demonstrated by ASR are erroneous as far as the claim of sound quality or recommendation for good sounding products. 50% of times misleading. Take all tube equipment. The evidence for ASR erroneous conclusions is the huge gap between what ASR recommends as good sounding equipment and the real world where satisfied customers made their own judgement that’s opposite to ASR. Either millions of audiophiles who chose product ASR rejected are right, or Amir is right. It’s mutually exclusive. That’s why ASR has such a problem with acceptance by mainstream hifi market, while AES.org with its true scientific approach doesn’t have this problem
M
5 Likes
My publication record and patent portfolio suggest that I know something about this, including that you are attempting to misdirect us with the non sequitur that because ASR has “science” in the name it is therefore a forum for scientific research. It’s not. This has been discussed above and dismissed.
It’s also a non sequitur to suggest that ASR is erroneous as far as the claim of sound quality. We keep going round and round on this but we don’t have to. Insofar as one values fidelity of reproduction to the original recording, then components that provide fidelity are desirable. The tube people can enjoy their tubes (they glow nicely). The vinyl people can spin a disk (very tactile). I heard a wax cylinder recording recently (historical, amazing). Those interested in high fidelity can choose modern components.
Finally, you are merging the notion of consumer preference with scientific…something, but it is incoherent. AES values scientific methodology as an organization. Many members are also industrial researchers and product oriented. But even so, I don’t think any member of AES would suggest that because consumers sometimes like low fidelity products that means there is no motive to push the limits of transparent, high fidelity reproduction and the science and engineering that makes it possible.
My summary stands.
19 Likes
Did the reconstruction filters still sound different after measurements found them to be identical?
12 Likes
That is not the summary of thread. It is your position repeated over and over again no matter how much I have explained it is wrong and anti-science and good engineering.
Addressed before, more than once. Measurements are there to find out if you as a designer did your job to reduce impairments in the audio pipeline or not. It seems you have gone 2/3 of the way there but not all the way. The problem is that you charge a massive premium for your product vs competitors that have gone all the way in making sure all possible sources of noise and distortion are reduced. You also seem to have no regards for bugs in your product, leaving filter selection broken no matter how many times it is brought to your attention.
I do this all the time. Take this random example of Fosi SK02: (DAC and HP Amp for $100):
And this comment below it: “Even the worst case intermodulation is at or below threshold of audibility.”
How am able to say that? Because if you have a peak playback level of 115 dBSPL, the intermodulation distortion per above graph lands around 0 dBSPL. This is about the full dynamic range of hearing. So even discarding masking, these distortion spikes are provably inaudible!
Let’s look at your Bridge II product in the same test:
First thing we notice is that there are spikes going up to -88 dB, far higher than the $100 device above. Now I can’t make a blanket statement of transparency. Under certain situations it may be possible to hear the power supply buzz. Such buzz is never preferred in music.
Here is the dynamic range of SK02 using balanced out:
Here is yours:
If the playback level is at 114 dBSPL, we can say the SK02 is essentially silent whereas your DAC will be at 4 dBSPL, potentially having audible noise. We pay 50 times more money for your server/DAC but don’t even get the same fidelity!
See how well we are able to apply audibility attributes to measurements? And importantly, now we can achieve absolute transparency and hence have no need for listening tests. What goes into that SK02 is what is in the content despite its absurdly low price.
The above analysis by the way is backed up by Louis Fielder’s excellent peer reviewed research (Dolby Fellow and ex president of AES):
See the keyword there: “noise-free reproduction of music.” That is our goal, absolutely transparency and we are able to get that for incredibly low cost to day.
There is also good work done in this paper:
Noise: Methods for Estimating Detectability and Threshold, ” Stuart, J. Robert, JAES Volume 42 Issue 3 pp. 124-140; March 1994
If you don’t have access to AES or don’t understand the paper, here is a video I produced on that topic and related:
We weren’t around a few years ago so manufactures went to town, ignoring good engineering and providing measurements, and instead selling people on stories. You know, like “Mytek Sound.” With no proof point, they make you think a $20,000 USB cable is needed for good sound. That your product doesn’t come with a proper power cable so let’s spend hundreds if not thousands of dollars. While we are at it, why don’t we produce DACs, AMPs, etc. that are distortion and noise factories but we sell them as taking said audiophile to live venue. Surround by such nonsense, even the more logical and rational people fell for this.
Then I come around and at first, folks like you are just flippant. Who is ASR? Who is this Amir guy? A few years later, and 1,500+ products objectively reviewed, and science of audibility explained, the market is transforming. It is a big ship so takes a long time to fully turn. But turn it will. People have seen the light and the old tactics of just claiming fidelity to the ear without a single controlled test to prove it, is starting to stink like old fish.
You have a choice here. Your competitors such as Schiit have retooled their development and are not producing excellent products recommended by me. Or you can sit it out hoping people continue to be irrational and purchase things on claims and not proofs.
What you can’t do in mixed company is claim you are on the side of science. You are the opposite. You have not cited a single piece of research to back any of your claims. You keep linking to AES as a whole whereas I provide specific research papers showing we know what is there and you seemingly not.
Instead of worrying about “science” that doesn’t exist, I suggest doing your job well. I know you can. Measure your products, find sources of noise and distortion and fix them. Spend a bit of resource doing proper listening tests with controls in place. You will quickly learn what science says: that what you believe is not there.
Anyway you look at it, you are spitting in the wind. Audiophiles have adopted this new method of informed product buying and you are missing out on that business.
19 Likes
The brush you’re painting with is way larger than the canvas you’re trying to paint on.
Measurements of all tube equipment cover the whole range from excellent to abysmal.
The straw you’re clinging on to, seems to have buckled long ago.
As I have quoted many times before in these fora, Toole‘s work on sound reproduction, loudspeakers, and rooms has shown, that under controlled conditions a significant majority of listeners from a broad range of demographic groups preferred the most linear and undistorted reproduction of music In listening tests.
Note, that loudspeakers and rooms, or even recording studio environments overwhelmingly contribute to the sound signature and usually totally swamp the comparatively small contributions of competently engineered electronics.
That said, I’m almost always painfully missing information about speakers and room acoustics measurements in discussions about electronics‘ contributions to the listening experience.
Only after having presented all relevant data, could we possibly discuss some minutiae like those „harmonics‘ timing issues“ …
7 Likes
I’m not sure companies like this will see much of a choice here. Are customers going to continue supporting them when they realize they have been duped? I think the best we can hope for is business to continue as usual until enough consumers start using sites like yours to make better informed purchases. Then these companies will have to decide how badly they want to stay in business.
6 Likes
I’m not sure Rob is your technical friend here 
2024 CanJam NY Talk
He says some whacky things but is producing state of the art gear at the same time (ASR measured Mojo2, Hugo2, Qutest, Hugo TT2 - only Dave is crap but is the oldest)
5 Likes
Agreed. Main problem for non loudspeaker high-end companies is the fact that DACs, streamers and amplifiers are a solved problem. Measurably solved. For street prices that boutique suppliers have to be scared of. These ships have sailed. Last line of defense in this lost battle is repeating the good old nonsense. Also, the typical customers get older and what comes after is more critical and more inclined to harvest info as offered by ASR.
So, not many choices left. Watching dinosaurs die.
8 Likes
Exactly. I believe there is confusion here about transparency meaning the best sound vs. preferences meaning the best sound. If the goal here is to learn towards achieving the best sound for yourself, then unentangling this confusion seems crucial. I don’t see this as a game. I’m not trying to win anything here other than learning about how to achieve the best sound. I’m not seeing where this type of effort is happening at ASR.