Audio Science Review Discussion

Wow, I come back after a few weeks away and your perspective is still wildly incoherent!

  1. You should send Amir a fixed version with a few notes on how to turn the HAT feature off.

  2. No one is imposing their will on unwilling participants. Don’t look at ASR if you are not willing. Others do the same: I implooooooore you.

  3. You make a wild range of assertions about the sounds of harmonics and the dry, clinical quality of certain high-measuring DACs and with no evidence to back up the claim. Still. It just keeps getting claimed, over and over again.

  4. You want to do a battle of the sound systems to prove a preference, but haven’t yet done a blind A/B to prove any of your listening quality claims.

  5. I do, however, like a diversity of audio equipment and design philosophies like you, but also want to understand how transparent, low-noise, non-distorting, well-implemented with proper filter selection, and exceptional each one is. Personally, I may choose sometimes based on aesthetics and features even when the product measures less well than competitors.

In any case, I highly recommend refining your products and your communications strategy instead of this oddly ineffective approach you are currently using.

15 Likes

Michal, OK, you’ve now regurgitated this text from @Jut. And then you say, “Exactly.” But what exactly do you think it proves, or demonstrates? So some reviewer in Hifi Choice used his ears and thought the Pioneer A-80 was to blame for what he perceived as a lack of “depth”, and – well, I don’t know what “lige” is – and a lack of “ambience”. So what? These are typical subjective perceptual reviewer wiggle words which have no definition. And what do the other reviews of the A-80 say? Was it widely hated? By itself, this text means nothing.

I suppose that’s the key issue. What someone thinks “the listening experience” is. Or should be. And that’s a personal preference choice.

The people who want all their gear to “measure well” are looking for an accurate rendition of the recording they’ve decided to listen to. The folks running things like tube amps and the like, the “euphonic distortion” crowd, are looking for zoom-wowie effects, probably for particular pieces of music. So, really, I think we’re all agreed that the “listening experience” should be the main goal.

8 Likes

A minor addition at the end of both those sentences would be, " and believe they’ve found it. I’m beginning to think it’s not about right or wrong, but just what you like that (you think) you hear.

From my review: " That [Stereophile] review was in September of last year. I checked my sample and firmware was up to date yet this bug remains. "

In all this time, you haven’t really read the review to see things like this. And what is wrong with you contacting me post review, showing me your measurements contrary to mine, and offering a new sample? You thought posting here, arguing about “sound” is better plan?

Stereophile reviewed a sample you sent them, right? How come the filter bug was there as well? How can you not have tested such a simple thing? Answer: you don’t measure. You don’t look for problems. You keep using your ear even though it is not capable of perceiving such filter settings so you ship the product broken.

You say a lot of things. Not one of them is backed by any independent proof. No research. No controlled listening tests. Nothing. You also misstate my position.

I haven’t told you switching power supplies are better. I test plenty of DACs with linear power supplies which perform far better than your product. What I have said is that I don’t care what you use. Measure it and make sure you are not going backward in performance when you go from switching to linear supply. Don’t go by lay intuition and audiophile speak that linear power supply is better. Measurements conclusively show that you have gone backwards here.

As to tape, it has hiss. If you don’t hear it, then you must have a loud AC next to you or something. CD is therefore superior in this regard.

You keep talking about SINAD. Yes, your product produced dismal performance in that department especially when considering its price. But I didn’t stop there. I ran a full suite of measurements where your box produced lower than expected performance. Jitter test for example is not SINAD. You did poorly here as well:

Notice the large increase in THD+N at sample rate of 48 kHz. That is because of the broken filter you have selected. The rise is due to out of band signal not being attenuated per sampling theorem. You are pumping ultrasonics into the audio system and you say that sounds better? Really?

Even at higher sampling rate where that becomes a non-issue, you have more broadband noise and distortion than the reference $99 DAC in dashed blue. How do you sell this with straight face? By claiming again that these tests don’t exist?

How about linearity?

That exponential rise says that measurement is dominated by power supply noise. Again, this is not SINAD.

In a controlled test, your customers wouldn’t be able to tell the difference between your $5,000 product and a $99 DAC if the rest of components are the same. But you are not going to go there. You want people to listen with their eyes, not ears.

My voice carries far because it is built on solid foundation of audio science, engineering and objective evaluation of audio gear. No one is listening to me just because I made claims like you are without proof points. For your part, your backers are confused audiophiles and subjective reviewers who don’t know that their brain is making stuff up that their ear did not deliver.

You mean in the crowded market where there are thousands of DAC offerings for example, you realized that making up a marketing story lets you differentiate your product and then charge huge sums of money for it. Higher end audiophiles and reviewers with no idea of how unreliable their hearing is, believed it and the market shifted. Now it is shifting back when the curtain is removed and we see that for all the extra money you spend, you get less objective performance. And those claims of extra fidelity are not worth the bits used to store them on your company web server. Now we are going through the protest phase. And pleading to be believed phase.

Meanwhile your competition is taking notice and upping their game. They can still tell the stories if they want. But but also properly delivering on objective performance, they get the people who follow audio science/engineering to buy from them as well. Everyone wins.

19 Likes

Any discussions with you end up with righteous monologues about your obsession with your crazy mission to save the hifi industry. You can’t even agree to disagree. A professional conversation with you is impossible. Let’s see what results you get 5 years from now going down this path.

M

3 Likes

Ah, yes, cryptic prognostications and foreboding! One day, maybe not today, but soon, and for the rest of your life, you will regret having measured stuff. We will all be naked and afraid, with only a collection of exceptional audio equipment to listen to. Crawling supplicants with brilliantly voiced IEMs stuck in our ears like silver drachmas, we will realize that we are like a tone-row composer on his deathbed…it was all a tragic mistake! Bring back distortion! Bring back noise! Bring back filter implementation errors and all the uninformed choices that the consumer once had for inferior technology. Not just vinyl: scratched vinyl. Not just tape: old, melted cassettes turned back to life with a ballpoint pen shaft. Not FM: foxhole AM.

A professional conversation with you is oxymoronic.

16 Likes

To be fair: The Topping D70 Pro SABRE ( and other models) offers 7 filters and a so called „Valve Mode“. I am pretty sure everyone will find something to suit there taste here.

I just do not know if everyone has the courage to test a different filter than number 3. :wink:

1 Like

In this case it is known as you have measured it: mainly 2 peaks @100Hz and @150Hz, maximum -115b below reference signal level.

You can choose any realistic room in a home environment and analyze the noisefloor spectrum. Set a reasonable max. level for 0dB and tell me please if the remaining humming is above the thresholds of audibility or not.

Thanks. So we agree on one thing that this seemingly inferior SNR of -115dB is not relevant when it comes to audibility.

I would like to have an honest estimation of what ´less performant´ actually means when you publish measurements, including the remark if that particular one is most likely not to be audible. Simply because if it is not audible, I personally would dismiss terms like ´less performant´ or ´underperforming´ and find them highly misleading.

I personally find both approaches legitimate, but similarly misleading, unrealistic and contradictive to my own vast experience when it comes to the quality of music reproduction.

The ´euphonic approach´ might lead to more obvious flaws in reproduction and it is absolutely nothing for me personally, but it is at least an honest view.

The ´measures well = accurate rendition´ approach I could accept in case the following aspects are fully implemented:

  • being aware of and relying on listening tests for reproduction quality aspects which cannot be measured/technically evaluated, such as transparency, resolution, localization, depth-of-field, ambience/reverb, bass character, dynamics.
  • honestly accepting and adding any conclusions which measurements are NOT audible and refraining from drawing any conclusions from these (such as SNR below audibility, THD below audibility, step response, in-room-overall frequency response and alike)
  • focussing on all available measurements and technical evaluation methods that are most likely to influence the actual sound quality, including those which are complicated to measure or complicated to optimize in a system (examples: everything related to room acoustic, group delay <100Hz, time- and direction-related issues, directivity index over a broad midrange/treble band and a lot more).

If these 3 points are neglected, I see a risk of publishing measurements and basing verdicts solely on them easily becoming misleading. This is pretty ironic as chasing for records in irrelevant or misleading measurements seems to be a similarly absurd phenomenon as the ´purity cult´ of esoteric highenders while the ´team measurements first´ was always claiming to counter esoteric beliefs by the power of science and technical knowledge. Discussing why -115dB of noisefloor is ´underperforming´ compared to -130dB, insisting on 99.9999% OFC wires instead of 99.99%, claiming that PCM352 sounds always better than PCM96, putting cables on little feet to avoid them touching the floor, counting inaudible spikes in the spectrum and THD way below 0.1% @10K - where is the difference between these different nonsensical beliefs and dancing around golden spec calfs?

To be honest, I find that very sad. I started my journey to hi-fi in times when esoteric high end craziness and arrogance was omnipresent while scientific approach and technical knowledge were widely ignored and dismissed. Lots of gear was crazily flawed and sounded terrible, you could not warn people enough to not throw out their money. An institution like ASR and more people basing their estimations on science and technical expertise would have been highly appreciated some 25years ago.

Today, most of gear, especially digital components, is meeting high standards and there is almost no need for such broad apodictic warnings (exempt from things like loudspeaker choice, room acoustics and room correction which are funnily met with little to no doubt). But it seems that with a delay of 25 years, the old verbal trench wars are arising again.

Nothing would be more needed today than learning from each other and finding an approach to better music reproduction. I can tell from own intense and decades-long experience, that both sides would find out surprising things.

4 Likes

What? I have always said that once you achieve total noise+distortion that is less than -115 dB, then you have provable inaudibility. There is no one asking for -130 dB, nor is that achievable. I would happily recommend a DAC that produced sum total of noise and distortion that is at -115 dB. That is not what we have here with Mytek Bridge II. We are talking sum total of noise+distortion landing at just -98 dB per SINAD:

Here is how a $15 Samsung phone dongle performed:

I don’t care what your view of audibility is. The designer of Mytek is claiming 40 years of experience yet can’t keep sum total of noise and distortion to be lower than a $15 dongle?

We are 45 years past the introduction of CD’s 16 bit format which means -96 dBFS noise floor. I don’t know how anyone making any DAC can say with the straight face they have done good engineering when they can hardly clear this hurdle let alone any catering to high res music.

You also are forgetting that we need headroom for DSP correction both for speaker/headphone and the room. This will cost you dynamic range.

That is totally backward. The high-end audio companies have made a mockery of audio performance. They destroy the fidelity of the audio pipeline and then ask you to pay 10 to 100 times more money for the privilege of it! Here is a $7,000 DAC by PS Audio:

SINAD of just 75 dB with distortion spike not at “-115 dB” you claimed but at -80 db!!! Crappiest first generation CD players would run circles around this DAC.

Here is a $10,000 DAC:

Granted, it has DSP but look at the harmonic spray there. Compare that to this $80 DAC:

Fully achieving transparency. If I had not measured, how the heck would you know that this DAC is something you should look at? And that the Weiss DAC is highly overpriced?

Here is a $7,500 optical player:

It underperforms that $80 DAC for heaven’s sake. The need for measurements has never been more important than it is now. People sell $100,000 DACs for heaven’s sake. There never was such excesses in the past.

That makes zero sense. What on earth you want me to learn from the other camp? That faulty listening tests are good? That price is proportional with performance? That empty claims are valid because they all repeat it?

You want to buy blind, be my guest. I and tens and thousands of other audiophiles want to optimize our budget and put the money where it is needed. We do that by measuring and understanding the science and engineering as it applies to them.

12 Likes

Iam still not quite sure what you are saying. Are you suggesting rather than use spinorama to shortlist speakers we get a group of engineers/accoustic engineers to work out what speakers we use at home??

By the way, what you suggest is not how any recording engineers I know work.

Can you share a speaker or speakers you think we should use at home?

1 Like

Which part of that would you describe as clearly negatively audible under home conditions and why? -98dB of THD to the reference signal translates to roughly 0.0015% of nonlinear distortion. From my understanding there is zero scientific evidence that is is recognizably audible as distortion. Or could you proof otherwise?

You have made that clear already. Thing is that most of people I talk to being in the market for hi-fi gear care for audibility and best sound. If you want to propagate buying decisions based on measurements instead which we are both agreeing on are inaudible, be my guest. But do not expect anyone to take this more seriously than people aiming for 99.9999% OFC wires or putting their cables on little feet to avoid them touching the floor.

So it is some matter of personal offense and revenge instead of honest consumer advice? In this case i cannot help.

So what? People are allowed to make a free choice of buying that gear or not. As far as you do not have a valid point about inherent audible flaws, I see absolutely no reason in issuing warnings or insulting people’s gear may be be overpriced in your eyes but nevertheless doing what it should. That’s in my understanding valid to most of digital components today and the majority of amplifiers.

What makes me sad is that at the same time inaudible specs of DACs are discussed, people are struggling with real acoustic disappointments, major flaws of hi-fi gear and confusion about making the right decision. I am talking about how loudspeakers and rooms are interacting with each other. My advice would be: if you want to save people from disappointing hi-fi decisions, focus on these aspects, do relevant measurements and educate them. There is really a demand for that while nowaday´s loudspeakers and rooms are becoming in the majority increasingly incompatible with each other and misinformation about room correction is everywhere.

First and foremost to accept which aspects of hi-fi reproduction are not measurable or determinable. You do not have to listen to high end electronic manufacturers or audiophiles, talk to leading pro-audio loudspeaker designers, room acoustic specialists or recording engineers instead.

Secondly to accept which major misconceptions and flaws of loudspeakers (speaking of directivity index) and room correction systems are still pretty common although there are technical solutions to these problems. You claim to apply science and engineering on the question what defines good sound, but when reading some of your reviews (KEF, Genelec and alike) it occurred to me you seem to avoid the real deals and focussing on completely irrelevant aspects instead in your reviews of DACs and alike.

Thirdly I sincerely recommend to talk to hi-fi users and music lovers in the real world. There is a lot to understand about what people really want and what their actual problems are. I cannot count how many times I have heard people complaining about booming bass, annoying, harsh hissing and no center localization. Never I have heard anyone whining about ´Help me, my DAC produces an unbearable amount of 0.0015% of THD!´ or ´PS induced noisefloor of -110dB is so annoying that I cannot listen to music anymore!´. These are simply not existing problems out there.

6 Likes

Nonsense gets somehow overwhelming in this thread. A manufacturer tries to sell his idea of needed distortions, so that his crapy product attracts some buyers. That “hiigh-end” market is nothing but crap, full of charlatans. I am happy ASR exists and takes that junk apart.

13 Likes

That´s for sure. But @Arindal makes some valid points to think about.

I like that, I want that too. But isn’t it most important to tell people what comes first and what matters most. You don’t need to spend a lot of money, just have a large room and no one that interferes with the interior of that room and build your own treatment.
I know audiophiles who spend hundreds of thousands of € and have a tall equipment rack between the speakers that destroys the center imaging. They don’t treat the room for low frequencies and don’t want to use the dsp for sound quality reasons, they say. I can tell them 100 times that they should do something about that. But I don’t think changing the high end dac for a Topping with less THD would ever help them getting a stable center image and good bass. When I see all these THD measurements I can only think “why”. When you and the members of ASR have fun with that it’s ok as long as there is some sort of disclaimer that ultra low THD doesn’t necessarily get you good sound but it’s just there for the fun of measuring stuff.

I am sorry, I was not allowed to edit my post (within the 4 h). It has to be “life”. By the way, this is from a group test of 75 amps with a panel partially blind listening.
,

1 Like

Well, different “audiophiles” want different things. Merely a matter of preference, and no objections to that on technical grounds. (Ethical grounds might have something to say, though.) In fact, there are probably a number of different classes of audiophile.

  1. The measurement fans. Maybe they’re uncertain of their ability to listen critically, maybe they don’t trust the shifting tides of perception, maybe they are trying to optimize value for money. But generally, they are looking for accurate rendition of the digital recording. They do care about speakers and room treatment/correction as well as low noise and distortion figures.

  2. The “euphonic distortion” crowd. They think it’s nifty to do things to the music, instead of just finding other music. They may be nostalgia fans. They may be tube-rollers, or op-amp rollers. They like to tinker with their systems.

  3. Guys who spent too much time reading Playboy in the 1970s. They think a stereo system can help them get laid. They like it to look all shiny and cool, everything on display in custom racks, bizarre-looking speakers, whatever they can find. Barry White plays at the drop of a switch. Blind testing would probably show that this approach is suboptimal.

  4. Guys who understand that an understated display of wealth actually can help them get laid. They want an elegantly displayed or even hidden system that dispenses not Barry White at the drop of a switch, but perhaps Johnny Hartman, and reeks of free cash flow,

  5. Guys who went to too many rock concerts and can only hear bass. They want the largest speakers that can fit in their rooms, and the most powerful amps. They don’t care about distortion because they can’t hear it.

So, who am I missing?

7 Likes

I would not rule that out as a source of bad center localization but according to my experience in most cases early and asymmetric reflections from side walls, uneven, low directivity index of the speakers, an overshoot of discrete reflexions in general combined with listeners sitting way too far from the speakers are the main roots of these annoying flaws.

Install 3 diffusion panels, get speakers with even directivity at medium index and adjust the listening distance accordingly - voilà!

Bad center imaging, intolerable level of discrete reflexions and booming bass are typical examples of annoying phenomena you in most cases cannot solve by EQing. You can reduce the level in frequency bands which are most annoying and thereby make it less dominantly audible by getting masked, but the risk is high you indeed have side effects which deteriorate the general sound quality. In many cases reproduction becomes subjectively ´lame´, lacking presence in certain instruments, revealing audible contradictions between frequency bands (mostly a bass separated from the rest of the spectrum) or just sounding dull.

That would not make sense as they choose speakers for completely different environment. But the example is useful to understand which aspects of evaluation are used in professional listening tests and are not determinable by measurements.

So I would rather suggest to reproduce the evaluation method than the results. When doing that, I found it pretty useful to have 1 or 2 recordings per quality aspect being evaluated.

If the listening situation would be defined by a full-band RT60 measurement, frequency response and waterfall plot below 300Hz at the main listening position and neighboring seats as well as parameters like listening distance, position relative to the walls and behavior of the walls in the low-frequency region, that would be possible.

For a large part of that group I would rather say: they claim that they care for speakers and room acoustics, but they trust measurements and automated routines more than their own ears or lack the experience to optimize a room without measurements (which is admittingly difficult and requires years of training). Many of them are negating the existence of non-determinable and non-measurable aspects of sound reproduction quality.

So I would come to the conclusion, yes you have missed at least 2 groups:

  • no. 6 almost the entirety of younger people, the ´Generation iPhone´ and ´generation Spotify´ being both pretty interested in music reproduction while having a completely different approach to gear. They are neither interested in measurements nor in big gadgets nor in technical toys, but see hi-fi gear rather as an everyday tool for enjoyment. While it is difficult to get them to like technical details, they have a pretty good intuitive understanding about things and how they want their music to sound like. If anything, they are mostly thrilled by cool features and software as well as tiny, shiny gadgets and simply good sound fitting this ´playlist´ approach.

  • no. 7 who are understanding and have managed to overcome the limitations and inherent flaws of group 1 & 2´s approaches to create some superior philosophy on optimizing sound while listening to music listener´s needs and preferences. They seem to have superior knowledge of measurements, acoustics and psychoacoustics, and a surprisingly high number of them are successful loudspeaker developers around the world - may it be consumer audio, high end, pro-audio or sound reinforcement.

2 Likes

This is great! I would read an Audio Room Science Review. I already use Dirac, REW, etc. and apply physical treatments to various degrees. There are some great REW discussions, tips, tutorials, pointers, etc. at ASR already, but consolidating and focusing would create a vibrant community.

What are “the real deals” you expected to see when reading KEF and Genelec reviews/measurements?

1 Like

They are told that. But that doesn’t mean we stop testing everything else. Plenty of broken products are shipped in all categories.

Don’t agree but not going to get into it now.

Again, don’t agree. Speaker radiation in that acute angle is mostly low frequencies which won’t be part of imaging. What there is there, will act as some scattering. Since both ears hear what comes from that angle equally, spatial qualities are not changed.

They don’t need to treat the room for low frequencies as that can’t be reasonably done. DSP is heavily talked about in ASR. People have strong preference for them. Almost every review I do of speakers and headphones is an advertisement for Roon and DSP.

No one is confused about that except people in the subjectivist camp who don’t look at measurements anyway. If you need a DAC, my measurements give you many options across the entire price range. You choose accordingly but know full well that there are important things like speakers and rooms.

You ask the wrong question. I hardly ever measure “THD.” Measurements are for THD+N or SINAD which includes the all important noise. You better pay attention to noise as I have repeatedly explained. Any boosting you do with DSP costs you headroom and more noise performance.

I don’t have to say sun comes out the east. Spend a bit of time on ASR and you quickly realize there is no confusion there. Just about every state of the art DAC I review gets multiple complaints saying, “why do I need a higher SINAD DAC?” If they ask me, the answer is some of us can afford the best there is and we may want to buy that. If not, here are 100 other DACs you could buy. No one is forcing you to buy this one.

12 Likes

Can this result in a loss of fidelity? In a room with speakers or headphones when you consider preference and perception, how would you know?