What kind of difference matters?

Roon Software Discussion
1

(I wrote this in a different thread, but it’s a general observation.)

Looking at all these discussions, the mind boggles.

Indulge me, play through some scenarios.

My son used to play trumpet, he switched to the standup bass, came to my house for a demo, I could clearly tell the difference, even when we did a double-blind test, his girlfriend replace the instruments, I could easily tell. Ridiculous, no?

To stay with the same kind of instrument, my friends and I did a blind test between Kenny G and Coltrane, we got 74% success rate on the difference. Ridiculous.

We did a blind test with recordings of Clapton on acoustic guitar, on his Unplugged album, and the electric he usually used, again we could tell with 62% success rate. Still ridiculous.

We tested speakers, the Wilson Alexia vs. the Bose Wave. Ok, I’m being silly with the Bose. Wilson Alexia vs. Magnepan 30.7, both highly regarded, but of course we could tell the difference, right?

Headphones, comparing the Sennheiser 800 with the RAAL SR1a: no question, both are great but there is no debate about whether we notice the substitution. Right? Or STAX 009, or Hifiman Susvara, or Abyss. Of course, the difference is dramatic.

DACs, it’s more subtle. In my experience, comparing the Chord M-Scaler with the Benchmark DAC3+HPA4, the initial impression is positive on both, pretty similar, but over time, with experience, the superiority of the Chord stack is evident. Ok, superiority is subjective, but the difference became quite obvious.

Ok, now let’s talk about AC power filters. USB direct-connect vs. networked. And optical vs. Ethernet. And USB regenerators and reclockers. And cables, USB and Ethernet. CD resolution or high-res, or DSD or MQA. I’m not going to argue about whether they make a difference. But when you read about people recommending one solution, you often see the phrase “we did this or that test and we could clearly tell the difference.” And people argue about that. And blind tests, and confirmation bias.

You don’t hear people talking about confirmation bias between a trumpet or a bass, or Kenny G or Trane, or Wilson or Magnepan. Or Lagavulin or Southern Comfort.

5 Likes
2

oh my… what did you fail on?

harder to test blind… feel is hugely different between headphones

1 Like
3

Yeah, but people in North America drink Lagavulin (and other malt Scotch whisky) with ice. That will get you deported from Scotland. As a Scot (expat) I’ve never done it of course, but I’d wager Lagavulin with a glassfull of ice tastes a whole lot more like Southern Comfort with a glassfull of ice than they both do with an equal volume of room temperature water. I’m not sure what sort of bias that is, but it’s clearly an independent variable.

5 Likes
4

I’m not sure what your point is.

If you could tell the difference between A and B in a blind test, you could certainly tell the difference in a sighted test. The converse is clearly not true: just because you could tell the difference in a sighted test does not necessarily mean you could tell the difference in a blind one.

Of course, in some happy circumstances you surely would be able to. When people express skepticism about the result of some sighted test (and suggest doing a blind one), they are implicitly suggesting that this is not one of those happy circumstances.

In all of the ‘controversial’ examples you cited, setting up a blind test is easy. Rather than arguing about whether a blind test is necessary (since the result is ‘obvious’), people should just get it over with and do one.

If you’re stuck at home because of the pandemic, what else have you got to do?

3 Likes
5

I spent most of my life building a deep understanding of networks, networking technologies, infrastructure, hardware, applications, etc. People have even paid me for this knowledge; some pretty generously! I still don’t understand how people are hearing differences in ethernet cables.

I’m not saying they are not hearing differences. I’m just saying I don’t understand it. This ends up with me investing just about $0 in ethernet cables. But, to your point, yeah I’ve never understood why people try so hard to convince others that they didn’t hear something. Hearing is a feeling. All the evidence in the world isn’t going to unconvince someone they didn’t feel something.

Also, if you have fancy ethernet cables I’m happy to test them. Make me a believer! I want to believe!

4 Likes
6

It isn’t networks that matter. It is the interface between the networked card and sensitive analogue circuitry. Very little in ordinary network practice has this particular set of challenges.

7

Really? Only 74%??? :astonished:

1 Like
8

You should. Water (and to some degree ice) helps add dynamic range to the experience. Not always what one wants, but it can be nice sometimes. Also, Scots that say there is one true way to drink Scotch should just give up now and hand the mantle over to the Japanese. I have no patience for that type of closed-mindedness.

You guys clearly haven’t tried any SoCo since you were in college :joy: It’s got some bourbon in it, but it’s neutral grain alcohol + fruit liqueur mostly. You might as well be comparing Bacardi to Lagavulin.

1 Like
9

Hey, I resent that! My most memorable Lagavulin 16 experience was with a bit of ice hacked away from a glacial cave in a Zermatt glacier ski tour. Glacial ice melts extremely slowly and delivers just what’s needed to open up the flavor. But I’d not use random tap water or tap water ice with quality single malt.

10

True, never tried Southern Comfort at all.
My most memorable youth drinking experience was during my compulsory army year in Sweden, the guys would drink screwdrivers by swigging some vodka from a bottle without swallowing, swigging some orange juice from another bottle, and then swishing it all around in the mouth.

1 Like
11

I was in Antarctica and the guides explained that the glaciers form from falling snow, initially white. It gets compressed by the weight of more snow, and after about 800 years all the air is pressed out and the ice is blue. But the pressure keeps on, and after 30,000 years the pressure has changed the molecular structure of the ice and it is glass clear. It eventually is “calved” out into the ocean. From a boat we picked up a chunk of the clear ice and brought it back to the bar of the ship. Excellent. Since then, I refuse to use any ice that isn’t 30,000 years old…

2 Likes
12

My fault. I want to make a point and I think “now hear this, just do what I say” may not be effective. It amuses me to try to trigger some new perspectives with crazy comparisons. (For the record, none of those comparisons happened.)

I am not entering the debate about what is audible. What I’m suggesting is that some things — speakers, rooms, headphones, the music — matter more than others. And yet, this forum, other forums, blogs and the commercial press is very focused on the tweaks.

1 Like
13

LOL… :clap:

14

That works but it also calls to mind a very evil concoction called a Cement Mixer.

15

Some of your silly examples raise an important point: familiarity. Being somewhat familiar with Coltrane and less with Kenny G., I would probably score much more poorly. If they’re all songs that you know, you should score 100%. Similarly, on the hi fi system that you know, you may be able to hear subtle differences when various components are swapped out.

I’m an amateur pianist and do not have perfect pitch, but I do know how to tune my instruments. When I hear a piece that I play from Bach’s Well-Tempered on an instrument tuned to A=415, thought to be closer to the historical pitches, I’m pretty sure I can sense the pitch is lower.

A long time ago, when my hearing was much better, I took home interconnect cables from a dealer and tried them out. I concluded that any differences I thought I might hear I did not care about sufficiently to spend big bucks on them. Certainly not the “night and day differences” the high end mags were touting.

Some of the criticisms levelled at double blind tests are valid. It is not how we normally listen, and in any event to evaluate fairly is a lot of work. Sometimes after a while, we just want to get back to listening to music. If you have skin in the game, especially if you make money by writing audio reviews, the pressure of the possible effect on your reputation could seriously influence the results.

Yet, if a difference really exists, some homo sapien on the planet should be able to detect it under double blind conditions. If nobody ever has, I don’t personally hear the differences and there is no plausible explanation for the differences based on accepted scientific principles, I don’t waste my time or money on them. Boutique AC power cords and digital cables fall into that category for me.

Double blind testing is the only way I know out of the point raised by the dude in the back of the crowd in the Firesign Theatre album: “That’s metapheesically absurd, mun, how can I know what you hear?”

2 Likes
16

Interesting. Does it sink in liquid water?

17

Pfft, people in N.A. drink blends.

Johnny Walker Black was always my downfall and I can tell the difference between Red, Black and Blue.

18

So, sometimes the difference is so big a ‘blinded test’ is ridiculous. It applies to the vast majority of observations/decisions we make every day.

Sometimes the differences are harder to discern, but the difference is important. Drug trials are done this way for this very reason. First, is there a really measurable improvement in outcomes.? Second, is the difference warrant attendant risks. The decision here is truly important for a population and will make a difference for a significant number of people. Blinded tests are necessary and the outcome matters.

But sometimes the difference is so small it ain’t worth the consternation. Even if there is a difference, it isn’t important. That’s where I am after doing this for fifty years. By now I have a pretty good sense of where you’ll find the important differences and where you are chasing ghosts. If people wish to get all worked up over what is basically nonsense, that’s fine. I’m not playing anymore.

3 Likes
19

Yes, exactly.

My simplest advice is, if you are assuring somebody (or yourself) that the difference is audible, it’s a waste of effort.

1 Like
20

Yep. It floated in the ocean, where we picked it up.

Also floated in gin and tonic, and in Campari.

Never found out if it floats in whisky.