The sorry state of equipment reviews


(Reader of the Internets) #1

Back in the 90’s, when I was reading those nit-picking reviews by Julian Hirsch of the latest audio equipment, I never thought I’d miss them. But the mainstream hi-fi press seems to have completely given up on them. At most they seem to re-iterate some specs provided to them by the manufacturer in press hand-outs! Along with playing some beautiful piece of music through it and providing a highly personal description of how they think it sounds.

I’d really like to see some exploration of how well parts of a piece of equipment do what they are supposed to do.

For instance, take a $5000 DAC. If it has an optical digital input, I’d like to have someone play some jittery source into it and measure how well it deals with that. If it has coax or USB, I’d like to see some connections to electrically noisy sources, and measurements of how well it deals with them. Ditto for power supplies – let’s put some line noise on the inputs and see how well it does. I’d like to have someone play a digital sound into it, capture the output waveform, and compare it to the supposed ideal version of that digital sound. This would all be very useful information in terms of matching it with suitable line conditioners and LPS systems etc.

As far as I can tell, the only ones doing this today are the founder of Audio Science Review, Amir Majidimehr, and the pseudonymous blogger Archimago. What Hi-Fi and Stereophile seem to have given up. Right now it seems that the only evaluation made possible is relative numbers of thumbs-up versus thumbs-down.


(Andrew Cox) #2

I’ve never been a fan of reviews in organs that take advertising from the manufacturers of the products under review. I accept that various responsible reviewers have valiantly fought to preserve their independence notwithstanding such sordid commercial arrangements, but as they have retired from the field or broadened their principles to ensure food on the table, the inevitable creeping “regulatory capture” eventuates.

Not everyone agrees with Archimago or Amir (would be surprising if they did !), but they are at least setting out to be independent.


#3

+1 on Julian Hirsch


(Henry) #4

The modern hifi press are sources of entertainment. Nit picking reviews piss off potential advertisers and bore the majority of readers.


#5

Good points all. However it was one of those reviews that introduced me to Roon.

Just saying.

–MD


(Henry) #6

I was just trying to find a plug and play way to listen to DSD! Roon was the first one I ‘just tried’ and it worked without sticking files in folders and associating plugins etc. That came later with ROCK! :joy:


(Ged) #7

I think hifi reviews are like all reviews - it is up to you to interpret what they mean and how it relates to what you want to know about the item under review. You get to know which publications you like more than others and individual reviewers and publication biases.
Personally, someone giving me jitter results would bore or terrify me depending on the context but nothing I would pay attention to necessarily. Magazines and reviewers have to make money and ensure that manufacturers will send them things to review so you have to take that into account.
I think industry reviewers are more consistent than “I scanned the market and I bought this for $10000 and now I’m going to give a review” gee that’s not going to be biased!
When we go on holiday my wife buys Vogue and the like - she’s never going to buy anything from the articles or advertisers. I buy HiFiXXX and very probably ditto as to purchase :slight_smile:


#8

I think there’s a lot of hocus pocus in computer audio so it would be great if someone sensible and independent could come up with a trustworthy view. So yeah, pick the nits.


(Henry) #9

I really don’t think there is. In fact it is astonishingly simple to get 95% of the available performance computer audio has to offer. The reality is a six grand music server still has a standard Mobo, memory and storage. You are effectively paying five and a half grand for power supplies and custom cases. That is the HiFi disease and reviews tend to contribute to that rather than eliminate it. Why? When was the last time you read a truly bad review? You won’t have read one in a magazine and even on line they are vanishingly rare.


#10

I’d be after a little more…

People are designing their own mobos I thought

Isn’t that the point, that we’d like them back.


(Henry) #11

No problem. I’ll sell you a OCXO clock upgrade to do that. It’ll cost more than the entire setup though!

Nobody is designing boards, just select the best one and buy off the shelf.

Reviews aren’t missed by many. If they were they’d actually be able to sell magazines or on line subscriptions.


(Reader of the Internets) #12

Magazines are of course in a tough place. They need manufacturers to buy ad space to keep going, and they don’t want to say something which will drive them away. And customers apparently won’t pay money to subscribe to webzines, so they don’t really care what the customer will or will not miss.

Amir and Archimago seem to largely buy the things they test, which necessarily limit them to lower-priced and less fluffy products. I did see a (very positive) review of the Allo DigiOne by Amir in which he states that Allo sent him the unit to test.


#13

This post was flagged by the community and is temporarily hidden.


(Reader of the Internets) #14

I suppose you’re right. Have to figure out a business model which would survive negative reviews. Tempting; about $20K in test equipment would set up a pretty good shop, I’d think.


(Henry) #15

Ha ha, well you certainly haven’t proved otherwise have you? I have been looking at servers recently. One of the most obvious things is they use quite obviously bog standard ATX motherboards. Some seem to favour AMD boards, probably because they offer the opportunity to under-clock the boards more routinely. But there are no linear supplies on board, no fancy on board clocks, no precious metals. Just good solid engineering in heavy casing with good thick interconnecting cables. Things like clocks are added as upgrades, ATX power supplies are bulky linear supplies. I have no doubt those who produce these items will impress upon us all of the work they have put in developing these products but the basis of these machines are high performance gaming boards onto which they bolt the items that make them ‘audiophile’ products. Perhaps that is tosh, but I try to keep myself informed. I am sure I’ll survive without your approval! :wink:


(Thomas Becker) #16

At this point, I tend to only follow, read and watch content produced by Darko.audio.

I find it entertaining but more because he realizes things such as Spotify and lossy music podcasts can bring great enjoyment. I would even enjoy a music-only podcast by @John_Darko as well.


(Reader of the Internets) #17

Perhaps on music. Not sure he knows much about equipment, beyond how to plug it together. Like most reviewers, his hardware reviews seem to contain much of what I believe our British cousins refer to as tosh and twaddle.


(Daniel Lundh) #20

I like Darkos stuff, mainly the videos. He usually starts out sensible and then ends up waxing poetical about how his braided USB cables enriches his life. It’s entertainment, I think it works. But honest reviewing? Nah.


#21

Hey, I post under my real name. :slight_smile:

I don’t think the traditional print magazines can deal with avalanche of audio products. There are literally thousands of DACs out there. I once counted over 1000 listed on Amazon alone! The magazine format of reviewing half a dozen products every month just doesn’t work. Right now, I am averaging one product test every other day. Each day I get one or two products I have either bought or has been sent on loan. I think I have produced 100+ reviews this year alone!

I also do fair bit of teardowns although not as much as I like to.


(James Vornov) #22

My sense after 30+ years of listening to systems big and small, cheap and dear, is that much of audio reproduction is a solved problem. Measuring equipment is good and cheap, plus we know how equipment needs to measure in order to sound good to most listeners. Then it’s tweaking response curves to provide a sonic signature. In the headphone world, brands provide inserts to let you choose your own sound to fit preference.

Back in the day, magazines got electronics with different levels of distortion, digital audio with artifacts and jitter, etc.

There’s not much nit-picking because everything measures more or less the same. What’s left besides trying to communicate the perceived sonic style of the unit? And these are so subtle, if they exist at all, that they can’t be detected in double blind tests.