Are Hifi Reviews Flawed?

[Welder]

http://theaudiocritic.com/plog/index...Id=18&blogId=1

It doesn’t really matter what some golden eared journalist, if I don’t write nice bullshit about this product I wont have a job next week thinks does it coz as all us subjectivists know, what he/she hears with his/her kit ain’t going to be nothing like what you hear is it? So it’s pointless unless there is some set of objective parameters right?

Wadaya mean NO!
You can’t have it both ways ffs. Either we trust out ears and apparently we don’t all hear the same, I mean, if we did that would be objective then wouldn’t it, or we trust to measurement which hopefully wont get altered by financial considerations.

[MartinT]

John - all magazines can do is give a guideline so as to create a shortlist. Otherwise you are left with a sea of hundreds of products and no way to filter.

The trick is to latch on to a reviewer whose musical and equipment tastes match your own.

[Neil McCauley]

John - all magazines can do is give a guideline so as to create a shortlist. Otherwise you are left with a sea of hundreds of products and no way to filter.

The trick is to latch on to a reviewer whose musical and equipment tastes match your own.

Ken Kessler in my case.

[Welder]

Hmm this find one you trust business sounds okay if you say it quick enough

Point is though, as the recent thread about Quad’s Peter Walker illustrates
http://theartofsound.net/forum/showthread.php?t=13291

the people that design Hi Fi components tend to design to a specification and like it or not that means measurement.

The very silly may choose to believe that once the required measured performance is attained the designer and a few trusted ears fine tune these components by ear but if that were in fact true, what was the point of the measuring bit if your going to say that doesn’t sound right with speaker A but is okay with speaker B but not with cable F but not on Wednesday?

Do you see where I’m coming from? You can fine tune by ear but only to a particular set of criteria after that its pointless because the likelihood of anyone buying that product and having exactly the same ancillary components, listening environment and hearing is non existent.

The next problem is of course if you’re a company, how do you instruct your technicians to build a component to “how it sounded last week at Bills place”?
You just can’t can you. You have to build to some specification.

Then of course there is the cost involved. Just how much time can you afford to spend listening to the various equipment permutations? What do you do, take two or maybe three people into a room and listen to music for a few weeks?

Of course the manufacturers say they’ve spent thousands of pounds and listening hours in developing their product. But, when I worked in the avionics industry and they have huge R&D budgets it went more like;
Get what the opposition are making.
Measure that.
Rip it to bits.
See what can be replaced at less cost with improved specification.
Make a prototype.
Make it look pretty for customer (yep, even the military like eye candy)
Get an order.
Get the product out the door as fast as possible coz you know your competitors are doing exactly the same.

There have been a few exaples here on AoS of a product going from conception to a manufactured reality. Anyone get asked to participate in any listening tests?

So the Hi Fi reviewer gets a unit. If you’re lucky they’ll actually listen to it for a couple of hours. What can they say about a product? The simple fact is the differences between one amplifier and another say in a properly set up review are going to be slight. After a couple of reviews you’re kinda stuffed for descriptive stuff which is why the reviewer writes so much crap. They can’t do much else can they coz describing how something sounds is hard just once let alone issue after issue.
They can’t write bad stuff about a product coz the publication just won’t get offered kit to review afterwards.
Most journalists don’t know shit about electronics/acoustics so the cant write technical stuff.
It seems from some of the reviews I’ve read that some of these journalists don’t even know how to plug the stuff in properly.
Above all what needs to be remembered even by those who believe in the Hi Fi fairy is that those in the Hi Fi media have a vested interest in perpetuating all and any myths regarding Hi Fi that can help keep the market ticking over.
Its fact. It’s the way of the world.

Once you’ve accepted the above then might be the time to try your hand at the win £25K in the pick that amp competition. Don’t tell me I’m wrong, prove it to me.

[Marco]

Hi John,

Hmm this find one you trust business sounds okay if you say it quick enough

Point is though, as the recent thread about Quad’s Peter Walker illustrates
http://theartofsound.net/forum/showthread.php?t=13291

the people that design Hi Fi components tend to design to a specification and like it or not that means measurement.

The very silly may choose to believe that once the required measured performance is attained the designer and a few trusted ears fine tune these components by ear but if that were in fact true, what was the point of the measuring bit if your going to say that doesn’t sound right with speaker A but is okay with speaker B but not with cable F but not on Wednesday?

Do you see where I’m coming from? You can fine tune by ear but only to a particular set of criteria after that its pointless because the likelihood of anyone buying that product and having exactly the same ancillary components, listening environment and hearing is non existent.

The next problem is of course if you’re a company, how do you instruct your technicians to build a component to “how it sounded last week at Bills place”?
You just can’t can you. You have to build to some specification.

I get where you're coming from, but you're missing the point somewhat, me old son.

I guess that when it comes to mass-produced equipment from huge mainstream manufacturers, particularly at the budget end of the market, there has to be some form of 'objective template', based on set measurement criteria, for the people who build the equipment to follow, in order for said products to be manufactured to a high standard of safety and consistency.

I'm guessing that's the case, but I've never been inside the relevant factories when equipment is being produced to find out. Perhaps someone who has could comment?

However, when it comes to smaller, more specialist manufacturers, it's a different kettle of fish. They most definitely listen to the results they achieve, building equipment based on objective measurement (in order that the equipment concerned meets certain basic electrical parameters), but they voice the final results by ear.

'Building to a specification' (referring to the part of your text I've highlighted in bold) is achieved by small, specialist, companies making bespoke equipment, through fulfilling the standards set by the designer in every instance when a product is made, i.e. once equipment has met its basic electrical parameters, it is then listened to and checked by the designer in a familiar test system, or someone he or she trusts, to ensure that the equipment passes the required standard, before it leaves the premises and is shipped to dealers or direct to end users.

*That* is their version of 'building to a specification'.

Perhaps that's why the best bespoke designed equipment often sounds superior to kit which has been churned out by major manufacturers in huge quantities? That's certainly mostly the case, in my experience!

Of course the customer cannot replicate the system testing conditions, in his or her home, used by a designer manufacturing bespoke equipment, and so the results they achieve in their own systems will always be different. But then that's the case with all equipment, both bespoke and mass-produced.

Also, bespoke equipment will never sound absolutely identical from box to box and achieve the same exacting standards of consistency maintained by large companies making mass-produced products, built to a standardised 'objective template', but such is the nature of the beast.... And their customers wouldn't want it any other way, because they trust the ears and expertise of their favourite audio designer!

I'll ask you a question... How do you think Anthony TD designs his superb sounding valve amps - do you think he simply solders a few bits together, measures the results, and as long as everything is 'A-ok' in a measurements and electrical sense, he just packs it up and ships it out?

Of course he doesn't. What happens if, despite what he's built fulfilling every known rule in the 'objectivist's book of measurements', it still sounds pants? Or rather, mediocre compared to how he thinks it should sound, or indeed how good it *could* sound with some minor adjustments?

Does he just shrug his shoulders and say to himself: 'Well, I've built the amp to the 'required specification', and it measures wonderfully, so fuck it, I'll sell it anyway even if I think it sounds pish...!' Of course he doesn't, he'll take it off the bench and tweak the design, making some subtle component changes, measure again, and then LISTEN to the results, until it SOUNDS right to his ears and meets his relevant safety criteria...... Then, and only then, does it get sold to customers.

*That*, my friend, is how the best audio equipment is made, and what Peter Walker should've done all those years ago with his sonically dreadful (in stock form) Quad IIs, but because he worshipped at the altar of measurements (as people like him often did), was too blinkered and arrogant to do so...!

However, perhaps 'adequate', to him, was enough? In my experience, it often is in audio with staunch objectivist types

Marco.

[The Grand Wazoo]

Interesting points, all of them, but here's a thought about old Mr Walker:
Given the time span he was operating in, & given the market he was selling into, do you think he would have sold any more amplifiers than the many thousands that he did sell over the 18 years they were on the market, if they had sounded better?
Personally, I don't think he would have (& I don't particularly like the II amp).

[Welder]

In general I accept what you’ve written Marco and I have no doubt that in fact a great deal of Hi Fi component manufacturers do in fact listen to their products
However, the designer still has to have an element of repeatability to his/her design and if some amplifier designer is prepared to contribute to this thread stating that each amp he/she builds is built in isolation from the last unit built (ie none of the specifications of a previous build are used) and the final configuration isn’t measured and recorded then I’ll have to take their word for it.

What I believe happens is, they may indeed initially design to a specification and then fine tune by listening but the changes have to be recorded and repeatable (?)and unless the designer has a phenomenal aural memory, they will build to a written and measurable specification finally, even if that final spec was established by listening and recording the changes that gave the required improvement.

Despite your views regarding Peter Walker and Quad amps they have a very high audiophile reputation so building by measurement and specification does at least seem possible even if you personally don’t find the results pleasing

I’m not sure the “best” audio components are bespoke built Marco. I wouldn’t argue that there are some very talented people involved in “bespoke” audio but I suggest the really large companies with their enormous resources tend to rule when it comes to quality audio reproduction. Bespoke is cool, you get something individual and possibly unique. If you’re lucky you get value because the small manufacturer doesn’t have corporate overheads etc. This doesn’t necessarily translate into best sounding available and I doubt very much if it applies at all to equipment using newer technologies.

[Reid Malenfant]

However, the designer still has to have an element of repeatability to his/her design and if some amplifier designer is prepared to contribute to this thread stating that each amp he/she builds is built in isolation from the last unit built (ie none of the specifications of a previous build are used) and the final configuration isn’t measured and recorded then I’ll have to take their word for it.

If you are going to produce a load of one type of thing then this is how it has to work..

Components are graded, you have different tolerance transistors (gain), capacitors & resistors etc (value +/-% of ideal) etc. As long as the components selected fall inside of the critical categories then there should be very little measurable or audible differences between each unit. 1% resistors kind of help here. Does someone honestly want to say that they can hear a difference between a 10K resistor & a 10K1 resistor in a circuit, come on..

Obviously there will be some on the margins (for what they are worth) that will be slightly better or worse than the unit that was initially listened to & put into production. The differences though should be pretty insignificant or there would be no production lines...

Excuse me, i'm on the lash As usual

[synsei]

A good equipment example for this thread are Stan's DAC's. I'm sure Stan would be the first to agree that they are not as good as he could make them if money was no object, but then that would be missing the point entirely. Stan has cleverly found the middle ground and has produced a DAC that sounds great, costs very little, and the icing on the cake is that he is engaged with his customer base in the development of upgrade mods. His DAC's are as good as he can make them for the price and no one could argue that he doesn't know what his products are capable of

[Marco]

Interesting points, all of them, but here's a thought about old Mr Walker:
Given the time span he was operating in, & given the market he was selling into, do you think he would have sold any more amplifiers than the many thousands that he did sell over the 18 years they were on the market, if they had sounded better?
Personally, I don't think he would have (& I don't particularly like the II amp).

Hi Chris,

The answer is very unlikely, because he successfully built a respected reputation, and through that, succeeded to surround his products in an air of mystique, which people bought into, rather like Tim de Paravicini does now. It's a significant reason why vintage Quad gear fetches the prices it does now on eBay, and such like.

And going back to the old days, when people like the BBC start to use your products, then that again adds further kudos and desirability to the Quad brand name, therefore it goes beyond the equipment itself achieving ultimate audio performance. It's about creating a respected and desirable marque, mainly through an audio designer's perceived 'guru status', which people will buy into.

Don't get me wrong, I think that Peter Walker was a talented audio designer; superb products like the ESL57s, amongst others, are testament to that.

I simply think that, like many others of his mindset, brought up on strict scientific values, he took his eye off of the ball, as it were, from being too influenced by the validity of objective measurements when designing audio equipment, the sonic reproduction of which ultimately must satisfy the scrutiny of the human ear more than it does the readout on an oscilloscope.

Marco.