And so the next major audiophile engineering challenge is, err, well what?

[Marco]

Tim,

This completely misses the point. If the "average person" or the highly-trained professional knows what they're listening to or what to listen for, the test is inherently flawed.

Yes I agree, but then so is the whole process of testing what people can hear. Listening for sonic differences in audio and attributing them to what is considered as 'real' in an objective sense is a complex, complicated, and quite frankly, almost impossible business quite simply because as human beings we don't 'listen like a computer' or analyse music signals in the way of test apparatus. I just don't get why people are so hung up on trying to 'prove' things all the time when there is next to no chance of ever being able to do this conclusively with a largely subjective entity such as audio, particularly when differences heard are subtle, as they often are with pretty much anything other than loudspeakers and some analogue equipment. Trust your ears and be happy! I do, and life is just 'peachy-keen' that way

And I suppose that's ok if the testing is truly blind; I'd have to consult some research professionals. But when you're testing hifi on your own, and you are insisting that you must know what is being tested and you must know what to listen for and you're aware of what you're listening to when, that truly is a useless model. You have opened the door to psychological bias so wide that just about any notion could walk through. You may trust your ears under such conditions, but no one else should.

I mostly agree, but that's when the opinions of people you trust come in and also the ability to have faith in your hearing acuity. If, for example, a friend of mine who has what I consider as discerning ears tells me that component X is amazing I would give much more credence to his or his opinion than I would to ten thousand "audiophiles" or recording engineers whom I don't know who think the opposite! And subsequently if my own listening experiences concluded the same as my friend then the opinions of the above 'experts' would be a complete and utter irrelevance. I'm afraid that I don't give much credence to the hi-fi opinions of people I don't know, no matter who they are. As far as I'm concerned that's exactly how it should be.

You've missed the point again. I'm not talking about testing hifi. I'm talking about testing whether or not a change in a system can be heard at all.

You wrote earlier:

the simple truth is that if a given tweak/component/difference cannot be heard often enough by a sufficiently large set of humans with normal hearing to exceed a margin for error (statistical sample), yet you still think you hear it, your ears are not to be trusted.

That's effectively 'testing hi-fi'. If not, what else is being used as the vehicle to determine differences with the tweaks or components you refer to above?

Also, I would refute your claim quite simply because if this "sufficiently large set of humans" doesn't possess the necessary discerning ears and hearing acuity to successfully identify genuine differences with given tweaks or components, and thus concludes that no difference exists, then the results of the test from a discerning audiophile's point of view are worthless. Dolph has already quite correctly pointed out that only a small percentage of audiophiles have the necessary skills.

To give you an example with forums, there are tens of thousands of people currently posting on the main UK audio sites which I've both participated in and observed for years. I can think of no more than maybe 20 people whose opinions I rate sufficiently and would trust with hi-fi - a tiny percentage of the total collective membership - so what would happen if the remainder were those who were doing the judging in your tests and they all concluded the opposite from me? Should I simply dismiss my own judgement based on my thorough listening experiences and conclude that I'm imagining things, or have faith in my own ears and those whose opinions I trust - and offer a two-finger salute to the rest?

I think you know my answer to that one!!

I suspect I could show the dedicated audiophile that his favorite component was, even in his own ears, inaudibly different from the one he replaced with it and he'd simply conclude that my test was flawed. He would always find some cause other than his own bias.

Possibly, or even probably, as there's every chance what he or she has concluded is right, despite the presence of bias or what scientific testing says to the contrary. Discerning real differences with hi-fi which cannot be measured is not the simple 'black & white' process you seem to be advocating.

Marco.

[Steve Toy]

There is, however, a class of very musical people who don't care about the audio quality, for they hear the music in their minds. What their ears receive is just a memory prompt. They might have a similar experience by reading the score: and that they might prefer, because they interpret the music as they wish, rather than have another opinion forced on them.

Good post. This explains why many musicians are so dismissive of hi-fi and fall into the objectivist camp despite having excellent hearing. Basically they hear what they want to hear, filling in the gaps along the way mentally.

[tfarney]

That's effectively 'testing hi-fi'. If not, what else is being used as the vehicle to determine differences with the tweaks or components you refer to above?

Well, this is half right. You can drop the "hi," as all we're testing is whether or not a change is audible at all.

It's not that I don't get your argument, Marco, I do. I've heard it many, many times before. It's just that if a statistical sample of people in blind testing hears nothing, and one guy with an emotional and financial investment in believing in what he is looking at as well as listening to swears he hears an improvement, I'm going with the controlled test. Take that one guy, put him through a blind AB/X with enough repetition to reduce the margin for error to insignificance, and if he can hear the component consistently, I'll seriously reconsider the position.

But that guy never seems to believes in AB/X testing and always refuses to participate in it.

Tim

[Labarum]

Good post. This explains why many musicians are so dismissive of hi-fi and fall into the objectivist camp despite having excellent hearing. Basically they hear what they want to hear, filling in the gaps along the way mentally.

Or maybe you are hearing an argument you want to hear.

They may be dismissive of HiFi fancies because they know what a violin sounds like, and they know that there is nothing to choose between system A's violin and system B's violin; or they know that A is more accurate than B.

This was the import of these words:


Brian:

The best persons to make an evaluation of a system's worth are those who play, or who are very familiar with the instruments and the style of music being played on the sound system at the time.
They could be completely ignorant of HiFi matters - probably better if they are - the question is straightforward: is this realistic?

Marco:

Brian, again I totally agree. What you're referring to essentially is a person's available 'benchmark' with audio. We all have one - but the fact is those belonging to some are better (read as more realistic) than others.

Although I would prefer Marco to have said "available 'benchmark' with music"

[Hozepipe]

This is jolly exciting isn't it

I can sympathize with both sides, Marco and Tim you both make good arguments.

I have a question for Tim though. I actually agree with you that most won't hear a difference in a blind test. I doubt I could, certainly not in an unfamiliar system.

What about a test that measured listening behavior - something that's a lot less subjective and can be easily measured without pressurizing the victim, I mean subject, and skewing the data? Lets just use Audiophiles and Audio Engineers as subjects to give the best chance of getting a positive result. Over a period of a few weeks say (a very expensive and improbable test, but also hypothetical so its free) measure how often and for how long each individual listens to the system as a measure of their satisfaction/enjoyment of it, half of the time with and half without the component change. Not forgetting to use a placebo control group where no change is made over the period. This would, in theory give you an objective measure of enjoyment that the system provides.

Do you think that this would be a fairer, more accurate test? It would show more conclusively any statistically significant difference between the component change group and the placebo group.

[Marco]

Good post. This explains why many musicians are so dismissive of hi-fi and fall into the objectivist camp despite having excellent hearing. Basically they hear what they want to hear, filling in the gaps along the way mentally.

Steve,

I agree (and also with what Gareth wrote). I would contend though that the majority of "very musical people" he describes simply haven't been exposed to how good their music can sound on a truly realistic, musical sounding hi-fi system. Once done, whilst agreeing with Gareth's sentiments and considering that they are highly relevant in most cases, I suspect that quite a few musicians would change their minds!

Again, it's simply a question of one's available benchmark with hi-fi (or music). People make claims about all sorts of stuff until their available benchmark is updated and improved. On a slightly different tact, I've known people who, for example, say that shop-bought pasta is every bit as good as home-made, until they taste the real thing made from scratch by an Italian or someone who knows how to cook proper Italian food! After that they proclaim something entirely different!

Marco.

[Labarum]

No No No. You cannot form a panel of Audiophiles and Audio Engineers. They are used to listening to what comes out of loudspeakers. They might easily have a very skewed view of what is accurate. Fashion can colour judgement.

Any panel must be of persons who know intimately the voices and instruments the audio system is attempting to mimic.

How you proceed when vocals are always amplified in live performance, and the backing is from electronic instruments that use loudspeakers, I don't know.

The concert goer and the competent performer (amateur or professional) are the only reliable judges. Audiophiles and Recording Engineers must stand aside. They cannot be objective judges.

[Hozepipe]

The concert goer and the competent performer (amateur or professional) are the only reliable judges. Audiophiles and Recording Engineers must stand aside. They cannot be objective judges.

Nah. I don't go with that at all. Hifi isn't real, but it has its benefits. You can't compare the two. There's all kinds of ways in which hifi is superior to live music, as well as being deficient. It's a false start IMO to assume that hifi is always inferior to live music. Sometimes it can be better. And vice verca.

I'm a musician myself so there

[Labarum]

Last week my son (now a 23 year old engineering graduate) took me to a couple of HiFi shops to listen to some 5.1 surround kit. He was auditioning B&W 684 (£700) and the centre speaker

http://www.bowers-wilkins.com/displa...fid=2375&sc=ht

at the second shop they did not have the £700 AV amp in stock so I suggested he heard them in stereo mode, having heard a similar setup at another shop.

We listened with a £350 Rotel integrated amp and a CD player by Rotel at about the same price

Simone Dinnerstein Goldberg Variations - opening aria.

His eyes filled up and he almost went into shock at the quality of the sound compared to the AV kit he had listened to an hour previously. While he recovered his composure I asked to up the game, and two £1000 boxes amp and CdP were added. The £700 B&W speakers were as good as my Quart (£5000 bracket by today's prices?).

The sound of my system he knows well, and he has the ears of a pretty competent Cellist. I changed CD to Saint-Saens Piano Concerto No 2 which starts off with gentle piano - and I said to him "These speakers will fail when the orchestra comes in". Well the orchestra does come in with a crash, but short of orchestra flat out those speakers were excellent - and at the price!

Well he has bought his AV amp, but now believes me about the limitations of AV kit.

This is a very personal post, but my lad knows what instruments sound like. I was amazed at his response to the more accurate stereo system. The musician and the techie in him very have an fight with each other.

A few days later he came on MSN - he had got his AV system up and running. He knows it is not as good at the pure stereo system he could have bought, but he wanted the technical gizmos. he then says to me "MP3s are rubbish, aren't they?"

On the cheap system he had as an undergraduate MP3 was good enough. On his present system, he now believes me - it has to be lossless.

Yes, musicians can differentiate.

[Marco]

Tim,

So you'll concede then that your earlier statement of:

the simple truth is that if a given tweak/component/difference cannot be heard often enough by a sufficiently large set of humans with normal hearing to exceed a margin for error (statistical sample), yet you still think you hear it, your ears are not to be trusted.

is in fact not the "simple truth" but merely your own opinion - the same as my opposite opinion is also merely that and not a fact?

It's not that I don't get your argument, Marco, I do. I've heard it many, many times before. It's just that if a statistical sample of people in blind testing hears nothing, and one guy with an emotional and financial investment in believing in what he is looking at as well as listening to swears he hears an improvement, I'm going with the controlled test.

That is of course your prerogative, but I most certainly wouldn't unless I knew them and rated their hi-fi opinions. That's not to say though that audiophiles never imagine things - we're all human and as such fallible. However, given that the appreciation of hi-fi is such a hugely subjective matter, and that the process thereof is by no means a perfect science, how can objective scientific testing consistently and reliably provide all the answers?

Take that one guy, put him through a blind AB/X with enough repetition to reduce the margin for error to insignificance, and if he can hear the component consistently, I'll seriously reconsider the position.

That simply wouldn't work. Like I said before, you need to be in a relaxed frame of mind when assessing hi-fi equipment or ancillaries to arrive at meaningful conclusions, not under pressure in a test environment. I sometimes wonder if people actually pay proper attention to what I (not to mention others) have written

But that guy never seems to believes in AB/X testing and always refuses to participate in it.

Probably because he thinks it's a fruitless and boring exercise and would rather just trust his ears and enjoy the music (like me!)

Marco.