View Full Version : Blind Testing
There a few fora where blind testing has a fervent or even fanatical following. Some people seem to be quite rigorous about this whereas others only seem to ask for it where they don't believe differences exist.
Personally I see it as a hassle and indeed there's no one else in my household who would willingly assist the process. Also I find short-term A/B comparisons only tell me about gross differences but many differences that really count need longer auditioning; this requires living with the product for a few days at least. It would be possible to blind test over matter of days but it becomes somewhat impractical.
Blind testing, speakers, arm and cartridge combinations etc would seem problematic too though any obstacles can theoretically be overcome with money and effort.
For my part A/B sighted comparisons combined with longer term listening work well.
Which camp do you sit in?
Maybe we need a blind test poll but we can't vote if we can't see the screen.
Aren't blind tests conducted by third parties, manufacturers, reviewers and allow a true freedom from coloration. influence of ones thinking, prejudices/favourites.
I'd love to do at least one just for the education it can provide. May golden eared persons have all sorts of reasons not to.
Why are you fussed?
Assess your new kit as best you can. Just avoid making it's amazing claims unless you want the chat
Aren't blind tests conducted by third parties, manufacturers, reviewers and allow a true freedom from coloration. influence of ones thinking, prejudices/favourites.
I'd love to do at least one just for the education it can provide. May golden eared persons have all sorts of reasons not to.
Why are you fussed?
Assess your new kit as best you can. Just avoid making it's amazing claims unless you want the chat
There seem to be a number of people insisting on blind testing for any statements made. I don't mean "wow this is amazing", I mean "this product is a little richer sounding" or whatever. Some of the comments on a few other fora essentially are "if you don't blind test then don't make any comparisons".
Why am I fussed; I'm not THAT fussed but these attitudes stifle discussion.
I do use my own method, as I described. It's not a tightly defined process but I've developed what works for me over 35+ years of building, buying and listening to music. I don't say this is what others must do, whereas the blind testers require others to follow their mantra. I don't find A/B testing (blind or otherwise) to be the only tool in the box though it has its place.
BTW, I tend to steer away from "amazing" claims, there's little that is truly amazing in reality. Everything has its place and price point.
There seem to be a number of people insisting on blind testing for any statements made. I don't mean "wow this is amazing", I mean "this product is a little richer sounding" or whatever. Some of the comments on a few other fora essentially are "if you don't blind test then don't make any comparisons".
Why am I fussed; I'm not THAT fussed but these attitudes stifle discussion.
I do use my own method, as I described. It's not a tightly defined process but I've developed what works for me over 35+ years of building, buying and listening to music. I don't say this is what others must do, whereas the blind testers require others to follow their mantra. I don't find A/B testing (blind or otherwise) to be the only tool in the box though it has its place.
BTW, I tend to steer away from "amazing" claims, there's little that is truly amazing in reality. Everything has its place and price point.
If you qualify your posts with In my Humble Experience or in my system, and say how you came to your view. The thing that pisses me off are folk who make a simplistic statement about something without trying it anywhere else then become offended because you don't accept their word as truth for all.
Gordon Steadman
20-05-2013, 11:04
I find the discussion a bit pointless for me.
My only criteria for my hi-fi is, when I sit down to listen, do I get drawn into the music and unable to leave or can I walk away and start talking about the system.
I have heard many systems over the years and very few have held my attention for very long. One of the worst I ever heard was Krell amps driving Watt speakers. I couldn't get away from it fast enough, no doubt cost a fortune. I don't really care how they measure or compare in the micro differences. Its all in the music.
My amp changes tomorrow. I sat in front of the Quads with the Firebottle OTL inserted rather than the Leak and it just rocked. Bach was pure jazz and I could not get away until it finished. First change in twenty years. We shall see how I feel long term.
Oldpinkman
20-05-2013, 11:10
My wife is always asking me "what am I supposed to hear" or "did I get it right". Well, was. She has realised this stuff does sound great and is learning to trust her own judgement. I think there is truth in both camps. The real test is extended listening pleasure. I think back to my early (and surprisingly current) digital experiences. I could listen to CD's and AB test, and hear stuff - but I never sat down and listened to a CD all the way through. If I was listening on CD, I would always need to make a cup of tea, or phone a friend, or air the dogs. I could listen to vinyl albums back to back for hours. That amounts to my description of digital "hardness". It shows up on a soak test like that - not a quick AB. That said, blind testing is useful for exposing delusions, including your (my) own. It is very easy to hear what you see, or think you should hear. Blind testing helps distinguish real differences from prejudice where a true AB is possible. Of course, you need a few others to make it work. But it was how we did it at Pink Triangle. And if the (immediate AB) differences are real, then they survive a blind test. But its more a tool for manufacturers. As a listener, what matters is what you hear. And if you like it - its good!
I try to use both - get a basically sound and well engineered design and then listen to music reproduced by it to see if the heart is engaged as well as the head. Good sounding products with obvious technical flaws tend to have been "voiced" around one particular parameter, the rest not being as good as this one particular aspect - IMO currently.. This is why good old gear can stand the test of time - decades in fact - and still do the business, because the design was a well balanced one, if a little more compromised than more recent ones made with up to date components. The Quad II is a classic case in point where EVERYTHING from the wires used to the values of some critical components was taken into account. Change these at one's peril as the (technical) balance goes and the results are readily audible.....
Some good points. I would concur that A/B testing, sighted or otherwise helps with identifying obvious technical flaws or inconsistencies, this relates to my head, not my heart. Longer term listening enables me to conclude whether my heart strings are tugged. Of course it's the music itself that moves us but the equipment has to allow it to happen.
Thanks for saying what I cannot :)
“Blinding” is a technique that is sometimes applied to an experimental design in order to control for various types of effects that would otherwise bias the results of the study. So, in order to answer the question of whether blinding is useful in a certain situation, it is important to be precise about what the question is in the first place!
If you have a question like “can people detect a difference between these two amplifiers” then there are a number of well-known effects that have to be controlled for in the experimental design. As examples, hearing is remarkably sensitive to amplitude effects - therefore levels must be matched very closely; auditory memory is very short term - therefore you need to use an ABX type design (or similar) that doesn’t stress the auditory memory too much; the sense of hearing does not operate independently of the other senses (or of higher order sense processing) – therefore you need to isolate the subject from these other sensory cues (using single blinding, for example); experimental subjects can be very sensitive to cues from the experimenter – therefore you need to isolate the subject from these experimental cues (by double blinding, for example).
However, there are other questions for which you positively don’t want the subjects to be isolated from certain effects. If you ask a question like “do you prefer this hifi setup to that one” then you’re asking about the complete experience, which includes all the sensory interactions that humans have. A pig ugly piece of equipment that sounds good may not be as acceptable as a beautiful piece of equipment that sounds just OK. Notice that we could, in principle, separate out the factors that contribute to the total experience; to do so would require a complex experimental design with some bits blinded and other bits not.
Furthermore there are some questions for which blinding is not possible. For example: “do you prefer the sound of this CD player or this turntable”. The sources can be reliably identified by the subject therefore they cannot be blind to them. This effect leads to some of the indirect experimental designs that are used to compare digital and analogue.
So, tl;dr version, whether or not you use blinding in an experimental design depends crucially upon the question you are attempting to answer. In some circumstances if it is not done, your experiment is worthless; in other circumstances it doesn’t matter at all.
So, in a way, being “fanatical” about blinding is neither here nor there; one either has a correct experimental design for the question in hand or one does not.
There is a big difference between an ABX test outside of the context of your own system, own room and own music and a more meaningful test where you do something like use Foobar2000's ABX plugin to compare files (where I did find that I could reliably spot the difference between FLAC and 320kbps files, and less reliably, between FLAC and AIFF), or conduct blind testing by switching between kit that you know, in your room, with your kit... I know that I couldn't hope to pass an ABX test in an alien environment, but I know 100% that I can spot the difference between level-matched DACs, amps and preamps in my own system blind, because I've tried it.
There is a big difference between an ABX test outside of the context of your own system, own room and own music and a more meaningful test where you do something like use Foobar2000's ABX plugin to compare files (where I did find that I could reliably spot the difference between FLAC and 320kbps files, and even between FLAC and AIFF), or conduct blind testing by switching between kit that you know, in your room, with your kit... I know that I couldn't hope to pass an ABX test in an alien environment, but I know 100% that I can spot the difference between level-matched DACs, amps and preamps in my own system blind, because I've tried it.
That's an interesting perspective. There are certain aspects which are so very easy for me to differentiate in my own system & room, eg sound stage width, depth, central focus and lots more. But I don't need a blind test to do this as I know my environment and system so well. Maybe a blind test is more applicable to an alien environment, but it maybe harder to do this well as you say. It certainly takes a while dial into a new environment and if ABX is harder in such an environment there's a dilemma as most of our listening is in our own rooms.
I am certainly in the trust my own ears camp and totally agree its about hearing it in the context of my own equipment and environment, with a selection of music I know really well. I try to keep a open mind and will go back and fourth a few times usually, but sometimes there is no need for this as the differences are so obvious.
For me the danger is in thinking what I hear applies to everyone tastes. equipment and environment. What works for me may not work for someone else.
walpurgis
21-05-2013, 16:49
There's a big difference between blind testing with members of 'Joe Public' and doing it with a bunch of Hi-Fi buffs. The results are interesting. I've seen this done years ago on a couple of occasions.
There's a big difference between blind testing with members of 'Joe Public' and doing it with a bunch of Hi-Fi buffs. The results are interesting. I've seen this done years ago on a couple of occasions.
Very interesting please elaborate? Part of me thinks Joe Public does better and the buffs over analyse...
Would Joe Public listen for the esoteric we do? No and maybe he's happier for it!
The more I think about it blind A/B testing can only pick out relatively gross differences. The DB ABXers who tell me they need my ABX results if they are to listen to me will now get from me, "if I'm to listen to you views you need to tell me about your long-term auditioning conclusions". Yes I pick out differences from an A/B comparison but those differences may not be important, more something you acclimatise to. Long term listening is worth doing once you're sure there's nothing major out of kilter.
Clive, what will your 2000th post be...? Make it good! :eyebrows:
Marco.
walpurgis
22-05-2013, 09:21
Very interesting please elaborate? Part of me thinks Joe Public does better and the buffs over analyse...
You have that about right. From what I recall, what the Hi-Fi buffs liked was described as "too bright" by 'Joe Public' and anything that 'Joe Public' described as having "a nice tone", was described by the Hi-Fi buffs as coloured or inaccurate.
There's a big difference between blind testing with members of 'Joe Public' and doing it with a bunch of Hi-Fi buffs. The results are interesting. I've seen this done years ago on a couple of occasions.
If I recall correctly, Harman go so far as to train up a listener panel in order to do their testing...yes this is it...
http://seanolive.blogspot.co.uk/2011/04/version-204-of-harman-how-to-listen-now.html
The rest of Mr Olive's column makes for an interesting read too.
The problem as I see it with any audio DB tests (including ABx) is that you'll never get consistency between tests nor anything like a large enough sample to really draw any meaningful conclusions other than "on this occasion this (small) group preferred A or B or C" unless there are significant differences between what you are testing for.
Things like DBT's for other research (medical for example) involve thousands of sample numbers.
At the end of the day it doesn't really matter what can or cannot be proved (this coming from a techy!) but what you personally prefer yourself as you are the ultimate arbitor, not a select group of golden eared testers (or not). All imho of course.
walpurgis
22-05-2013, 10:25
Quite right. It's your own ears that should make the judgements!
Yes, but unfortunately, some people insist on employing the logic/thought processes of an emotionless, pre-programmed robot. Spock would be proud!
Marco.
The problem as I see it with any audio DB tests (including ABx) is that you'll never get consistency between tests nor anything like a large enough sample to really draw any meaningful conclusions other than "on this occasion this (small) group preferred A or B or C" unless there are significant differences between what you are testing for.
Things like DBT's for other research (medical for example) involve thousands of sample numbers.
At the end of the day it doesn't really matter what can or cannot be proved (this coming from a techy!) but what you personally prefer yourself as you are the ultimate arbitor, not a select group of golden eared testers (or not).
Post of the year, perhaps? :clap:
Unfortunately, Paul, the problem outlined in your first paragraph doesn't deter 'objectivists'/'measurists' from preaching to all and sundry that the proof obtained is somehow conclusive and irrefutable... :doh:
Marco.
Surely it is true that objectivists as well as subjectivists are guilty of 'preaching'?
There's just no way to ever be sure, the overconfident objectivist is as obnoxious as the overconfident subjectivist. Both extremes smack of egotism.
Measurements are needed in the designing of products. Specifications are useful in helping us select products which are likely to appeal to our personal taste, at least to some extent.
There seems to be a group of objectivists who are only happy when measurements and tests correlate with their model of how systems work. Do they feel emotion from music and if so can they only achieve this when the numbers add up for them? I can't help feeling there's a parallel with an artist using a painting-by-numbers kit.
Happy 2000th post, Clive. It *was* a good one! ;)
Marco.
Happy 2000th post, Clive. It *was* a good one! ;)
Marco.
It's ironic to have reached an objective metric on this thread. I'll go with the subjective comment!
The problem as I see it with any audio DB tests (including ABx) is that you'll never get consistency between tests nor anything like a large enough sample to really draw any meaningful conclusions other than "on this occasion this (small) group preferred A or B or C" unless there are significant differences between what you are testing for.
Things like DBT's for other research (medical for example) involve thousands of sample numbers.
snip
DB trials for medical research can involve sample sizes in the thousands but trials this big tend to be infrequent (and very expensive). When one is designing such a study from the point of view of sample size the key question concerns the anticipated effect size. If you're looking for a large effect size (the drug cures everybody suffering from the disease, for example) then the sample size can be really quite small; on the other hand, if you're looking for a small effect size (the drug improves the cure rate from 67% to 68% for example) then you need a very large sample.
The same would hold true in audio; if you're looking for a large effect size ("the difference is like night & day") then a small sample size is likely to be enough.
The problems in audio experiments tend to stem from other aspects of the experimental design. In particular, the need to control for confounding factors that would otherwise bias the results. These confounding effects can be quite severe. I suggested a few of these confounders earlier in the thread; others have observed that the experimental setting may be detrimental to the subject's ability to perform; even replicating the position of the head from trial to trial may be key!
walpurgis
22-05-2013, 11:59
Some of the nicest sounding equipment measures badly enough to be dismissed if assessed on purely technical merit.
DB trials for medical research can involve sample sizes in the thousands but trials this big tend to be infrequent (and very expensive). When one is designing such a study from the point of view of sample size the key question concerns the anticipated effect size. If you're looking for a large effect size (the drug cures everybody suffering from the disease, for example) then the sample size can be really quite small; on the other hand, if you're looking for a small effect size (the drug improves the cure rate from 67% to 68% for example) then you need a very large sample.
The same would hold true in audio; if you're looking for a large effect size ("the difference is like night & day") then a small sample size is likely to be enough.
The problems in audio experiments tend to stem from other aspects of the experimental design. In particular, the need to control for confounding factors that would otherwise bias the results. These confounding effects can be quite severe. I suggested a few of these confounders earlier in the thread; others have observed that the experimental setting may be detrimental to the subject's ability to perform; even replicating the position of the head from trial to trial may be key!
This is very true, but to a large extent, DBT seems most often used in audio for very small differences where your own ears, eyesight and wallet are the only really meaningful deciders on choice.
I agree with Clive's posting too that you need measurements in design and specifications to help system match (I use the term "system match" as it seems less arbitrary) but you assemble a system with the heart (ok ears :lol:) as well as the head. We're going through quite a few body parts here so it may as well follow that there are some other "body parts" that insist that measurements tell you everything when they very clearly do not.
For example, two different loudspeaker systems. One with an 8 inch full range driver and another 3-way with a 12 inch woofer, 6 inch mid and whatever tweeter.
Both could be designed to measure anechoically similar, eg 45Hz - 15KHz +/-5dB although efficiency would be different.
To argue that because both measure similarly that they must both sound the same is clearly nonsense since there is far more involved. OK, you could then argue that not enough variables are being measured and it's here that the point starts to become obvious; few different pieces of kit are similar enough to measure identically using the same set of measurement criteria (as far as speakers are concerned anyway) so the measurement argument becomes less important in choice than looking at the specification and then auditioning. It remains very important for designing the speakers in the first place though.
There will always be those (no names but we know a few!) who will insist that measurements are everything and they've even built their systems around measuring everything (not always successfully though...).
I do think it is important for accuracy (as the only variable in an ideal world should be the recording) that systems are specified as close to ideally measuring flat (FR) with low distortion but ultimately that is no guarantee you'll enjoy the end result any more than something with a little colour. Providing a system allows you to engage emotionally with the music then there's no right and wrong.
It happens all the time in photography where a photographer produces a final image that engages with the senses but may not resemble very closely the actual scene for example. You're a long time dead so don't let anyone tell you how your system should sound. Enjoy the journey and assemble to your own taste.
sq225917
22-05-2013, 15:31
It's pretty simple for me. I know my eyes fool my ears so if I'm making a 'do I pay thousands for the difference in sound' I want to make sure the only variable is what I hear, not what I see.
In those areas that are on the edge of science fiction then i want to make sure a piece of kit does what is claimed of it, and in that instance a blind test is the only reliable tool.
I'm not prepared to waste good money on hearsay and marketing rhetoric or to allow myself to be lulled into believing something makes a difference when common sense or an ABX proves it audibly does not.
The phrase in Simon's post sums it all up......
"common sense"
Only place for blind tests is in reviewing kit
The phrase in Simon's post sums it all up......
"common sense"
Only place for blind tests is in reviewing kit
Spot on David.
Rothchild
22-05-2013, 21:31
It's pretty simple for me. I know my eyes fool my ears so if I'm making a 'do I pay thousands for the difference in sound' I want to make sure the only variable is what I hear, not what I see.
In those areas that are on the edge of science fiction then i want to make sure a piece of kit does what is claimed of it, and in that instance a blind test is the only reliable tool.
I'm not prepared to waste good money on hearsay and marketing rhetoric or to allow myself to be lulled into believing something makes a difference when common sense or an ABX proves it audibly does not.
Sound understanding, succinctly put. +1
Edit: Another interesting thing is that your beliefs can fool your ears too!
It's pretty simple for me. I know my eyes fool my ears so if I'm making a 'do I pay thousands for the difference in sound' I want to make sure the only variable is what I hear, not what I see.
In those areas that are on the edge of science fiction then i want to make sure a piece of kit does what is claimed of it, and in that instance a blind test is the only reliable tool.
I'm not prepared to waste good money on hearsay and marketing rhetoric or to allow myself to be lulled into believing something makes a difference when common sense or an ABX proves it audibly does not.
I know you don't believe usb cables can sound different. I see HFN this month blind tested some and found "significant and repeatable differences". There's some blind testing that I agree with.
Alan Sircom
22-05-2013, 22:33
I know you don't believe usb cables can sound different. I see HFN this month blind tested some and found "significant and repeatable differences". There's some blind testing that I agree with.
And the measurebators will simply say that it 'wasn't blind enough'.
I have performed a stack of blind and level matched tests over the years. They are extremely useful, but not a panacea.
They do not necessarily make a good analogue to the real world because - for good or ill - we buy and listen under heavily biased conditions. I have been in blind tests that contradict the received wisdom of how something is supposed to sound, and the received wisdom pushes the results of that blind test out of the way.
That holds for both the subjective types who already have already made up their minds how something sounds and will not be persuaded otherwise, and by the object-to-it-all-ivists, who are convinced that the test is irrelevant because their Bumper Book of Big Audio Science (1948 edition) said so.
And the measurebators will simply say that it 'wasn't blind enough'.
I'm sure they will but I have more faith in Paul Miller!
Hi Alan,
Surely it is true that objectivists as well as subjectivists are guilty of 'preaching'?
There's just no way to ever be sure, the overconfident objectivist is as obnoxious as the overconfident subjectivist. Both extremes smack of egotism.
To an extent I agree with you. There are faults on both sides - and they are generally represented by the extremists in both 'camps'. However, elsewhere, it's gotten to stage now where subjectivists (who, let's not forget, represent the significant majority of the membership of pfm and Wigwam, and certainly here), cannot relate the legitimate experiences and observations they've had in audio, for the benefit of other like-minded individuals on a forum, without 'consumer saviour-type' objectivists jumping on them belligerently and shouting 'prove it!'
Now, *if* the subjectivists concerned were foolishly attempting to portray FACT, rather than merely expressing their legitimate OPINIONS on a particular subject, then I would consider them as fair game, for objectivists to ask for proof of said supposed 'facts', derived from scientific evidence. However, as I've already mentioned, that's not what is happening.
What's happening is that people's perfectly valid subjective opinions are being arrogantly stamped on, by objectivists, keen to 'educate' those that they consider need 'educating', by pointing out the 'error of their ways', to the extent that it's stifling discussion, as people aren't posting because they're afraid of being unjustifiably attacked - and this problem is getting worse all the time. It completely ruins a forum for the majority, who simply want to exchange their ideas and experiences with other enthusiasts, in a friendly environment, without fear of being interrogated or ridiculed! If allowed to continue, that type of behaviour kills forums.
The crux of the matter is that it should be taken for granted that, unless otherwise stated, whatever you write on a forum, is simply an expression of your opinion, based on your relevant experience to date of a particular subject - nothing more than that. Therefore, given that's the case, people should be able to express their opinions freely, without being badgered for 'scientific proof', to confirm whatever it is that they may have genuinely experienced.
By all means challenge people's opinions, robustly if necessary, if you think they're wrong, and contest pseudo-science, but do it considerately, and by playing the ball, not the man - and most importantly - without trying to 'convert' people to an opposing mindset. Realistically, the latter is very unlikely to happen. That is something which we absolutely insist on, here on AoS (it is at the core of our ethos), along with treating people with politeness and respect. If the same sentiments were practiced elsewhere, you would virtually eliminate, overnight, the nonsense and endless circular arguments that occur on other forums! :exactly:
Marco.
The brain can fool us but for me its the experience of time with different bits of kits that allows me to make a choice. In the end I have to live with my choices.
I agree Marco that debate is being stifled by objectivists insisting of DB testing. I would go as far as saying it's a form of censorship. Some more extreme individuals involved comment without knowing anything about the products involved. The extreme individuals in my view are probably not fully well, in much the same way that someone with extreme OCD is ill. They will of course say extreme subjectivists are as bad and they could well be correct. The vast majority of us however are simply passionate about our interest, just as enthusiast groups are about cars, cameras, malt whisky, gardening, sport etc.
As for other fora; I think WAM is a lost cause for other reasons, Tony at PFM might like to consider an objectivists room or instead publicly declare PFM an objectivist forum, he should get off the pot and decide.
Clive, before I comment, what do you mean by the bit in bold:
Tony at PFM might like to consider an objectivists room or instead publicly declare PFM an objectivist forum, he should get of the pot and decide.
:)
Marco.
Clive, before I comment, what do you mean by the bit in bold:
:)
Marco.
Should be get OFF the pot - sorry typo. You're not familiar with the expression? Probably my typo confused....
http://www.urbandictionary.com/define.php?term=shit%20or%20get%20off%20the%20pot
Ah, right. Yup, I've heard of that. It just didn't click with your typo. Okies, I shall now 'pen' my reply.
Marco.
I agree Marco that debate is being stifled by objectivists insisting of DB testing.
Blind testing, in order to 'prove' the validity of one’s subjective experiences, is not how most normal people behave. You can't help chuckling at the irony of objectivists, who claim that 'normal' people would laugh at the ridiculous antics audiophiles get up to, which are so far-removed from how 'normal' people behave in the real world.
The fact is, if you think that those outside of our hobby (i.e. 'normal' people) would think that we're crazy, when reading what we write on forums about bits of wire, etc, improving the sound of our stereos, they’d be even more likely to think that we’re certifiable, if they read that none of us ever believed anything we heard, unless it had been subjected to the rigours of DBT – that would REALLY give them a laugh!!
The reality is that there is nothing remotely *normal* about such behaviour. In the real world, 'normal' people trust their ears, as it were. They don't continually question the reliability of their senses (unless, as you say, they are suffering from some form of illness). So who exactly are the 'loons'? ;)
I'll get to your comments about pfm and Wigwam now, as I wanted to address those in a separate post.
Marco.
The Grand Wazoo
23-05-2013, 08:06
.....they’d be even more likely to think that we’re certifiable, if they read that none of us ever believed anything we’d heard, unless it had been subjected to the rigours of DBT – that would REALLY give them a laugh!! There is nothing remotely *normal* about such behaviour.
That's a ridiculous statement Marco! How can it be true?
Please repeat it. Clive could you please refute it? Someone else too! But not in that order. I'll shut my eyes while you're speaking and only then will I decide whether Marco is telling the truth.
That's a ridiculous statement Marco! How can it be true?
Please repeat it. Clive could you please refute it? Someone else too! But not in that order. I'll shut my eyes while you're speaking and only then will I decide whether Marco is telling the truth.
I've tried typing blind but it results in errors which rightly confuses people. So I have to stop typing now as I must keep my eyes shut. It makes test driving cars difficult too.
That's a ridiculous statement Marco! How can it be true?
Please repeat it. Clive could you please refute it? Someone else too! But not in that order. I'll shut my eyes while you're speaking and only then will I decide whether Marco is telling the truth.
:lolsign:
Marco.
I now have an objectivist on pfm telling me he doesn't need to read about the blind test in HFN because he knows what they will say.
I really don't get objectivists coming along with pre-conceived ideas. No wonder they have to DB ABX their life. It seems their pre-conceptions are so strong they can't trust themselves when making decisions.
Like you say, a psychologist (or perhaps rather a psychiatrist) would have a field day ;)
Marco.
I don't know about normal people trusting their ears, as most of 'em don't care in all honesty. "Trusting their ears" usually leads "normal" people into Bose and so on :eek:
I've said it many times before, but my take is that the very best and most endearing audio gear which stands the tests of time seems to have been designed using both methods, objective and possibly blind testing to get the basics right, with subsequent fine tuning done by ear to fine-tune the basic product.
An example, very over-simplistic but still valid I think - two loudspeakers, both offering a pretty flat response with low distortion and high power handling. The "pretty flat" response could still have up to 5db of balance differences in various parts of the range. If marketed to advantage - the AVI "ALL passive and many active speakers that aren't ours, boom!" could be seen to others outside of the AVI clan as "Our speakers have good bass weight and power but AVI speakers don't have any!" The truth is probably somewhere in the middle, but you see how spin can be applied to any of the methods we use to justify our audio beliefs and/or systems used at the time...
I don't know about normal people trusting their ears, as most of 'em don't care in all honesty. "Trusting their ears" usually leads "normal" people into Bose and so on :eek:
'Trusting their ears' would be an automatic, normal and everyday human reaction, which they probably wouldn't even think about.
The point is, Dave, 'normal' people don't fret about being 'fooled' by their senses, when making typical decisions about audio (i.e. do I consider that what I'm hearing is real and is it enjoyable to listen to?) that, on forums, appears to be the case with some diehard objectivists - and remember I'm not talking about people who design equipment professionally, who as far as I'm concerned, are perfectly entitled to be as obsessively 'objective' as they wish!
I'm referring to ordinary end-users of product and their mania with DBT :mental:
I honestly think that these people act the way they do just to get a reaction and wind up 'the opposition'. Quite frankly, there can be no other reasonable explanation for their behaviour, other than the desire simply to be an irritant and flood discussion sites with their prejudices and blinkered dogma. I suspect it acts as a form of comfort blanket.
Marco.
I'd like to ask this question, and direct it particularly at the OP, Clive, (but it's also open for anyone else to answer):
What is the motivation for diehard objectivists to post on forums on audio topics of a highly subjective nature?
Is it some twisted form of 'sport', poking fun at those of a different mindset, to try and score points? Is it the desire to, unselfishly, genuinely help and/or educate people who they think may have made a mistake, is simply an ego trip - or is it more an attempt to convert others to their own way of thinking, in order to satisfy their sensibilities and help cope with the discomfort of their Cognitive Dissonance?
An explanation of the term 'Cognitive Dissonance', for those who are unaware of the concept: http://psychology.about.com/od/cognitivepsychology/f/dissonance.htm
To address my question above, if anyone thinks that it’s for the second reason listed then, in most instances, they are living in cloud-cuckoo land.
Marco.
I'd like to ask this question, and direct it particularly at the OP, Clive, (but it's also open for anyone else to answer):
What is the motivation for diehard objectivists to post on forums on audio topics of a highly subjective nature?
Is it some twisted form of 'sport', poking fun at those of a different mindset, to try and score points, the desire to, unselfishly, genuinely help and/or educate people who they think may have made a mistake, simply an ego trip, or is it more an attempt to convert others to their own way of thinking, in order to satisfy their sensibilities and help cope with the discomfort of their cognitive dissonance?
An explanation of the term 'Cognitive Dissonance', for those who are unaware of the concept: http://psychology.about.com/od/cognitivepsychology/f/dissonance.htm
To address my question above, if anyone thinks that it’s for the second reason listed then, in most instances, they are living in cloud-cuckoo land.
Marco.
I don't know much about the individuals involved. I expect it's all of the above. There's one of two who simply like stirring things and have little going well in their lives, probably unfulfilled .and angry Many seem to have a view of the world based on what they see as factual and that experienced-based judgements are valueless. For some reason these types won't live-and-let-live; they want to educate or impose their will because subjectivists are so very misguided and misled. Whilst I'm typing this I'm recognising ingredients that make up radical fundamentalists. Yes that's it, they are the radical fundamentalists of the audio world.
I don't know much about the individuals involved. I expect it's all of the above. There's one of two who simply like stirring things and have little going well in their lives, probably unfulfilled .and angry Many seem to have a view of the world based on what they see as factual and that experienced-based judgements are valueless. For some reason these types won't live-and-let-live; they want to educate or impose their will because subjectivists are so very misguided and misled. Whilst I'm typing this I'm recognising ingredients that make up radical fundamentalists. Yes that's it, they are the radical fundamentalists of the audio world.
Oh, there is such a rich vein of dark humour here for the tapping but perhaps now is not the time to air it considering recent events :eyebrows:
Oh, there is such a rich vein of dark humour here for the tapping but perhaps now is not the time to air it considering recent events :eyebrows:
Those events did of course come to mind. Intolerance is one common behaviour I perceive, clearly I'm not suggesting that anything other than irritating statements on fora are being made.
Those events did of course come to mind. Intolerance is one common behaviour I perceive, clearly I'm not suggesting that anything other than irritating statements on fora are being made.
Of course Clive, please don't think I was suggesting otherwise, it's my warped SOH I was questioning and nobody elses ;)
I don't know much about the individuals involved. I expect it's all of the above. There's one of two who simply like stirring things and have little going well in their lives, probably unfulfilled .and angry Many seem to have a view of the world based on what they see as factual and that experienced-based judgements are valueless. For some reason these types won't live-and-let-live; they want to educate or impose their will because subjectivists are so very misguided and misled. Whilst I'm typing this I'm recognising ingredients that make up radical fundamentalists.
Interesting, and I don't disagree. What's your view of the Cognitive Dissonance aspect?
I read with interest this response to Serge, on the 'Turntables comparison' thread, on pfm, which the last time I looked, he hadn't yet responded to:
100 million Christians may believe....on the basis of no evidence
Those who suggest differences in turntable sound (mostly) base their belief on (measured) evidence...however it seems that this evidence is not acceptable to you because of your subjective findings
You seem to accept that there are measurable differences between turntables...but because you cannot hear them (others claim they can) you reject these differences...which actually represents an interesting about -face for you.
In other cases you harp on about others reporting audible differences, which you claim you would only accept if these reports are backed up by measurements!
You cant` have it both ways...I think you are really just trolling.
For me, the guy has nailed the sheer hypocrisy in Serge's argument. Furthermore, Serge's behaviour, IMHO, is that of classic Cognitive Dissonance - and he's not the only one there guilty of that type of behaviour. What's your view?
I also read this on the TQ thread, there:
Trust your ears at your peril.
That final line Strikes me as a little bit sad; I don't pretend to have golden ears etc, but if something makes me happy, I'll go for it instead of worrying I might be wrong.
I fully agree with the response of Martin (realysm42), above, as that's also how I feel. However, at first I laughed out loudly when I read the bit in bold (I've seen this type of thing before, but it always shocks/amuses me, in equal measure). Then, however, I felt genuinely sad at how apparently disconnected some people are from normal human behaviour, due to brainwashing by science...
Fundamentally they're refusing to accept their humanness (as they seemingly can't cope with the concept of our innate fallibility) - and *that* I find truly sad.
I have no doubt whatsoever that, sometimes in audio I will make mistakes, as a result of fooling myself (and my ears), due to expectation bias, or whatever, but I have no problem with that and simply accept it as part of being a human being. If I'm wrong, I just put it down to experience and learn from it - but it never prevents me from prioritising what my ears tell me over measurements or scientific wisdom, in terms of what my instincts tell me exists.
People like Serge, however, appear to behave more like robots, programmed to obey their 'logic command centre', driven by worshiping at the altar of science, and seem totally incapable of countermanding that control...
Very, VERY weird!!!
Marco.
Interesting, and I don't disagree. What's your view of the Cognitive Dissonance aspect?
I believe it covers a very big part of this. From your link, this chimed well for me:
Why is Cognitive Dissonance Important?
Cognitive dissonance plays a role in many value judgments, decisions and evaluations. Becoming aware of how conflicting beliefs impact the decision-making process is a great way to improve your ability to make faster and more accurate choices.
It as though some of the individuals haven't progressed in their understanding of themselves so they need data to back up their views, indeed they almost don't hold views, they simply repeat what they think are truisms.
Trust your ears at your peril.
Do what? That is an utterly ridiculous statement... :lol::lol::lol:
I think that the problem arises Marco when people like Serge and others (the author of the "trust your ears at your peril" quote) use such phrases out of context. I think that Serge underneath it all is actually a very decent fella and truly believes what he's saying, but unfortunately, he did seem to want it both ways on that thread.
If the context was correct, ie you need to know how something measures; then don't trust your ears, measure it!...then that would to me be a perfectly reasonable thing to say.
Most kit today is pretty damn good, performance-wise, so why should the consumer want to know about anything other than "will it fit into the system"(physically and specification-wise) and "what does it sound like"...end of.
I read the thread where Serge admitted not hearing differences between TT's yet admitting they do measure differently (resonance control, rumble, pitch stability etc etc). I've also read threads where other measurebaters also insist that many carts are specified as flat on FR so there shouldn't be any differences there either. Not if you're willing to discount compliance, stylus shape, generator design, flux damping and distortion LoL!
This makes such people either ignorant of the facts they're claiming to posses knowledge of or quite simply, hypocrites.
As a designer, I am interested in measurements because they tell me something important about the design and performance of a component, ditto specifications. That's where it ends though and the ears take over once the measurements have been made. Many speakers (crossover design for example) are adjusted only after just as many hours of subjective listening as there has been anechoic measurement.
The only exception I can think of where measurements may be important to consumers of kit is loudspeaker/room interaction. Sometimes it's useful to learn how the room affects response so that something can be done to tackle problem areas.
Do what? That is an utterly ridiculous statement... :lol::lol::lol:
I know!! :mental:
But it's also rather sad when you realise that the people who spout that stuff are, seemingly, DEADLY SERIOUS........... :doh: :doh:
Paul (and Clive), some good points there. I'll get to them later.
Marco.
I know!! :mental:
But it's also rather sad when you realise that the people who spout that stuff are, seemingly, DEADLY SERIOUS........... :doh: :doh:
Paul, some good points there. I'll get to them later.
Marco.
The psychological aspect of that statement is incredibly interesting. It is as if such people are denying what they hear with their own ears. Perhaps it is an attempt to justify why they have spent so much money on their kit, who knows? With the greatest respect (and I am not being sarcastic here) I wonder whether these people are true lovers of music.
Before I get a chance to comment again in detail, here's some more interesting reading on the subject of Cognitive Dissonance:
http://psychcentral.com/blog/archives/2008/10/19/fighting-cognitive-dissonance-the-lies-we-tell-ourselves/
This bit is particularly apt:
Not everyone feels cognitive dissonance to the same degree. People with a higher need for consistency and certainty in their lives usually feel the effects of cognitive dissonance more than those who have a lesser need for such consistency.
And more (it gets better):
Cognitive-dissonance is just one of many biases that work in our everyday lives. We don’t like to believe that we may be wrong, so we may limit our intake of new information or thinking about things in ways that don’t fit within our pre-existing beliefs. Psychologists call this “confirmation bias.”
;)
I could almost have written that myself!! If 'objectivists' on hi-fi forums think that they're immune from the above, then they're even more blinkered than I thought...!
Marco.
Hi Alan,
To an extent I agree with you. There are faults on both sides - and they are generally represented by the extremists in both 'camps'. However, elsewhere, it's gotten to stage now where subjectivists (who, let's not forget, represent the significant majority of the membership of pfm and Wigwam, and certainly here), cannot relate the legitimate experiences and observations they've had in audio, for the benefit of other like-minded individuals on a forum, without 'consumer saviour-type' objectivists jumping on them belligerently and shouting 'prove it!'
Now, *if* the subjectivists concerned were foolishly attempting to portray FACT, rather than merely expressing their legitimate OPINIONS on a particular subject, then I would consider them as fair game, for objectivists to ask for proof of said supposed 'facts', derived from scientific evidence. However, as I've already mentioned, that's not what is happening.
What's happening is that people's perfectly valid subjective opinions are being arrogantly stamped on, by objectivists, keen to 'educate' those that they consider need 'educating', by pointing out the 'error of their ways', to the extent that it's stifling discussion, as people aren't posting because they're afraid of being unjustifiably attacked - and this problem is getting worse all the time. It completely ruins a forum for the majority, who simply want to exchange their ideas and experiences with other enthusiasts, in a friendly environment, without fear of being interrogated or ridiculed! If allowed to continue, that type of behaviour kills forums.
The crux of the matter is that it should be taken for granted that, unless otherwise stated, whatever you write on a forum, is simply an expression of your opinion, based on your relevant experience to date of a particular subject - nothing more than that. Therefore, given that's the case, people should be able to express their opinions freely, without being badgered for 'scientific proof', to confirm whatever it is that they may have genuinely experienced.
By all means challenge people's opinions, robustly if necessary, if you think they're wrong, and contest pseudo-science, but do it considerately, and by playing the ball, not the man - and most importantly - without trying to 'convert' people to an opposing mindset. Realistically, the latter is very unlikely to happen. That is something which we absolutely insist on, here on AoS (it is at the core of our ethos), along with treating people with politeness and respect. If the same sentiments were practiced elsewhere, you would virtually eliminate, overnight, the nonsense and endless circular arguments that occur on other forums! :exactly:
Marco.
Indeed, we are in broad agreement on many things. This is why I too will always be subjective, in my appreciation of music. The thing is, you (and many others) claim 'objectivists' are arrogant (or even suffering a disorder) in dismissing our subjective assessments, perhaps due to the absolutist nature if the rebuttal. However, many 'subjectivists' are hardly being reasonable, in claiming to be able to discern remarkably tiny differences in sonic signature, even in individual electrical components in a larger, more complex circuits - and then crying foul when an objective poster (reasonably) scoffs at such a notion.
Before anyone feels defensive, let me be clear: It isn't for either side to say the other cannot possibly be right - yet this is what continually happens. This is not a comment on which 'side' is more credible; it is a comment that each 'side', with it's more extreme 'ists', is often as arrogant as the other. The arrogance in both sides is in the absolutism, the insistence that the other 'side' can be dismissed.
'Objectivists' look on the claims of us subjective enthusiasts with amusement and occasionally derogatory dismissiveness. It irks me too, but there it is.
Yet the so called 'innocent' party, the 'subjectivist', relegates the observations of the other party with derogatory comments along such lines as 'measurbator', and 'people who have to double blind test their lives'.
To the 'normal' person, one who has little interest in hifi beyond their MP3 player, which would be more ridiculous? The nerd who measures everything with oscilloscopes, and arranges blind tests on Friday evening for interconnect cables? - Or the nerd who claims he hears the differences in wires costing hundreds of pounds, and can tell the difference between resistors and capacitors in his CD player?
Neither side has much to shout about.
Alan, the biggest problem is not in WHAT people write on forums, but HOW they COMMUNICATE, i.e. with an almost complete contempt and disrespect for their fellow man - and yes, both objectivists and subjectivists are guilty of this behaviour.
Therefore, it's not the message that's necessarily the problem, but rather the messenger!
I'll get to the rest of your post later :)
Marco.
Alan, the biggest problem is not in WHAT people write on forums, but HOW they COMMUNICATE, i.e. with an almost complete contempt and disrespect for their fellow man - and yes, both objectivists and subjectivists are guilty of this behaviour.
Therefore, it's not the message that's necessarily the problem, but rather the messenger!
I'll get to the rest of your post later :)
Marco.
There's not much need, the rest is simply observations designed to support the point, which you have clearly understood.
Perhaps the problem is that part of human nature which looks for resolution, where there is none; fact, where none exists.
There are some conclusions being jumped to here, partly due to the ridiculous polarization of 'subjectivist' vs 'objectivist'. Those taking a wholly subjectivist position do seem to repeatedly fail to understand what is being said. For example Serge Auckland (who has his own blog laying out his approach - worth a read - so I don't have aproblem using him as an example) has an approach which is logically self-consistent if considred quite extreme by the average enthusiast who don't seem to get it or don't want to get it.
Essentially it is about achieving as flat a frequency response and as low distortion as possible and using measurements to achieve this. Whys is this desirable? Because it eliminates, as much as possible, colouration in the replayed sound. Consequently you will not end up with a system that plays some music really well and makes a dog's dinner of other material. And we have all heard many systems like that. Now there are plenty of people who swear by a bit of colouration, especially the harmonic distortion that valve amps add. How many times have you made a change to your system that you initially decided was a big improvement, only to decide a week or two later, having trawled through a chunk of your music collection, that it does, on the whole, make things sound worse? That has happened to me on more than one occasion - the Belkin Pure AV interconnects spring to mind.
As to blind ABX yes it is a useful tool in some circumstances. Some differences are so small as to be inconsequential and certainly not worth spending money on. Upgrading a transistor amp from a 40 watt £200 amp to a 60watt £2000 amp - something I've seen on a few occasions on the forums - is IMO a complete waste of money and blind ABX has shown this to be the case repeatedly. Indeed challenges exist in which you can win a lot of money by identifying the difference to a statistically significant extent. No-one has succeeded - in fact in the last example offered by speaker company Harbeth no-one accepted the challenge. There may/will be a difference BUT - If the difference is so small you cannot spot it in an ABX then why are you spending all that money? Think about that a bit and see what conclusion you come to.
There are some conclusions being jumped to here, partly due to the ridiculous polarization of 'subjectivist' vs 'objectivist'. Those taking a wholly subjectivist position do seem to repeatedly fail to understand what is being said. For example Serge Auckland (who has his own blog laying out his approach - worth a read - so I don't have aproblem using him as an example) has an approach which is logically self-consistent if considred quite extreme by the average enthusiast who don't seem to get it or don't want to get it.
Essentially it is about achieving as flat a frequency response and as low distortion as possible and using measurements to achieve this. Whys is this desirable? Because it eliminates, as much as possible, colouration in the replayed sound. Consequently you will not end up with a system that plays some music really well and makes a dog's dinner of other material. And we have all heard many systems like that. Now there are plenty of people who swear by a bit of colouration, especially the harmonic distortion that valve amps add. How many times have you made a change to your system that you initially decided was a big improvement, only to decide a week or two later, having trawled through a chunk of your music collection, that it does, on the whole, make things sound worse? That has happened to me on more than one occasion - the Belkin Pure AV interconnects spring to mind.
As to blind ABX yes it is a useful tool in some circumstances. Some differences are so small as to be inconsequential and certainly not worth spending money on. Upgrading a transistor amp from a 40 watt £200 amp to a 60watt £2000 amp - something I've seen on a few occasions on the forums - is IMO a complete waste of money and blind ABX has shown this to be the case repeatedly. Indeed challenges exist in which you can win a lot of money by identifying the difference to a statistically significant extent. No-one has succeeded - in fact in the last example offered by speaker company Harbeth no-one accepted the challenge. There may/will be a difference BUT - If the difference is so small you cannot spot it in an ABX then why are you spending all that money? Think about that a bit and see what conclusion you come to.
Way back at the start of the thread it was said that A/B comparisons and long term listening are both useful. You yourself siad that long term listening can show that a change may not be as significant as was first thought. It works the other way too, sometimes long term listening demonstrates how it's possible to miss something seemingly minor which turns out to be important. A/B or ABX is just one tool, I certainly wouldn't decide on an amp change or indeed not changing an amp due to a short term A/B test only.
Way back at the start of the thread it was said that A/B comparisons and long term listening are both useful. You yourself siad that long term listening can show that a change may not be as significant as was first thought. It works the other way too, sometimes long term listening demonstrates how it's possible to miss something seemingly minor which turns out to be important. A/B or ABX is just one tool, I certainly wouldn't decide on an amp change or indeed not changing an amp due to a short term A/B test only.
You are not Blind ABX testing to see if there is an improvement or not. You can't really use it for that for the reasons you state. It only determines if a difference is audible. You don't even have to like the music being played!
You are not Blind ABX testing to see if there is an improvement or not. You can't really use it for that for the reasons you state. It only determines if a difference is audible. You don't even have to like the music being played!
There's so much more to musical enjoyment than is there more or less bass, is there more treble is or there more detail. There are very subtle nuances which over time are very important to some. Long term auditioning is more balanced but I use both except I do need to do blind; that's just my way, I don't insist others do sighted. Whatever works for the individual.
It as though some of the individuals haven't progressed in their understanding of themselves so they need data to back up their views, indeed they almost don't hold views, they simply repeat what they think are truisms.
Absolutely - hence my reference to 'robots' and programming. Robotically trotting out 'accepted' truisms is exactly it.
For me, anyone who ultimately (and that is the key word here, which allows someone first to test for things objectively or subjectively, using their own assessment procedure, before forming a conclusive opinion on whatever it is being tested), can't trust their senses, is betraying themselves as a human being...
The above is true if, after having applied the above test procedure, and the results are inconclusive, and so don't entirely confirm what your ears are quite clearly telling you - yet you STILL refuse to believe what you're hearing, simply because it doesn't 'fit' with currently accepted scientific wisdom.
Yes, you *could* be imagining what you're hearing, due to the influence of a number of factors, but at what point, after having exhausted all available (to you) objective test procedures, in order to confirm the existence of a phenomenon, which have proved unsuccessful, yet your senses are still clearly telling you that said phenomenon exists, do you trust the human instincts you were born with??
*THAT* is the key question... For those who, under those circumstances, would still refuse to trust their senses, I would suggest that they seek help, 'cos they sure as hell have a problem! ;)
Marco.
Yes, it's good to hear sensible & reasonable views being expressed here. All too often, one finds on forums a type of censorship applied by a small but vocal group who infiltrate any thread with the same old line of argument revolving around blind-testing & measurements to quell any discourse or open-discussion of topics. In the main I consider this group to be comprised of pseudo science aficionados who believe that they hold a superior view of audio & under the guise of trying to protect the general populace of poor un-informed souls from wasting their money, spout their pseudo-scientific views.
Firstly, there is, as yet, no set of measurements that tells how something sounds - so providing or not providing measurements is immaterial to judge the sound of a device or system.
Secondly, I'm no expert in blind testing but my small bit of knowledge of it has led me to the conclusion that their idea of real blind-testing is naive & uninformed. The scientific approach to double blind testing is well documented & contained in a number of papers which these people would do well to read before spouting the pseudo-tests that they are suggesting. there are a number of methodologies for conducting such tests depending on what is to be evaluated & what level of confidence is acceptable. In general these methods go to great lengths to eliminate ALL biases, not just sighted biases & require much more formalism than is suggested by the pseudo-scientists. The results use statistical methods for evaluation particularly when small differences are being tested for, as is typical in audio. This entails a large number of trials/participants with rigorous pre-qualifications & anchors to ensure that the testing is valid.
The accusation that is often used by such a group are that anybody who doesn't agree with their viewpoint/analysis is a FOO-spreader. If I understand the term correctly, this means a spreader of mis-information. To my viewpoint, it is they that are the major offenders & mis-informers with viewpoints ranging from - the arch-measurebater himself, Winer that all things in audio reproduction have been solved 50 years ago to the same view but expressed less contentiously - there is no difference between DACs/TTs/amps. As an aside, it's interesting to note that a number of these people are involved in the speaker area or room treatment field, which, of course, to them is the only area left for improvement.
Yes, the world is a place full of contradictions - we internalise many of these contradictions & live with them holding conflicting views at the same time - yet the vast majority of us we seem to be able to cope.
If the sense of hearing was so unreliable, I doubt our species would have survived to this day!
Edit: Of course I have two flaws in my post above - 1) I'm in the industry 2) I'm in the contentious field that attracts the FOO accusers i.e computer audio.
So why would my view be anything but skewed?
Hmmm..can't go along with all of that but I appreciate the sentiments you're making there John. Most things that affect how something sounds can be measured, from FR, phase accuracy and distortion, SPL etc etc, so I can't hold as firmly to the belief that we don't understand what makes something sound the way it does as firmly as perhaps you do. We may not be measuring the right things or we may not even know what it is we need to measure! However, what is also true I suspect to a majority of people is that many of these measurements on paper in isolation can't tell you what you need to know and more importantly, many people can't associate a series of numbers or graphs on paper with actual sound, so listening is the obvious answer!
Where these measurements do come in handy is in comparative assessment of specific areas of performance. Eg, if you would like to try a valve amp, it's very useful to have a plot of frequency and phase response along with impedance for your speakers to see what sort of a match it might be; I wouldn't ever discount something because it didn't seem to measure well enough for synergy, I'd try it in the system, but measurements at least give an indication of what you might expect.
The other area is in trying to achieve a 20-20KHz system with low distortion so that the only possible variable is the recording (as has already been mentioned) and in terms of the absence of any sort of standard definition of "High Fidelity" then this would appear to be as good a place to start as any.
Ultimately though I agree that there are too many within both camps who steadfastly refuse to believe in anything other than some fairly narrow and fundamental position on the subject. On one side there are those who perhaps pride themselves in a total lack of knowledge (which I do find strange as if interested in something surely finding out about how it works is part of the fun!) and in the other camp, there are those who deride the viewpoints (most of which are perfectly valid personal subjective assessments) in favour if some pretty ignorant viewpoints passed in pseudo scientific ways...a classic has to be the old chestnut "there's no difference between a piece of barbed wire fencing or wet string to a high quality interconnect" which to me demonstrates that they don't perhaps fully understand the subject they claim to be "experts" at.
Excellent last couple of posts, chaps! :)
Paul, just to answer this quickly:
On one side there are those who perhaps pride themselves in a total lack of knowledge (which I do find strange as if interested in something surely finding out about how it works is part of the fun!)
Yup, if your brain works that way... As a staunch subjectivist, I certainly don't "pride myself in a total lack of knowledge". My position is that, technically, I know most of what, for my needs I need to know, and for everything else, I have very capable people, many of whom are personal friends, that I can call on to help me out.
My main passion is music, not messing around with the boxes that reproduce it (far less measuring their technical performance), so the FUN bit, for me, is in expanding my music collection, whilst ensuring that the music I've got sounds as good as possible, through my hi-fi system. In order to ensure the latter, I'll employ whatever skills I have, together with those of my more technically minded friends, and that system works very well indeed.
Therefore, not being particularly interested in how the kit I've got works, most certainly doesn't spoil the fun factor of owning it! ;)
Marco.
Excellent last couple of posts, chaps! :)
Paul, just to answer this quickly:
Yup, if your brain works that way... As a staunch subjectivist, I certainly don't "pride myself in a total lack of knowledge". My position is that, technically, I know most of what, for my needs I need to know, and for everything else, I have very capable people, many of whom are personal friends, that I can call on to help me out.
My main passion is music, not messing around with the boxes that reproduce it (far less measuring their technical performance), so the FUN bit, for me, is in expanding my music collection, whilst ensuring that the music I've got sounds as good as possible, through my hi-fi system. In order to ensure the latter, I'll employ whatever skills I have, together with those of my more technically minded friends, and that system works very well indeed.
Therefore, not being particularly interested in how the kit I've got works, most certainly doesn't spoil the fun factor of owning it! ;)
Marco.
I can totally see where you're coming from with that Marco but I guess that my response was more aimed at those debating hifi issues rather than enjoying listening to music! I'm someone who needs to know how things work (for my business) but am with you in that it's all a means to an end to enjoy music. Personally I don't fret over measurements with my own system (bar needing to measure the loudspeaker's response for commercial purposes) and am more concerned with tracking down new titles and expanding the music collection.
Paul, don't get me wrong - I'm not saying measurement is not needed or important - what I'm saying is that at the stage audio is now at all the MAJOR FR, THD, etc. issues are well known & measurable & they give us a very, very rough guide to performance & indeed can alert us to some potential issues in system building. But they don't tell us how a particular device will sound. Indeed as most modern devices measure well, the conclusion might be that all should therefore sound the same - as is the position taken by some, who also use this cohesion in the measurements to underpin their viewpoint. The resultant argument being that if you can't produce the measurement that shows how A differs from B therefore A sounds the same as B.
Their real catch 22 is that when any such measurement is produced they then claim that measurements are not necessarily audible so you now need to produce blind listening results, fully documented as proof. No doubt, if anybody was foolish enough to succomb to this, they would have issues with the tests validity/results.
The fact of the matter is that, yes audio has progressed to the point where gross, measurable differences between devices are the exception yet all devices do not sound the same. I believe we are at the very fine level of measurable differences certainly in the area of digital audio.
On that other forum, I posted measurements from Jim Lesurf showing the procedure & results for two USB audio devices that showed some very fine level of detailed differences between them - differences that are not picked up by the standard set of measurement techniques. Apart from one poster (who disagreed with the value of the results but gave good logical arguments for why), the results just generated the type of replies I listed above.
Paul, don't get me wrong - I'm not saying measurement is not needed or important - what I'm saying is that at the stage audio is now at all the MAJOR FR, THD, etc. issues are well known & measurable & they give us a very, very rough guide to performance & indeed can alert us to some potential issues in system building. But they don't tell us how a particular device will sound. Indeed as most modern devices measure well, the conclusion might be that all should therefore sound the same - as is the position taken by some, who also use this cohesion in the measurements to underpin their viewpoint. The resultant argument being that if you can't produce the measurement that shows how A differs from B therefore A sounds the same as B.
Their real catch 22 is that when any such measurement is produced they then claim that measurements are not necessarily audible so you now need to produce blind listening results, fully documented as proof. No doubt, if anybody was foolish enough to succomb to this, they would have issues with the tests validity/results.
The fact of the matter is that, yes audio has progressed to the point where gross, measurable differences between devices are the exception yet all devices do not sound the same. I believe we are at the very fine level of measurable differences certainly in the area of digital audio.
On that other forum, I posted measurements from Jim Lesurf showing the procedure & results for two USB audio devices that showed some very fine level of detailed differences between them - differences that are not picked up by the standard set of measurement techniques. Apart from one poster (who disagreed with the value of the results but gave good logical arguments for why), the results just generated the type of replies I listed above.
That's a perfectly reasonable viewpoint John. The only thing that tempers it (with my "devil's advocate" hat on) is that if such differences are minute in terms of measurement, one might expect then that sonically the differences are also very small. If measurable differences are high then it usually follows that sonically, differences are normally quite obvious.
I'm with you though as you can just as easily hear differences between digital kit that supposedly measure the same, albeit more subtle differences. I think that extended listening is needed to really get to the heart of any device and subtlety in differences doesn't always correlate with unimportance in differences, as it depends just what those differences are. For example, frequency response doesn't always tell us everything we need to know about phase accuracy (which affects things like timbral accuracy) so decay of notes, ambience of the recording venue, certain nuances in the voice or an instrument and placement in terms of stereo presentation can be presented with marked differences between different pieces of equipment that have very similar (good) specifications.
Yes, Paul but again just to clarify what I mean - I'm not saying that LeSurf's measurement (http://www.audiomisc.co.uk/Linux/Sound3/TimeForChange.html)actually fully explains the audible difference between the devices (Halide Bridge USB-SPDIF bridge Vs DACMagic's own inbuilt USB receiver) rather that it is a measured difference betwen the devices that isn't shown using the standard set of measurement tools. I posted it to see what the response would be to the challenge often made to back up audible differences with some/any measurement, otherwise it's all just self-delusion.
It may or may not be of significance - it requires a larger body of example measurements to derive any correlation with audible differences. I'm aware that correlation does not = causation & also that there are some uncontrolled parameters between the two devices being measured i.e using the DACmagic's SPDIF input (with an external USB-SPDIF converter) Vs using the DACmagic's internal USB receiver - to name just one.
Yes, Paul but again just to clarify what I mean - I'm not saying that LeSurf's measurement (http://www.audiomisc.co.uk/Linux/Sound3/TimeForChange.html)actually fully explains the audible difference between the devices (Halide Bridge USB-SPDIF bridge Vs DACMagic's own inbuilt USB receiver) rather that it is a measured difference betwen the devices that isn't shown using the standard set of measurement tools. I posted it to see what the response would be to the challenge often made to back up audible differences with some/any measurement, otherwise it's all just self-delusion.
It may or may not be of significance - it requires a larger body of example measurements to derive any correlation with audible differences. I'm aware that correlation does not = causation & also that there are some uncontrolled parameters between the two devices being measured i.e using the DACmagic's SPDIF input (with an external USB-SPDIF converter) Vs using the DACmagic's internal USB receiver - to name just one.
Agreed. I'm surprised given how you've explained this that those "on the other forum" would find grounds to argue with what you wrote in the context that it was written. It makes sense & I get it:thumbsup:
Werner posted something on pfm which switched on a light bulb in my head. I sort of knew it anyway but he helped me rationalise it better.
There are many categories of listener; I happen to be someone who value soundstage, image and central focus. There are many other aspects I value but but the point here is that some don't value spatial effects or simply can't or won't hear them.
Werner's point (badly paraphrased by myself) was that it's hard to hear the spatial clues in an alien system & room. It takes time to acclimatise to pick up on the subtle clues our brains use to create a 3D representation of a band. When I swap a piece of equipment in my system it is very obvious to me if the spatial portrayal has altered. Also if you're tired or have your ears ringing from travel you won't hear with such acuity either. The upshot is that auditioning in an alien room is not likely to be productive in spatial terms, not unless you spend a couple of days acclimatising. Probably you can assess frequency differences and few otherwise easily measurable aspects but this is only a subset of what many of us listen for.
I wonder if this partially explains some of the heated debates around blind testing and equipment SQ differences.
I agree that people have different value systems when it comes to listening - I have come across people who seem to think that unnatural ultra-detail is hearing into the music rather than it being a digital distortion. I have come across people who think boomy bass is retrieving more from the music. I've never come across people who can't hear sound stage but then I don't mix with those sorts of people :)
I don't think we need to acclimatise ourselves over a long period to hear sound stage - I believe the ear/brain system does this in milli-seconds - otherwise our survival would be threatened not knowing from which direction a sound is emanating unless we had spent a long time in a particular place. I have never had a problem going into a strange room & hearing sound stage in a recording/system.
I think the problem with blind testing/debates is that some people have pre-conceived notions & they skew their listening to this pre-conception. It's like drawing - some people can analyse what they see & commit it to paper, others can only draw what's in their head. Children can only do this - their pre-conceptions get in the way of their actual perceptions.
I do however agree that the blind-testing usually suggested is pseudo-science - people pretending that they know what they are talking about/doing - playing a at doing science like the grown-ups do :). Personal blind testing is fine as a personal sanity check but this mass blind testing is just crap. They pretend that the biggest bias factor is sightedness & then ignore all other biases & checks & statistics & run a half-arsed blind test for proof of their contention that "all DACs/amps/etc are the same". Why not try running a similar pseudo-test on speakers & see what sort of results are obtained?
Having read Werner's post I can agree with him - we can tell a good imaging from bad imaging room immediately, we don't need to acclimatise ourselves. Can we adjust our perceptions over time to a room, yes but that's not the same thing. Does sound stage change between different rooms, yes! Can sound stage be killed by a bad room, yes but I would suggest that that room's intelligibility factor is very low i.e the ability to hear & understand speech in the room.
Do some people have difficulty with sound stage, yes, it would seem so either because of their system/room/preconceptions.
Thinking more about possible acclimatisation this may be more to do with the other effect Werner mentioned - not being rested enough to pick up fine cues. I can only speak from my experience - I know that if I'm weary (as opposed to a little tired) I can't focus on music enough to enjoy it. When doing comparisons I wait until I'm in a receptive mood too. Almost invariably when I listen to a system not in my room I have traveled a reasonable distance or sometimes had a boozy night previously. This doesn't mean I can't listen critically but it may take the edge off my abilities.
What's in the back of my mind....is there a correlation between those advocating blind testing (mostly poorly constructed) and those who primarily are happy as long as frequency response is good and noise is low?
For those that don't rate soundstage, here's a recent experience I had. I was trying a USB to SPDIF device. When USB powered from my computer the sound stage width stopped in the inner edges my speakers, ie it was totally between the speakers. Adding an external ps to the converter allowed the soundstage to exceed the width on my speakers. I expect this is a big reason for John designing battery powered converters and dacs.
Yes, Clive & I believe most of the sound stage illusion comes from temporal & low-level cues within the sound field - the area where good PS design has the most impact. It's funny sometimes when some people say that ALL your doing is reducing the background noise - this has such a profound effect on everything we hear in the sound field that it is amazing. That's one of the reasons why I call my range of products Ciúnas - the Irish for silence.
Ali Tait
29-05-2013, 16:37
Aye, can agree with that, it's not until you hear what the absence of noise does that you truly appreciate this.
I do however agree that the blind-testing usually suggested is pseudo-science - people pretending that they know what they are talking about/doing - playing a at doing science like the grown-ups do. Personal blind testing is fine as a personal sanity check but this mass blind testing is just crap. They pretend that the biggest bias factor is sightedness & then ignore all other biases & checks & statistics & run a half-arsed blind test for proof of their contention that "all DACs/amps/etc are the same".
Spot on, John. All done simply to bolster their own (usually very ingrained) prejudices! :rolleyes:
If only things were that simple.
Marco.
Aye, can agree with that, it's not until you hear what the absence of noise does that you truly appreciate this.
I hear what you guys are describing. Space, sound stage, 'quiet space'.
There is something similar in TVs and photography - when the black is 'really black' the image jumps off the screen in an almost 3D way, giving the depth and reality.
I have run a few blind listening tests with groups of people and once the word "test" is mentioned the entire exercise is fooked from the word go in my opinion.
To prove this theory, we did a mains cable blind test at one of my bakeoffs with six or seven people present and while it was a planned out formal test regime the results were all over the place, totally inconclusive. Once that stage was completed, I then said we will do a simple quick blind A-B with no pressure to determine any outcome, more of a giggle really than anything serious. The result was unanimous because that word "test" was left out of the scenario.
Umm..whilst interesting, that doesn't prove anything Frank.
With regard to an earlier comment (by John) regarding the blind testing of speakers - this is probably the single component that is the subject of most blind testing. In fact, some manufacturers use it as a primary development tool.
A couple of years ago I was at one of the hi-fi shows and Alan Sircom did a very interesting dem regarding the power of suggestion. First he told us all that was the focus of his dem, then he told us that it was quite easy to influence a dem. Then he said he was going to demonstrate two interconnects.The first he said was OK and plugged it in and played an expert of music - during which he sat motionless and read a paper. He then got a second interconnect, told us this was the best, plugged it in, turned up the volume a bit and proceeded to click his fingers and tape his feet to the music. He then asked the audience (of about 30 people) which they preferred. Even though they had been told about the blooming trick most still chose the second cable! The two cables were the same.
Umm..whilst interesting, that doesn't prove anything Frank.
It of course "proves" nothing and never will, but it is an important factor if the intention is to conduct a formal and objective blind test of any description, then it will probably be equal to, or less than reliable than a subjective sighted test because the participants have already been tainted.
It of course "proves" nothing and never will, but it is an important factor if the intention is to conduct a formal and objective blind test of any description, then it will probably be equal to, or less than reliable than a subjective sighted test because the participants have already been tainted.
Agree "proof" is not posssible in a blind test - a well conducted one returns a statistical confidence level of the result, not proof.
Eliminating sighted bias without realising or caring about other biases is a sure way to invalidate a test.
A formal & scientifically correct DBT is a difficult & expensive thing to organise. Anything less is a pretense!
A formal & scientifically correct DBT is a difficult & expensive thing to organise. Anything less is a pretence!
Nail > head.
The misconception is that the results of a flawed (non-formal and non-scientifically correct) DBT test are more conclusive and/or irrefutable than those of a flawed (in the commonly recognised way) sighted test. The fact is that, with audio, neither test 'proves' much beyond the most basic of assessment criteria, despite what some would like to think.
There are simply too many variables, necessary to consider (some possibly currently unknown), for the methodology or results of any such test to be genuinely accurate, and so either way, both types of tests are compromised. Therefore, the reality of the matter is that you simply have to choose your compromises and go with whatever results YOU consider are liable to be the most accurate (basically by satisfying your own subjective and/or objective judgement criteria)...
That's why, in the final analysis, I will always trust my ears, as more often than not, I find them perfectly adequate for the job that's required. There is absolutely no need for this mania with DBT. Dare I say it, some folk should get a life or seek medical help! ;)
Marco.
Ali Tait
01-06-2013, 19:16
I really don't get why some folk get so het up over this. At our Scottish Mafia get togethers, we just try stuff. Some sounds better. Some doesn't. Who gives a toss what anyone else thinks, if it sounds better to you then it IS better. Simples.
I completely agree, Ali. The problem is that some folk have an unhealthy and obsessive need to 'irrefutably prove' their life experiences before accepting that they actually exist!! :doh: :mental:
Marco.
Oldpinkman
06-06-2013, 12:09
With apologies to Ali Tait who has seen my original post on the Dac thread, for more repeats than the BBC, I had an "aha" on a beach in Norfolk this weekend, chatting power amp design to someone who knows what he's talking about. We recalled a lot of nonsense we had experienced with people hearing what they thought they should. As he put it, referring to research he had been doing on power amps, "the only positive correlation I can find between a design feature and perceived sound quality in reviews in magazines is price".
When you are designing a piece of kit, and developing product, blind testing helps eliminate bullshit and wishful thinking. However, this is only necessary if the difference is fairly small, and by definition, not that important. If a difference matters, you can hear it, and you know you can hear it. Some differences are "deal-breakers". Elsewhere I have quoted products which, without them, I am just not happy listening to music. I don't need to do A:B's, blind or otherwise to confirm that. The proof is I get bored, stop listening to the music, and go off and do household chores in preference. The thing about a "deal breaker" is it is fundamental to your enjoyment of the music, not just something you can spot as a party trick in an A:B comparison.
Indeed - for differences that matter, the "dealbreakers" A:B is probably not only unhelpful, but misleading more often than not. A long term fatigue test is the ultimate test of a dealbreaker (back to going off to do chores instead).
My best example must be Funk Firms F5 tonearm. I have only heard it twice - on a Funk LSD, with a Decca London, through a technics amp and speakers on chairs in a container used as a factory (you know - the lorry /ship things). 2nd time fitted to my PT with a Goldring 1012 and all my other toys, including my living room. Absolutely fundamental unmissable sound quality - perhaps the ultimate "dealbreaker" in my system. And easily recognised in both those VERY different systems. No need to rip it off and A:B with the Mission 774 it replaced. Quite happy to trust my hearing and memory.
When you are designing a piece of kit, and developing product, blind testing helps eliminate bullshit and wishful thinking.
Absolutely - I couldn't agree more!
However, I'm referring to the roasters (there's a nice bit of Scottish vernacular for you), who are merely end users of product, not designers, and don't have the gumption, ultimately, to trust the human instincts they were born with, when deciding if something that they've heard with their hi-fi system is real, and so need flawed bollocks like DBT to 'confirm' it.
They're married to measurements... Honestly, these nutters crack me up!! :lol:
Marco.
Powered by vBulletin® Version 4.2.3 Copyright © 2024 vBulletin Solutions, Inc. All rights reserved.