Blind Testing

[Yomanze]

There is a big difference between an ABX test outside of the context of your own system, own room and own music and a more meaningful test where you do something like use Foobar2000's ABX plugin to compare files (where I did find that I could reliably spot the difference between FLAC and 320kbps files, and less reliably, between FLAC and AIFF), or conduct blind testing by switching between kit that you know, in your room, with your kit... I know that I couldn't hope to pass an ABX test in an alien environment, but I know 100% that I can spot the difference between level-matched DACs, amps and preamps in my own system blind, because I've tried it.

[Clive]

There is a big difference between an ABX test outside of the context of your own system, own room and own music and a more meaningful test where you do something like use Foobar2000's ABX plugin to compare files (where I did find that I could reliably spot the difference between FLAC and 320kbps files, and even between FLAC and AIFF), or conduct blind testing by switching between kit that you know, in your room, with your kit... I know that I couldn't hope to pass an ABX test in an alien environment, but I know 100% that I can spot the difference between level-matched DACs, amps and preamps in my own system blind, because I've tried it.

That's an interesting perspective. There are certain aspects which are so very easy for me to differentiate in my own system & room, eg sound stage width, depth, central focus and lots more. But I don't need a blind test to do this as I know my environment and system so well. Maybe a blind test is more applicable to an alien environment, but it maybe harder to do this well as you say. It certainly takes a while dial into a new environment and if ABX is harder in such an environment there's a dilemma as most of our listening is in our own rooms.

[John]

I am certainly in the trust my own ears camp and totally agree its about hearing it in the context of my own equipment and environment, with a selection of music I know really well. I try to keep a open mind and will go back and fourth a few times usually, but sometimes there is no need for this as the differences are so obvious.
For me the danger is in thinking what I hear applies to everyone tastes. equipment and environment. What works for me may not work for someone else.

[walpurgis]

There's a big difference between blind testing with members of 'Joe Public' and doing it with a bunch of Hi-Fi buffs. The results are interesting. I've seen this done years ago on a couple of occasions.

[Yomanze]

There's a big difference between blind testing with members of 'Joe Public' and doing it with a bunch of Hi-Fi buffs. The results are interesting. I've seen this done years ago on a couple of occasions.

Very interesting please elaborate? Part of me thinks Joe Public does better and the buffs over analyse...

[Clive]

Would Joe Public listen for the esoteric we do? No and maybe he's happier for it!

The more I think about it blind A/B testing can only pick out relatively gross differences. The DB ABXers who tell me they need my ABX results if they are to listen to me will now get from me, "if I'm to listen to you views you need to tell me about your long-term auditioning conclusions". Yes I pick out differences from an A/B comparison but those differences may not be important, more something you acclimatise to. Long term listening is worth doing once you're sure there's nothing major out of kilter.

[Marco]

Clive, what will your 2000th post be...? Make it good!

Marco.

[walpurgis]

Very interesting please elaborate? Part of me thinks Joe Public does better and the buffs over analyse...

You have that about right. From what I recall, what the Hi-Fi buffs liked was described as "too bright" by 'Joe Public' and anything that 'Joe Public' described as having "a nice tone", was described by the Hi-Fi buffs as coloured or inaccurate.

[Eremite]

There's a big difference between blind testing with members of 'Joe Public' and doing it with a bunch of Hi-Fi buffs. The results are interesting. I've seen this done years ago on a couple of occasions.

If I recall correctly, Harman go so far as to train up a listener panel in order to do their testing...yes this is it...

http://seanolive.blogspot.co.uk/2011...isten-now.html

The rest of Mr Olive's column makes for an interesting read too.

[Reffc]

The problem as I see it with any audio DB tests (including ABx) is that you'll never get consistency between tests nor anything like a large enough sample to really draw any meaningful conclusions other than "on this occasion this (small) group preferred A or B or C" unless there are significant differences between what you are testing for.

Things like DBT's for other research (medical for example) involve thousands of sample numbers.

At the end of the day it doesn't really matter what can or cannot be proved (this coming from a techy!) but what you personally prefer yourself as you are the ultimate arbitor, not a select group of golden eared testers (or not). All imho of course.