View Full Version : For those who can hear a difference between Wav/AIFF/FLAC
Rothchild
16-10-2013, 16:35
Science needs you!
Please enroll for this simple 10 minute test to demonstrate that it's actually possible:
http://www.trustmeimascientist.com/2013/09/03/think-you-have-golden-ears-take-the-scientist-challenge/
They've made it really easy for you as you'll 'only' be spotting the difference between a 320kbs Mp3 and a 16bit 44.1 wav (or higher resolution) which for anyone capable of hearing the difference between wav and aiff (for instance) should be a freaking doddle. They'll help out by supplying the files (of music of your choosing) you can use your own prefered system and you can practice as much as you like. You can use Foobar's ABX tool or there's a neat open source tool here: http://abx-comparator.berlios.de/
Now is the chance to put all those terrible, cynical neighsayers in their place and prove once and for all that it really does matter and that it's clearly audible.
There's a nice follow up piece here: http://www.trustmeimascientist.com/2013/10/07/update-on-the-golden-ear-challenge-who-will-conquer-audio-mount-everest/ which you'll also have the opportunity to prove wrong.
NB: I appreciate that they've used an 'offensive' term in the title of their article but I beg that you don't let it distract you from the chance to settle this debate once and for all and prove that there is a difference that you can repeatably hear.
I'm not sure that I will hear a difference but in the spirit of investigation I'm going to take the test and see how I do, I urge you to also. The greater the body of research the better the quality of the claims it's able to make.
Ali Tait
16-10-2013, 17:13
I doubt anyone can tell the difference reliably TBH.
SteveTheShadow
16-10-2013, 19:38
TBH my CD collection lies idle as I have no CD player. I have ripped the ones I want, including a 20th Anniversary edition of Floyd's DSOTM. This is a CD I had forgotten all about. I only found it after I had bought the download from iTunes.
I have tried a few times to A/B the two recordings.
There is no difference whatsoever, between 256K AAC and 16/44.1 that I can hear.
I used to believe I could hear big differences. I was fooling myself and am no longer ashamed to admit it. There are far greater audible differences between digital music server/software, amps, DACs and speakers than there are between format resolutions. The recording is the least of our worries, when it comes to high quality audio. Of course vinyl is reckoned to be the ultimate high resolution format, but I continue to loathe and despise it and have never been afraid to say so.
My listening these days is almost exclusively 256K AAC iTunes plus a number of 320K Mp3 downloads and I don't feel I am missing anything.
JRiver Media Center for Mac reports over 17,000 songs in my library, and a large number of my jazz, blues and classical, recordings have stunning sound quality, whilst the rest are perfectly good.
There go all my hi-fi credentials and therefore anything I might say can now be considered unworthy of even cursory attention.
But I couldn't give a monkeys TBH.
I believe I can tell the difference, but who knows, I could be fooling myself.....dont need to prove it though.I will stick to flac thanks.....
I am definitely missing enough between lossless / WAV / AIFF and 320kbps as I found during my 100% confidence tests using Foobar2000's ABX plugin. However I did also find that I could spot the difference between FLAC and WAV using the same plugin, but I quickly 'fatigued' and I repeatedly found that I could spot the differences at first, but then started drawing blanks. ABX tests often have too poor conditions to cater for our extremely poor and rapidly fading auditory memory, but if you can instantly switch it makes things a whole lot more meaningful a test.
StanleyB
17-10-2013, 02:57
You can use Foobar's ABX tool or there's a neat open source tool here: http://abx-comparator.berlios.de/
Is Foobar the only option as far as the decoding medium is concerned, and how accurate is it? A chain is after all only as good as its weakest link, and the quality of decoding and playback is of the utmost importance in such a test. Messing around with ASIO and WASAPI for different audio players is the stuff of legends. No two people can agree which driver, decoder, or audio player is more accurate. That in turn will impact on the accuracy of any set of files that is run through them.
I don't use Foobar myself. I think it is lousy. But if there is a Media monkey ABX plug-in I am up for the challenge.
Rothchild
17-10-2013, 06:35
A chain is after all only as good as its weakest link, and the quality of decoding and playback is of the utmost importance in such a test..... That in turn will impact on the accuracy of any set of files that is run through them.
Sorry for clipping your quote, I hope I'm not changing your sense too much, but I have to ask if you're seriously suggesting that you'll only be able to hear the difference between a 320Kbs Mp3 and a Redbook Wav using a particular player? Or that Foobar is so poorly/well implemented that it makes a wav sound like an Mp3 (of visa versa)?
It seems that many who have an interest in this arena are moving to citing differences in ever increasingly esoteric places. Yet people have claimed on these very boards that there is a repeatably recognisable difference between Wav/FLAC/AIFF yet there's a dearth of takers when presented with an opportunity to shut up every single neighsayer on the internet (and have a flattering article written about you) with what must to these folk be a comparatively easy test; to spot the difference between an Mp3 and a straight PCM Wav. The people who publicly make the claims should have no problem publicly proving them, surely?
It's such a great chance to put this stupid debate to bed once and for all, why can't we find amongst our hallowed ranks even just one person to put the cat out in to the pigeon field?
I'll have a go at the weekend. I reckon I can tell the difference even through my cheap little powered PC speakers. Of course I could be disasterously wrong about that and I'll be happy to admit it if I am.
The difference between 320kbps pretty small it has to be said. If someone knows what to listen for then it should be possible to hear a difference to the tails of hf sounds but this would be much easier with lower bit rate mp3.
If the challenge had been about different versions of the experimental MQn player, this would have been a far easier test. It's a bind test I have taken and have run with 4 individuals.
StanleyB
17-10-2013, 08:11
Sorry for clipping your quote, I hope I'm not changing your sense too much, but I have to ask if you're seriously suggesting that you'll only be able to hear the difference between a 320Kbs Mp3 and a Redbook Wav using a particular player? Or that Foobar is so poorly/well implemented that it makes a wav sound like an Mp3 (of visa versa)?
:scratch:. So your initial statement that "you can use your own prefered system" is in fact untrue. Are you a serving or former police officer by any chance?
According to the information that I have on the Foobar ABX plug-in, it levels out the sound between the two audio files. Now in order to do that, the bitrate has to be altered. If the output files you are supposed to be judging are not the same as the original files due to alterations of the bitrate by Foobar, how can judgement be made on tampered evidence?
sq225917
17-10-2013, 08:16
I don't think there's any difference between bit perfect software players, I've never been able to detect any, though the nature of the beast requires long switching times where I much prefer an instantaneous switch. I've never been able to hear any difference between any of the lossless file formats through a bit perfect player either, and that has been via instantaneous abx. I can however tell the difference between mp3 at 320k and cd quality, but it really does depends on the music. Any well recorded acoustic, classical or live stuff I can always pick out, crappily produced, and or heavy music with a lot of studio trickery and awful compression is much harder to discern.
I'd sure love to meet someone with magic ears who can tell lossless formats apart.
I did a little experimentation R.E. FLAC v WAV & the MDAC bit perfect test last year...
http://theartofsound.net/forum/showthread.php?21132-Squeezebox-FLAC-Wi-Fi-and-the-M-DAC-bit-perfect-test!
...the only thing that's changed is my MDAC is now upgraded to 'L2 Toy', and I have an iFi USB Power in place of the galvanic isolator...and it's still bit perfect, but sounds even better...;)
I'd sure love to meet someone with magic ears who can tell lossless formats apart.
I don't think I can if things are set up properly, but there are a lot of variables otherwise. I've a feeling I missed something when I was comparing and in retrospect won't be losing any sleep on FLAC vs. WAV. Lossless and 320kbps is a different story, and readily identifiable on my STAX headphones for instance.
:scratch:. So your initial statement that "you can use your own prefered system" is in fact untrue. Are you a serving or former police officer by any chance?
?
That's very funny if a little bit unkind.
Rothchild
17-10-2013, 15:48
Geez, yeah! I'm not sure who's least trustworthy now? The Police have entered the ranks of Politicians and Estate agents! (and it's endlessly funny to me that 'our' confidence in the police shouldn't have been shaken when they killed that bloke at G20, wasted our money spying on hippies on behalf of the energy companies or kettling legitimate protestors but the moment one of the posh boys cops a bit of stick from one (having admitedly called him names just not 'that' name) it calls the whole service's intergrity into doubt!)
Anyway!
I appreciate that the statement is a little misleading, I just took it to mean your own amp, speakers, room, cable, dac, computer etc rather than the software so yes, on a small point of order I was a little misleading. This was not my intention so I'm sorry about that.
Nevertheless, the point remains and my question still stands: Are you suggesting that only certain players can present the difference between a 320K Mp3 and a Wav file? I still contend if there are people able to clearly determine a difference between Wav/Flac/Aiff they should walk this test even if the source were transcribed back to a wax cylinder (ok maybe I'm exagerating...). The folk who can have nothing to lose by taking the test and once they've proven that it is, of course and indeed, possible there will be a nice article that we can refer back to each time this debate rears its ugly heads and all the cynics can be put in their places once and for all.
Thanks for the pointer about the Foobar ABX tool, I will research, test and attempt to feedback my findings this evening.
realysm42
17-10-2013, 15:50
I'll have a go this weekend.
TBH my CD collection lies idle as I have no CD player. I have ripped the ones I want, including a 20th Anniversary edition of Floyd's DSOTM. This is a CD I had forgotten all about. I only found it after I had bought the download from iTunes.
I have tried a few times to A/B the two recordings.
There is no difference whatsoever, between 256K AAC and 16/44.1 that I can hear.
I used to believe I could hear big differences. I was fooling myself and am no longer ashamed to admit it. There are far greater audible differences between digital music server/software, amps, DACs and speakers than there are between format resolutions. The recording is the least of our worries, when it comes to high quality audio. Of course vinyl is reckoned to be the ultimate high resolution format, but I continue to loathe and despise it and have never been afraid to say so.
My listening these days is almost exclusively 256K AAC iTunes plus a number of 320K Mp3 downloads and I don't feel I am missing anything.
JRiver Media Center for Mac reports over 17,000 songs in my library, and a large number of my jazz, blues and classical, recordings have stunning sound quality, whilst the rest are perfectly good.
There go all my hi-fi credentials and therefore anything I might say can now be considered unworthy of even cursory attention.
But I couldn't give a monkeys TBH.
Agreed!
Frankly I couldn't give a toss about the 'packaging', if it sounds good to me and I enjoy the music I am happy to listen on any format. Much of my listening is done via Spotify and I can honestly say that I am perfectly happy with what emanates from my speakers during these sessions. Am I getting the best out of my system listening this way? Perhaps not but neither I nor Jo are in the slightest bit bothered about it, so if that strips me of any audiophile pretensions in the eyes (ears?) of others then so be it. :)
magiccarpetride
18-10-2013, 22:11
Science needs you!
Please enroll for this simple 10 minute test to demonstrate that it's actually possible:
http://www.trustmeimascientist.com/2013/09/03/think-you-have-golden-ears-take-the-scientist-challenge/
They've made it really easy for you as you'll 'only' be spotting the difference between a 320kbs Mp3 and a 16bit 44.1 wav (or higher resolution) which for anyone capable of hearing the difference between wav and aiff (for instance) should be a freaking doddle. They'll help out by supplying the files (of music of your choosing) you can use your own prefered system and you can practice as much as you like. You can use Foobar's ABX tool or there's a neat open source tool here: http://abx-comparator.berlios.de/
Now is the chance to put all those terrible, cynical neighsayers in their place and prove once and for all that it really does matter and that it's clearly audible.
There's a nice follow up piece here: http://www.trustmeimascientist.com/2013/10/07/update-on-the-golden-ear-challenge-who-will-conquer-audio-mount-everest/ which you'll also have the opportunity to prove wrong.
NB: I appreciate that they've used an 'offensive' term in the title of their article but I beg that you don't let it distract you from the chance to settle this debate once and for all and prove that there is a difference that you can repeatably hear.
I'm not sure that I will hear a difference but in the spirit of investigation I'm going to take the test and see how I do, I urge you to also. The greater the body of research the better the quality of the claims it's able to make.
With all due respect, this challenge is meaningless. People can't seem to be able to clue in to the fact that we are not listening to sounds, we are listening to music. Listening to music is akin to taking drugs. Music affects our moods, it affects our perception, it affects our interpretation. We are not mechanical devices nor are we measuring instruments who can produce consistent, measurable results.
If I claim to prefer one format (say mp3) over another format (say lossless 16 bit), I do it based on my overall experience obtained through prolonged listening. Even if I cannot pinpoint with robotic precision whether mp3 or WAV format is playing, I get the overall feeling, the mood changer, that's telling me that one format definitely works better for me.
Of course, it would be arrogant of me to presume that what works best for me must by necessity also work for others. But the fact remains that certain formats suit my constitution better.
For example, I focus a lot on listenability; to me, analog-like sound, similar to the quality I get from a decently pressed LP spinning on a better-then-average turntable, is the most listenable sound. I can spend hours listening intently to that type of sound and never experience any listening fatigue.
On the other hand, if I play the same material in the mp3 format, even if at first it may sound more exciting or more kick-ass than vinyl, in the longer run I find it to be tiring, exhausting, fatiguing.
I agree with magiccarpetride and add that comparative A/B tests emphasise our analytical abilities at the expense of using our brain's Limbic System. Yet when we listen for enjoyment over the long-term the Limbic System is very involved, to shut it out by performing short A/B tests frankly results in the tests not being fit for purpose.
Rothchild
19-10-2013, 07:41
I'd agree that subjective preference is entirely untestable (and any concept of a test is pointless, it's the sort of experiment that gets psychologists and sociologers a bad name). However, the subjective determination of a difference is entirely testable, with the test specified in the OP as being one of the simplest ways of doing so. Indeed not hearing a difference still may not affect the subjective preference but it will help those of us trying to extract maximum SPP to know where our resources might be best expended.
I wonder if the fact that I said that the test can only take 10 minutes has perhaps misdirected people somewhat as it is also the case that you can practice as much as you like and take as long as you like over it. I just thought that given that it's such an easy test (in the scale of these things, a lossy codec vs real pcm) that those who can would just blast through it but it doesn't preclude taking your time and letting which ever bit of you brain that does the listening, do the listening.
Rothchild
19-10-2013, 07:41
Sorry, double post. On a flaky connection today.
I think also for me I really need to know the music before trying a test like this
I will record something I know well both in FLAC and WAV and report back
I certainly notice a difference when I play with files sometimes positive sometimes negative and more than welcome to demonstrate this on my own system for those that doubt this
I'd agree that subjective preference is entirely untestable (and any concept of a test is pointless, it's the sort of experiment that gets psychologists and sociologers a bad name). However, the subjective determination of a difference is entirely testable, with the test specified in the OP as being one of the simplest ways of doing so. Indeed not hearing a difference still may not affect the subjective preference but it will help those of us trying to extract maximum SPP to know where our resources might be best expended.
I wonder if the fact that I said that the test can only take 10 minutes has perhaps misdirected people somewhat as it is also the case that you can practice as much as you like and take as long as you like over it. I just thought that given that it's such an easy test (in the scale of these things, a lossy codec vs real pcm) that those who can would just blast through it but it doesn't preclude taking your time and letting which ever bit of you brain that does the listening, do the listening.
I'll give a go later if I'm not all tested out. I'm spending time with many different versions of a player currently, which really are quite different to each other. There's only so many tests I can do in a day.
SteveTheShadow
19-10-2013, 08:40
For example, I focus a lot on listenability; to me, analog-like sound, similar to the quality I get from a decently pressed LP spinning on a better-then-average turntable, is the most listenable sound. I can spend hours listening intently to that type of sound and never experience any listening fatigue.
On the other hand, if I play the same material in the mp3 format, even if at first it may sound more exciting or more kick-ass than vinyl, in the longer run I find it to be tiring, exhausting, fatiguing.
I agree with this entirely.
Only difference in my case is that it is the distortions present in vinyl playback that drive me crazy.
When I had my LP12 I was on edge all the time and didn't enjoy music at all.
Other people's high end TTs drive me nuts too so I can't simply blame the LP12.
I have striven to get my system to sound like reel to reel analogue, and have only recently succeeded, with computer based audio.
Listenability is the key, not trying to hear nigh on undetectable differences between digital formats.
This kind of crap is what gives audiophiles a bad name in the real world. But it suits the marketing numbers game to a tee.
We should be promoting high quality playback of music in the home as something to aspire to and enjoy,
something that is worthwhile; not an impenetrable load of pseudo-scientific, marketing-driven nonsense.
I feel the reason why many of us like the older amplifier/speaker driver/ concepts from the 50s and 60s
matched up with modern computer audio systems or high end vinyl, if that's your poison, is the way they put you in touch with the heart and soul of the music; something that a great deal of modern gear spectacularly fails to do.
The golden-eared cognoscenti can argue all they want that it's coloration; it makes not one jot of difference.
They can fool, themselves that they hear a gnat fart in the second row on hi-rez, whilst you or I can't appreciate that
degree of subtlety with 16/44.1. Cobblers I say!
Excellent post, Steve. I concur completely. However, with this bit:
Only difference in my case is that it is the distortions present in vinyl playback that drive me crazy.
When I had my LP12 I was on edge all the time and didn't enjoy music at all.
Other people's high end TTs drive me nuts too so I can't simply blame the LP12.
...I'd be very interested in your opinion of listening to my own T/T, playing mint condition (well-produced) vinyl, as I can detect none of the distortions you mention, although I know exactly what it is you're referring to.
One of the best things about high-quality direct-drive turntables, such as modified SL-1200s/1210s, is that when optimally partnered and set-up (playing the aforementioned pristine vinyl, which 85% of my record collection consists of), is that they sound remarkably neutral and 'master tape-like', or if you prefer, 'digital', in the sense of when digital is done well, not badly, as is no doubt the case with your music streaming.
Like you, I utterly detest listening to music if it sounds like, in a bad sense, it's being reproduced by a turntable :spew:
Therefore, because I can detect no such distortions or colorations from my T/T, I often forget whether I'm listening to a record or a CD! However, your ears may pick on something that mine don't, so it would be interesting for us to have a sesh sometime (either at your place or mine) where we can analyse for that effect.
Either you or me, or both of us, may learn something new :)
Marco.
SteveTheShadow
19-10-2013, 15:05
Excellent post, Steve. I concur completely. However, with this bit:
However, your ears may pick on something that mine don't, so it would be interesting for us to have a sesh sometime (either at your place or mine) where we can analyse for that effect.
Either you or me, or both of us, may learn something new :)
Marco.
Hi Marco, I'm up to my eyes in work at the mo as the assessment season gets underway so results can be in before Christmas.
Maybe in the new year.
Rothchild
19-10-2013, 18:13
I think also for me I really need to know the music before trying a test like this
I will record something I know well both in FLAC and WAV and report back
I certainly notice a difference when I play with files sometimes positive sometimes negative and more than welcome to demonstrate this on my own system for those that doubt this
John, you can use any music you like and we're talking about reliably calling the difference between a 320k Mp3 and a 16/44.1 Wav (or higher indeed) rather than FLAC (I put that in the title as some people have claimed difference so this is an easy starter, perhaps if lots of folk can regularly and demonstrably spot this contrast we might step it up to FLAC / AIFF etc)
Take all the time you need, find an ABX tool that you get on with and let us know how you get on.
Thanks for taking part.
What I do is record the same tracks in WAV and FLAC and let you know how I get on
With all due respect, this challenge is meaningless. People can't seem to be able to clue in to the fact that we are not listening to sounds, we are listening to music. Listening to music is akin to taking drugs. Music affects our moods, it affects our perception, it affects our interpretation. We are not mechanical devices nor are we measuring instruments who can produce consistent, measurable results.
I'm struggling with my memory here, but is it Glenn? (magiccarpetride) . . anyway, I'm with you fella. Music to me has never been a test signal and never will be. I have said it a number of times, but I personally cannot reliably tell the difference between WAV, FLAC or AIFF and spent a good deal of time researching this before embarking on the mammoth task of ripping my CD library, which is now around 3,000 files. I wouldn't like to take a 'one on one' Pepsi challenge against mp3 (320) and a lossless file, but I do know I don't enjoy listening to it for extending periods, but I can listen to FLAC all day long and regularly do. I'm not sure if that's down to my system, the way its configured or my preferred style of music, but I don't really care too much about that either, as I don't listen to my system anymore I listen to bass lines, musical composition, lyrics, vocal harmonies and try to understand what the artist is portraying and what lies behind the music, rather than how much this new cable may have 'altered' the sound - to me that can be rather counter productive and gets in the way of what my end goal is, but that's just me.
If you can tell the difference and its important to you, then work away as its all about what makes you happy, music is great for that as it reaches so many different people, for so many different reasons and I love that. As for me, well you may have noticed I don't pop up that often now, but its due to being totally content and happy to spend my time listening to music or going to shows, which is what gives me the buzz from music, not messing about with gear. So this test is moot for me, but I agree with what the OP is saying to a point.
Of course it could also be because I'm a little deaf, but what the heck I'm happy with my sounds and that's all I really care about now . . .
:comatose:
brucew268
19-10-2013, 19:34
Last year as I was moving from CD to server-based source, I was curious about this and ripped the same tracks to MP3, WAV, and to FLAC. In a blind test I don't know that I could reliably tell you which format I was listening to. But I could tell which one I liked best each time, and it usually ended up being a WAV file. Happily, JRMC and dbPoweramp support the same method of tagging WAV files, so I ripped everything to WAV and FLAC,with FLAC being the backup and WAV being the working file. The differences were subtle but perceptible on well recorded music. I also have some music that is beautifully performed and satisfying to listen to, but where I cannot tell the difference between MP3 320CBR and WAV fullres. I assume that has something to do with the recording process and am happy that it still turned out a musically satisfying result.
BTW some will argue that if they are both bit-perfect, they will sound the same. Others will argue that the computer processing of each algorithm can cause differences in sound, just as two CD players with similarly low levels of jitter/distortion can sound different. Others claim that there is something poor in the design of the FLAC 'shell' (who knows what). All I know is that in my system the sound I slightly but consistently prefer is coming from WAV files.
I have tried a few files and struggling to hear a difference between FLAC and WAV
I shall still continue to use WAV myself as see no point in changing format and will try a few more files on Sunday
I can still hear a difference when I change the file from standard CD to 24bit 96khz usually for the better but not always
magiccarpetride
21-10-2013, 19:25
I'd agree that subjective preference is entirely untestable (and any concept of a test is pointless, it's the sort of experiment that gets psychologists and sociologers a bad name). However, the subjective determination of a difference is entirely testable, with the test specified in the OP as being one of the simplest ways of doing so. Indeed not hearing a difference still may not affect the subjective preference but it will help those of us trying to extract maximum SPP to know where our resources might be best expended.
I wonder if the fact that I said that the test can only take 10 minutes has perhaps misdirected people somewhat as it is also the case that you can practice as much as you like and take as long as you like over it. I just thought that given that it's such an easy test (in the scale of these things, a lossy codec vs real pcm) that those who can would just blast through it but it doesn't preclude taking your time and letting which ever bit of you brain that does the listening, do the listening.
My point is this: even if I would score success in the proposed test, it wouldn't mean anything. The same test subject may score near 100% success one day and then fail miserably doing the exact same listening test the next day (or, within the next hour). Human propensity for perceiving and detecting nuances fluctuates wildly from one minute to the next. Not reliable in the least, and thus it is absolutely meaningless to try and pinpoint such a wishy-washy thing down to exact science. Even legally speaking, it is a recognized fact that humans accounting for experiences that transpired in their world is a very unreliable venture. Hence the emergence of elaborate legal systems attempting to curb this inherent flakiness.
We're humans, we're not robots. Furthermore, most of us consume music from the neck down (except audio geeks who do it from the neck up).
Originally ripped all my music to Flac.
Rainy Sunday afternoon messing about and I converted a couple of albums I knew well to uncompressed WAV and AIFF in Jriver.
Was able to hear improvements on the vast majority of the tracks to set about converting them all to uncompressed format.
Have the storage space so why not?
Further, to me, AIFF sounded better (more relaxed & open but losing no focus or detail) than WAV.
This was more evident in hires files.
Job done, dont need to think about it any more, all Flacs converted to AIFF.
Like most things, its probably ears/system/level of arsedness dependent, so Ymmv may vary.
Try these things for yourself in your own system, its worth a go I think, even if just to rule It out.
Rothchild
27-10-2013, 11:45
Hi Gary,
Please contact Justin at 'trustmeimascientist' (he seems to be quite a nice chap) and arrange for some verified files and an agreed test process, this argument needs to be put beyond the scope of claim and counterclaim. Once you've done it you'll save everyone who can hear the difference from having to make unsubstantiated claims on internet messageboards and there'll be a single reference (that Justin will write celebrating your ability) that all doubters can be referred to.
If you 'can't be bothered' or 'don't need to prove yourself' (or some other variation of the argument that will prevent you proving your claim) then I would humbly suggest that you also shouldn't be wasting your time making the claim publicly.
As someone who can hear the difference between wav and aiff the difference between wav and 320kbs Mp3 should take you virtually no time or effort at all.
brucew268
27-10-2013, 12:12
Discussion boards are discussions... and Gary made no claims except for what HE heard on HIS system. Perhaps, though, he would have done better starting a separate thread since as OP you seem intellectually hostile to his perspective.
Rothchild
27-10-2013, 12:45
Discussion boards are discussions... and Gary made no claims except for what HE heard on HIS system. Perhaps, though, he would have done better starting a separate thread since as OP you seem intellectually hostile to his perspective.
Given that the topic is about generating enough evidence to substantiate such claims I think it's fair for me to maintain this focus. The point is: If it's audiably different one should be able to pick up on the difference at a better rate than chance. (and I would add as a corrolory as the debate has developed that if the difference is only audiable on a given setup, in a given room, to a given listener etc then the difference is quite probably not in the source material).
Hostile (intellectually or otherwise) is a bit strong, fair enough I'm not just settling in to a cosy concensus but I feel I'm being polite and discursive, if anyone were to bring another method or tool that would repeatably allow us run a similar test to check claims and assumptions I'd be very happy to hear/try it. The internet is, after all, just a great big advertising hoarding so having repeatable tests and other mechanisms to verify the claims found therein can only be a good thing surely?
I have at no point suggested that anyone isn't hearing what they are hearing on their systems, what I'm seeking is some amount of verification to the claims in keeping with the ethos of the boards of determining how to derive the best value for money from my HiFi.
sq225917
27-10-2013, 15:17
People think lots of things in life that aren't true yet they effect their daily comings and goings, no reason HiFi should be any different.
Rothchild
27-10-2013, 16:28
People think lots of things in life that aren't true yet they effect their daily comings and goings, no reason HiFi should be any different.
I don't disagree with this statement, indeed I think it nails the nub of the issue at hand.
When the effect of believing something that isn't true results in spending unnecessary money on HiFi gear then the outcome is relatively trivial.
But, this propensity, when scaled up to more important matters, can become exceedingly dangerous. So it's useful to have methods and mechanisms to verify our amazing abilities (and I mean that genuinely, people can perceive and act in amazing ways) against our acknowledged susceptibility to believing/accepting untruths (which there are rather too many people in the world willing to exploit for their own gain).
brucew268
27-10-2013, 18:29
Thank you for the clarification on where you are coming from.
The point is: If it's audiably different one should be able to pick up on the difference at a better rate than chance. (and I would add as a corrolory as the debate has developed that if the difference is only audiable on a given setup, in a given room, to a given listener etc then the difference is quite probably not in the source material).
Might I offer a additional perspective. Yes, one should have a better rate than chance at identifying difference. Agreed. Yet, if the difference is not in the source material (bits lost or added), that does not mean as some might imply that it is entirely subjective. We know that assembling a great and musically satisfying system is more than just finding the best components; it is a matter of the synergy of the various components electrically and sonically. We could find several components that are judged very accurate yet which have different voicing from each other and which do not have identical electronic measurements. Some will work well with each other and some we might judge not to work well together. In addition one of the most significant components is the room and how the hi-fi is interacting with the room (speakers, space, boundaries, etc.). We haven't even mentioned whether the server and DAC handle the different format algorithms (WAV, FLAC, etc.) with equal aplomb.
Yet, I agree that in the system I have assembled for my tastes, I should be able to determine a difference at a greater rate than chance, and so should most on this board with fairly well developed and resolving systems. If not, we can stop 'worrying'.
...having repeatable tests and other mechanisms to verify the claims found therein can only be a good thing surely? Agreed. I just have to decided whether I care enough to take the time to download Foobar and set it up properly and then run the test. Certainly don't have the time in the next 3 weeks, and not sure I'm bothered enough to prove it to naysayers by doing proper double-blind... about which arguments are rife and some boards have specifically forbidden discussion of the method due to the angry discussions that tend to ensue.
Marc,
Imo your base assumption that you can form valid test results from this is flawed.
You cant form a valid test for something that is subjective.
All you have is a set of numbers, x number of people say they can hear a difference, y say they cannot.
Then you draw some conclusion that fits.
As this is a discussion board, can I also humbly suggest you refrain from suggesting what people should or should not share their opinions on, thanks.
If youve been around long enough on hifi forums youll see that the subjective/objective argument comes up all the time, has been for years.
No point going round in circles mate.
Rothchild
27-10-2013, 18:44
Gary, you're right. I shouldn't have suggested you not share your opinion, it's not really what I meant but I can see how it was easily misinterpreted, so I'm sorry about that.
Re: not testable due to subjective nature please see post #22
This thread is not a subjective / objective debate, it's about using a repeatable testing methodology to validate the quality of subjective judgements.
To be fair this has always been a put your money where your mouth is thread and it,s not subjective. If you asked people to see the difference between two colours it would be the same. I'm suprised some of the people that can hear the difference in digital cables have not stepped up to the challenge.
Gary, you're right. I shouldn't have suggested you not share your opinion, it's not really what I meant but I can see how it was easily misinterpreted, so I'm sorry about that.
Re: not testable due to subjective nature please see post #22
This thread is not a subjective / objective debate, it's about using a repeatable testing methodology to validate the quality of subjective judgements.
No harm done Marc. :)
Don't run Foobar on my system (and I'm a lazy git) so cant help, sorry.
Rothchild
28-10-2013, 07:27
Well, thanks for taking the time to reassert your claims Gary, I'm glad you're not so lazy that you didn't miss that opportunity.
Andrew, thanks for summing it all up so succinctly. It is interesting that we regularly see so many claims for the ability to resolve difference but when provided with an opportunity to prevent those claims being challenged there is a dearth of takers and the claims are replaced with even more complex posturing and ever more esoteric rationalising as to why a simple 'is same, is different' test won't tell us anything.
5 pages and no one willing (or able?) to put it to rest once and for all (and make themselves an internet hero in the process).
I saw this elsewhere but it's a nice addendum to the issue raised by Simon in #34
Does square B look:
A: Lighter
B: Darker
C: The Same
as square A?
https://upload.wikimedia.org/wikipedia/commons/thumb/6/60/Grey_square_optical_illusion.PNG/220px-Grey_square_optical_illusion.PNG
Marc, that is an excellent illusion and very clearly illustrates how our senses can be fooled. Even though I know A and B are the same shade I cannot convince my eyes that they are.
I will have a go at the test in a minute. However, I must admit that I don't really care what the results are as I have almost zero invested in digital; perhaps that makes me a good candidate...
Hmm... so first you have to search for the Foobar2000 (which I happen to already have) ABX add on, then you have to download it, then you have to install it - then it doesn't work (for me any way). To be blunt, I'm not surprised no one has done this.
Ha, have got it all working now - I rather like it as a bolt on :).
So, am i right in thinking you have to create your own files to do this 'test'?
Ha, have got it all working now - I rather like it as a bolt on :).
So, am i right in thinking you have to create your own files to do this 'test'?
Yes, but to keep it reliable you'd need to use something like http://www.exactaudiocopy.de/ to get bit perfect rips.
Blimey, there is rather a lot of hoops and jumping involved! I already have some digital files I could convert to the two formats (as I have very few CD's), but my interest is waning. I do like the ABX software though and can see myself using it to compare files I have made of turntable modifications etc.
Well, thanks for taking the time to reassert your claims Gary, I'm glad you're not so lazy that you didn't miss that opportunity.
Andrew, thanks for summing it all up so succinctly. It is interesting that we regularly see so many claims for the ability to resolve difference but when provided with an opportunity to prevent those claims being challenged there is a dearth of takers and the claims are replaced with even more complex posturing and ever more esoteric rationalising as to why a simple 'is same, is different' test won't tell us anything.
5 pages and no one willing (or able?) to put it to rest once and for all (and make themselves an internet hero in the process).
I saw this elsewhere but it's a nice addendum to the issue raised by Simon in #34
Does square B look:
A: Lighter
B: Darker
C: The Same
as square A?
https://upload.wikimedia.org/wikipedia/commons/thumb/6/60/Grey_square_optical_illusion.PNG/220px-Grey_square_optical_illusion.PNG
Assuming that first line is slight sarcasm Marc?
Too much "complex posturing and ever more esoteric rationalising" already for my tastes. :)
Good luck though with this, good debate always makes for interesting reading.
Gaz.
realysm42
28-10-2013, 12:34
Has anyone gone to the trouble of making the files to A/B, so then all I have to do is get Foobar and set it up :eyebrows:
I will get round to this, eventually.
I will create some this week, can share the files here.
Too much hassle to much mucking about with computers. If I wanted to do that I'd be making a fortune in I.T - If someone wants to set up a proper test where we just turn up and do it, no problem with that, I will be happy to contribute towards the cost.
I will create some this week, can share the files here.
I doubt that you can I'm afraid - unless you are going to use music you own the publishing rights to.
Yes sorry we cannot allow files to be shared here Last time we had issues for months because of that
realysm42
28-10-2013, 18:43
Thanks for the offer Neil.
And apologies guys, I understand what you're saying. I'll still get round to it, was just being lazy...
Rothchild
29-10-2013, 16:55
Gary, it was only slight ;-) Despite our disagreements here I'm sure we'd get on over a :cool: and some choons sometime.
Mark, glad you got it working, it's not that complex!
If anyone wants to play along:
Foobar can be downloaded for free from here: http://www.foobar2000.org/
The ABX comparator can be snagged here: http://www.foobar2000.org/components/view/foo_abx (when I downloaded it and clicked on it it knew what to do and just installed itself in to foobar)
For now you can select any of your own music from CD or native wav. This is the part of trying to make it easy to pass the test, your own (digital) music on your own rig (apart from the player as earlier clarified)
You may also need something to convert wavs / cds to mp3 - I snagged a copy of LAME from: http://www.rarewares.org/mp3-lame-bundle.php
If you can derive test result better than chance then Justin at trustmeimascientist will engage and provide you with 3rd party verified sources for the sake of confirmation, but for now there is an aspect of trust that participants will play fair.
I just ripped a track from a cd (Betty Wright - Cleanup Woman, hot damn!) did it once to wav and once to 320kpsCBR (just wind the quality slider up to the right hand end in the encoder settings) and dropped both resultant files in to the playlist in Foobar, make sure both are selected and then right click and go 'utilities' / 'ABX Two Tracks' left both options unticked (so there's no funny business going on) and away you go. You can select loop points, start / stop points and can flick back and forth in a virtually seamless fashion to your hearts content (or until you think you can hear a spot if x or y is a or b) then press the button and move on.
For the purposes of 'good science' you should tick 'hide results' and commit to doing a certain number of itterations of the test before you look (10 is a fair minimum) otherwise the risk is that you just do enough tests chase a result rather than it being an honest piece of research.
Maybe it would be fun to share a playlist of tracks that we think folk might have access to and make it easy to recognise the difference in? Potentially we could take an amount of this 'off board' for the aspect that John refers to. If folk are up for that let me know and I'll make some arrangements.
Gary, it was only slight ;-) Despite our disagreements here I'm sure we'd get on over a :cool: and some choons sometime.
Marc,
I always play the hurt card :)
Naa, yer fine mate...choons n beer cool, mind you, your buying you shandy drinking southerner.
Genuinely interested in reading the results, more as a psychological experiment mind you than anything hifi related...
Three pages later and none have at least tried to test it and kill the argument stone dead. I spent a good amount of time a year or so ago using Foobar2000's ABX plugin, using FLAC, AIFF, LAME-encoded 320kbps etc. I couldn't reliably tell the difference between FLAC and WAV/AIFF blind, but I did between lossless and 320kbps, admittedly I was using STAX electrostatic 'phones, and at the time not convinced I could have noticed through the speakers.
As it stands I sold my V-Link 192 and stuck with CD playback, if someone could lend me a USB to SPDIF converter then would happily repeat the test.
I converted a file using lame to 320 mp3. Conducted 35 tests in total, some short, some switching a lot. At the start it was difficult, I had to try and locate a reference point where one sounded slightly rougher than the other and switch between them. All in all it was a pretty tedious session, and tiring on the ears listening to the same 5 seconds again and again (red lorry yellow lorry comes to mind). It was going well (down to 7.6%) then I lost my mojo. Results are below.
19:02:27 : Test started.
19:07:17 : 01/01 50.0%
19:07:43 : 01/02 75.0%
19:10:44 : 01/03 87.5%
19:15:35 : 01/04 93.8%
19:16:04 : 01/05 96.9%
19:20:01 : 01/06 98.4%
19:23:53 : 02/07 93.8%
19:27:36 : 02/08 96.5%
19:29:43 : 03/09 91.0%
19:30:38 : 04/10 82.8%
19:31:24 : 05/11 72.6%
19:31:41 : 06/12 61.3%
19:33:11 : 06/13 70.9%
19:34:17 : 07/14 60.5%
19:36:01 : 08/15 50.0%
19:38:06 : 08/16 59.8%
19:38:35 : 09/17 50.0%
19:40:01 : 10/18 40.7%
19:42:50 : 11/19 32.4%
19:45:08 : 12/20 25.2%
19:46:12 : 13/21 19.2%
19:46:51 : 14/22 14.3%
19:48:09 : 15/23 10.5%
19:58:28 : 16/24 7.6%
20:00:17 : 16/25 11.5%
20:05:13 : 16/26 16.3%
20:06:54 : 16/27 22.1%
20:09:51 : 17/28 17.2%
20:12:28 : 17/29 22.9%
20:13:54 : 18/30 18.1%
20:14:59 : 19/31 14.1%
20:21:37 : 19/32 18.9%
20:23:37 : 19/33 24.3%
20:25:56 : 19/34 30.4%
20:26:43 : 19/35 36.8%
20:26:46 : Test finished.
----------
Total: 19/35 (36.8%)
Rothchild
30-10-2013, 20:42
Wow Rob, that's pretty impressive thanks for sharing.
What was your reference track?
I've got find a decent reference to do this, it's been fun playing with Betty Wright but as that track sounds like it was recorded on (soulful) sandpaper at the best of times I don't think it's going to give me a chance of hearing a real difference!
The track was Trafik - Indestructible. I'm sure there are more suitable better mastered tracks to test with though.
realysm42
30-10-2013, 20:51
Thanks Rob.
Do you think you started to score worse because you got tired/lost interest?
Thanks Rob.
Do you think you started to score worse because you got tired/lost interest?
Possibly yes. At times I even thought A matched up with B and couldn't shake that out of my head which needed more time to jar that loose. It's certainly no way to listen to music.
Rothchild
30-10-2013, 22:13
There's an interesting cycle in the numbers though isn't there? It gets significantly worse before it gets better (and then gets worse again). Half way through the test it's at 50/50 (having dragged itself back up from nearly total guess work (at test 8). Indeed it's interesting as much for its symmetry as anything: Test 8 96% chance of guessing, Test 15 50/50, Test 24 7% chance of guessing, Test 32 it's cycling back up towards evens.
If anything it looks more like practice paid off rather than boredom set in, as the run from test 24 to 32 is where you manage to hold the most consistent score.
I'm not a statistician but I am aware that this sort of attempt to interpret is dangerous so I will restrain myself until I've done some more research about how we're supposed to interpret these figures, but I think the point I'm trying to make is that whilst the figure did go down at one point to 7% chance of guess work it did also go as high as 96% chance of guesswork. Nevertheless on the assumption that the test was played fairly (insofar as 35 was decided at the outset and the ongoing results were hidden) I think 36% chance of guesswork represents a significant one in the eye for the doubters.
This is really interesting and did give it a shot on my Mac... Anyhoots, i compared an AIFF RedBook file to the very same file compressed to 320kbps AAC (.m4a)
And, to my surprise, i didnt score enough correct decisions for it to be a relevant result! I am, however, not good at comparisons om the fly.. I know, after a few days if i LIKE what i hear.
I will give it another shot though, upgraded to Mavericks before and that crippled my M2Tech USB2SPDIF adapter so this was done using only the ABX tester, via optical output to my DAC.
(In the long run i prefer Amarra via the M2Tech adapter, AND uncompressed files. Or to be absolutely correct i prefer uncompressed files via UPnP, and no fecking computer at all;))
Regars Mike
Nevertheless on the assumption that the test was played fairly (insofar as 35 was decided at the outset and the ongoing results were hidden) I think 36% chance of guesswork represents a significant one in the eye for the doubters.
To be honest I never intended the test to go that far. I didn't hide the results but I'm not convinced this should make a difference, either the difference is audible and can be judged, or is not and is ruled by random chance alone, and there is no way you can train for a null difference. Whether it would have got worse I'm not sure, I ended it when I got fed up.
The percentages seem a little misleading anyway, 50% should be the statistically perfect representation of random guessing surely. It makes sense in the context it is used I suppose, but may make a lot of people think 50% is significant.
Powered by vBulletin® Version 4.2.3 Copyright © 2026 vBulletin Solutions, Inc. All rights reserved.