Thought I'd start a new thread about this. For quite some time I've been intrigued by blind testing, and whether that's the end all be all when it comes to audible differences in audio (or preferences). A couple of years ago, when I was still struggling in the passive quagmire, this led me to sell off a very expensive amplifier, and buy a very cheap one. After all, there shouldn't be audible differences between amplifiers according to ABX-tests, right?
But somehow, the speakers didn't sound as right with the new and cheap amplifier as with the more expensive one. I was never sure whether to trust my own hearing on that, or assume that it was just a placebophile illusion. Then I got active monitors, and that rendered the whole discussion moot for the time, for me at least.
But now I've been thinking about it again, as I might change my system, and proably will buy some new electronics. Should one trust blind tests blindly - or not? The subjectivists, of course, have long been skeptical of blind tests because it invalidates basically all of their beliefs. And I guess that almost all of us on this forum share the belief that sighted listening can be unreliable. For me, that's something that goes without saying. I will never trust my own sighted listening without hesitation - neither do I trust the sighted listenings of others, unless it can be backed up by measurements and/or reasonable theories about what happens that don't invalidate any fundamental physical laws. But even though sighted listening is unreliable, does that mean that blind listening necessary is completely reliable? Lately I've been inclined to say no.
----------------------
The subjectivist criticism of blind testing has been that it is "unnatural", and is different from "normal" listening, and that this masks real and objective differences. Another line of criticism has been that the brain might confuse things - that the brain mixes everything into a diffuse sensory soup when trying to compare listening impressions analytically. This is the view of Siegfried Linkwitz, for example, and the blogger The Rational Audiophile. I would say that both of these theories are possible. But until recently, I wasn't aware of any systematic data to back this up. However, I recently came across this peer-reviewed article from 2013, by two French researchers:
https://hal-institut-mines-telecom....ile/index/docid/842647/filename/APAC_5172.pdf
In that article, they do an experiment where they compare different procedures for blind testing preferences about loudspeakers. One procedure is the most common one in ABX tests, in which short excerpts are played after one another. Procedure two is closer to normal listening - where long excerpts are played, and the listeners can switch themselves. Here, however, the volume was fixed and matched. In the third procedure, the listeners listened to long excerpts, AND could adjust the volume themselves. This procedure was deemed to be the closest to normal listening.
The very intriguing finding was that procedure 3 turned out to be the most discriminating. Under this procedure, there were statistically significant differences in ratings between loudspeakers that weren't there under procedure 1 and 2. This means that loudspeakers that it was not possible to discriminate between quality wise under the ordinary testing procedure, actually were perceived as different when the test resembled "normal" listening. The researchers also explain that this was not because of any loudness mismatch - the test subjects didn't use the volume button that much, even though they could. It was the mere possibility of changing the volume that made them able to discriminate more finely between the loudspeakers.
That raises the question: Are there other kinds of objective differences that may be heard under normal listening, but that disappear under some blind test procedures?
-----------------------
What's my take-away? Ok, assuming that this study is valid, of course - hopefully it will be replicated. I think blind tests are valuable when it comes to identifying the differences that are most obvious, and which might be the most important when listening. If a difference is objectively minimal, and never shows up on a ABX test, it's also an indication that it might no be that important. Still: Increasingly I'm taking a positive ABX result as a confirmation, but I don't necessarily take a negative ABX result as a disconfirmation.
So I no longer take for granted that an objective difference that has yet to be ABXed is inaudible. I am even more concerned with measurements and objective differences, and less so with the results of ABX tests which explore audibility. I also trust my subjective listening a tiny bit more than before, even though I still try to be aware of my biases.
I have therefore, for example, gone back to streaming from Tidal in hifi/CD quality, even though most ABX tests show that few people can ABX lossless from 320-bit. In case I will ever need to drive a speaker system with external amps again (with active crossovers of course), I think I'll probably get amps that measure really well (like Hypex), and not settle for the cheapest things I find in a garage. In a way I'm endulging myself with some audiophilia nevrosa.
-----------------------
Any comments or thoughts? I'd love input on this - I'm not very entrenched in my views on this matter.
But somehow, the speakers didn't sound as right with the new and cheap amplifier as with the more expensive one. I was never sure whether to trust my own hearing on that, or assume that it was just a placebophile illusion. Then I got active monitors, and that rendered the whole discussion moot for the time, for me at least.
But now I've been thinking about it again, as I might change my system, and proably will buy some new electronics. Should one trust blind tests blindly - or not? The subjectivists, of course, have long been skeptical of blind tests because it invalidates basically all of their beliefs. And I guess that almost all of us on this forum share the belief that sighted listening can be unreliable. For me, that's something that goes without saying. I will never trust my own sighted listening without hesitation - neither do I trust the sighted listenings of others, unless it can be backed up by measurements and/or reasonable theories about what happens that don't invalidate any fundamental physical laws. But even though sighted listening is unreliable, does that mean that blind listening necessary is completely reliable? Lately I've been inclined to say no.
----------------------
The subjectivist criticism of blind testing has been that it is "unnatural", and is different from "normal" listening, and that this masks real and objective differences. Another line of criticism has been that the brain might confuse things - that the brain mixes everything into a diffuse sensory soup when trying to compare listening impressions analytically. This is the view of Siegfried Linkwitz, for example, and the blogger The Rational Audiophile. I would say that both of these theories are possible. But until recently, I wasn't aware of any systematic data to back this up. However, I recently came across this peer-reviewed article from 2013, by two French researchers:
https://hal-institut-mines-telecom....ile/index/docid/842647/filename/APAC_5172.pdf
In that article, they do an experiment where they compare different procedures for blind testing preferences about loudspeakers. One procedure is the most common one in ABX tests, in which short excerpts are played after one another. Procedure two is closer to normal listening - where long excerpts are played, and the listeners can switch themselves. Here, however, the volume was fixed and matched. In the third procedure, the listeners listened to long excerpts, AND could adjust the volume themselves. This procedure was deemed to be the closest to normal listening.
The very intriguing finding was that procedure 3 turned out to be the most discriminating. Under this procedure, there were statistically significant differences in ratings between loudspeakers that weren't there under procedure 1 and 2. This means that loudspeakers that it was not possible to discriminate between quality wise under the ordinary testing procedure, actually were perceived as different when the test resembled "normal" listening. The researchers also explain that this was not because of any loudness mismatch - the test subjects didn't use the volume button that much, even though they could. It was the mere possibility of changing the volume that made them able to discriminate more finely between the loudspeakers.
That raises the question: Are there other kinds of objective differences that may be heard under normal listening, but that disappear under some blind test procedures?
-----------------------
What's my take-away? Ok, assuming that this study is valid, of course - hopefully it will be replicated. I think blind tests are valuable when it comes to identifying the differences that are most obvious, and which might be the most important when listening. If a difference is objectively minimal, and never shows up on a ABX test, it's also an indication that it might no be that important. Still: Increasingly I'm taking a positive ABX result as a confirmation, but I don't necessarily take a negative ABX result as a disconfirmation.
So I no longer take for granted that an objective difference that has yet to be ABXed is inaudible. I am even more concerned with measurements and objective differences, and less so with the results of ABX tests which explore audibility. I also trust my subjective listening a tiny bit more than before, even though I still try to be aware of my biases.
I have therefore, for example, gone back to streaming from Tidal in hifi/CD quality, even though most ABX tests show that few people can ABX lossless from 320-bit. In case I will ever need to drive a speaker system with external amps again (with active crossovers of course), I think I'll probably get amps that measure really well (like Hypex), and not settle for the cheapest things I find in a garage. In a way I'm endulging myself with some audiophilia nevrosa.
-----------------------
Any comments or thoughts? I'd love input on this - I'm not very entrenched in my views on this matter.
Last edited: