• WANTED: Happy members who like to discuss audio and other topics related to our interest. Desire to learn and share knowledge of science required. There are many reviews of audio hardware and expert members to help answer your questions. Click here to have your audio equipment measured for free!

Blind test: we have a volunteer!!!

Status
Not open for further replies.

ABall

Senior Member
Forum Donor
Joined
Jan 5, 2020
Messages
329
Likes
159
Location
Reading (UK)
Correct. We never converged on how to do the AB testing physically. The number of ways there could be a "tell" was quite large and if the goal is reduced to just differentiate between two items, the job would be done. That wasn't the spirit or intent of my challenge.

Do you think his proposed testing protocol backs all that was said in that video? How would you take the leap from telling two amps apart for any reason to backing all the subjective commentary in there?

I have passed some incredibly difficult double blind tests. The difference in all those cases was extremely small. So me passing them doesn't mean all the things people say about those two samples are correct.

I am guessing you measured a difference between the 2 devices you correctly identified or was it a test of music bit rate etc?, even so I would be interested in the results, apologies if this has already been documented, I am fairly new here.
 

restorer-john

Grand Contributor
Joined
Mar 1, 2018
Messages
12,580
Likes
38,282
Location
Gold Coast, Queensland, Australia
The only way as I see it to do any form of amplifier blind testing that will stack up and not be shot down by the membership as flawed, is to construct an ABX comparator that uploads the test result data along with video streamed footage in realtime. That way, the test subject can't use analysis software on the samples and the results can be verified.

No, I'm not signing up to build it. :)
 

Chocomel

Active Member
Joined
Feb 18, 2019
Messages
106
Likes
322
Do you think his proposed testing protocol backs all that was said in that video? How would you take the leap from telling two amps apart for any reason to backing all the subjective commentary in there?

Definitely not, it would basically just be about if he can perceive any difference at all but we would not know what the cause of that would be, of course trying the restrict it to the actual performance of the products so we might feel confident in saying it's a difference he heard due to the performance of the products . If he passed he might say it was because of the things he mentioned in the video but we wouldn't be able to confirm that.

It is not a leap i would personally take. It would take a boatload more of research and testing.
 

Blaspheme

Senior Member
Joined
Apr 14, 2021
Messages
461
Likes
515
Definitely not, it would basically just be about if he can perceive any difference at all but we would not know what the cause of that would be, of course trying the restrict it to the actual performance of the products so we might feel confident in saying it's a difference he heard due to the performance of the products . If he passed he might say it was because of the things he mentioned in the video but we wouldn't be able to confirm that.

It is not a leap i would personally take. It would take a boatload more of research and testing.
If all of that is true, then the idea to test the claims via ABX was either flawed from the outset, or not proposed/considered in good faith.

I'll ask the same question again though: if both amps are transparent according to the measurements—barring glitches in the test procedure that lead to unblinding—how does the slightest difference exist?
 

BrEpBrEpBrEpBrEp

Active Member
Joined
May 3, 2021
Messages
201
Likes
245
I'll ask the same question again though: if both amps are transparent according to the measurements—barring glitches in the test procedure that lead to unblinding—how does the slightest difference exist?

Give the below post by @amirm a check, where he explained the issue.

They are actually. I am just not happy with the state of doing a remote physical test. We would really need to build an automated ABX box and send it to you that removes all doubt about the validity of the protocol. This post really set me back on complexity of doing it otherwise:

There you go - remotely setting up an ABX test with no tells would require an automated box. So the issue seems to be possible glitches unblinding, as you said, @Blaspheme .
 
Last edited:

Chocomel

Active Member
Joined
Feb 18, 2019
Messages
106
Likes
322
I'll ask the same question again though: if both amps are transparent according to the measurements—barring glitches in the test procedure that lead to unblinding—how does the slightest difference exist?
If that were actually the case it would indicate that the measurements that were done are not extensive enough, it could also be that the threshold(s) that's used for transparency is too lenient.
 

Blumlein 88

Grand Contributor
Forum Donor
Joined
Feb 23, 2016
Messages
20,524
Likes
37,057
Any thoughts on using a series Amp test? For whatever reason they seem much better at hearing differences in my experience. Put an Amp in and out of circuit to see if you can hear it.
 

Blaspheme

Senior Member
Joined
Apr 14, 2021
Messages
461
Likes
515
If that were actually the case it would indicate that the measurements that were done are not extensive enough, it could also be that the threshold that's used for transparency is to lenient.
I agree, so is that actually the case, and are the measurements insufficient? We need a test ...
 

markus

Addicted to Fun and Learning
Joined
Sep 8, 2019
Messages
646
Likes
653
Sure, which ones you want to pic? Here is my original post in the other thread:



The "I can tell there is slightest difference between the two" which is what he wants to show now was not part of the equation.

But if he fails it would dismiss ALL of his subjective claims. If he doesn't fail we have learned something. Isn't that how science works?
 
Last edited:

restorer-john

Grand Contributor
Joined
Mar 1, 2018
Messages
12,580
Likes
38,282
Location
Gold Coast, Queensland, Australia
Any thoughts on using a series Amp test? For whatever reason they seem much better at hearing differences in my experience. Put an Amp in and out of circuit to see if you can hear it.

The giveaway with series is residual noise increase.
 

MDT

Member
Joined
Apr 13, 2021
Messages
63
Likes
216
But if he fails it would dismiss ALL of his subjective claims. If he doesn't we have learned something. Isn't that how science works?

Correct. Step one is to prove that a difference can be detected. Step two is to identify why, assuming step one is passed.

The boring result is if step one is failed. There should be no audible difference between two transparent amps. So the interesting result is if he passes the test, because then we get to learn something new.

Unfortunately there seems to be a bit too much ego floating around that some posters are losing sight of the interesting possibilities in this endeavour.
 

Blaspheme

Senior Member
Joined
Apr 14, 2021
Messages
461
Likes
515
Correct. Step one is to prove that a difference can be detected. Step two is to identify why, assuming step one is passed.

The boring result is if step one is failed. There should be no audible difference between two transparent amps. So the interesting result is if he passes the test, because then we get to learn something new.

Unfortunately there seems to be a bit too much ego floating around that some posters are losing sight of the interesting possibilities in this endeavour.
Regarding the steps, exactly. Regarding the interesting result (or not) yes that's what I was waiting for. Regarding ego, also yes—see also 'animus' mentioned previously—but I was preferring to be wong there.
 

Iving

Member
Joined
Nov 18, 2019
Messages
31
Likes
95
This thread is an object lesson in how *not* to approach a problem armed with science.

A research question has to be formulated. It has to be precise and specific, such that the experiment tests *only* the (relationship between the) variables under scrutiny.

The logic and thinking behind development of the research question has to be ultra-disciplined - the opposite of what is happening here.

You don't start with a social bluff, adjusting your position as you play to an evolving public gallery. This particular process is a charade. A public spectacle. It's not doing anyone any favours.

The whole experimental environment is protected in science - such that interpretation of results is not compromised. The opposite is happening here.

There was never a properly-conducted blind test in prospect. It is right to recognise that, even if a good one had been designed (and protected from undue influence), it would not address all the things a particular party is vexed about in Golden's review (assuming that was what this was all about in the first place). Chagrin is not a good basis for scientific endeavour.

Good luck resolving your differences. You will not do so on this horizon. You can't (and shouldn't be able to) police subjective audio experiences on the internet. And the debate regarding relationship of equipment measurements (and brain phenomena such as expectation bias) to SQ is far from finished.

It won't be completed in this thread. Nor any that doesn't adopt a fresh approach. That fresh aporoach requires open-mindedness, dignified conduct - and *utter* methodological rigour.

Why doesn't the audiophile community deserve that?

No doubt the dogmatism and the antagonism here will continue ad nauseum.

Instead of pearl clutching all round, why not just admit it's really about popcorn and site stats?

My serious point, in case anyone wonders, is that we could do a lot better to improve our audio understandings, and that it might be worth it.
 
Last edited:

Blaspheme

Senior Member
Joined
Apr 14, 2021
Messages
461
Likes
515
Guys, I am getting royally confused... Which of the two bloggers is 'our' blogger? (Whose blogger is Amir? I want to be on the winning side!)
GO is a vlogger, surely.
 

Thomas savage

Grand Contributor
The Watchman
Forum Donor
Joined
Feb 24, 2016
Messages
10,260
Likes
16,298
Location
uk, taunton
I predict lots of chat and enough word salad so both sides can claim the other impossible to collaborate with and no actual useful test .

Some bodge test might happen where OP claims victory and ASR claims foul . Likewise we might ' go our own way ' and do a test and those roles get reversed.

This is very reminiscent of boxing fight negotiations that also play out on social media .

If there were a Olympics for passive aggressive online posting this thread and the MQA one would serve as the gold standard.

Of course I might be wrong..,
 
Last edited:

BrEpBrEpBrEpBrEp

Active Member
Joined
May 3, 2021
Messages
201
Likes
245
I'd agree that this has turned into essentially a farce. I'm still interested in those digital captures anyways - to show that there's no perceptible difference between 2 transparent amps captured on the same ADC.

And if people "don't trust" DACs and ADCs... I guess they should probably listen to only Vinyl cut from analog masters.
 

BrEpBrEpBrEpBrEp

Active Member
Joined
May 3, 2021
Messages
201
Likes
245
I predict lots of chat and enough word salad so both sides can claim the other impossible to collaborate with and no actual useful test .

Some bodge test might happen where OP claims victory and ASR claims foul .

This is very reminiscent of boxing fight negotiations that also play out on social media .

If there were a Olympics for passive aggressive online posting this thread and the MQA would serve the gold standard.

Of course I might be wrong..,

Agreed. This was entertaining for the first couple days, but it's getting old real fast.

Besides, I'm not sure what the point even is in the end. Similar tests have been done, and the results are as boring as they are predictable.
 

Thomas savage

Grand Contributor
The Watchman
Forum Donor
Joined
Feb 24, 2016
Messages
10,260
Likes
16,298
Location
uk, taunton
Agreed. This was entertaining for the first couple days, but it's getting old real fast.
Trust me when I say it's definitely not entertaining for a those that have spent years helping build ASR , formative members and retired mods alike . This thread and the MQA thread are a sad state of affairs.

If something like this is to stand a chance first thing would be to talk in private offline and agree on test aims and protocols , to then set that in a email and both publish it on respected platforms independently. Then you have solid ground to open a thread like this , with a shared statement to be ' peer reviewed ' by the membership .

This is just a circus, both sides thinking they hold the hill. Our vlogger friend can afford such game we need to be better imo .
 
Status
Not open for further replies.
Top Bottom