• WANTED: Happy members who like to discuss audio and other topics related to our interest. Desire to learn and share knowledge of science required. There are many reviews of audio hardware and expert members to help answer your questions. Click here to have your audio equipment measured for free!

Blind test: we have a volunteer!!!

Status
Not open for further replies.

pozz

Слава Україні
Forum Donor
Editor
Joined
May 21, 2019
Messages
4,036
Likes
6,825
Do two level matched, equal duration recordings into an ADC and use foobar's ABX plugin.

Anyone with the facility to do precise level matching should create the files. @KSTR, would you mind doing that? That way a lot of the setup and provenance wouldn't be in question and the files would be publicly available. The units in question could be sent to you. Your nulling method and @pkane's null software can be used to characterize the difference.

Unless of course ADCs like the RME ADI-2 Pro are thought to pollute the signal chain.
 

KSTR

Major Contributor
Joined
Sep 6, 2018
Messages
2,690
Likes
6,013
Location
Berlin, Germany
Unless of course ADCs like the RME ADI-2 Pro are thought to pollute the signal chain.
Yep, this will be the bail out for such a scenario when it fails.
OTOH, I've already reported about this incident, which might suggest a) that the RME is transparent enough (both DAC and ADC) and b) that a R2R-DAC has enough coloration to be ABX'able even with such an indirect scenario.
Basically we'd then be back at a process I've tried to initiate here, a while ago: first confirm that enough people (even if it's only GO) are 100% sure they can detect the difference in sighted comparision with the remote file approach, then try actually nail it down in a ABX, then look for root causes.
 

pozz

Слава Україні
Forum Donor
Editor
Joined
May 21, 2019
Messages
4,036
Likes
6,825
https://www.audiosciencereview.com/...on-tidal-to-test-mqa.22549/page-7#post-747236
So I guess in summary. I'd consider myself an objectivist, but I think many other objectivists expectation-bias themselves into being unable to hear a difference between two given devices because they don't expect that X level of the metric they're testing for would be audible.
An ABX test CAN be used to prove a difference exists. But it cannot be used to prove a negative, that one does not exist.
False equivalence here. Blind ABX can be used to prove either that a difference exists, or that the difference is below threshold for perception (for pitch, loudness, tone, etc., whatever's being tested). The latter case is when you break out the machines to characterize what difference does exist. And then take such data and compare it to published studies about perception thresholds, playback conditions and the like.
 

TK750

Active Member
Joined
Apr 19, 2021
Messages
228
Likes
408
Location
UK
Surely, if we have to go to such great lengths to ensure a legitimate result, the differences must be completely inconsequential, and the characteristics described in the GO review must be wholly imagined (I suppose this is why Amir is willing to gamble $1K). That said, I do enjoy a bit of theater, which is mostly what we have here.

That doesn't really seem like a logical conclusion at all to me, going to great lengths to ensure a legitimate result doesn't really indicate the magnitude of potential differences (or lack thereof). You go to great lengths to ensure that it is exactly the thing you are testing and not any other outside influence/factor that is generating the difference (or lack of).
 

direstraitsfan98

Addicted to Fun and Learning
Forum Donor
Joined
Oct 1, 2018
Messages
826
Likes
1,225
I also had the thought that GO wins here regardless of the outcome. If he passes the test he’ll be a certified golden ear, if he fails, the amount of traffic this drives to his YT channel will be worth the embarrassment.
Isn’t the stuff uploaded to his YT already embarrassing to him [in your guys eyes]?
 

Zensō

Major Contributor
Joined
Mar 11, 2020
Messages
2,740
Likes
6,740
Location
California
Isn’t the stuff uploaded to his YT already embarrassing to him [in your guys eyes]?
His content is similar to much of the subjective review content on YouTube. I would think failing a controlled ABX in front of his subjective audience is another thing though.
 

Hotwetrat

Active Member
Joined
Mar 26, 2021
Messages
231
Likes
132
Location
UK
one can use the SU-9 and M500 in that case :D




Who is going to administer, witness and check validity of the test ?

I secretly hope he can pass the test with scientific valid evidence. That would be good for charity and future discussions on the sound of DACs.

Are comparable filter settings going to be used or are different filters allowed ?

Asking questions I didn't want to ask....

If someone was motivated to 'fudge' a test like this, it's not difficult - and there's plenty of motivation. Call me skeptical AF.

I struggle to trust anyone who knowingly profits/has profited from perpetuating audiophile snake oil, and those subjective youtube reviewers lose a lot of business if nobody wants to hear them talk about air and soundstage and separation and 'cleaner low end' from a £2500 DAC.

*Enter Mr 1 post Audioquest/PS audio emplyoyee*

Where you at Paul.. I know you lurking....
 
Last edited:

maverickronin

Major Contributor
Forum Donor
Joined
Jul 19, 2018
Messages
2,527
Likes
3,308
Location
Midwest, USA
His content is similar to much of the subjective review content on YouTube. I would think failing a controlled ABX in front of his subjective audience is another thing though.

There's always the usual special pleading so there won't be any negative repercussions when he fails.
 

Emlin

Addicted to Fun and Learning
Joined
Jul 8, 2018
Messages
748
Likes
1,047
Yep. Often it's stress. He will claim that the thought of $1000 potentially not going to charity would be so...

Challenging myself. On the other hand, maybe he can do it. I think he can't, but maybe he really is special.

We should together ensure that he can do a fair test in a fair way, without prejudgement. THAT is science.

And if we don't, he'll have a way out, should he need one. and nothing will have been proven.

That would be bad for the cause of this site, and only give succour to the subjectivists.
 
Last edited:

maverickronin

Major Contributor
Forum Donor
Joined
Jul 19, 2018
Messages
2,527
Likes
3,308
Location
Midwest, USA
Yep. Often it's stress. He will claim that the thought of $1000 potentially not going to charity would be so...

Ha!

That's actually a really "good" excuse. It has emotional impact. It paints the testee a a selfless underdog working for the good of others and the tester as a malevolent miser snatching away food and clothing from starving children. Much better than the usual "too stressful" defense.

I was just thinking of the usual backsies on whether or not the rest of the supporting equipment is "resolving" enough to hear the difference between the devices actually under test. The testee "admits" that he made a mistake in agreeing to the other supporting equipment. All the other subjectivists say, "I could have told you that ahead of time". Cognitive dissonance is resolved.
 

Emlin

Addicted to Fun and Learning
Joined
Jul 8, 2018
Messages
748
Likes
1,047
Ha!

That's actually a really "good" excuse. It has emotional impact. It paints the testee a a selfless underdog working for the good of others and the tester as a malevolent miser snatching away food and clothing from starving children. Much better than the usual "too stressful" defense.

I was just thinking of the usual backsies on whether or not the rest of the supporting equipment is "resolving" enough to hear the difference between the devices actually under test. The testee "admits" that he made a mistake in agreeing to the other supporting equipment. All the other subjectivists say, "I could have told you that ahead of time". Cognitive dissonance is resolved.

Maybe, but let's do the test correctly so that any confounding variables are eliminated. Let's do this scientifically, or we are nothing.
 

maverickronin

Major Contributor
Forum Donor
Joined
Jul 19, 2018
Messages
2,527
Likes
3,308
Location
Midwest, USA
Maybe, but let's do the test correctly so that any confounding variables are eliminated. Let's do this scientifically, or we are nothing.

Yep. No point in trying if you aren't going to give it your best.
 
OP
amirm

amirm

Founder/Admin
Staff Member
CFO (Chief Fun Officer)
Joined
Feb 13, 2016
Messages
44,368
Likes
234,386
Location
Seattle Area
Do two level matched, equal duration recordings into an ADC and use foobar's ABX plugin.
Now there is an idea. I could record the output of Magnius and whatever else he said sounds better and have him go through this checkpoint first. Good thinking!

He had so many issues with Magnius sound that he better be able to hear it in such a test.
 
OP
amirm

amirm

Founder/Admin
Staff Member
CFO (Chief Fun Officer)
Joined
Feb 13, 2016
Messages
44,368
Likes
234,386
Location
Seattle Area
Unless of course ADCs like the RME ADI-2 Pro are thought to pollute the signal chain.
He better not complain about that after saying this in his MQA write-up:

"I've included analog recordings of a "Full Decode", outputting from an ifi iDSD Diablo into an RME ADI-2 Pro FS R ADC at 24 bit 768khz. "

I have that ADC and can even encode at that exact rate if he wants.
 
OP
amirm

amirm

Founder/Admin
Staff Member
CFO (Chief Fun Officer)
Joined
Feb 13, 2016
Messages
44,368
Likes
234,386
Location
Seattle Area
Anyone with the facility to do precise level matching should create the files. @KSTR, would you mind doing that? That way a lot of the setup and provenance wouldn't be in question and the files would be publicly available. The units in question could be sent to you. Your nulling method and @pkane's null software can be used to characterize the difference.
I am happy to do that. I can even loan him the RME ADC if he doesn't have one. Agree pkane is best at creating such files. I know I don't have the patience. :)
 
Status
Not open for further replies.
Top Bottom