Schiit's Jason Stoddard on blind testing

[pkane]

https://www.head-fi.org/threads/sch...bable-start-up.701900/page-3798#post-15486474

 

[Blumlein 88]

Attempt to co-opt blind listening to support sighted listening impressions. The guy is good at telling the stories. Have your cake and eat it too.

 

[majingotan]

To us, it’s level-matched listening to hidden audio components in a low-stress environment. Afterwards, you might pick a favorite, comment on any differences you might hear, or try to identify the product. But there’s no forced choices, no time limit, no pressure…and “there is no difference” is a totally cool answer

This is his fallacy for his argument/hypothesis. Our memory is proven not to be reliable for long term deduction, and having a song that you know will spawn preconceived biases that will skew the results to something imaginative

This is Stoddard's hypothesis in quote:

Stoddard’s Hypothesis

There are small audible differences in audio electronics that cannot be readily explained by measurements, or that exist below the level commonly accepted as inaudible; furthermore, some people can hear these small differences, and some of the people who can hear them may consider them important.

Pretty much saying that our hears hear small differences when comparing amps/DACs that have 120 dB SINAD. He forgot that our brain is not reliable enough to perform blind test according to his methodology: "But there’s no forced choices, no time limit, no pressure"

 

[Grattle]

He seems very open minded and sincere. I especially like the Moffat Corollary.

 

[Grattle]

This is his fallacy for his argument. Our memory is proven not to be reliable for long term deduction, and having a song that you know will spawn preconceived biases that will skew the results to something imaginative

Your specific memory may act that way, but with time, your may find that you would rather listen to Amp A for longer than Amp B with the same music. Maybe the cumulative cognition of differences is a stronger indicator than a smaller window of time.

I'm sitting here now listening to a coaster. I kid you not. I'm listening to a coaster, and I like it - or maybe more accurately, I don't dislike it.

 

[majingotan]

I literally use mine as a coaster as I don't have the parts here in my country to build one. Hopefully, I can make mine functional someday

your may find that you would rather listen to Amp A for longer than Amp B with the same music.

This I don't disagree, but I know consciously that I'm preferring my Saga in Tube mode as opposed to passive mode despite the poor measurements and bass roll-off in tube mode per Amir's review

Maybe the cumulative cognition of differences is a stronger indicator than a smaller window of time.

That's definitely cognitive bias which we want to avoid when performing DBT ABX tests.

 

[pkane]

This is his fallacy for his argument. Our memory is proven not to be reliable for long term deduction, and having a song that you know will spawn preconceived biases that will skew the results to something imaginative

I don't have a problem with someone trying to find differences in a long-term listening session, as long as it's blind. It's harder to get statistically significant results if you are doing a week-long session with each DUT, but assuming someone is spending a few hours with each, level-matched and blind, then all the power to them! If they can produce a 9/10 or 10/10 ABX match or preference score, then there's likely an audible difference.

 

[Ron Texas]

The biggest mystery of the Schiit saga is how the Yaggi, an expensive DAC which measures like a cheap motherboard DAC received glowing recommendations from so many paid reviewers.

 

[Grattle]

The biggest mystery of the Schiit saga is how the Yaggi, and expensive DAC which measures like a cheap motherboard DAC received glowing recommendations from so many paid reviewers.

Can be said of many expensive DACS. By some standards (not mine), the Yaggi isn't expensive. I don't trust very many any paid reviewers at all. I don't trust pure measurements either, but rather the measurements combined with the discussion triggered by the measurements.

I agree with those who say spend the bulk of your budget on the transducers. I've found this to be the case in my own experience and it's not even close.

 

[Grattle]

I don't have a problem with someone trying to find differences in a long-term listening session, as long as it's blind. It's harder to get statistically significant results if you are doing a week-long session with each DUT, but assuming someone is spending a few hours with each, level-matched and blind, then all the power to them! If they can produce a 9/10 or 10/10 ABX match or preference score, then there's likely an audible difference.

I agree that blind listening is necessary, no doubt.

 

[jsrtheta]

I get a little tired of the persistent claim that ABX tests are all done quickly and under great pressure. As Peter Aczel explained years and years ago, this is nonsense. There is no reason it has to be done this way, and often isn't. But that would undermine Stoddard's ability to advance a caricature.

 

[Grattle]

The biggest mystery of the Schiit saga is how the Yaggi, and expensive DAC which measures like a cheap motherboard DAC received glowing recommendations from so many paid reviewers.

It's well liked by many non-reviewers too. I've never heard one, so I'm not qualified to say much about the thing.

 

[Grattle]

I get a little tired of the persistent claim that ABX tests are all done quickly and under great pressure. As Peter Aczel explained years and years ago, this is nonsense. There is no reason it has to be done this way, and often isn't. But that would undermine Stoddard's ability to advance a caricature.

Some people are just never happy. Here the man is saying that his cheaper amps are all you need and that is somehow a bad thing? What is it that you want from him? Schiit sells an amp that measures better than the Drop THX, and for 1/3 of the price, and that's not good enough. Geez, some people are never going to be satisfied. He never said that all abx tests are done quickly and under pressure. He said that for some, they can be stressful. What's wrong with you?

 

[direstraitsfan98]

The biggest mystery of the Schiit saga is how the Yaggi, and expensive DAC which measures like a cheap motherboard DAC received glowing recommendations from so many paid reviewers.

They were paid for their reviews. So they are obligated to give a glowing recommendation. Or else no more free gear and money in the future. What's the mystery?

 

[Sgt. Ear Ache]

Some people are just never happy. Here the man is saying that his cheaper amps are all you need and that is somehow a bad thing? What is it that you want from him? Schiit sells an amp that measures better than the Drop THX, and for 1/3 of the price, and that's not good enough...

which one?

 

[k3nb5t]

I enjoyed the read, and I appreciated his call to civility between the two camps. The introduction of blind listening at a company like Schiit can only be a good thing, both for Schiit and for the consumer base. I don't agree with his hypothesis, but I do respect the approach. I interpreted this as a journey that Schiit has undertaken. Their transition from sighted, to blind testing is an important step. They started with a primitive setup and are increasingly adding complexity and controls. While they didn't start with a fully controlled test in mind, they do seem to be heading in that direction. Perhaps we, the community and customers, can continue to push them

 

[raistlin65]

Some people are just never happy. Here the man is saying that his cheaper amps are all you need and that is somehow a bad thing? What is it that you want from him? Schiit sells an amp that measures better than the Drop THX, and for 1/3 of the price, and that's not good enough. Geez, some people are never going to be satisfied. He never said that all abx tests are done quickly and under pressure. He said that for some, they can be stressful. What's wrong with you?

He doesn't say that. In fact, he goes so far as to say you might not hear much of a difference now, but it's going to matter to you over time:

Moffat’s Corollary

Human hearing seems to be more integrative than differential, so those small differences between components may be magnified over time, and therefore seem larger and more important than during rapid switching.

That leaves plenty of room for his customers to imagine that they might better get that more expensive amp. Then due to expectation bias from brand loyalty--and some cognitive dissonance since someone did spend extra money--they will be convinced of what he said, that their amp sounds even better over time.

Thanks to the fleeting nature of audio memory, I think it's BS. Here's what I would like to test. Take the Magni 3+ and Heresy. Have someone do blind listening (as Schiit defines it) as much as they want in a day with the two of them. Pick their favorite.

Now wait a week without listening to either one. Give them one blind to listen to as long as they want. Record which one it is. Do that again a week later. And again a week later. And so forth. Then evaluate if the listener was accurately able to specify which they were listening to. I bet most people cannot reliably do so.

 

[raistlin65]

I enjoyed the read, and I appreciated his call to civility between the two camps. The introduction of blind listening at a company like Schiit can only be a good thing, both for Schiit and for the consumer base. I don't agree with his hypothesis, but I do respect the approach. I interpreted this as a journey that Schiit has undertaken. Their transition from sighted, to blind testing is an important step. They started with a primitive setup and are increasingly adding complexity and controls. While they didn't start with a fully controlled test in mind, they do seem to be heading in that direction. Perhaps we, the community and customers, can continue to push them

This is not a call to civility. He has purposely created a new term "blind listening" that subjectivists can use to say that they don't believe in ABX testing. And then, even though he claims initial testing only shows minor differences, that better amp or DAC is going to sound better to you over time. We could say the same thing about expensive cables, too. Even though an ABX test will show no difference between inexpensive and expensive cables, that expensive cable will sound better to you once you have used it at home.

Way to sell your product with a new snake oil pitch, Jason.

 

[Sgt. Ear Ache]

He doesn't say that. In fact, he goes so far as to say you might not hear much of a difference now, but it's going to matter to you over time:



That leaves plenty of room for his customers to imagine that they might better get that more expensive amp. Then due to expectation bias from brand loyalty--and some cognitive dissonance since someone did spend extra money--they will be convinced of what he said, that their amp sounds even better over time.

Thanks to the fleeting nature of audio memory, I think it's BS. Here's what I would like to test. Take the Magni 3+ and Heresy. Have someone do blind listening (as Schiit defines it) as much as they want in a day with the two of them. Pick their favorite.

Now wait a week without listening to either one. Give them one blind to listen to as long as they want. Record which one it is. Do that gain a week later. And again a week later. And so forth. Then evaluate if the listener was accurately able to specify which they were listening to. I bet most people cannot reliably do so.

Of course they can't. it's ludicrous.

There's nothing particularly stressful about a blind test. It's a total red herring...more obfuscation and BS from people who know they can't actually hear what they say they can hear.

 

[Matias]

I see this article and their lately shift to "herectic" opamps as a very targeted response to this forum. This is true marketing: selling people what they want.