Schiit's Jason Stoddard on blind testing

[gikigill]

I lived in California from 1962 until 2012. No basements in any of my residences.

I have lived in Panama since 2012. No basements.

I feel deprived having to use my computer while gazing out at the high-altitude tropical rainforest.

View attachment 52119

Thats one depravity i,ll gladly join

 

[Grattle]

The biggest mystery of the Schiit saga is how the Yaggi, an expensive DAC which measures like a cheap motherboard DAC received glowing recommendations from so many paid reviewers.

Have you listened to one? I have not, therefore anything I say about it is meaningless. You make the inference....

 

[Ron Texas]

Have you listened to one? I have not, therefore anything I say about it is meaningless. You make the inference....

I don't know what inference you mean, but I have never even seen one. It measures terribly but "professional" reviewers love it. Someone please explain it to me.

 

[Grattle]

I don't know what inference you mean, but I have never even seen one. It measures terribly but "professional" reviewers love it. Someone please explain it to me.

You say it's no better than motherboard audio. You haven't heard one. You say it measures terribly. How does it sound to your ears and your brain? That's what matters. Most of the things that are slammed here for "bad measurements" have no audible issues. Any measurements issues beyond audibility are meaningless in a listening context. Everyone here seems fixated on things that they can't hear, to a fault.

 

[Ron Texas]

You say it's no better than motherboard audio. You haven't heard one. You say it measures terribly. How does it sound to your ears and your brain? That's what matters. Most of the things that are slammed here for "bad measurements" have no audible issues. Any measurements issues beyond audibility are meaningless in a listening context. Everyone here seems fixated on things that they can't hear, to a fault.

The Yaggi measures so badly it is above the known threshold for audible distortion. It's not necessary to audition one to make that statement. I never said that I heard it, only that it measures no better than motherboard audio. Actually, it's quite a bit worse. Stop inferring that I am responsible for all the problems of the world.

Have you auditioned a Yaggi? I hope you didn't waste your money on one.

 

[direstraitsfan98]

I had a Schiit Yggdrasil for many years in my reference system of a Pass Labs amp and pre, and Harbeth 30.2. At one point I decided to do a comparison to a babyface RME and Topping D30. I tried to blind AB test as well as control the volume levels but it was very difficult because no matter what we did the Yggy always sounded a little louder. I attribute the Yggdrasil sounding “better” solely to this simple reason. As for distortion, I heard nothing from either DAC. All of them sounded excellent. I do however think the Yggy looks more handsome on an audio rack. It’s big, it’s beefy, and makes you feel better looking at it then some dinky little thing that you hide on your pc desk.

I sold it about 2 years after I got it because everyone of my peers made fun of me for owning it. I couldn’t even show off my system without someone bullying me about owning it. I found the whole experience bizarre considering these same people liked to listen to bass heavy EDM on their headphones and small bookshelf monitors and these monitors no doubt measured double digits in distortion at times. Oh well. Every community has nasty people.

 

[Grattle]

The Yaggi measures so badly it is above the known threshold for audible distortion. It's not necessary to audition one to make that statement. I never said that I heard it, only that it measures no better than motherboard audio. Actually, it's quite a bit worse. Stop inferring that I am responsible for all the problems of the world.

Have you auditioned a Yaggi? I hope you didn't waste your money on one.

Please show me the audible distortion.

 

[Ron Texas]

Please show me the audible distortion.

Please show me a little civility. Nobody likes to be badgered.

 

[Grattle]

I had a Schiit Yggdrasil for many years in my reference system of a Pass Labs amp and pre, and Harbeth 30.2. At one point I decided to do a comparison to a babyface RME and Topping D30. I tried to blind AB test as well as control the volume levels but it was very difficult because no matter what we did the Yggy always sounded a little louder. I attribute the Yggdrasil sounding “better” solely to this simple reason. As for distortion, I heard nothing from either DAC. All of them sounded excellent. I do however think the Yggy looks more handsome on an audio rack. It’s big, it’s beefy, and makes you feel better looking at it then some dinky little thing that you hide on your pc desk.

I sold it about 2 years after I got it because everyone of my peers made fun of me for owning it. I couldn’t even show off my system without someone bullying me about owning it. I found the whole experience bizarre considering these same people liked to listen to bass heavy EDM on their headphones and small bookshelf monitors and these monitors no doubt measured double digits in distortion at times. Oh well. Every community has nasty people.

I'm not being nasty. I don't understand the Schiit hate here. Especially when people are just looking at the graphs. I don't agree with the group think that happens here. I know that the Schiit Heresy measures better than the Magni 3+. I prefer the sound from the Magni 3+. I prefer the sound of my DIY OPA1622EVM even more than the Magni 3+. All three sound great for the price though. I also LOVE my Modi Multibit - EVEN though it doesn't measure up to ASR standards. I hear no distortion. It's the best sounding DAC that I've heard. I just bought a 2nd one for my office. I'm not going to handcuff myself because of some graphs that some rando puts up on his site. I'll judge it based on my ears, and I'll support Mike Moffat's attempts at pushing the envelope. Doing the same old things over and over doesn't advance anything. At least Schiit is exploring and creating and learning. I just don't get the hate, and the denigration of the company and people who buy their products because some graph says that it has more inaudible noise than some other product from a manufacturer who kisses my ass. ASR claims to be so scientific and open minded, but are anything but. Inaudible is inaudible.

 

[pkane]

I'm not being nasty. I don't understand the Schiit hate here. Especially when people are just looking at the graphs. I don't agree with the group think that happens here. I know that the Schiit Heresy measures better than the Magni 3+. I prefer the sound from the Magni 3+. I prefer the sound of my DIY OPA1622EVM even more than the Magni 3+. All three sound great for the price though. I also LOVE my Modi Multibit - EVEN though it doesn't measure up to ASR standards. I hear no distortion. It's the best sounding DAC that I've heard. I just bought a 2nd one for my office. I'm not going to handcuff myself because of some graphs that some rando puts up on his site. I'll judge it based on my ears, and I'll support Mike Moffat's attempts at pushing the envelope. Doing the same old things over and over doesn't advance anything. At least Schiit is exploring and creating and learning. I just don't get the hate, and the denigration of the company and people who buy their products because some graph says that it has more inaudible noise than some other product from a manufacturer who kisses my ass. ASR claims to be so scientific and open minded, but are anything but. Inaudible is inaudible.

You're a few years too late to the fight

Schiit picked up their game when they were shown how poorly their devices measured by ASR. Since then, they invested in measurement equipment and are doing a good job of producing decent, often inexpensive, and well measuring devices. Good engineering is not a bad thing and is to be encouraged. Why argue about it?

 

[Grattle]

Please show me a little civility. Nobody likes to be badgered.

I said please.

 

[Grattle]

You're a few years too late to the fight

Schiit picked up their game when they were shown how poorly their devices measured by ASR. Since then, they invested in measurement equipment and are doing a good job of producing decent, often inexpensive, and well measuring devices. Good engineering is not a bad thing and is to be encouraged. Why argue about it?

I disagree. They aren't stupid though, and are making products for those who do prefer the numbers. I don't care if you buy something based on it's measurements. If having great measuring gear makes you happy, then great. Everyone wins. I prefer to judge with my ears, it's way more fun for me that way. To each his own. Live and let live, yadda yadda yadda. I'm not going to call a group of strangers foolish for doing what they want with their own money though, and I don't think the folks here should either. The snarkiness here is what I'm trying to address.

 

[pkane]

I disagree. They aren't stupid though, and are making products for those who do prefer the numbers. I don't care if you buy something based on it's measurements. If having great measuring gear makes you happy, then great. Everyone wins. I prefer to judge with my ears, it's way more fun for me that way. To each his own. Live and let live, yadda yadda yadda. I'm not going to call a group of strangers foolish for doing what they want with their own money though, and I don't think the folks here should either. The snarkiness here is what I'm trying to address.

You are addressing snarkiness and I'm addressing proper engineering. We're on different wavelengths.

 

[Inner Space]

... I don't understand the Schiit hate here. Especially when people are just looking at the graphs. I don't agree with the group think that happens here ... I'm not going to handcuff myself because of some graphs that some rando puts up on his site. I'll judge it based on my ears, and I'll support Mike Moffat's attempts at pushing the envelope. Doing the same old things over and over doesn't advance anything. At least Schiit is exploring and creating and learning. I just don't get the hate, and the denigration of the company and people who buy their products because some graph says that it has more inaudible noise than some other product from a manufacturer who kisses my ass. ASR claims to be so scientific and open minded, but are anything but. Inaudible is inaudible.

Interesting to observe, in a wider context, how Schiit (and PS Audio) developed (or fell into) a wholly new marketing concept of first co-opting and then steering internet forum passions. Both sellers have created loyal on-line fanbases, who buy both the product and the BS, and who energetically campaign elsewhere on the sellers' behalf. Looks like @Grattle, for one, has bought into the model completely. Well done, Schiit.

 

[pozz]

Have you guys heard about “slow listening” theory by Genelec researchers? It tries to explain why we can hear some differences in short listening tests, but for others longer listening sessions are needed. Here’s their AIS paper, would be great to know what Jason thinks about it.

The paper really doesn't explain anything of the sort. It needs to be read carefully. Foremost it's a conceptual piece and manifesto-ish, and provides no direct experimental evidence. The way it presents long and short listening session is also not intuitive, nor does it say that all these sessions let one hear all kinds of heard differences equally.

I highlight this passage: "slow listening could take as long as it would take for the subject to learn a new language, maybe more." I will come back to this claim later. For now, let's add some substance to it. Earlier on, there's a sort of industry criticism:

The claim around "anyone interested in sound today" is weak, or at least poorly written. I could generally agree, allowing a lot of slack, since the cognitive aspects of hearing are well-known, as is the increasing standard for "life-like sound" when looking at historical commentary on various recording and reproduction technologies (a little technological leap gets people saying that the music is there with them in the room or the recording has come to life, and later on, after the enthusiasm fades, deficiencies are better appreciated). That's not the same thing as claiming that psychoacoustic encoding is ineffective, or that older codecs are worse than newer ones. This is where the paper should have quoted or looked for research on this issue specifically. It's too bad it didn't.

Regardless, let's take the mention of the 400ms grey zone mention at the end of that passage and find a more detailed description:

"Phoneme discrimination" comes from linguistics. A phoneme is a unit of uttered language, kind of like a syllable, but related to speaking and hearing rather than written or grammatical language. The difficulty described is like learning to recognize the tonal components of languages like Vietnamese, or to hear the differences in pronunciation of various vowels across regions and accents. One of the standard fields in linguistics is the physiology of language and which parts of the mouth, tongue, throat and nose are used during speech (this research was used to establish minimum standards for intelligibility in communications systems in the 1940s and earlier by Fletcher, e.g., the frequency content of speech). This then used to represent acoustic differences using notation like IPA and technical vocabulary.

Cognitive linguistics steps a bit further back and notes that acoustic differences of vocalization are often not enough to help listeners recognize and pick out phonemes (from the linked paper: "Listeners, who are misinformed about a speaker’s (socio-)linguistic background, are more inclined to perceive the incoming stimulus according to their sociolinguistic expectations than to the acoustic characteristics of the stimulus."). Listeners often need context, like what to listen for, or where the speaker is coming from. It's like trying to understand the accent, grammar and vocabulary of Jamaican or Scottish English if you hail from elsewhere. It's not easy, but after a while in the country it becomes second nature. Same goes for sound and music.

Let's come back to this statement: "Slow listening could take as long as it would take for the subject to learn a new language, maybe more." So, all in all, the paper's emphasis is on learning, the idea that certain perceptions may not accessible to a listener immediately, but may be easily recognizable later. That's straight out of psychology (note that psychoacoustics is considered a branch of psychology, not its own field), and well-designed experiments record not only subject responses but subject responses over long periods. (My favorite study on memory, by Luria, took place over 30 years!) Note that the timescale is not defined beyond making this general claim.

As such the paper really doesn't focus on long listening sessions per se, but on what it has taken, historically speaking, to recognize what are now known as commonplace problems in audio. There is really no basis for concluding that the long term review and impressions-type publications are in the right, or have any validity beyond the accidental or circumstantial. That this paper is used to defend those kinds of uncontrolled listening comparisons is simple misreading.

This is the paper's conclusion:

The final sentence is the key. It says: don't take shortcuts in recording, reproduction or manufacturing technologies based on a simple idea of psychoacoustics, like accepting lenient standards for lossy compression or distortion or loudspeaker design. The implication that there are potential issues and differences between gear that we are not completely aware of is a pretty fair conclusion. But note that it does not support or anywhere say that those who are claiming to hear differences are in the right. All it says is not to take the easy way out when it is possible to do better, especially if the current research does not have all the answers about what is acceptable or what isn't. Note this, for example:

Seems to be pretty clear cut. Using new knowledge, rigorously test existing industry standards and see if they hold up to the science. If you have to use short listening tests, make sure that:

Which means that, as a manufacturer or researcher, you can't conclude that your listeners' reports are reliable until you take their frame of reference and abilities into account, and how you might bias the results by having too narrow a focus when designing the experiment.

The paper is mostly a demand for better mindfulness and higher quality research from an industry that tends to value extremely specialized knowledge and an economic small-minded sort of practicality. An engineer is more likely to be able to quote you Newton's laws of motion than to have read any of his writings (@andreasmaaan this is what I meant before when I said that textbooks don't help knowledge—most rip out the idea, formula, fact from the context of its invention, and present it as is, without any acknowledgement of what it took to come upon it—sorry I didn't reply before).

 

[pozz]

Oh, and to be clear, I think the long post by Jason of Schiit, quoting myself, is an example of an "economic small-minded sort of practicality". It opportunistic to take a cursory look at the science and dive headstrongly to a self-serving conclusion. Like others have said, it's talented marketing. Rather than being inclusive, it reinforces an older position with new vocabulary.

The difference between subjective and objective is not a free spirit of inquiry vs. stiff bureaucratic study, and the merger of the two is not the random comparison of listening impressions to measurements or partially-related studies. A lot of work has to be done to make listening impressions and personal experience matter and to learn to interpret the data, i.e., you have to know when to step back and shut your piehole.

The really radical thing is... taking the time to read the existing literature. Slowly.

 

[direstraitsfan98]

I'm not being nasty. I don't understand the Schiit hate here. Especially when people are just looking at the graphs. I don't agree with the group think that happens here. I know that the Schiit Heresy measures better than the Magni 3+. I prefer the sound from the Magni 3+. I prefer the sound of my DIY OPA1622EVM even more than the Magni 3+. All three sound great for the price though. I also LOVE my Modi Multibit - EVEN though it doesn't measure up to ASR standards. I hear no distortion. It's the best sounding DAC that I've heard. I just bought a 2nd one for my office. I'm not going to handcuff myself because of some graphs that some rando puts up on his site. I'll judge it based on my ears, and I'll support Mike Moffat's attempts at pushing the envelope. Doing the same old things over and over doesn't advance anything. At least Schiit is exploring and creating and learning. I just don't get the hate, and the denigration of the company and people who buy their products because some graph says that it has more inaudible noise than some other product from a manufacturer who kisses my ass. ASR claims to be so scientific and open minded, but are anything but. Inaudible is inaudible.

By the way I wasn’t talking about you or anyone here. I was referring to the people from before who were bullying me for owning the Yggy. And the ridicule didn’t start until after Amir published his measurement. Go figure.

 

[raif71]

You're a few years too late to the fight

Schiit picked up their game when they were shown how poorly their devices measured by ASR. Since then, they invested in measurement equipment and are doing a good job of producing decent, often inexpensive, and well measuring devices. Good engineering is not a bad thing and is to be encouraged. Why argue about it?

I'm one of the people who's late to the fight. Wonder how I'd missed this thread coz it's really relevant at least to me. Enjoyed reading the posts here.

 

[amirm]

ASR claims to be so scientific and open minded, but are anything but. Inaudible is inaudible.

You can't waive your hands and declare some artifact as inaudible. You need to prove that. I tested the Yggy at low amplitude and then turned the volume up. There was definitely audible artifacts but obviously you would not hear them unless you set your reference to very high playback level. so no, it is not guaranteed inaudible. It is only so in "typical" usage scenario. If you want it to be provably inaudible, you need lower distortion than what I measured.

And no, we are not plain "open minded." Don't know where that comes from. No astrophysicist is open minded about the earth being flat.

 

[Wombat]

Some science contrarians expect others to be so open-minded that there is a risk of brain spill.