Review and Measurements of Schiit Jotunheim and iFi iDSD Black Label DACs and Headphone Amps

[Blumlein 88]

Didn't mean to poop in your measurements church, just providing some science since, you know, that's in the name of this web site.

You have provided nothing of the sort.

No one here is assuming measurements are perfectly conducted. No one is thinking the measuring gear is perfect. In fact there is much discussion about the ins and outs, pitfalls and gotchas involving how measurement is done. Ditto about how it relates or fails to relate to what is heard. That is why your entire previous post is of no use of any kind. Nothing about it is a reasonable description of measuring, its uses, or how it is used or considered. It is devoid of any informational content. The only content is the straw man you are making of such a pursuit in order to change the discussion from measurements I'm guessing you don't like. You aren't believable as thread pooping is exactly what you were doing and you know it.

 

[derp1n]

Didn't mean to poop in your measurements church, just providing some science since, you know, that's in the name of this web site.

Did you get day release from SBAF?

 

[amirm]

Thus I think measurements are just as likely to induce "crap"rather than cut it.

Is that what you think of your car's speedometer? You prefer to use a wet thumb in the air for how fast you are going?

 

[Thomas savage]

Of course that opinion assumes that measurements are perfectly conducted, by perfect equipment, interpreted perfectly, and perfectly translate to audible differences uniquely and specifically to our individual ears ... which, of course, is the entire purpose of the products being measured: good audio to our individual ears.

Thus I think measurements are just as likely to induce "crap"rather than cut it.

Belief in the perfection of measurements is, therefore, religious, not scientific.

Err, ok ..

All these measurements are just data points for folks to consider. A objective appraisal, the value people put in them is entirely up to them.

I’d look for other areas in your life to excise your rebellion as what you have posted here is pure fantasy and no one here is buying.

 

[Thomas savage]

Is that what you think of your car's speedometer? You prefer to use a wet thumb in the air for how fast you are going?

No need, the scientific thing to do is just feeeeeel how fast your going . Speedometers are redundant not that I’d know as I’m banned from driving coz the unscientific buggers with the laser gun said I was going too fast.

I told the judge it felt like the correct speed but no dice.

Heretics.

 

[amirm]

 

[Gruss Gott]

Is that what you think of your car's speedometer?

Great example Amir! Let's discuss. With a speedometer:

(1.) We've all agreed that the objective measure - velocity in m/h - has a value which uniquely, specifically, and, therefore, definitely determines legality.
(2.) By convention, we've also agreed to "slop" in both our vehicle's measuring accuracy (speedometer) and the accuracy of police tools.
(3.) However my speedometer's value is not predictive of whether I'm going to get a ticket, i.e,, have a bad outcome, even though I might look at the measure and know that it's "bad".

So what's the scientific key there? The defined link between the objective measure and the outcome: is it predictive of an outcome? Our speedometers aren't (I'll skip the relativity point where we can't prove if the speedometer is measuring my speed or yours). For example your speedometer might measure "good" (under the limit) and mine might measure "bad", but neither of us get a ticket and I'm also having more fun and getting more done than you - certainly I'm faster. So that speedometer wasn't predictive of anything. Doesn't mean it's useless information, though, just not predictive by itself (as pointed out in your cartoon).

Now let's translate that to audio:

(a.) We'd have to agree on a set of measures and values that uniquely, specifically, and definitively predict an audio outcome; every single time for every device, for every single person, and

(b.) We'd have to scientifically conduct those measurements ... e.g., one person does the set up, one person validates, three people take the measures, another validates, etc ... and then we'd have to do that roughly 10-20 times per measure.

If we had those things (a & b), then we'd be scientifically predictive, but since we don't, we're not. Which means we haven't cut any crap ... at least not definitely.

That's not to say it's not valuable data - I've defended you many times Amir, and will continue to partially because I like you but also because I think you do good work. (so the sbaf hater up there can stick it up his butt)

That said, measurements don't "cut the crap" because they're not very predictive of an outcome any more than my speedometer is - I speed all day every day and haven't gotten a ticket in 6 years, so while my speedometer provides a value it's not reliably predictive of a "good" or "bad" outcome.

QED.

 

[Frank Dernie]

Great example Amir! Let's discuss. See, with a speedometer:

(1.) We've all agreed that the objective measure - velocity in m/h - has a value which uniquely, specifically, and, therefore, definitely determines legality.
(2.) By convention, we've also agreed to "slop" in both our vehicle's measuring accuracy (speedometer) and the accuracy of police tools.
(3.) However my speedometer's value is not predictive of whether I'm going to get a ticket, i.e,, have a bad outcome, even though I might look at the measure and know that it's "bad".

So what's the scientific key there? The defined link between the objective measure and the outcome: it's predictive of an outcome - and our speedometers aren't. Yours might measure "good" and mine might measure "bad", but neither of us get a ticket and I'm also having more fun and getting more done that you - certainly faster. So that speedometer wasn't predictive of anything.

Now let's translate that to audio:

(a.) We'd have to agree on a set of agreed measures and values that uniquely, specifically, and definitively predict an audio outcome; every single time for every device, and

(b.) We'd have to scientifically conduct those measurements ... e.g., one person does the set up, one person validates, three people take the measures, another validates, etc ... and then we'd have to do that roughly 10-20 times per measure.

If we had those things (a & b), then we'd be scientific but since we don't we're not.

That's not to say it's not valuable data - I've defended you many times Amir, and will continue to partially because I like you but also because I think you do good work.

That said, measurements, especially don't "cut the crap" because they no really predictive or an outcome any more than my speedometer is - I speed all day every day and haven't gotten a ticket in 6 years, so while my speedometer provides a value it's not reliably predictive of a "good" or "bad" outcome.

QED.

Your comparison makes the serious mistake of assuming that a speedometer is a device to measure whether you will get a ticket or not.
It is a device for measuring speed.

 

[Gruss Gott]

Your comparison makes the serious mistake of assuming that a speedometer is a device to measure whether you will get a ticket or not.
It is a device for measuring speed.

Exactly, and by itself, speed is a meaningless measure as it predicts nothing. So less a "mistake" and more trying to extend the metaphor ... and nothing in the entire topic of consumer audio should be considered "serious", which was your mistake.

We can weigh our DACs. That's a measure. But weight isn't predictive of audio quality. That's the thing with science - its goal is to be predictive.

My only point was responding to the dude who said that measurements "cut the crap" - no they don't, they may even confuse things assuming your goal is great audio ... cause in that case the only measure that counts is what you hear.

Again, I'm not saying measurements are useless, just that they're neither definitive, nor predictive, nor scientifically valid (at least the way they're done here with consumer audio electronics and probably just about everywhere else)

Personally I use all manner of measurements as factors in my purchases (eg., Tyll's and Amir's), but just like with my speedometer, I break the measurement law every day too.

 

[March Audio]

Great example Amir! Let's discuss. With a speedometer:

(1.) We've all agreed that the objective measure - velocity in m/h - has a value which uniquely, specifically, and, therefore, definitely determines legality.
(2.) By convention, we've also agreed to "slop" in both our vehicle's measuring accuracy (speedometer) and the accuracy of police tools.
(3.) However my speedometer's value is not predictive of whether I'm going to get a ticket, i.e,, have a bad outcome, even though I might look at the measure and know that it's "bad".

So what's the scientific key there? The defined link between the objective measure and the outcome: is it predictive of an outcome? Our speedometers aren't (I'll skip the relativity point where we can't prove if the speedometer is measuring my speed or yours). For example your speedometer might measure "good" (under the limit) and mine might measure "bad", but neither of us get a ticket and I'm also having more fun and getting more done than you - certainly I'm faster. So that speedometer wasn't predictive of anything. Doesn't mean it's useless information, though, just not predictive by itself (as pointed out in your cartoon).

Now let's translate that to audio:

(a.) We'd have to agree on a set of measures and values that uniquely, specifically, and definitively predict an audio outcome; every single time for every device, for every single person, and

(b.) We'd have to scientifically conduct those measurements ... e.g., one person does the set up, one person validates, three people take the measures, another validates, etc ... and then we'd have to do that roughly 10-20 times per measure.

If we had those things (a & b), then we'd be scientifically predictive, but since we don't, we're not. Which means we haven't cut any crap ... at least not definitely.

That's not to say it's not valuable data - I've defended you many times Amir, and will continue to partially because I like you but also because I think you do good work. (so the sbaf hater up there can stick it up his butt)

That said, measurements don't "cut the crap" because they're not very predictive of an outcome any more than my speedometer is - I speed all day every day and haven't gotten a ticket in 6 years, so while my speedometer provides a value it's not reliably predictive of a "good" or "bad" outcome.

QED.

QED

Well in that case please present your case in specific detail as to what is wrong with any measurements performed or conclusions drawn from them and 2hy that isn't an indication of audio qaulity. If you can't , and I am absolutely confident that you have no cogent response, then I would suggest the case has been proven.

If you have no response then you are simply in the area of criticism without understanding, in other words, ignorant.

 

[RayDunzl]

A traffic cop pulled over a speeding motorist and asked, "Do you have any ID?"

The motorist replied, "About what?"

 

[Thomas savage]

Someone call 911 , We need the metaphor police and the logic ambulance.

 

[Blumlein 88]

Exactly, and by itself, speed is a meaningless measure as it predicts nothing.

No, speed can be used to meaningfully predict many things. You unfortunately misused it for a purpose it is not fit for.

We can weigh our DACs. That's a measure. But weight isn't predictive of audio quality. That's the thing with science - its goal is to be predictive.

So do we have to read the other million ways you can pick the wrong measure for the wrong purpose?

My only point was responding to the dude who said that measurements "cut the crap" - no they don't, they may even confuse things assuming your goal is great audio ... cause in that case the only measure that counts is what you hear.

Again, I'm not saying measurements are useless, just that they're neither definitive, nor predictive, nor scientifically valid (at least the way they're done here with consumer audio electronics and probably just about everywhere else)

So you think intentionally abusing a couple of metrics allows you this logical leap? You could take the approach from what works as the list is shorter than what doesn't work. I get the impression that isn't where you'll go with this next.

 

[Wombat]

I believe Gruss Gott usually means good-day but in lesser usage it also means good-bye. Entering this forum, so forcefully, with two left-feet puts the latter use into perspective.

 

[Gruss Gott]

Well in that case please present your case in specific detail as to what is wrong with any measurements performed or conclusions drawn

I already did that, but happy to repeat it for you:

(1.) We have no set of measurements that predict experienced audio quality for all similar devices for all listeners 100% of the time, thus the measurements we have are information only, not scientifically definitive nor predictive.

(2.) Even if we did have scientifically predictive measurements, those measurements would have to be performed scientifically to be valid:

• Equipment and environment calibrated and validated,
• One person/team does the set up,
• Another validates the set up,
• Multiple people/teams take the measurements,
• Multiple devices of the same manufacturer from separate batches are tested at the same time
• Multiple teams analyze the data for conclusions
• etc etc etc

(3.) And let's say we have and do all of that and let's be positive: it's the greatest device ever! BUY BUY BUY! Well, due to manufacturer quality control, variance in human hearing, and our lack of understanding of exactly how human hearing works (e.g, why do we instantly know live music vs reproductions?) there's still no guarantee an individual buyer would experience great audio, just a higher probability. maybe.

Thus measurements, especially when done unscientifically, are interesting but certainly not definitive nor predictive of any listener's experience.

and therefore measurements haven't "cut the crap".

QED.

(although I feel obliged to say again, that doesn't mean not useful and I personally love the work @amirm does - just encouraging some perspective here)

 

[Veri]

That's a whole lot of nothing just for a "there's no guarantee". An objectively 'sound' device is always preferable in my mind..

We already have SBAF if we want to hear "church of Amir" nonsense

 

[andreasmaaan]

(1.) We have no set of measurements that predict experienced audio quality for all similar devices for all listeners 100% of the time, thus the measurements we have are information only, not scientifically definitive nor predictive.

We actually have a lot of quite good data on the thresholds at which particular kinds of distortion become audible, accumulated through years of research. You're correct that this data is not definitive, but we can look at reasonably comprehensive measurements of a given device and reach an informed view as to whether or not it will colour the sound in any audible way.

We also have a a number of controlled blind tests of good-measuring DACs in which listeners were unable to discern one from the other. More importantly, to my knowledge there has never been a controlled blind test in which listeners have been able to discern a difference between two DACs that measure in such a way that they would be expected to be transparent. When that comes along, I'll be willing to revise my opinion.

There's always room for doubt, but I don't think it takes much to put 2 and 2 together...

 

[Thomas savage]

We are not building a spacecraft here, we are just exploring the performance of audio kit using well documented measurement techniques.

They are open to scrutiny in reasonable fashion, Iv no clue what @Gruss Gott thinks he’s arguing against .

Individuals will all have their own thresholds when it comes to how they chose to evaluate information, I think members here have a higher threshold than many elsewhere.

If we had more cooperation ( less egotism and commercial influence) between those measuring stuff we might achieve a higher ‘ scientific ‘ standard..

But we don’t so we do what we can.

Where’s the lack of perspective?

 

[andreasmaaan]

We are not building a spacecraft here

Are you suggesting that home audio enthusiasts are less serious about their hobby than NASA engineers are about their work?

Iv no clue what @Gruss Gott thinks he’s arguing against

He's got a most excellent username / profile pic combo though

 

[amirm]

Belief in the perfection of measurements is, therefore, religious, not scientific.

An important clarification here: our measurements are not intended to create new science. They are basic measurements intended for design verification. No new science pops out of them. Instead, we test to see if a device is truly hi-fi and is transparent as countless audio products claim to be.

All the measurements are canned and standardized. The device being used by me is the best in class and literally the standard in the industry by which others are measured.

Members and manufacturers routinely comment on my results and at the end, the process goes through a lot of iterations before we are done. This is in sharp contrast to measurements in print magazines and other online sites where it is take it or leave it. I am not bound by commercial interest to move on to review some other expensive product so I can take the time to test and re-test until high-confidence truth about the fidelity of the product is determined.

In my reviews I do go beyond measurements, explaining often how the results correlate with how we hear using psychoacoustics. There, we are using science but again, not inventing any.

This is audio engineering at play for the most part. And it is something all manufacturers should be doing. But they are not as customers are not pushing them to do even basic measurements let alone the work I do. So if you want your complaint to have value, send the note to every manufacturer and ask them to measure as you state. It will fall on deaf ears but it is better than stating your case here.