General debate thread about audio measurements

[FrivolsListener]

Maybe this for 16 bit. The Bifrost looks nothing like this -90.3 dbFS waveshape in Atomic Bob's view of it.

Does any real 16 bit DAC produce that, or is it from a textbook? Not being snarky, being curious.

 

[FrivolsListener]

I've not been following this thread for a while, but a brief dip in now reveals that the exact mains voltage is being questioned?!! It might make a difference as to whether a regulator is or is not dropping out..??!!! I am having trouble in expressing my disbelief. It is cognitive dissonance or something, where a discussion is taking place in all seriousness about something that no engineer would ever have trouble with when designing a low power device that runs on the mains. Why does audiophilia result in 'designs' that are so amateurish and shambolic that people might even raise it as an issue?

Yes, I asked.

The reason I asked is because Jason showed very similar bad behavior from an experiment that he had undersupplied.

I'm really not caring one way or the other -- as I said several times in this latest spurt of activity, I bought a DAC based on Amir's review to see if I could subjectively hear what he objectively measured. I'm mostly curious why this corner of the web is where nothing Schiit makes work and why that doesn't exist on, say, SBAF. (Head-fi's reasons need no further exploration.)

I am also interested in better equipment, no matter what the brand is on the front. But, unlike a lot of people, I do have to consider a budget, so 5-figure DACs are right out.

 

[FrivolsListener]

Hmmm. Seems like you have not read much of my reviews as I show and explain that signal in everyone. Here it is from my latest review: https://www.audiosciencereview.com/...gemini-2000-dac-and-headphone-amplifier.2556/

The one on the left is what it is supposed to look like.

Now compare that to the Schiit Modi 2 Multibit from abob. And you tell me if you did a review and got that, you would say nothing.

Actually, I have.

What my reaction would be is to say that the result of the DX-7 is "too perfect." I wonder what happens if you test on off frequencies, like 2 kHz or 3.1415927 kHz or whatever. It wouldn't be the first time a product was engineered to overperform on a benchmark by designing it to the benchmark: nVidea did that; Intel did that. (Contrast with @Blumlein 88 's plot above, which is what I'd expect.)

As site-owner, you should be able to look at my activity on the site and which pages I've read.

 

[Cosmik]

I'm mostly curious why this corner of the web is where nothing Schiit makes work and why that doesn't exist on, say, SBAF.

The review of the amplifier with 34% distortion was Stereophile, though. It follows the same theme as the smoking headphone amplifier: it might just about work if you are careful not to put steady tones into it.

 

[Thomas savage]

Actually, I have.

What my reaction would be is to say that the result of the DX-7 is "too perfect." I wonder what happens if you test on off frequencies, like 2 kHz or 3.1415927 kHz or whatever. It wouldn't be the first time a product was engineered to overperform on a benchmark by designing it to the benchmark: nVidea did that; Intel did that. (Contrast with @Blumlein 88 's plot above, which is what I'd expect.)

As site-owner, you should be able to look at my activity on the site and which pages I've read.

That’s a bit ridiculous, when we start going down this road I think all value in the exchange is lost.

No one here is trying to prove something is bad or something is good in regard to our published measurements at least. Stuff just gets tested and the information put forward, open to question of course but we are not here trying to trip up products.

There’s often time limits too, amir has a window to measure stuff so in order for there to be a coherent history of product measurements we need to keep to set testing parameters.

That’s how I understand it at least , everyone is free to interpret the information as they wish. The whole this forum vs that or this guy’s measurements vs amirs is only really of value if you have deep understanding of those measurements and that seems rarely to be the case.

For the layman it can be a bit difficult, you just have to take in the information the best you can and make your choice but there’s plenty of educational content here and the members that do understand things are more than willing to help..

 

[garbulky]

I agree on the question how relevant is this stuff really? -90 db sound is audible but how audible in music? The answer is probably not very or not at all. Can you make out a distortion IN the -90 db sound in music? Well, I doubt it.
We get that it is possible to distinguish up to 120 db but is that really in music? No music masks a lot of stuff.
So then it comes to how useful are these measurements in real world usage? And that's important because people are calling companies all kinds of garbage for stuff that's pretty tiny in terms of actual real world usage. I see the term "shown the door," and all kinds of descriptive terms like "handily beaten".

When in reality, "barely any difference if that, other than for a graph," is a more apt term.
A THD of 0.01% and 0.00001% is an exponential difference. It's massive. When compared between the two we should naturally call a company whose device that does 0.01% THD to be shelling out incompetently designed garbage. But in real world use, it's almost no difference. So the relevance of measurements must come in to play at some point.

The basic point argument I see being made is
If you can buy a much better measuring device for the same price or less from another company why the heck wouldn't you. See the Topping DAC? It's beaten everything out there! Or see these tiny spikes at -130 db? The Topping Dac doesn't do that! Look at this pile of Schiit that does! What a turd.

Well the answer is....if you are only interested in measurements predicting outcomes, then you must look at what the measurements say in outcomes of music listening. Not tones. it doesn't really matter if the Topping DAC does it if you can't actually hear it in music.

OTOH, there are people that are genuinely interested in getting the best gear they can bar none, forget audibility. For them, they want to know that their DAC is the best in terms of engineering even if it makes no real world difference. As long as they know what they are doing when they proclaim major differences in DACs - what the actual implications of it are.

 

[amirm]

True enough; it is an opinion. So tell me: Can you provide peer reviewed research that -90 dBm signals can be heard on any transducer by a human?

Sure. Look at the references in my article on that very topic: https://www.audiosciencereview.com/forum/index.php?threads/dynamic-range-how-quiet-is-quiet.14/.

They are both from Journal of Audio Engineering Society which requires peer review.

 

[RayDunzl]

Does any real 16 bit DAC produce that, or is it from a textbook?

I think "yes".

I'll see if I can measure it here (probably not, tools lacking).

---

Stereophile posts those waves, sometimes:

https://www.stereophile.com/content/nagra-dac-da-processor-measurements

They aren't always so "textbook".

https://www.stereophile.com/content/lector-strumenti-audio-digitube-s-192-da-converter-measurements

 

[FrivolsListener]

The review of the amplifier with 34% distortion was Stereophile, though. It follows the same theme as the smoking headphone amplifier: it might just about work if you are careful not to put steady tones into it.

And I have said upthread (and in other ASR threads) that I wouldn't use that amp. However, driving an amp to clipping with a sub 1 ohm output impedance and yanking out the dummy load (which shorts the TRS jack) and being shocked -- SHOCKED! -- that it causes a problem is a little disingenuous. Would you do that to your amp? (I wouldn't. I'd turn it down and pull the thing out. And that's when listening to music or content other than a continuous clipping sweep.)

Jason has noted in his blog that there are amplifier topologies that he won't consider because they'll dump so much current that they'll weld a TRS plug and jack together.

 

[amirm]

I agree on the question how relevant is this stuff really?

It is quite relevant in how purchase decisions are made.

In absence of measurements like mine, you are driven hard by the marketing of manufacturers. Collectively for example, they have conditioned consumers to think that R2R ladder DACs are more accurate. That then preconditions listening tests people do. They compare their current DAC to R2R and arrive at the false conclusion that they sound better. That word then spreads and more and more people believe this to be true.

What measurements have shown so far with respect to Schiit DACs is that their R2R DACs are less accurate. This is news to many people who heretofore thought the truth was the other way around.

Here is the simple truth and there is no way of getting around it. A DAC's job is to accept digital samples and produce analog voltages corresponding to it. I bet if I interviewed 100 audiophiles, 99 would say that is what they expect or think their DAC should do, or is doing. The other person is probably drunk so that doesn't count. We measure to see if that hypothesis is true. To the extent DACs relative to their competition fall substantially worse, then that matters to people on the basis of purchasing gear.

Importantly, it is never the case where you are better off knowing less about a product you are going buy.

In the specific case of a -90 dB test, we have picked a very old standard where the assumption was that reproducing even 16 bits which was in the bit depth of CD was hard. Today, there is content and of course every DAC that plays up to 24 bits and beyond. Many people think that 24 bits is what they are buying. So we could have picked a test at -110 or even -120 db out of -144 floor of 24 bits. But we have not. We are not going to extremes to differentiate. We are going to a reasonable place in fidelity that doesn't even exceed CDs and ask: "does this DAC do what I think DACs should do?"

Finally, important part of purchase decision is which product to buy. Given a choice of one that mangles everything at -90 dB and one that doesn't, it makes little sense to buy the former. Even if you don't need the extra fidelity, why reward the company that built the sub-par device? Worse yet many times the subpar device actually costs more!

So, be an informed consumer. Don't let people convince you need to know less. You don't.

 

[FrivolsListener]

That’s a bit ridiculous, when we start going down this road I think all value in the exchange is lost.

Fair enough. But a statement implying that I haven't read reviews here from someone who should know better wasn't helpful, either.

 

[FrivolsListener]

Sure. Look at the references in my article on that very topic: https://www.audiosciencereview.com/forum/index.php?threads/dynamic-range-how-quiet-is-quiet.14/.

They are both from Journal of Audio Engineering Society which requires peer review.

Thank you.

 

[amirm]

Does any real 16 bit DAC produce that, or is it from a textbook? Not being snarky, being curious.

When you see that little "AP" symbol on top right of a graph, it means Audio Precision analyzer. As such, they are always measurements.

In this case you are being given standard DAC measurements that stereophile performs. See this among many examples: https://www.stereophile.com/content/bryston-bda-3-da-processor-measurements

So yes, that is real output, not textbook. The digital samples/textbook don't have that ringing.

 

[garbulky]

It is quite relevant in how purchase decisions are made.

In absence of measurements like mine, you are driven hard by the marketing of manufacturers. Collectively for example, they have conditioned consumers to think that R2R ladder DACs are more accurate. That then preconditions listening tests people do. They compare their current DAC to R2R and arrive at the false conclusion that they sound better. That word then spreads and more and more people believe this to be true.

What measurements have shown so far with respect to Schiit DACs is that their R2R DACs are less accurate. This is news to many people who heretofore thought the truth was the other way around.

Here is the simple truth and there is no way of getting around it. A DAC's job is to accept digital samples and produce analog voltages corresponding to it. I bet if I interviewed 100 audiophiles, 99 would say that is what they expect or think their DAC should do, or is doing. The other person is probably drunk so that doesn't count. We measure to see if that hypothesis is true. To the extent DACs relative to their competition fall substantially worse, then that matters to people on the basis of purchasing gear.

Importantly, it is never the case where you are better off knowing less about a product you are going buy.

In the specific case of a -90 dB test, we have picked a very old standard where the assumption was that reproducing even 16 bits which was in the bit depth of CD was hard. Today, there is content and of course every DAC that plays up to 24 bits and beyond. Many people think that 24 bits is what they are buying. So we could have picked a test at -110 or even -120 db out of -144 floor of 24 bits. But we have not. We are not going to extremes to differentiate. We are going to a reasonable place in fidelity that doesn't even exceed CDs and ask: "does this DAC do what I think DACs should do?"

Finally, important part of purchase decision is which product to buy. Given a choice of one that mangles everything at -90 dB and one that doesn't, it makes little sense to buy the former. Even if you don't need the extra fidelity, why reward the company that built the sub-par device? Worse yet many times the subpar device actually costs more!

So, be an informed consumer. Don't let people convince you need to know less. You don't.

First, I absolutely think it's a good idea you do all the tests. It does show things for what they are - like R2R DACs and their linearity vs DS DACs.
So for instance right now, I'm not sure where the "mission critical" tagline on multibit holds value if their linearity is off.
That way when you buy you can make a more informed decision.

But if you can't hear it in music, how is it relevant to a purchase? My things is that the measurement differences have to be correlated to the real world. To your credit you do listening tests, though you haven't mentioned your procedure. But in your graph you talk about "far superior" and "far better". Well true when you look at a graph it does look "far superior". But when you actually take it in context, it looks to me like it's not really any different as far as how you'd use the DAC for.
In your listneing tests, I noticed you've remarked that for most of them you haven't heard a difference. Or you mention I heard this one's treble was a bit more colored. And then you just leave it there. Well if you didn't hear a difference, that's probably the biggest take away of the experiment imo as far as purchase decisions go. And if you did hear a difference, how much was that difference worth to you? But you just sort of make a brief mention about it. But in terms of measurements you have a lot more descriptive things to say.

 

[amirm]

Actually, I have.

Then why did you say whether the output is supposed to look like a square wave???

What my reaction would be is to say that the result of the DX-7 is "too perfect." I wonder what happens if you test on off frequencies, like 2 kHz or 3.1415927 kHz or whatever. It wouldn't be the first time a product was engineered to overperform on a benchmark by designing it to the benchmark: nVidea did that; Intel did that. (Contrast with @Blumlein 88 's plot above, which is what I'd expect.)

Those are different tests. -90 dB sine wave uses 24 bit format so won't look like steps. It is way above the minimum samples in 24 bit format (to the tune of 50+ dB).

As to cheating, it is not possible. The DAC has no idea if you are feeding it music or tones. It has no intelligence that way. Nvidia drivers sit inside the operating system, running code that can analyze what it is being told to do. And generate specific responses without actually doing the work. Here, we are measuring the analog output of the DAC. It cannot be faked that way.

As site-owner, you should be able to look at my activity on the site and which pages I've read.

I can't and I wouldn't. Can we dial back this type of back and forth?

 

[garbulky]

The DAC has no idea if you are feeding it music or tones. It has no intelligence that way.

Yes I doubt that you can make DACs to cheat the system per se

 

[Blumlein 88]

Does any real 16 bit DAC produce that, or is it from a textbook? Not being snarky, being curious.

That's from a Benchmark tested by Stereophile when fed a 16 bit signal.

 

[amirm]

But if you can't hear it in music, how is it relevant to a purchase?

I don't know how to define "music" for all content and all people.

What if I have a piece of music that has -10 dB for peaks as to have some headroom (i.e. not compressed to hell). Now -90 dB is -80 dB down from the peaks. You would be turning up the volume higher on your amp than you would with music that is maxed out at 0 dB. So what do we do now? Make the statement that -80 dB is all we need?

People want to have the freedom to buy whatever music, listen at whatever level. Let's give them the best fidelity we can. It is not like it is hard or impossible to get a clean -90 dB. DACs at $100 are doing that day in and day out. So it is not like we are asking for automakers to make cars that get 200 miles to the gallon and they say it is impossible. What we are asking here is readily available and indeed, is way worse than what the DAC silicon itself is able to do.

 

[amirm]

In your listneing tests, I noticed you've remarked that for most of them you haven't heard a difference. Or you mention I heard this one's treble was a bit more colored. And then you just leave it there. Well if you didn't hear a difference, that's probably the biggest take away of the experiment imo as far as purchase decisions go. And if you did hear a difference, how much was that difference worth to you? But you just sort of make a brief mention about it. But in terms of measurements you have a lot more descriptive things to say.

The level of rigor in my linsteing tests is not remotely the same as the measurements.

As to what it is worth to me, in many comparisons I have done, the cheaper DAC is the one that has won out with better performance! Without measurements people would automatically assume the other way around. So measurements are saving people, not costing them. To the extent there is a counter example, then sure, people can make judgements. But they would be doing so with real data, not suppositions.

 

[FrivolsListener]

I think "yes".

I'll see if I can measure it here (probably not, tools lacking).

---

Stereophile posts those waves, sometimes:

https://www.stereophile.com/content/nagra-dac-da-processor-measurements

View attachment 11868

They aren't always so "textbook".

https://www.stereophile.com/content/lector-strumenti-audio-digitube-s-192-da-converter-measurements

View attachment 11869

About what I'd expect for a 24 bit DAC showing 16 bit patterns. Good. (And the last, not so much.) But I wonder what a 16 bit DAC would do.