Audibility thresholds of amp and DAC measurements

[j_j]

Amir does these tests of DACs.
X axis is the gozinta, and Y axis is the gozouta. Relative in terms of relationship to input at very low levels. Measured tone seems to be 1kHz, bandwidth unstated in anything I can find but it apparently catches the effect of band pass filters so probably at least 40 kHz.

Rick “this is something Amir provides” Denney

It seems somewhat less than extremely revealing for distortions, but does expose monotonicity issues, I suspect.

 

[Rja4000]

expose monotonicity issues

It would show any attempt to cheat with level.
Compression, gate,...

 

[j_j]

Yes, that it would. So would basic least mean squares measurement.

 

[SIY]

It seems somewhat less than extremely revealing for distortions, but does expose monotonicity issues, I suspect.

It also shows noise, redundantly.

 

[j_j]

It also shows noise, redundantly.

This is where a 10 second "bong" tone (t e ^ (-t/constant) window is very handy. Using a Hilbert envelope processing, you can see just how the attack and decay work, cleanly, and with very simple processing).

 

[RichB]

It seems somewhat less than extremely revealing for distortions, but does expose monotonicity issues, I suspect.

In instant A/B listening to DACs that are level matched using 1 1kHz tone to within hundredths of a volt (0.00x), I have perceived differences in DACs.
The dominant view is that the DAC tests here with 1kHz and multi-tone is sufficient to characterize performance, so that audible differences between high performance (green) DACs.

I expect that many believe I am imagining audible differences and want a blind test. which is fine.
Is it possible, that audible differences exist but are not fully charactered by the current suite of measurements?
Timing between channels or distortion with complex tones, or perhaps others.

- Rich

 

[pma]

The dominant view is that the DAC tests here with 1kHz and multi-tone is sufficient to characterize performance

Why do you think it is a “dominant view”?

 

[RichB]

Why do you think it is a “dominant view”?

Let's just say, there is loud opposition to audible differences for green-zone DACs.

- Rich

 

[j_j]

In instant A/B listening to DACs that are level matched using 1 1kHz tone to within hundredths of a volt (0.00x), I have perceived differences in DACs.
The dominant view is that the DAC tests here with 1kHz and multi-tone is sufficient to characterize performance, so that audible differences between high performance (green) DACs.

Could you explain how you heard this difference? (Note, it's not impossible if there is some frequency shaping outside the 1kHz point, duh.)

I expect that many believe I am imagining audible differences and want a blind test. which is fine.

Nobody thinks you "imagine" anything, but the case of inadvertent use of other senses, or even pure expectation, is completely demonstrated, and there is nobody, anywhere, who can hear, at least, who has demonstrated an ability to avoid the fundamental cognitive effects of perception.

Is it possible, that audible differences exist but are not fully charactered by the current suite of measurements?
Timing between channels or distortion with complex tones, or perhaps others.

- Rich

Since one can not prove a negative proposition, ever, it's a logical impossibility, yes, there COULD be differences. On the other hand, sensitivity to level, loudness, frequency analysis, time arrival, etc, are very well characterized. It seems quite clear that anything more will be a very fiddly edge case.

As the commercial says: Where's the beef? You need that DBT (with controls) to make your point.

 

[Mort]

Let's just say, there is loud opposition to audible differences for green-zone DACs.

I always thought of that zone as 'likely inaudible', genuinely allowing for a few that might be able to differentiate.

 

[RichB]

Could you explain how you heard this difference? (Note, it's not impossible if there is some frequency shaping outside the 1kHz point, duh.)



Nobody thinks you "imagine" anything, but the case of inadvertent use of other senses, or even pure expectation, is completely demonstrated, and there is nobody, anywhere, who can hear, at least, who has demonstrated an ability to avoid the fundamental cognitive effects of perception.


Since one can not prove a negative proposition, ever, it's a logical impossibility, yes, there COULD be differences. On the other hand, sensitivity to level, loudness, frequency analysis, time arrival, etc, are very well characterized. It seems quite clear that anything more will be a very fiddly edge case.

As the commercial says: Where's the beef? You need that DBT (with controls) to make your point.

Yes, I can do a DBT when I get a willing accomplice.
I have the switches in place. I also have two of the same DACs, oh my.

- Rich

 

[Old_School_Brad]

Why do you think it is a “dominant view”?

It's a test tone that plays to the strengths of the equipment. In fact, it sits right in the sweet spot of audio electronics, where most gear can reproduce it without any difficulty.

Much like selling cars on a rainy day. Scratches in the paint are not visible until the water dries.

But you already know this. What you might have meant, rather, is whether the prevailing view is that it's truly sufficient.
I don’t think so. Perhaps many simply choose not to engage in discussions about it and just accept measurements as they are.

 

[j_j]

It's a test tone that plays to the strengths of the equipment. In fact, it sits right in the sweet spot of audio electronics, where most gear can reproduce it without any difficulty.

Which test tone? The 1kHz tone is only one test signal out of many that I use, at least. I saw others mention a buzz tone, maybe (or not I don't know) one I made a while ago that very quickly exposes IMD anywhere from near-DC to 20kHz, by causing very specific distortion products around 500Hz. That's a very revealing test. That tone can be used for the level test just like any other tone. So I think you're poking at a straw man here, by proposing there's only one test, when there isn't.

But you already know this. What you might have meant, rather, is whether the prevailing view is that it's truly sufficient.

That's not relevant, it's not the only test used here, now, is it?

I don’t think so. Perhaps many simply choose not to engage in discussions about it and just accept measurements as they are.

Quite aside from what you think, how about evidence?

 

[Old_School_Brad]

Which test tone? The 1kHz tone is only one test signal out of many that I use, at least. I saw others mention a buzz tone, maybe (or not I don't know) one I made a while ago that very quickly exposes IMD anywhere from near-DC to 20kHz, by causing very specific distortion products around 500Hz. That's a very revealing test. That tone can be used for the level test just like any other tone. So I think you're poking at a straw man here, by proposing there's only one test, when there isn't.


That's not relevant, it's not the only test used here, now, is it?


Quite aside from what you think, how about evidence?

I believe you might be misinterpreting my comment. I was specifically referring to situations where only a 1 kHz test tone is used -nothing more, nothing less.

We all know this is sometimes used in discussions and debates to either support or debunk a claim.
For example, when amplifier manufacturers list a power rating at 1 kHz in their specs, or when a single 1 kHz distortion test result is mistakenly taken as conclusive evidence of a product’s quality, whether positive or negative.

 

[j_j]

I believe you might be misinterpreting my comment. I was specifically referring to situations where only a 1 kHz test tone is used -nothing more, nothing less.

We all know this is sometimes used in discussions and debates to either support or debunk a claim.
For example, when amplifier manufacturers list a power rating at 1 kHz in their specs, or when a single 1 kHz distortion test result is mistakenly taken as conclusive evidence of a product’s quality, whether positive or negative.

Evaluation based strictly on a 1kHz tone is "a good start". That, for example, does not test frequency response. The graphs upthread show level roving, etc, as well. So that's more than a 1kHz sine wave test. It's still insufficient, of course. A buzz tone that reaches to 20kHz with equal spectra across the top, now, is a much more useful test for many reasons. It's still only part of a test suite.

 

[donjoe0]

Really the issue to me isn't even that multiple tests with different stimulus types aren't being done. As I said over in the more appropriate thread for this - the one about DAC sound signatures - multitone tests are mechanically being performed, only for the inconvenient part of their results to go completely ignored and the device to be absurdly declared "audibly transparent" anyway. Inconvenient part i.e. that the tone amplitudes do not come out equal out of the device, suggesting there will be some FR issues depending on what materials you're putting through the device.

I see people keep talking about "burden of proof" over there - well, anyone using the expression "audibly transparent" is taking on the heaviest burden of proof of all; they should produce world-class ABX tests to prove it, or stop using that expression to describe devices ever again.

 

[j_j]

Really the issue to me isn't even that multiple tests with different stimulus types aren't being done. As I said over in the more appropriate thread for this - the one about DAC sound signatures - multitone tests are mechanically being performed, only for the inconvenient part of their results to go completely ignored and the device to be absurdly declared "audibly transparent" anyway. Inconvenient part i.e. that the tone amplitudes do not come out equal out of the device, suggesting there will be some FR issues depending on what materials you're putting through the device.

I see people keep talking about "burden of proof" over there - well, anyone using the expression "audibly transparent" is taking on the heaviest burden of proof of all; they should produce world-class ABX tests to prove it, or stop using that expression to describe devices ever again.

All I can say is that when differences between two signals in the atmosphere are below the noise floor of the atmosphere, which isn't that small, really, you're in very good shape. Meters are much, much more sensitive than ears. So at some point the "transparent" claim is on good ground, with an ABX test, unless you're proposing that there is some supernatural something at play.

Now, if a multi-tone shows errors in frequency response, obviously they should be examined.

 

[Old_School_Brad]

Really the issue to me isn't even that multiple tests with different stimulus types aren't being done. As I said over in the more appropriate thread for this - the one about DAC sound signatures - multitone tests are mechanically being performed, only for the inconvenient part of their results to go completely ignored and the device to be absurdly declared "audibly transparent" anyway. Inconvenient part i.e. that the tone amplitudes do not come out equal out of the device, suggesting there will be some FR issues depending on what materials you're putting through the device.

I see people keep talking about "burden of proof" over there - well, anyone using the expression "audibly transparent" is taking on the heaviest burden of proof of all; they should produce world-class ABX tests to prove it, or stop using that expression to describe devices ever again.

Hmm. It seems like you may have just made a claim. I highlighted that part because I’m interested in understanding exactly what you mean or which measurement you’re referring to, so we can take a closer look.

 

[donjoe0]

Hmm. It seems like you may have just made a claim. I highlighted that part because I’m interested in understanding exactly what you mean or which measurement you’re referring to, so we can take a closer look.

Hidizs S8 multitone was one example I found a while back. But there's plenty. Almost every time Amir measures an affordable DAC/amp with multitone, the test tones come out with a visibly non-flat envelope, even at the bad zoom level he shows them (because he's only using them as noise tests), with spike heights that cross into potential audibility by having more than 0.1-0.2 dB of difference, yet this goes completely ignored in the results discussion.

In my limited explorations of this, the amplitude differences get even worse when you test with hundreds of tones and large FFT sizes. This requires some investigation, I only got to trying such measurements after hearing DAC/amp differences in (admittedly sighted) A/B comparisons. There could be something there, but I'm not seeing a lot of people discussing it to clarify, only to handwave it away.

 

[Old_School_Brad]

Hidizs S8 multitone was one example I found a while back. But there's plenty. Almost every time Amir measures an affordable DAC/amp with multitone, the test tones come out with a visibly non-flat envelope, even at the bad zoom level he shows them (because he's only using them as noise tests), with spike heights that cross into potential audibility by having more than 0.1-0.2 dB of difference, yet this goes completely ignored in the results discussion.

In my limited explorations of this, the amplitude differences get even worse when you test with hundreds of tones and large FFT sizes. This requires some investigation, I only got to trying such measurements after hearing DAC/amp differences in (admittedly sighted) A/B comparisons. There could be something there, but I'm not seeing a lot of people discussing it to clarify, only to handwave it away.

Thank you. -Reading..

Edit; Stopped reading. Did @pkane later make a comment to your measurements back then?