• WANTED: Happy members who like to discuss audio and other topics related to our interest. Desire to learn and share knowledge of science required. There are many reviews of audio hardware and expert members to help answer your questions. Click here to have your audio equipment measured for free!

Audibility thresholds of amp and DAC measurements

makmeksam

Member
Joined
Nov 13, 2019
Messages
59
Likes
44
Many subjectivists argue that measurements can not capture the audio quality of an AMP or a DAC.

We can accept that there are lot of variables that come into play when measuring a device. Input level, input signal (1kHz sine/CCIF tones/SMPTE tones etc), gain/output level, load resistance are some of them. If we were to capture the distortions with respect to at least the above variables, we have to create at least a 5 dimensional plot. But that is not practical and we never do that.

Because of this limitation can there be things that we miss with measurements but subjectivists can possibly sense by listening? (Consider IMD for an instance. We mostly use two test tones to measure IMD. But real music can be much more complex. So real music may give more distortions even if the measurements of a given device are very good?) Or else can we theoretically argue that measurements can capture the audio quality of an AMP or a DAC with a very good certainty even with the limited amount of measurements that we consider?
 

solderdude

Grand Contributor
Joined
Jul 21, 2018
Messages
15,891
Likes
35,912
Location
The Neitherlands
So a state of the art measurement system that can measure far beyond studio quality ADC and DACs and can expose issues with amps/DACs even below noise levels cannot capture what much lesser studio equipment can when music is recorded, processed, and reproduced ?
The folks making these claims don't realize the 'wonderful recordings' they listen to comes from equipment that performs less than measurement gear.
Just realize the recording and playback equipment uses the same techniques but with SOTA performance and engineering.

It is all a bunch of nonsense which they claim must exist because measurements don't jive with what they think they are hearing.
All it means is that their ears are nowhere as good as measurement equipment and some of those folks have a preference for certain types of 'changes' made to the recorded signal.

Do some audibility tests.
The only caveat is you need top notch headphones/speakers with well performing equipment otherwise you are listening to deficiencies of the transducers.
This is why with some of the online tests you find some people cannot listen past a certain level where at the maximum level there is an unexpected rise in performance.
Are these the golden eared folks ?
No.. they are using good transducers and good equipment.

So do audibility tests but not on average equipment.
 
Last edited:

majingotan

Major Contributor
Forum Donor
Joined
Feb 13, 2018
Messages
1,509
Likes
1,781
Location
Laguna, Philippines
Keep in mind that subjectivists create biases of their own. Take a look at Magni 3+ and Magni Heresy comparison: https://www.headphones.com/blogs/ne...-heresy-headphone-amplifier-review-comparison

On the overall preference section:

This is based on both the outputs of my blind comparison as well as spending multiple days with these units simply listening for pleasure and at length, with the latter being a much more significant factor.

He was performing sighted test much more significantly than objective double blind test which he also admit that it's not, "necessarily to the standards of a full-blown clinical-grade double-blind test, it did involve level matching to within 0.1 dB, I (sic) did not know which amplifiers were being compared in any given iteration and the setup allows for a single amplifier to be both the “A” and “B” units at the same time." Also a key section on his A/B test is missing: where is the EVIDENCE of the A/B test, just like @amirm pointed out on the Heresy thread?

I can smell how biased that blogger is over 0.0008x% THD+N performing amps with op-amp gain stage and prefer the "exciting sound signature" of a discrete amp.

There are users here who have proven and proven by even just sighted volume matched A/Bing, there are no differences in amp that measures beyond our normal listening level.

But real music can be much more complex. So real music may give more distortions even if the measurements of a given device are very good?) Or else can we theoretically argue that measurements can capture the audio quality of an AMP or a DAC with a very good certainty even with the limited amount of measurements that we consider?

Recorded music have FAR less dynamic range than DACs can achieve. Just look at loudness wars database and see for yourself. Only those incredibly well recorded and mastered classical music with very high dynamic range can really take advantage of a DAC's SINAD performance, and even then you'll be hard pressed to find a difference with well engineered DACs on a DBT volume matched test. A DAC's job is to convert 0s and 1s to analog and measurements capture all that analog signal beyond our hearing capability. IMO, all the other subjective differences cannot be validated until there are sufficient evidence proven and the test is validated officially by a third party organizer that aren't affiliated with either sides.
 

j_j

Major Contributor
Audio Luminary
Technical Expert
Joined
Oct 10, 2017
Messages
2,267
Likes
4,758
Location
My kitchen or my listening room.
The comment a few articles up is quite mistaken as to what we can measure, and how we can evaluate it. It is not hard to do SNR tests and calculate error spectra on real signals. So, of course, we do that. It is also possible to contrive nasty "buzz" tones that have fifty harmonics or so, and that create a setup where EVERY nonlinearity creates a tone at one particular place, too.

Measuring noise floor isn't hard any more.
Measuring useless things like "THD" and "SNR" aren't hard any more, on complex, simple, or any other known signal. Sorry, but that's how it is.

If your stereo's maximum reproduction level is known knowing the all-encompassing SNR and knowing it's below the threshold of hearing really does suffice. You can do all of this with matlab and some simple, short programs, using a super-high quality DAC and ADC.

It's not hard.

No, this isn't a simple THD mesaurement.
 

Blumlein 88

Grand Contributor
Forum Donor
Joined
Feb 23, 2016
Messages
20,521
Likes
37,049
The comment a few articles up is quite mistaken as to what we can measure, and how we can evaluate it. It is not hard to do SNR tests and calculate error spectra on real signals. So, of course, we do that. It is also possible to contrive nasty "buzz" tones that have fifty harmonics or so, and that create a setup where EVERY nonlinearity creates a tone at one particular place, too.

Measuring noise floor isn't hard any more.
Measuring useless things like "THD" and "SNR" aren't hard any more, on complex, simple, or any other known signal. Sorry, but that's how it is.

If your stereo's maximum reproduction level is known knowing the all-encompassing SNR and knowing it's below the threshold of hearing really does suffice. You can do all of this with matlab and some simple, short programs, using a super-high quality DAC and ADC.

It's not hard.

No, this isn't a simple THD mesaurement.
So can you lay out the outline for what measurement protocol would be? It isn't a simple THD measurement.............what is it?

My speakers are capable of about 105 db SPL at the listening position. The electronics all together at the speaker can manage an SNR of roughly 100 db. What kind of testing do I need to do for my or other systems? The wideband ambient noise level is mid 30 db SPL at quiet times.

@j_j
 

j_j

Major Contributor
Audio Luminary
Technical Expert
Joined
Oct 10, 2017
Messages
2,267
Likes
4,758
Location
My kitchen or my listening room.
If you have a wideband SNR of 100dB you should be pretty much good to go, assuming you have a flat noise floor in the system.


What you do, simply put, is to put in a signal you know capture the signal out, equalize it for whatever gain error and frequency response errors you find, and then take the difference. That's the noise.

Simpler to explain than do, but not horribly hard.

The noise of the atmosphere is about 6dB SPL, give or take, white noise, at 20-20K. You can't hear it because the energy density in an ERB/Critical Band isn't enough to raise it above absolute threshold. Barely.

Your ambient level you have of 30dB SPL does not quite suffice, you also need the noise spectrum. But it's a pretty quiet room in the modern world.
 

Spocko

Major Contributor
Forum Donor
Joined
Sep 27, 2019
Messages
1,621
Likes
2,999
Location
Southern California
So a state of the art measurement system that can measure far beyond studio quality ADC and DACs and can expose issues with amps/DACs even below noise levels cannot capture what much lesser studio equipment can when music is recorded, processed, and reproduced ?
The folks making these claims don't realize the 'wonderful recordings' they listen to comes from equipment that performs less than measurement gear.
Just realize the recording and playback equipment uses the same techniques but with SOTA performance and engineering.

It is all a bunch of nonsense which they claim must exist because measurements don't jive with what they think they are hearing.
All it means is that their ears are nowhere as good as measurement equipment and some of those folks have a preference for certain types of 'changes' made to the recorded signal.

Do some audibility tests.
The only caveat is you need top notch headphones/speakers with well performing equipment otherwise you are listening to deficiencies of the transducers.
This is why with some of the online tests you find some people cannot listen past a certain level where at the maximum level there is an unexpected rise in performance.
Are these the golden eared folks ?
No.. they are using good transducers and good equipment.

So do audibility tests but not on average equipment.
This is why I love waterfall decay graphs and off-axis measurements, as these data points are so indicative of a speakers' room interaction.
 

makmeksam

Member
Joined
Nov 13, 2019
Messages
59
Likes
44
Many subjectivists argue that measurements can not capture the audio quality of an AMP or a DAC.

We can accept that there are lot of variables that come into play when measuring a device. Input level, input signal (1kHz sine/CCIF tones/SMPTE tones etc), gain/output level, load resistance are some of them. If we were to capture the distortions with respect to at least the above variables, we have to create at least a 5 dimensional plot. But that is not practical and we never do that.

Because of this limitation can there be things that we miss with measurements but subjectivists can possibly sense by listening? (Consider IMD for an instance. We mostly use two test tones to measure IMD. But real music can be much more complex. So real music may give more distortions even if the measurements of a given device are very good?) Or else can we theoretically argue that measurements can capture the audio quality of an AMP or a DAC with a very good certainty even with the limited amount of measurements that we consider?


The comment a few articles up is quite mistaken as to what we can measure, and how we can evaluate it. It is not hard to do SNR tests and calculate error spectra on real signals. So, of course, we do that. It is also possible to contrive nasty "buzz" tones that have fifty harmonics or so, and that create a setup where EVERY nonlinearity creates a tone at one particular place, too.

Measuring noise floor isn't hard any more.
Measuring useless things like "THD" and "SNR" aren't hard any more, on complex, simple, or any other known signal. Sorry, but that's how it is.

If your stereo's maximum reproduction level is known knowing the all-encompassing SNR and knowing it's below the threshold of hearing really does suffice. You can do all of this with matlab and some simple, short programs, using a super-high quality DAC and ADC.

It's not hard.

No, this isn't a simple THD mesaurement.
With my question I was referring to commonly published measurements (like this or this) of individual devices by manufacturers or independent people like @amirm. Can those measurements accurately predict the performance of the given device when it is connected with other devices with real music as input?

Btw. I am not a subjectivist. I am just trying to figure out the limitations of common measurements if there are any and understand what commonly published measurements of individual devices actually can tell about the device.
 
Last edited:

j_j

Major Contributor
Audio Luminary
Technical Expert
Joined
Oct 10, 2017
Messages
2,267
Likes
4,758
Location
My kitchen or my listening room.
With my question I was referring to commonly published measurements (like this or this) of individual devices by manufacturers or independent people like @amirm. Can those measurements accurately predict the performance of a given device when it is connected with other devices with real music as input?

Aside from power amplifiers and speakers, what would you suspect would go wrong with properly made measurements that have meaning?

Now, simple THD is "mostly useless". Most manufacturer specs are, likewise, "mostly useless". I don't actually know what measurements Amir is making, but I am aware he knows that distortion spectra are required for any meaningful examination, for instance.

The one thing, with modern solid-state equipment, at least, that might be an issue would be power amplifiers interacting with poorly designed speaker impedance. That can be determined for a speaker with a relatively simple measurement, best done at 2 or 3 different power levels. An amplifier can be tested for a given resistive impedance, and likewise measured for phase margin, so an amplifier that isn't off the charts that way should be fine within its current ratings.

But there are some very, very absurd speaker impedances out there. Those should be noted, laid down, and avoided, if you want my opinion. I can easily make a speaker with a well-behaved impedance fairly simply, even with a passive crossover. I won't comment on engineering practices that give an 8 ohm speaker (so called) a 3 ohm mostly inductive impedance at 3kHz or so, except to note that I was impressed, but not necessarily in a positive fashion.
 

makmeksam

Member
Joined
Nov 13, 2019
Messages
59
Likes
44
Aside from power amplifiers and speakers, what would you suspect would go wrong with properly made measurements that have meaning?
Most published measurements including the ones from @amirm have the following measurements in most cases.
  • FFT @ 1kHz sine
  • Frequency response
  • THD+N vs power plot @ 1kHz
  • SMPTE or CCIF plot
  • Output and input impedance
Let’s say there is a device that measures very well below the hearing thresholds with above measurements. Now, can’t someone argue that the device may perform bad if the input signal was 5kHz in THD+N plot? Or it will perform poor with any other input signal other than 1kHz? Or it will perform poor with a different resistive load? I am concerned about any other case that is not tested in the published set of measurements. Is there any theory that says “if a device measures well in above test cases it is very likely to perform well with any other test cases”. Otherwise, what is the purpose of published measurements? Is it just a benchmark war?
 

trl

Major Contributor
King of Mods
Joined
Feb 28, 2018
Messages
1,967
Likes
2,523
Location
Iasi, RO

j_j

Major Contributor
Audio Luminary
Technical Expert
Joined
Oct 10, 2017
Messages
2,267
Likes
4,758
Location
My kitchen or my listening room.
Most published measurements including the ones from @amirm have the following measurements in most cases.
  • FFT @ 1kHz sine
  • Frequency response
  • THD+N vs power plot @ 1kHz
  • SMPTE or CCIF plot
  • Output and input impedance
Let’s say there is a device that measures very well below the hearing thresholds with above measurements. Now, can’t someone argue that the device may perform bad if the input signal was 5kHz in THD+N plot? Or it will perform poor with any other input signal other than 1kHz? Or it will perform poor with a different resistive load? I am concerned about any other case that is not tested in the published set of measurements. Is there any theory that says “if a device measures well in above test cases it is very likely to perform well with any other test cases”. Otherwise, what is the purpose of published measurements? Is it just a benchmark war?


I'm not sure quite what you're asking.

First, a pseudo-periodic set of broadband tones with a common difference ought to be one of the test signals, and the error spectrum from that would be the diagnostic. This would also show frequency response in one way. The use of an allpass to also find frequency response would be a second thing I'd prefer to use.

Power bandwidth is required for amplifiers.

But, THD is "mostly useless", and only at 1kHz does not show the internal gain structure of anything much.
 

j_j

Major Contributor
Audio Luminary
Technical Expert
Joined
Oct 10, 2017
Messages
2,267
Likes
4,758
Location
My kitchen or my listening room.
But AFAIK Amir also provides THD+N vs. frequency as well, for both DACs and amps as well.

Much more useful. I'd also use a broadband signal, a good choice is sine waves at 227+n*500, equal levels, phases scrambled at startup. Use 229+n*500 in the other channel.

You can extend this signal to 20kHz with no trouble, and you'll see a fast result for distortion from a broadband signal immediately at 500Hz. Crosstalk between channels will show some rather odd beating, usually at 2Hz, but can be 1 or 4 if nonlinearity is involved.

This is used into a power resistor for an amplifier. No human in their right mind would ever want to hear this.
 

fredoamigo

Addicted to Fun and Learning
Forum Donor
Joined
Mar 11, 2018
Messages
600
Likes
987
Location
South East France
So a state of the art measurement system that can measure far beyond studio quality ADC and DACs and can expose issues with amps/DACs even below noise levels cannot capture what much lesser studio equipment can when music is recorded, processed, and reproduced ?
The folks making these claims don't realize the 'wonderful recordings' they listen to comes from equipment that performs less than measurement gear.
Just realize the recording and playback equipment uses the same techniques but with SOTA performance and engineering.

It is all a bunch of nonsense which they claim must exist because measurements don't jive with what they think they are hearing.
All it means is that their ears are nowhere as good as measurement equipment and some of those folks have a preference for certain types of 'changes' made to the recorded signal.

Do some audibility tests.
The only caveat is you need top notch headphones/speakers with well performing equipment otherwise you are listening to deficiencies of the transducers.
This is why with some of the online tests you find some people cannot listen past a certain level where at the maximum level there is an unexpected rise in performance.
Are these the golden eared folks ?
No.. they are using good transducers and good equipment.

So do audibility tests but not on average equipment.

I like this test. Who's tried it? https://hearingtest.online/
 

makmeksam

Member
Joined
Nov 13, 2019
Messages
59
Likes
44
But AFAIK Amir also provides THD+N vs. frequency as well, for both DACs and amps as well.
In THD+N vs. frequency plot only one output level is used. So, I guess, someone can argue that the THD+N vs. frequency curve will be completely different for a different output level etc.
 
Last edited:

Serge Smirnoff

Active Member
Joined
Dec 7, 2019
Messages
240
Likes
136
In df-metric I also stumbled upon the question of audibility of distortions. I think my findings could be helpful for this discussion.

My current working definition of the threshold of distortion audibility (s-level in df-metric) is based on the statement, which seems to me reasonable but I can't prove it; so, it's my initial assumption:

If a sound of some low pressure level is inaudible then it doesn't change perception of any other sound, to which it is added.

Literally:
0dB(SPL)sound1 + XdB(SPL)sound2 = XdB(SPL)sound2

which means that everything below 0dB(SPL) doesn't exist for the hearing and distortion can safely reside there imperceptibly.

In order to map the 0dB(SPL) to the voltage of a real signal (which will be inaudible), we need the one more parameter - max SPL of the listening environment where that signal will be auditioned. Yes, the threshold of distortion perception depends on dynamic range of a listening environment. All distortions of a music signal that are beyond that dynamic range are inaudible. In other words, in order a difference signal to be imperceptible it should be below a noise floor of listening environment. For example listening of portable players has noise floor around 30dB(SPL) [bedroom at night time] and max recommended sound pressure level for the case is 85dB. Dynamic range of this listening environment is 55dB. Hence, any uncorrelated difference signal below -55dB will be inaudible (-55-3.01=-58.01dB to be correct).

If somebody uses noise canceling headphones with noise floor around 0dB(SPL) and max SPL around 100dB then imperceptible level of difference signal is -103.01dB. Using of IEMs (or any efficient noise canceling headphones) requires the most accurate sources of signal. The extreme (the lowest possible) level of distortion perceptible by humans then is defined by the max dynamic range of hearing, which is 139dB [-9dB(SPL) -- 130db(SPL)]; its Df = -142dB (hardly practical value though )).

(Note1) The thresholds (s-levels) defined this way are absolute - the type of difference signal doesn't matter, it will be inaudible in any case. But there are cases where a DUT can reach transparency with higher levels of difference signal. In such cases the diff.signal turned out to be imperceptible due to features/bugs of human hearing system. Psychoacoustic encoding is a good illustration of this phenomenon. Df levels with such encoding are relatively high [-40dB, Lame V0] but still inaudible thanks to special “distortion shaping” accounting numerous peculiarities of our hearing. Many other techniques of hiding distortion (or making it more pleasant) are used in audio industry with more or less success (because such hiding depends on music material).

(Note2) For simplicity I do not consider accuracy of transducers here but it is not hard to expand this approach to them as well.

Thus, my current working equation for defining s-level in df-metric for a particular listening environment:

S-level = -DynamicRange - 3.01 [dB]
 

j_j

Major Contributor
Audio Luminary
Technical Expert
Joined
Oct 10, 2017
Messages
2,267
Likes
4,758
Location
My kitchen or my listening room.
It is necessary to consider the spectrum of the noise, not just the overall level. a 2khz sine wave at 2dB SPL is much different than 2dB SPL white noise.
 

Serge Smirnoff

Active Member
Joined
Dec 7, 2019
Messages
240
Likes
136
Yes, we can consider whatever peculiarities of our hearing if we think they are relevant to the task. And what is the task? I see it as developing a measurable criterion, which guarantees that a DUT is sonic-transparent for 99.7% of population. In other words to define a class of audio equipment that would be accurate enough not to be discussed in terms of psychoacoustics. Exactly like nobody discuss lossless codecs. Bit-perfectness in analog audio is waveform-perfectness. To define reliable standard(s) for the latter is not hard indeed. If my approach to defining threshold(s) of distortion audibility is correct then the only questionable issue is defining correct noise floors for various listening environments. I think prog. simulation noise is the best signal for the purpose. The measurements with sine signals will show lower SPL values (which in turn will result in more strict requirements to accuracy of audio equipment) but they are irrelevant for the case as difference signal (distortion) is always a wide-band noise-like signal.
 
Top Bottom