• WANTED: Happy members who like to discuss audio and other topics related to our interest. Desire to learn and share knowledge of science required. There are many reviews of audio hardware and expert members to help answer your questions. Click here to have your audio equipment measured for free!

Audibility thresholds of amp and DAC measurements

Serge Smirnoff

Active Member
Joined
Dec 7, 2019
Messages
240
Likes
136
Here is the problem... there is no 'we'. There are many we. All of them ask (not demand) different things from manufacturers and some of them are happy to oblige. Some manufacturers cater for audiophools, others for audiophiles, others like to play the measurement game and yet others follow their own path, often supported by enough sales.
So, we, audio consumers, have a problem. Audio manufacturers know much more about real performance of their products than consumers. This, along with absence of clear audio metric motivates manufacturers to sell normal audio products as exceptional ones (profit). Most of the "work" then is done by marketing departments, which are very active now on various forums (headfi as an example) where they actually maintain some “safe” level of misinformation of their customers, participating in customers' layman technical discussions based on misconceptions. In other words manufacturers learned to form/control the DEMAND as well. Resulting prices on the audio market are no longer the result of consensus between demand and supply - they are dictated/controlled by the supply side. Customers have no impact on what is manufactured and how because they are misinformed and can not send clear signal to the supply side. That is called manufacturer-driven market. [my article about this from 2007 - http://soundexpert.org/articles/-/blogs/the-1st-of-april-audiophiles-day]

I propose a solution for this problem.
 
Last edited:

MC_RME

Addicted to Fun and Learning
Technical Expert
Audio Company
Joined
May 15, 2019
Messages
854
Likes
3,564
I seriously doubt that a DF metric would convince any of the typical 'marketing infected' customers to no longer believe that a device can sound 'better' than another one, although they have the same DF value.
 

solderdude

Grand Contributor
Joined
Jul 21, 2018
Messages
15,891
Likes
35,912
Location
The Neitherlands
You can choose on this page the devices of your interest, I will compute df levels for them from supplied recordingd. Also you can do this yourself but I'm ready to help.

How do you separate the source from the ADC when it comes to the df ?
 

solderdude

Grand Contributor
Joined
Jul 21, 2018
Messages
15,891
Likes
35,912
Location
The Neitherlands
So, we, audio consumers, have a problem.

I don't think so. Consumers have a choice. They can buy whatever they want based on whatever measurements are available or (worthless) subjective reviews. Even df metrics if they so desire. Whatever they believe in.

What is your proposed solution ?
 

Serge Smirnoff

Active Member
Joined
Dec 7, 2019
Messages
240
Likes
136
How do you separate the source from the ADC when it comes to the df ?
My current solution is to use much better (-12dB) audio interface for df measurements. After some low level accuracy all measurement interfaces produce the same df levels for DUTs. DAPs are pretty bad DUTs and can be measured with just good interface+ADC. Can be discussed/researched further.
 

Serge Smirnoff

Active Member
Joined
Dec 7, 2019
Messages
240
Likes
136
I seriously doubt that a DF metric would convince any of the typical 'marketing infected' customers to no longer believe that a device can sound 'better' than another one, although they have the same DF value.
It is not easy and it is a matter of time and consumer education. Somebody has to start the process.


Consumers have a choice. They can buy whatever they want based on whatever measurements are available or (worthless) subjective reviews.
Consumers can not take advantage of their possibility to choose as they are full of false ideas prepared by marketers. So, step by step, it is a continuous process.

What is your proposed solution ?
In short: to develop a standard for audio quality on consumer side. It will be based on slightly inflated numbers for accuracy compared to the current ones on the market.

In details: market of DAPs is the best to start with; I will show how to increase the role of customers and motivate manufacturers to produce real-HQ products on this market. Df metric is designed for the purpose.

1. We need a picture of accuracy of current best players on the market. Df measurements of a few dozens of DUTs will be enough to understand the variance of the parameter among them.
2. To find theoretical value(s) of s-level for the market.
3. Based on both the theoretical values and the real ones it will be easy to determine the required/sufficient level of accuracy with m-signal for DAPs.
4. Communicate to the manufacturers the found s-level and procedure of its measurement.

As I already have a lot of various df measurements I expect the following results. Normal-like distribution of df levels among DAPs with the best ones showing Df = [-40dB -45dB]. Theoretical value I expect within [-50dB -60dB] range. Resulting accuracy demand could be around -55dB. Then we, most educated and organized audio consumers from ASR forum/SEproject, recommend manufacturers to produce audio players, having df level with m-signal (or power simulated noise) below -55dB. As there is a lot of complications in assessing various types of distortion and their impact on perceived audio quality we decided to establish this strict and pure technical parameter for accuracy of m-signal which guarantees transparent operation of the player for majority of population. All other solutions result in marketing speculations and increase prices for DAPs. Also we suggest all music lovers who care about SQ to join us in our efforts to clean the DAPs market from unfair manufacturers and to buy DAPs operating below the found s-level. What can manufacturers answer to this demand? - Your standard is too high and lower values are sufficient for you?
 

solderdude

Grand Contributor
Joined
Jul 21, 2018
Messages
15,891
Likes
35,912
Location
The Neitherlands
Consumers can not take advantage of their possibility to choose as they are full of false ideas prepared by marketers. So, step by step, it is a continuous process.

Audio worlds consists largely of false ideas (seen from measurement side).
One would have to 'forbid' all subjective tests and magazines/websites. @Thomas savage is going to have to boot currently half of the ASR members playing with subjective thoughts or that love to listen to vinyl. How will one Df vinyl ?

Education of the public is impossible. Even in a heavily censored police state this is impossible because you cannot force people in thoughts that aren't theirs. NwAvGuy, ASR and quite a few others have been educating folks and almost daily new folks register here to tell people that sighted subjective listening is the holy grail because measurements say nothing.

I hope you don't mind if I wait till the Df metric has been proven to be valid (and what the magic number is below a certain threshold) to be audible transparent and what exactly the pitfalls of the metrics are.
It would be neat if one number actually could correlate with how people perceive the device in blind conditions under actual loads (how do you plan this with speaker amps and the infinite speaker combinations and at different volume levels ?)
 

Serge Smirnoff

Active Member
Joined
Dec 7, 2019
Messages
240
Likes
136
How will one Df vinyl ?
)) If the sourse music material is digital and published also on Vinyl, then it is just a combination of various types of distortion, creative listening, no need to measure it. But if one really measure it, df level will be around, say, -15dB and probably artifact signature will be far from all others, so that df level (-15dB) will have no sense. But when another Vinyl with the same music be added, df values of both cases will be indicative because artifact signatures of both Vinyls will be close.

If Vinyl is the only source then it is a reference itself, nothing to measure as well. So, it depends on what question we are going to answer.
 

Serge Smirnoff

Active Member
Joined
Dec 7, 2019
Messages
240
Likes
136
I hope you don't mind if I wait till the Df metric has been proven to be valid (and what the magic number is below a certain threshold) to be audible transparent and what exactly the pitfalls of the metrics are.
Instead of waiting you can participate in the process of proving/disproving the new metric. After collection of data (1. and 2.) it will be quite obvious whether df-metric works or not. If there is no consensus about validity at this point then experiment will be finished. No animals will be killed during the try.
 

solderdude

Grand Contributor
Joined
Jul 21, 2018
Messages
15,891
Likes
35,912
Location
The Neitherlands
What's the point in guessing where Df would be. There are lots of alterations to the signal before it can be written in vinyl plus endless variances in cartridges and pre-amps in so many aspects.
Having a CD with a possibly same master won't be a good reference for sure.
Just admit that you cannot Df this ... ;)
Yet it can sound excellent.
 

Serge Smirnoff

Active Member
Joined
Dec 7, 2019
Messages
240
Likes
136
Just admit that you cannot Df this ... ;)
Yet it can sound excellent.
I cannot. Fortunately there is no need to measure Vinyl. For me Vinyl is a variant of creative listening and it can sound excellent (especially for those who started listening music in Vinyl era).
 

solderdude

Grand Contributor
Joined
Jul 21, 2018
Messages
15,891
Likes
35,912
Location
The Neitherlands
Instead of waiting you can participate in the process of proving/disproving the new metric. After collection of data (1. and 2.) it will be quite obvious whether df-metric works or not. If there is no consensus about validity at this point then experiment will be finished. No animals will be killed during the try.

I have done a fair amount of electrical nulling (not possible with DACs) and know the limits of nulling.
For myself I can deduct enough from blind listening tests I have done with a few DAC's and reading a suite of plots.
I can't hear differences between DAC's that measure differently (just not in FR, no broken DACs were ever used by me) that even when all DACs would (and will) measure differently and have different Df to me there is no relation to sound in level matched blind tests.
So I am far from the perfect candidate to help.

My interest in transducers as that's where the real audible differences are found. To limit myself I just concentrate of headphones as speakers/rooms is quite another level.

To me the electronic side is already more than transparent and 'fixed' for many many years and no more gain can be had sonically beyond a certain point. If manufacturers want to sell excellent or broken devices I don't care and am not on a crusade either.
I very well cannot tell others not to like broken DACs or amps with a deliberate 'character' just because they don't measure near perfect or have poor Df metrics.

I am quite sure if I were to dedicate myself to something like this (proving metrics to really say something about perceived sound quality) I would need to quit my job and dedicate many years and shitloads of money, organize blind test panels and keep folks interested.

I do wish you and others participating lots of fun with the project though and hope something comes out of it.
 

pkane

Master Contributor
Forum Donor
Joined
Aug 18, 2017
Messages
5,632
Likes
10,205
Location
North-East
I have done a fair amount of electrical nulling (not possible with DACs) and know the limits of nulling.
For myself I can deduct enough from blind listening tests I have done with a few DAC's and reading a suite of plots.
I can't hear differences between DAC's that measure differently (just not in FR, no broken DACs were ever used by me) that even when all DACs would (and will) measure differently and have different Df to me there is no relation to sound in level matched blind tests.
So I am far from the perfect candidate to help.

My interest in transducers as that's where the real audible differences are found. To limit myself I just concentrate of headphones as speakers/rooms is quite another level.

To me the electronic side is already more than transparent and 'fixed' for many many years and no more gain can be had sonically beyond a certain point. If manufacturers want to sell excellent or broken devices I don't care and am not on a crusade either.
I very well cannot tell others not to like broken DACs or amps with a deliberate 'character' just because they don't measure near perfect or have poor Df metrics.

I am quite sure if I were to dedicate myself to something like this (proving metrics to really say something about perceived sound quality) I would need to quit my job and dedicate many years and shitloads of money, organize blind test panels and keep folks interested.

I do wish you and others participating lots of fun with the project though and hope something comes out of it.

I've found the DF metric, as well as the DeltaWave rms null metric, to be useful as a better 'engineering' metric rather than one used to judge subjective sound quality. To me, both are better in quantifying the accuracy of reproduction/recording than SINAD, THD or SNR metrics, since they include a lot more of the possible distortions in the computation. Since they both use real music signal to compute the metric rather than test tones, this has a better chance of capturing unexpected behavior at different frequencies or different levels. Here's a recent comparison I ran using DeltaWave for someone who was in the market for a decent audio interface using waveform captures posted on the Gearslutz DAC/ADC loopback thread:

1579453701149.png


The RMS difference in the last column indicates the RMS level of the error signal, in dBFS. This is with a 24/44.1kHz recording. The larger the Correlated Depth, the better time/phase match between the two files.
 

solderdude

Grand Contributor
Joined
Jul 21, 2018
Messages
15,891
Likes
35,912
Location
The Neitherlands
Yes, I agree with such a metric (or single number) to show more than just a SINAD. I don't think a SINAD number is a valid number for sound quality.
Nor THD or multitone nor just squarewave response or frequency response.
One must look at the entire measurement suite to be able to tell something about the level of fidelity/signal integrity.

The 'danger' of nulling lies in the fact that benign phase or amplitude differences at the extremes of the audible spectrum have too much influence on the actual 'number'.
Of course when one has the input signal and output signal one can analyze what is amplitude related and maybe phase related and compensate for this (partially) so you can get a 'better' number.

I would not easily say though that the number generated is closer to perceived sound quality (as originally claimed by Serge).
What Df would be considered 'transparent' ? Can one have 2 different sounding devices with a similar Df but clearly different 'classic' measurements ?

It is a fun exercise and can generate a certain number using different stimuli (which Serge has shown). The amount of work in it is impressive.
The conclusions drawn may need some more thought and usability as well as correlation with properly tested observations is needed.
Certainly when it comes to audibility thresholds (what this thread is about).
 

Blumlein 88

Grand Contributor
Forum Donor
Joined
Feb 23, 2016
Messages
20,523
Likes
37,056
I've found the DF metric, as well as the DeltaWave rms null metric, to be useful as a better 'engineering' metric rather than one used to judge subjective sound quality. To me, both are better in quantifying the accuracy of reproduction/recording than SINAD, THD or SNR metrics, since they include a lot more of the possible distortions in the computation. Since they both use real music signal to compute the metric rather than test tones, this has a better chance of capturing unexpected behavior at different frequencies or different levels. Here's a recent comparison I ran using DeltaWave for someone who was in the market for a decent audio interface using waveform captures posted on the Gearslutz DAC/ADC loopback thread:

View attachment 46520

The RMS difference in the last column indicates the RMS level of the error signal, in dBFS. This is with a 24/44.1kHz recording. The larger the Correlated Depth, the better time/phase match between the two files.
Why the big difference in the M4 and the M2 from Motu? I know you used someone else's captures so maybe you can't say.
 

pkane

Master Contributor
Forum Donor
Joined
Aug 18, 2017
Messages
5,632
Likes
10,205
Location
North-East
Why the big difference in the M4 and the M2 from Motu? I know you used someone else's captures so maybe you can't say.

Don't know. It's possible that the M2 capture was under less than ideal conditions or settings. The owner's first attempt at capture produced a much better (but still not perfect) result. This turned out to be using a fully digital path -- with no analog loopback :) If the digital capture wasn't perfect, perhaps there was something set in the audio path that altered the signal (some filter, mixer, or an effect). M4 capture by another owner does appear very significantly better. That's the danger of using someone else's captures for analysis -- one doesn't always know if these were done properly or under what conditions/settings.
 

pkane

Master Contributor
Forum Donor
Joined
Aug 18, 2017
Messages
5,632
Likes
10,205
Location
North-East
One must look at the entire measurement suite to be able to tell something about the level of fidelity/signal integrity.

In my opinion, this is where the DeltaWave null actually does provide a benefit. The null isn't just captured but also analyzed to produce various metrics that tell the large part of the story of what is or isn't right with the DUT.

The 'danger' of nulling lies in the fact that benign phase or amplitude differences at the extremes of the audible spectrum have too much influence on the actual 'number'.
Of course when one has the input signal and output signal one can analyze what is amplitude related and maybe phase related and compensate for this (partially) so you can get a 'better' number.

Well, that's what DeltaWave excels at, IMHO :) Finding, measuring, and eliminating amplitude and phase differences that are not important for a listener, but are significant when producing a null.

The conclusions drawn may need some more thought and usability as well as correlation with properly tested observations is needed.
Certainly when it comes to audibility thresholds (what this thread is about).

Agreed. This is the reason I don't normally push DeltaWave single metric numbers as the be-all measure of a DUT, as I don't believe these are descriptive enough. That's why there are many tools built-in to DW, including audibility testing using ABX, a blind preference test, as well the ability to listen to the error signal amplified by up to 100dB to determine audibility. And even a frequency shifter capability that can let you listen to the ultrasonic content by shifting it into the audible range -- all so one can decide for themselves if the differences are significant, audible, and could potentially cause problems in playback equipment. Also available are a number of visual tools, from actual waveform to spectrum to cepstrum to clock drift visualization plots and even a lissajous and linearity.
 
Last edited:

Blumlein 88

Grand Contributor
Forum Donor
Joined
Feb 23, 2016
Messages
20,523
Likes
37,056
Just to add to your data. I ran Deltawave on the Gearslutz loopback again using the latest version on the Zen Tour.
Difference RMS 88.29 and correlated null 91.65 db. No weighting. Switching from Kaiser to Blackman_Harris 7 on the FFT window improved results slightly.

Focusrite Scarlett 18i20 1st gen.
Difference RMS 70.92, and corr. null 73.51 db.
 
Last edited:

pkane

Master Contributor
Forum Donor
Joined
Aug 18, 2017
Messages
5,632
Likes
10,205
Location
North-East
Just to add to your data. I ran Deltawave on the Gearslutz loopback again using the latest version on the Zen Tour.
Difference RMS 83.08 and correlated null 89.33 db. No weighting.

Focusrite Scarlett 18i20 1st gen.
Difference RMS 70.38, and corr. null 73.51 db.

I discussed updating the results on the Gearslutz thread with Didier. He is reluctant to change the existing data or to add to it because he thinks it will confuse readers even more.

Maybe it’s worth while to create a DeltaWave DAC/ADC loopback results thread so these can be compared. Maybe hosted here, on ASR or on Gearslutz. Of course, I’d ask Didier for permission before doing this.
 
Top Bottom