Alternative method for measuring distortion

[solderdude]

Thank you for your support. I need it now as never before.

When you start to view comments of others not as negative and 'bashing' you might see you have been getting support but not in a form you would want it to be (praise only).
Instead of calling people like J_J names you could also reach out to people like him who dedicated a big portion of their lives dealing with perception and measurements.

Maybe it's your wording, maybe its your line of thinking that differs. We are all looking for the same thing here.

Be like a sponge not a firewall that only wants to hear/see their side. There are enough of those on both sides of the fence.
Learn from both sides.

 

[Serge Smirnoff]

The limitation of such a metric is that the distance between any two measurements is not tied to the perception of such a difference. You stated a df audibility threshold number of -50dB as I recall. The problem is, df doesn't tell me if a distortion with a df metric of -35dB is just as inaudible, or much more audible, or if I can even tell the difference between -35dB and -45dB. As far as I can tell, DF doesn't provide any guidance on relative audibility of the error/distortion it measures because it's not tied to audio perception. I think this is why j_j and others were objecting to your definition.

Would be great if there were a link with subjectively perceived sound quality.

Exactly. Df level is a very simple objective parameter. And though it correlates to perceived audio quality better than any other measurable value, it can not be used for the purpose alone. It needs to be connected with perception in some way. I proposed two such connection mechanisms: (1) by means of listening tests [http://soundexpert.org/documents/10179/11017/se_igis.pdf] and (2) by means of comparison of artifact signatures [#55]. Without connection to perception df levels are useless for reliable measurements of perceived audio quality. I think it is absolutely clear, I say this every time when I describe df-metric (last time here and here). The main idea is different - to make df level low enough to exclude the necessity of researching distortion completely. And df-metric offer the methods for discovering such low levels - in theory and by measurements. These low levels depend on listening environments [#177]. Df level -50dB is for mass portable audio market. For other listening environments it could be, say, -70dB for critical listening environments, -90dB for studio equipment, -120dB for laboratory equipment (just rough estimations as an example). The solution is to make them very low, where audibility of distortion does not matter.

 

[Serge Smirnoff]

When you start to view comments of others not as negative and 'bashing' you might see you have been getting support but not in a form you would want it to be (praise only).
Instead of calling people like J_J names you could also reach out to people like him who dedicated a big portion of their lives dealing with perception and measurements.

Maybe it's your wording, maybe its your line of thinking that differs. We are all looking for the same thing here.

Be like a sponge not a firewall that only wants to hear/see their side. There are enough of those on both sides of the fence.
Learn from both sides.

I do not need "prise" for sure. I know very well that while support creates comfortable psychological environment for a research, only opposing moves it further. Actually j_j helped me a lot in this sense. The problem with the discussion of my df-metric is different. What I offer is effectively cancel most of their ("people like him who dedicated a big portion of their lives dealing with perception and measurements") past research efforts. I propose to solve the problem of audio quality on syntactic level by pure engineering methods. Such radical change of the audio paradigm makes most of their knowledge of perception not important. There is very little hope for their support in the case. My only hope is that, as you said, we are all looking for the same thing here - to have perfectly sounding devices at reasonable/cheap prices. And my research clearly shows to me that this is easily doable. But for the purpose the old paradigm should be dropped.

 

[solderdude]

Needless to say you will need to 'disect' the diff file and isolate the benign differences and the sound degrading differences. Only then you can create a 'number' relating to sound quality. Only when this is done and people accept the generated values you will get the recognition you seem to desire.
Thus effectively split the diff file up compared to the original signal and add some weighing to come up with something useful.
For this you need knowledge about perception and split the diff file up in aspects.
The "people like him who dedicated a big portion of their lives dealing with perception and measurements" already researched those aspects and wisely used different analysis methods to look at the different aspects separated instead of measuring them as a whole and pick apart aspects later.

You are both looking for similar things using different methods. Both of you compare the original signal to the resultant and try to find relations between the measurements and what is perceived. One measures aspects separately and looks at the whole picture. You do one measurement and need to pick apart the aspects and understand their influence on the perecieved SQ.

I applaud your wish and efforts to use a much simpler measurement method if that is possible using software and a soundcard.
IMO you are still far removed from linking a generated number to a preferred sound quality.
Just like Olive his research in preferred sound signatures which obviously deviate from flat.
So are there also aspects in the diff file that may be more or less objectionable and maybe are even preffered.

"people like him who dedicated a big portion of their lives dealing with perception and measurements" already paved the way. Instead of alienating them I suggest you make use of the years of accumulated wisdom (the word knowledge may not cover it) and choose your words carefully instead of calling "people like him who dedicated a big portion of their lives dealing with perception and measurements" whatnot.

 

[Serge Smirnoff]

Thus effectively split the diff file up compared to the original signal and add some weighing to come up with something useful.

That is exactly what I want to avoid. If you have, say, accuracy of -100dB with real signal (or simulated one) there is no need to split anything. My real-world measurements show that required numbers of the accuracy are much higher, not -100dB. This is the essence of singularity point - we just need to find the level of accuracy which does not require further distortion analysis in regards to its audibility.

 

[Serge Smirnoff]

Just like Olive his research in preferred sound signatures which obviously deviate from flat.
So are there also aspects in the diff file that may be more or less objectionable and maybe are even preffered.

Such "preferred" distortions should be added on top of transparent audio path. Just like in photo. Analog films, each had its visual "signature". Today sensor matrices have no that personal touch, it is applied later during processing.

 

[pkane]

That is exactly what I want to avoid. If you have, say, accuracy of -100dB with real signal (or simulated one) there is no need to split anything. My real-world measurements show that required numbers of the accuracy are much higher, not -100dB. This is the essence of singularity point - we just need to find the level of accuracy which does not require further distortion analysis in regards to its audibility.

You can't avoid psychoacoustics, even with DF. When you are trying to determine the threshold of DF where the distortions become inaudible, you'll need to deal with the audibility of various distortions. In order for the DF metric (and whatever threshold you pick) to become accepted widely, you'll need a psycho-acoustic proof of its effectiveness. This requires a lot of testing with large number of samples, variously mixed distortions, different DUTs and a large number of listeners. Until you do this (and most likely, after) people will continue to remain skeptical that the result is as universal as you claim. Even those, like me, who are very much a believer in measuring the error signal

 

[solderdude]

That is exactly what I want to avoid. If you have, say, accuracy of -100dB with real signal (or simulated one) there is no need to split anything. My real-world measurements show that required numbers of the accuracy are much higher, not -100dB. This is the essence of singularity point - we just need to find the level of accuracy which does not require further distortion analysis in regards to its audibility.

I am not talking about -100dB as we all know this is far below any audible limits... I am talking measured -35 where part of it is inaudible so that number should be -50dB or so when it comes to audible effect distance.

 

[solderdude]

Such "preferred" distortions should be added on top of transparent audio path. Just like in photo. Analog films, each had its visual "signature". Today sensor matrices have no that personal touch, it is applied later during processing.

What if someone added it in the device (say a Pass amp) that measures 'bad' (and thus nulls bad) yet many folks like the sound ? Then where is the relation and how can you justify the generated number in that case. This is the hard part IF you want to come up with a new metric with high correlation to perceived SQ. Of course there will be a certain correlation to measured metrics but I thought your goal was perceieved sound correlation.

 

[Serge Smirnoff]

You can't avoid psychoacoustics, even with DF. When you are trying to determine the threshold of DF where the distortions become inaudible, you'll need to deal with the audibility of various distortions. In order for the DF metric (and whatever threshold you pick) to become accepted widely, you'll need a psycho-acoustic proof of its effectiveness. This requires a lot of testing with large number of samples, variously mixed distortions, different DUTs and a large number of listeners. Until you do this (and most likely, after) people will continue to remain skeptical that the result is as universal as you claim. Even those, like me, who are very much a believer in measuring the error signal

I can tell you how to avoid psychoacoustics )) The threshold should be set not according to audibility of various distortions but according to absolute threshold of audibility for human hearing.

I am not talking about -100dB as we all know this is far below any audible limits...

... and at this df level psychoacoustics is irrelevant. Even more, it becomes irrelevant at much higher df levels. These df levels of irrelevancy depend on dynamic range of listening environment. For the case of IEMs in unechoic chamber )) theoretical safe df level can really be -103dB (according to absolute threshold). In other cases it is much higher. Ok, lets define it -100dB. Do you think that it is not possible to manufacture such devices or that this will be economically inefficient? What do you want to save/economize? Why are you afraid to set really high (overkill) standard? And, instead, prefer to set lower standard but perform a huge amount of really complicated and expensive research of audibility of distortion?

 

[Serge Smirnoff]

What if someone added it in the device (say a Pass amp) that measures 'bad' (and thus nulls bad) yet many folks like the sound ? Then where is the relation and how can you justify the generated number in that case.

For sure some folks will love some distortion, others will love another one. Some distortion will be pleasant with one type of music, another with another. Also some folks sometimes change their minds and listening tastes with age or due to change of a lifestyle ..... This is our reality and in such reality any audio standards are impossible as well as any objective audio metrics, because it is an area of creative listening. In such a mess the only reliable reference point is the waveform of an audio signal. The standard for its accuracy should satisfy all the cases above including critical listening. On top of this one can add whatever he wants. Some folks will add vinyl distortion and will listen all their tracks in this mode - no problem, transparent chips will be cheap anyway because they will be a mass product (commoditization).

 

[Serge Smirnoff]

In the end the question is asked in this way: why to have expensive audio of mediocre quality when it is possible to have cheap and perfectly transparent one? Looks like a joke isn't it? And how many efforts is required to convince people to choose the second!!! ))

 

[pkane]

I can tell you how to avoid psychoacoustics )) The threshold should be set not according to audibility of various distortions but according to absolute threshold of audibility for human hearing.

So how do you explain this threshold choice to those who claim that they can hear differences below -200dB? It's not that I even remotely believe Rob's claims, but this is the reality of the industry that you are trying to convince to use the DF metric. Here's Rob Watts, of Chord Audio:

The problem with noise floor modulation is the ear/brain is extremely sensitive to it, and certainly can detect levels of noise floor modulation that is below the ability to measure. My own tentative conclusions (or rule of thumb) are that one can hear levels of noise floor modulation down to -200dB - currently we can measure noise floor modulation at -180 dB, and Dave has zero measured noise floor modulation. In terms of SQ, if noise floor modulation is say around -120 dB (typical class D) you get considerable hardness and glare; at -140 dB its grain in the treble; below -160 dB then things sound much smoother with better instrument separation and focus. This continues until about -200dB (and perhaps even lower - reducing RF noise is not something that has an acceptable limit).

 

[solderdude]

transparent chips will be cheap anyway because they will be a mass product (commoditization).

These already exist. Atom and Heresy show that this is possible for $ 100.- and is even manufactured in the US. Chinese companies will be able to produce it even cheaper. One can also build cheap and well performing DACs.
It isn't a problem at all.
It's the people buying crap that need education. Manufacturers just cater for them. DF nor measurements cannot convince those that have their own 'belief' and manufacturers will always make stuff these people want. They sure as hell do not want -100dB nulling devices because it is not what they want.

The standard for its accuracy should satisfy all the cases above including critical listening.

It doesn't now and never will. There are far far more crappy devices than near perfect devices. The vast majority of people wanting to buy devices do so on looks, price, functionality and positive reviews or dealers demoing and telling them what to buy.
Only very few people really care for perfect waveform reproduction. For them devices already exist in all price ranges.
The rest really, really does not give a damn about any measurements.

I understand your crusade and frustration.

 

[pkane]

... and at this df level psychoacoustics is irrelevant. Even more, it becomes irrelevant at much higher df levels. These df levels of irrelevancy depend on dynamic range of listening environment. For the case of IEMs in unechoic chamber )) theoretical safe df level can really be -103dB (according to absolute threshold). In other cases it is much higher. Ok, lets define it -100dB. Do you think that it is not possible to manufacture such devices or that this will be economically inefficient? What do you want to save/economize? Why are you afraid to set really high (overkill) standard? And, instead, prefer to set lower standard but perform a huge amount of really complicated and expensive research of audibility of distortion?

So, how do you know that -100dB (as an example) is achievable with current electronics? I can measure DF using DeltaWave for various pro audio interfaces in loopback mode, including some well-measuring and considered transparent. Here's an example of df computed for an RME ADI-2 Pro:

Assuming this is accurate, it doesn't seem all that great, does it? Is this audible?

If we now correct the non-linear phase differences due to filter response (less than 8 degrees maximum deviation below 500Hz), the result is very different:

Looks like this is telling me that a small 10 degree phase droop below 500Hz is worth a whopping 38dB difference in the DF metric? How do you explain this? Here's the phase difference (blue=original, pink=after DeltaWave corrections):

Adding delta spectrogram (diffogram):

 

[Serge Smirnoff]

So how do you explain this threshold choice to those who claim that they can hear differences below -200dB? It's not that I even remotely believe Rob's claims, but this is the reality of the industry that you are trying to convince to use the DF metric. Here's Rob Watts, of Chord Audio:

It's the people buying crap that need education. Manufacturers just cater for them. DF nor measurements cannot convince those that have their own 'belief' and manufacturers will always make stuff these people want. They sure as hell do not want -100dB nulling devices because it is not what they want.

Yes, I know about these claims. For those people Rob Watts will manufacture devices with -200dB accuracy. That simple ))

 

[Serge Smirnoff]

It doesn't now and never will. There are far far more crappy devices than near perfect devices. The vast majority of people wanting to buy devices do so on looks, price, functionality and positive reviews or dealers demoing and telling them what to buy.
Only very few people really care for perfect waveform reproduction. For them devices already exist in all price ranges.
The rest really, really does not give a damn about any measurements.

I understand your crusade and frustration.

May be we should stop all research and measurements? What the audio science and this forum are for then? Good questions BTW. We can ask @pkane: why you develop your application, what will be the result of research of audibility of distortion? Some audio metric? What is your goal, pkane?

 

[Serge Smirnoff]

Looks like this is telling me that a small 10 degree phase droop below 500Hz is worth a whopping 38dB difference in the DF metric? How do you explain this? Here's the phase difference (blue=original, pink=after DeltaWave corrections):

I explain this as follows: indeed, some phase differences can spoil df values, I propose to ignore this. There will be some overkill. FLAC is also produce heavier files in comparison with, say, AAC@640kbit/s, but people prefer FLAC because they do not want to deal with the uncertainty. "Lossless" analog audio could be the same - you just sure that it is transparent.

 

[Serge Smirnoff]

So, how do you know that -100dB (as an example) is achievable with current electronics?

These already exist. Atom and Heresy show that this is possible for $ 100.- and is even manufactured in the US. Chinese companies will be able to produce it even cheaper. One can also build cheap and well performing DACs.
It isn't a problem at all.

On that page with DiffMaker samples the best one has df=-70dB:

Forssell DAC from AD245(master) bis -20 dbu.flac_cut.wav(44)__ref_diffmaker.flac(44)__mono_400-74.8529-69.9477-54.6983

Manufacturers never designed their devices for the goal of increasing accuracy with real signal. Nobody asked them for this.

 

[Serge Smirnoff]

The vast majority of people wanting to buy devices do so on looks, price, functionality and positive reviews or dealers demoing and telling them what to buy.

DF nor measurements cannot convince those that have their own 'belief' and manufacturers will always make stuff these people want.

All those audio magazines and sites with reviews exist on the money from manufacturers, who "educate" customers and create their wishes/beliefs. If manufacturers managed to control the market, why customers can not do the same? Only customers themselves can change the game, nobody else. Somebody has to start. BitTorrent protocol changed the game on the music market. Audio metric could do the same on the market of consumer audio.