Speaker Equivalent SINAD Discussion

[MZKM]

I thought I'd create a thread where suggestions/comments can be made on if Amir were to rank speakers based on measurements, how would he do so.

As of right now, since Amir wants his rating to be based on listening tests and not just opinions, Sean Olive's Predicted Preference rating is likely what will be used for now as it is decently accurate (probability 0.86). I think before suggesting alterations/alternatives, that we should first understand Olive's algorithm; he has filed a patent for it and is available as a searchable PDF (warning: may download instead of view).

Please bear with me.
His algorithm is such:

Here is it as a percentage:

NBD (Narrow Band Deviation): Average Narrow Band Deviation (dB) in each 1/2-octave band from 100 Hz-12 kHz

NBD_ON: On-axis​

NBD_PIR: Predicted In-room Response​

• y-bar is the average amplitude value within the 1/2-octave band n
• y-sub(b) is the amplitude value of band b within the 1/2-octave band n
• N is the total number of 1/2-octave bands between 100Hz-12kHz
• NBD can be a good metric for detecting medium and low Q resonances.

​

​

LFX (Low Frequency Extension): Log10 of the -6dB point (below 300Hz) in the Sound Power curve, relative to average Listening Window (300Hz-10kHz) SPL.

Easy to understand; turning this into a formula:​

​

As noted, for rear-firing speakers, making the -6dB point in the Sound Power curve be relative to the average Sound Power SPL may be better than the average Listening Window SPL.​

SM_PIR (Smoothness of Predicted In-room Response): Smoothness in SPL based on a linear regression line (at least 1 square error) thru 100Hz-16 kHz.

It is simply R^2 (correlation squared):​

• n is the number of data points used to estimate the regression curve
• X and Y represent the measured versus estimated amplitude values of the regression line.
• A natural log transformation is applied to the measured frequency values so that they are linearly spaced. Smoothness (SM) values can range from 0 to 1, with larger values representing smoother frequency response curves.

___________________________

Observations & Critiques

1. THD is not included. Rating THD performance is hard to do as we would have to agree on what is audible. I'll start with a suggestion of a downward slope where audibility thresholds is set to 10% THD @ 20Hz and 0.1% THD @20kHz; this is already considerably lower that what I consider audible, but I reduced it to please others.
2. Besides log-scaling, the preference rating does not weight frequencies differently.
3. I think NBD_ON may have too much of an effect on the score, we almost never listen solely on the direct axis. I can't alter the algorithm and see if it improves as I don't have Harman's data, but I think it should be changed to a NBD on the Listening Window, or maybe a +/-5° window to not deviate too much.
4. Working on the assumption that on-axis is the intended axis, some speakers are designed to have no or little toe-in, so that 15° to 30° off-axis is the reference axis. Same for vertical on-axis.

That's all I got for now.

Since Dr. Olive has all the data to check, and it is his work being critiqued (possibly improved), I wonder if he would be wiling to run any altered version to see if it obtains a higher correlation.

EDIT: Working on the calculation based off the NHT's text files Amir provided.

 

[617]

1. THD is not included. Rating THD performance is hard to do as we would have to agree on what is audible. I'll start with a suggestion of a downward slope where audibility thresholds is set to 10% THD @ 20Hz and 0.01% THD @20kHz; this is already considerably lower that what I consider audible, but I reduced it to please others

THD is simply not a useful descriptor of sound quality. In designing speakers, I and other designers use THD as a way of hinting at the frequencies and levels where a transducer is running up to mechanical limitations, but other than that it simply does not correlate to what we hope it does except in extreme cases.

See Geddes' papers on the subject:
http://www.gedlee.com/Papers/Distortion_AES_I.pdf
http://www.gedlee.com/Papers/Distortion_AES_II.pdf

Powerpoint summary:
http://www.gedlee.com/Papers/The Perception of Distortion.pdf

"THD and IMD have no correlation to the perception of the distortion that they are intended to represent.
Correlation is possible with a metric that takes into account the way the ear actually functions
One of the most important implications is that distortion in loudspeakers could well be an insignificant factor"

Great summary article from Multimedia Manufacturer:
http://www.gedlee.com/Papers/THD_.pdf

I have no idea how easy it would be to derive this value from Klippel data but it would be the way to go in my opinion.

 

[amirm]

Did Earl ever rate his own speakers on his metric?

 

[Krunok]

THD is simply not a useful descriptor of sound quality. In designing speakers, I and other designers use THD as a way of hinting at the frequencies and levels where a transducer is running up to mechanical limitations, but other than that it simply does not correlate to what we hope it does except in extreme cases.

See Geddes' papers on the subject:
http://www.gedlee.com/Papers/Distortion_AES_I.pdf
http://www.gedlee.com/Papers/Distortion_AES_II.pdf

Powerpoint summary:
http://www.gedlee.com/Papers/The Perception of Distortion.pdf

"THD and IMD have no correlation to the perception of the distortion that they are intended to represent.
Correlation is possible with a metric that takes into account the way the ear actually functions
One of the most important implications is that distortion in loudspeakers could well be an insignificant factor"

Great summary article from Multimedia Manufacturer:
http://www.gedlee.com/Papers/THD_.pdf

I have no idea how easy it would be to derive this value from Klippel data but it would be the way to go in my opinion.

From Toole's book "Sound reproduction .." - only 2 pages about non-linear distorion, and here is the summary (page 453):

"In loudspeakers it is fortunate that distortion is something that normally
does not become obvious until devices are driven close to or into some limiting
condition. In large-venue professional devices, this is a situation that can occur
frequently. In the general population of consumer loudspeakers, it has been very
rare for distortion to be identifi ed as a factor in the overall subjective ratings.
This is not because distortion is not there or is not measurable, but it is low
enough that it is not an obvious factor in judgments of sound quality at normal
foreground listening levels."

 

[Krunok]

Did Earl ever rate his own speakers on his metric?

Did you?

 

[617]

Did Earl ever rate his own speakers on his metric?

Earl made home speakers with pro sound drivers, and his comments on their distortion characteristics mimic what Toole says above; he says they they will go loud enough to cause hearing damage before hd becomes a factor.

I'm building a pair of Abbey's right now and I'll let you know if that is accurate.

 

[amirm]

Did you?

What?

 

[amirm]

Earl made home speakers with pro sound drivers, and his comments on their distortion characteristics mimic what Toole says above; he says they they will go loud enough to cause hearing damage before hd becomes a factor.

I'm building a pair of Abbey's right now and I'll let you know if that is accurate.

Thanks. Still, I was hoping he had rated his own speakers using that metric so we had an example to look at.

 

[TimDH]

Thanks. Still, I was hoping he had rated his own speakers using that metric so we had an example to look at.

Amir, I raised a question before: wouldn't the directivity of the speaker enter into how you want to estimate the in-room performance (weighting of direct-reflected-power metrics)?

thanks for the great work

 

[MZKM]

Did Earl ever rate his own speakers on his metric?

Just thought of this, how difficult is it actually to use Olive’s algorithm? I don’t know how Klippel stores the data, so how difficult is it to calculate something like NBD?

 

[MZKM]

it has been very rare for distortion to be identified as a factor in the overall subjective ratings.
This is not because distortion is not there or is not measurable, but it is low
enough that it is not an obvious factor in judgments of sound quality at normal
foreground listening levels."

Right, I’m suggesting a real low impact, like maybe 4%. Using that, going from 0-4, and if subtracting this parameter in the formula:

• 0% for vanishing low THD
• 1% for low THD
• 2% for THD target
• 3% for decently above target
• 4% for way past target

 

[617]

Thanks. Still, I was hoping he had rated his own speakers using that metric so we had an example to look at.

Unfortunately on further review it's not obvious how Gm would even be tested in a speaker. It's basically a system which weights HD by order and level if I'm not mistaken, and the studies were carried out by distorting a reference recording with different distortion profiles. These were then auditioned and ranked by test subjects using IEMs.

Absolute and comparable HD distortions of speakers are really hard to do and I'm aware of no real standard. Unlike FR which we can cleverly separate from room response, harmonic distortion changes quite a bit depending on where you measure (not aware how directivity plays into this although I would assume higher orders get quieter off axis).

The best HD tests I've seen rely on steady state stepped sine signals which of course become mingled with room response. The traditional way to deal with this is by close mic'ing, which works well except for the fact that if the speaker is loud enough for distortion to be a major factor, and the mic is 1" away, the test signal at the capsule is very very loud and the mic itself contributes distortion.

It may turn out you need to build an anechoic chamber after all. I will increase my patreon support if you decide to do so.

 

[amirm]

Just thought of this, how difficult is it actually to use Olive’s algorithm? I don’t know how Kippel stores the data, so how difficult is it to calculate something like NBD?

The high level formula is known as you wrote. In detail though, I lost out of time to focus on what the specific computation is. Klippel fully exports graph data and I am able to open them in Excel so we can compute what we want.

 

[amirm]

The best HD tests I've seen rely on steady state stepped sine signals which of course become mingled with room response. The traditional way to deal with this is by close mic'ing, which works well except for the fact that if the speaker is loud enough for distortion to be a major factor, and the mic is 1" away, the test signal at the capsule is very very loud and the mic itself contributes distortion.

To this end, I do have the laser module for the Klippel analyzer. I don't have the actual laser fixture, nor do I know what it cost. But if we can detect distortion using that, it would eliminate the issues with room noise and such.

 

[617]

To this end, I do have the laser module for the Klippel analyzer. I don't have the actual laser fixture, nor do I know what it cost. But if we can detect distortion using that, it would eliminate the issues with room noise and such.

That's obviously the way to do it. I think that the HD graphs have the potential to be really misleading and I would recommend not publishing them as I believe many readers would latch onto them and make incorrect assumptions about the DUT performance.

 

[dwkdnvr]

I'm not sure that measuring distortion is any more difficult than measuring frequency response in terms of room impacts. Using the log-sweep sine method, you can capture both IR and HD in a single sweep, and the auto-correlation provides a fair amount of noise immunity and SNR. Room influenced reflections and modes would still need to be handles though, but it 'should be' viable for >200Hz or so testing. This is most likely what sites like HiFiCompass ect are using for individual driver measurements and these are seen as very useful.

The volume problem is tougher. Unless you want to perform a series of close-mic sweeps covering each driver and then merge, you're going to have to do it at 1M or farther to allow for driver integration, and that means full volume testing.

 

[617]

I'm not sure that measuring distortion is any more difficult than measuring frequency response in terms of room impacts. Using the log-sweep sine method, you can capture both IR and HD in a single sweep, and the auto-correlation provides a fair amount of noise immunity and SNR. Room influenced reflections and modes would still need to be handles though, but it 'should be' viable for >200Hz or so testing. This is most likely what sites like HiFiCompass ect are using for individual driver measurements and these are seen as very useful.

The volume problem is tougher. Unless you want to perform a series of close-mic sweeps covering each driver and then merge, you're going to have to do it at 1M or farther to allow for driver integration, and that means full volume testing.

I think we need to get back to what we want the measurements for. To my mind what we are getting at is output level - how loud can the speaker go. The Kali specification lists distortion figures at three frequencies, and I think this is sensible.

Distortion in speakers is similar to amps where there is an inflection point where it starts to increase much faster. A distortion sweep could show that frequency.

I think we need more data and maybe some other people to chime in here.

 

[Wombat]

It be would be interesting to see how driver suspension characteristics affect very low level drive performance, e.g. at threshold of hearing levels and how this relates to relevance of S/N performance of preceding devices.

 

[amirm]

I'm not sure that measuring distortion is any more difficult than measuring frequency response in terms of room impacts.

Well, for frequency response we are using the analyzer's advanced math and double scan to remove the room influence. In contrast, the distortion measurements are done like you would do with just a scan and no processing. So room modes and reflections get in the way and screw up the results. Imagine if you have a room mode at second harmonic. It could add up to 20 dB to that amplitude.

 

[amirm]

It be would be interesting to see how driver suspension characteristics affect very low level drive performance, e.g. at threshold of hearing levels and how this relates to relevance of S/N performance of preceding devices.

It is best to think of these measurements as "large signal" analysis, not small. The latter is an entirely different thing and may very well require state of the art in anechoic chambers and such. We lack data for large signal which is by far the most important thing in speakers. Let's get that under our belt and over time we can see what drill downs can be done.