Speaker Equivalent SINAD Discussion

[amirm]

That is more of an acoustics issue than speaker. My suggestion to put a thick carpet on the floor. The effect is above a few hundred hertz so can be effectively restrained. That said, there is other research that says the floor bounce may not be bad. See this post from me on testing from Fraunhofer Institute and comment from Dr. Toole: https://www.audiosciencereview.com/...y-theory-without-measurement.7127/post-162900

 

[Blumlein 88]

I think some serious context is missed here. The SINAD graph for electronics has created huge value for consumers. Company after company is going back to proper engineering, redesigning their audio products and producing far lower distortion and noise (hence higher SINAD). All of this has occurred at zero cost to consumers. Take the ESS IMD Hump. Adjustment of a couple of passive components results in reduction of some 20 dB in intermodulation distortion! Schiit has produced the same $99 DAC and Amps before with 20 to 30 dB improvement in SINAD.

At the same time, extremely low SINAD in 40s and 50s is shining a huge light on how bad some devices are. Think of Pass ACA amplifier kit. Many people were shocked with how bad those results were. Yet, Nelson Pass had already post a bunch of graph measurements to show the same but because it was not summarized in a simple to read comparison graph like SINAD, no one took notice. Even I didn't until I tested it.

Stereophile has been testing speakers and electronics for decades. Yet it has not had the effect that our measurements here have. Again, the fault is that the data lacks any kind of summary that shows comparative performance.

Here, the situation is far, far better than using SINAD. The work by Sean Olive is 100% based on double blind controlled testing to make sure the score correlates with listening test results. This is no simple measurement from 50 years ago that SINAD is. Here are some bits from the paper:
A Multiple Regression Model for Predicting
Loudspeaker Preference Using Objective
Measurements: Part II - Development of the Model
Sean E. Olive, AES Fellow

View attachment 45625

Correlation of 1.0 means perfection. That the model predicts listening test results with 100% accuracy. Here is the actual graph relative to results:
View attachment 45626

As you see the experimental results closely hug the linear prediction.

View attachment 45627

And how the tests were conducted:
View attachment 45628

If this kind of scoring is not good enough for you all, I don't know what is. This research is a gift. I highly suggest reading it in detail before scuffing at it.

And it is not like we can go without. I just measured another speaker that i will post soon. I am sitting here, seeing some anomalies in the measurements but no way of characterizing it at all in relative scale to what I have measured before.

At the end, the scale may just be good for showing the best and the worst. This is what SINAD is doing and is a great service and outcome. I don't care if someone argues between a speaker that gets a score of 6 or 7. I care about clearly identifying the dogs and heros.

As with speaker testing, there are many reasons not to do something. Get on board to solve this problem. The consumer needs a scale. It doesn't have to be perfect. It is not like he has any scale whatsoever to use right now. A compass is not as good as GPS but it can sure tell you more or less which way to walk if you are lost.

Is there somewhere that we can download a copy of this paper other than the AES?

As an example of something that appears just wrong is the Zu Audio Soul Supreme at $4500/pr. Pretty much a poster child for what Harman says would score very poorly vs a good speaker. Like I'd imagine heads up with $149 LSR 305 this $2250 speaker would lose badly. Maybe you can put a Zu of some sort on the Klippel machine. They all seem to have similar problems.
Stereophile measurements:

 

[amirm]

Is there somewhere that we can download a copy of this other than the AES?

If you have Dr. Toole's book, it is summarized there. A lot of Harman papers used to be public but I don't think that is the case anymore. Let me look....

 

[DDF]

That is more of an acoustics issue than speaker. My suggestion to put a thick carpet on the floor. The effect is above a few hundred hertz so can be effectively restrained. That said, there is other research that says the floor bounce may not be bad. See this post from me on testing from Fraunhofer Institute and comment from Dr. Toole: https://www.audiosciencereview.com/...y-theory-without-measurement.7127/post-162900

I'm familiar with this and responded to Dr. Toole about it here at ASR (no response).

The FHG test is valid but is far from universally representative of hif-fi playback. For example, reproduction of an anechoically recorded voice via a loudspeaker no doubt sounds more realistic with the floor reflection included upon playback, given listener expectation. It's far from established that symphonic works would be similarly better served with local floor reflection present. The floor reflection doesn't exist with these time stamps or characteristics in a live symphonic event. A similar argument can be made for a live recording where the recording environment's floor reflection is already present (though the spatial component is admittedly missing). I don't believe Dr Toole or Harman ever studied this.

Carpet doesn't reduce the floor reflection in a timbre-neutral way given its effect is limited towards higher frequencies. Side wall absorption is often detrimental since its effect is not broad band, and this (can) distort the perceived timbre in room. Since the floor reflection has a much greater impact on perceived timbre than side wall (see Bech), I think the same argument applies even more strongly to the floor reflection. Maybe its better to leave the floor reflection in all together, or deal with its effect through carpet (mid/high freq) + DSP (lower freq)

I was mistaken, I've collected 40+ papers (gulp, why do I do this to myself ) directly related to this. If I come to any helpful conclusions, I'll post.

 

[bobbooo]

Is there somewhere that we can download a copy of this paper other than the AES?

That particular paper is available for free here (scroll down for the correct paper):

A Multiple Regression Model for Predicting Loudspeaker Preference Using Objective Measurements: Part II - Development of the Model

It's essential reading really for this thread.

 

[restorer-john]

The problems with any attempt to reduce performance indication to a metric or group of metrics is that it risks reducing things too much and distorting perception

I Australia, we had a respectable magazine for many years, Australian HiFi. Around 1990 they embraced the presentation of "simplified metrics" and bar-graphs instead of comprehensive measurements. Initially, they presented proper measurements alongside a bar graph summary, then the measurements gradually disappeared and they ranked using just the bar graphs. Their credibility went totally out the window and we regarded their reviews as a joke for decades. Quoting an excellent Australian HiFi review was not a good move to a technical buyer, their eyes would roll and the sale would be lost...

It was a pity because they were trying to summarize the extensive measurements they were undertaking in the lab, into bite-sized, easy to comprehend visuals for uneducated consumers. It alienated the technical people completely. Eventually they went back to a basic dumbed down review with recommendations and a second technical review for the "geeks". I never even looked at the first part of the review, just went straight to the measurements section. Maybe a two piece review structure could work?

The bar graphs for SINAD on ASR have achieved little as far as I am concerned, but I guess I'm not the guy looking online to buy a cheap D/A converter from China. I don't even look at it. It's already falling apart with products with zero THD, falling behind more distorting products due solely to mains spurs that are totally inaudible.

I understand the sheer amount of data the new Klippell system creates and the desire to drill down to a basic rating, but beware of arbitrary ranking and simplification.

 

[SIY]

I'd like to see the algorithm used for successful prediction, determined experimentally. There's a lot of adjustable parameters, which concerns me.

edit: I found the linked paper where they did that. So NM.

 

[Sancus]

Carpet doesn't reduce the floor reflection in a timbre-neutral way given its effect is limited towards higher frequencies. Side wall absorption is often detrimental since its effect is not broad band, and this (can) distort the perceived timbre in room. Since the floor reflection has a much greater impact on perceived timbre than side wall (see Bech), I think the same argument applies even more strongly to the floor reflection. Maybe its better to leave the floor reflection in all together, or deal with its effect through carpet (mid/high freq) + DSP (lower freq)

I was mistaken, I've collected 40+ papers (gulp, why do I do this to myself ) directly related to this. If I come to any helpful conclusions, I'll post.

Do the studies indicating that floor bounce have so much influence focus purely on its existence/reduction in SPL? Or is the issue with floor and ceiling bounces also solvable with speakers that have good vertical off-axis response, so that the floor bounce at least sounds consistent, even if it is present?

 

[Sancus]

The bar graphs for SINAD have achieved little as far as I am concerned, but I guess I'm not the guy looking online to buy a cheap D/A converter from China. I don't even look at it. It's already falling apart with products with zero THD, falling behind more distorting products due solely to mains spurs that are totally inaudible.

An imperfect measure is way better than none, for me at least SINAD is hugely valuable because it makes it possible to point less and non-technical people at the reviews here. And then perhaps they even learn something. But without that initial simplification, most people aren't going to be interested when they figure out exactly how much research and time is required to understand all the graphs and highly technical measures. And so they won't even be present to learn something in the first place.

 

[Blumlein 88]

That particular paper is available for free here (scroll down for the paper): A Multiple Regression Model for Predicting Loudspeaker Preference Using Objective Measurements: Part II - Development of the Model

It's essential reading really for this thread.

Thank you. And yes needed to discuss this in an informed manner.

 

[restorer-john]

An imperfect measure is way better than none, for me at least SINAD is hugely valuable because it makes it possible to point less and non-technical people at the reviews here

That's the problem. It's like recommending a daily-driver car based on the 0-100 time.

Tell me, what does a SINAD of 110dB tell you?

 

[amirm]

Carpet doesn't reduce the floor reflection in a timbre-neutral way given its effect is limited towards higher frequencies.

Which is all you need per Bech's research: Spatial aspects of reproduced sound in small rooms
Soren Bech)

Anyway, this is really off-topic as I mentioned. Room acoustics is a nature of the room, not speaker. While you can get speakers that interact with room less or more in this manner, it goes beyond a single metric of performance.

 

[amirm]

That's the problem. It's like recommending a daily-driver car based on the 0-100 time.

Tell me, what does a SINAD of 110dB tell you?

No more discussion about SINAD please. If you don't like this work, let the rest of us who are interested to discuss relevant metrics for speaker performance.

 

[restorer-john]

No more discussion about SINAD please. If you don't like this work, let the rest of us who are interested to discuss relevant metrics for speaker performance.

I brought up bar charts for speaker characteristics Amir, and presented a considerably thought-out post, complete with historical examples. We know you've hung your hat on a particular metric for other devices, one that even you have now admitted has its flaws.

My post was a warning that the same type of criticism may appear down the track, if using an untested or deliberately simplified metric as a universal way to rank speakers. Can you not see that?

 

[amirm]

My post was a warning that the same type of criticism may appear down the track, if using an untested or deliberately simplified metric as a universal way to rank speakers. Can you not see that?

Father is sitting with his friend and asks his son to take a vase and put it on the other side of the room. As soon as the kid takes the vase, he smacks him on the back of the head. The kid starts crying as he carries the vase to the other side of the room. Friend asks him why he did that? He says just in case he breaks the vase as it won't do any good after the fact!

So no, I don't want to hear dire prediction of what could happen in your mind and deal with the complaints now. Nothing of the sort has happened with electronics measurements. You have no history with me to base such predictions.

I like to see people work on problems than just complain about what others are doing. We are trying to utilize state of the art in sound reproduction and psychoacoustics and you complain like it is some random thought. If this doesn't interest you, move along. It is interfering with good effort others are putting forth.

 

[DDF]

Which is all you need per Bech's research: Spatial aspects of reproduced sound in small rooms
Soren Bech)

View attachment 45638

Anyway, this is really off-topic as I mentioned. Room acoustics is a nature of the room, not speaker. While you can get speakers that interact with room less or more in this manner, it goes beyond a single metric of performance.

Please bear with me, this is definitely not off topic.

Your quote from Bech is for a spatial aspect, not timbral. That paper studies spatial effects. 2 kHz affects spatialization because those frequencies align with the signatures of the external ear/torso spectral notches, used in part as the mechanism for spatial localization.

Bech's paper assessing timbral impacts from reflections is "Timbral Aspects of reproduced Sound in Small Rooms I",

and can be downloaded here:
https://backend.orbit.dtu.dk/ws/portalfiles/portal/4415806/Bech.pdf

The results here identify multiple reflections as affecting timbre but the floor had strongest impact

Here is why this is a loudspeaker design attribute, as much as a room acoustic attribute.

Floor reflection response is a loudspeaker design trade off. Conventional wisdom (DIY world) is to design for best driver phase matching on axis. This is often a mistake. Changes in (vertically aligned) driver phase matching directly affects where the inter-driver cancellation notch is. By allowing some relatively small driver phase mismatch on axis (and compensating for with modest adjustments to the knee's initial roll off), a design can be audibly improved by steering the cancellation notch away from the floor and providing a timbrally more accurate floor reflection.

The direction of the floor notch radiation has to be assumed, but its no different than Harman's assumptions about the same (or side wall early reflection). If you wanted to get really slick, you could even calculate it for the specific loudspeaker with fixed assumptions about listening distance, using
https://www.diyaudio.com/forums/multi-way/284567-measurement-utility-micheight.html
or another on line calculator.

This is purely a science based observation derived from DBTs in the literature, not an opinion. Its as relevant a metric, perhaps more so (given its impact) than the spinorama's adoption of side wall reflection into its score.

I think it would be a mistake to disregard it and am happy to discuss further.

 

[MZKM]

As an example of something that appears just wrong is the Zu Audio Soul Supreme at $4500/pr.

Those are aimed at those with 2W tube amps. I doubt many people with solid state amps own them. Sure there are alternatives like from JTR, but those are very much not WAF-friendly.

 

[Blumlein 88]

Those are aimed at those with 2W tube amps. I doubt many people with solid state amps own them. Sure there are alternatives like from JTR, but those are very much not WAF-friendly.

A bad colored speaker is a bad colored speaker.

And the ZU was listed as 97 db/watt, but tested at 91.5 so it won't do for 2 watts.

Meanwhile a slightly cheaper Klipsch Heritage Forte III tested at 95 db/watt with these better plots. How pretty you think they are is of course up to anyone. The beauty of good measurements is you can ignore the false ad copy of the Zu and for $500 less get a better performing more efficient speaker. Not shown is the CSD plot which is horrid, hashy and at least 3 times worse for the Zu vs the Klipsch.

 

[bobbooo]

Also, if using NBD, I was looking over the formula and for

I have no clue what “band b” is and it isn’t mentioned. Do you know or could you ask Sean?

Oh, and 100Hz-12kHz can’t be separated into whole 1/2 octave bands, so not sure what happens with that:
100Hz-150Hz
150Hz-200Hz
200Hz-300Hz
300Hz-400Hz
400Hz-600Hz
600Hz-800Hz
800Hz-1200Hz
1200Hz-1600Hz
1600Hz-2400Hz
2400Hz-3200Hz
3200Hz-4800Hz
4800Hz-6400Hz
6400Hz-9600Hz
9600Hz- ?
Does he just end it at 12000Hz instead of 12800Hz, or does he do something else?

If you look at the description for Narrow Band Deviation in Olive's paper, he says:

The mean absolute deviation within each ½-octave band is based on a sample of 10 equally log-spaced data points...Whereas AAD measures deviations from flatness relative to the average level of the reference band 200-400 Hz, NBD measures deviations within a relatively narrow ½-octave band.

This, combined with the similarity in structure of the NBD formula to that for AAD (Absolute Average Deviation), suggests to me that y-sub(b) is the amplitude of each of these "10 equally log-spaced data points" within each band, where b is an index that sums over these data points. So y-bar minus y-sub(b) is the amplitude deviation of each of these 10 equally log-spaced (in frequency) points from their average amplitude in a particular band. This is then averaged to arrive at a mean deviation of these points from the average in that band. This is done for each band in the 100 Hz to 12 kHz range, then finally the mean of all these average deviations is taken, to arrive at the Narrow Band Deviation metric for the speaker as a whole.

As for the separation of the 100 Hz to 12 kHz range into bands, you're not using the correct 1/2-octave divisions. As an octave increase is a factor of 2(^1) difference in frequency, a 1/2-octave increase is a factor of 2^(1/2) i.e. a factor of the square root of 2. So these would be the correct 1/2-octave bands from 100 Hz to 12 kHz (to the nearest Hz):

100Hz -141Hz
141Hz - 200Hz
200Hz - 283Hz
283Hz - 400Hz
400Hz - 566Hz
566Hz - 800Hz
800Hz - 1131Hz
1131Hz - 1600Hz
1600Hz - 2263Hz
2263Hz - 3200Hz
3200Hz - 4525Hz
4525Hz - 6400Hz
6400Hz - 9051Hz
9051Hz - 12800Hz

However, this also doesn't divide exactly up to 12 kHz. Maybe Olive is actually referring to the approximate center frequency when talking about bands between 100 Hz and 12 kHz, not the lower and upper bound of the first and last bands respectively. Perhaps he means the commonly used bands in a 1/2-octave equalizer i.e. those in this table, which have center frequencies ranging from 125 Hz to 11314 Hz (nicely within the 100 Hz to 12 kHz range).

 

[Wombat]

Father is sitting with his friend and asks his son to take a vase and put it on the other side of the room. As soon as the kid takes the vase, he smacks him on the back of the head. The kid starts crying as he carries the vase to the other side of the room. Friend asks him why he did that? He says just in case he breaks the vase as it won't do any good after the fact!

So no, I don't want to hear dire prediction of what could happen in your mind and deal with the complaints now. Nothing of the sort has happened with electronics measurements. You have no history with me to base such predictions.

I like to see people work on problems than just complain about what others are doing. We are trying to utilize state of the art in sound reproduction and psychoacoustics and you complain like it is some random thought. If this doesn't interest you, move along. It is interfering with good effort others are putting forth.

That first paragraph is rather disturbing in terms of rational communication. The rest of the post is 'put-down'. Not a usual example of your responses to long-contributing members.