Master Preference Ratings for Loudspeakers

[MZKM]

Wait, what? Why would you ever divide by frequency? Frequency has nothing to do with this.

I think @pierre has it right - he's getting the same results as Klippel and his code follows the Klippel formula to the letter. (Whether Klippel themselves have the right formula is still up for debate, but given their reputation I would be inclined to trust them by default…)

I'm not sure what n stands for, and one excerpt I think stated the sample period, which I thought meant frequency.

In any case, the weighted one is what is being used, and I don't understand why it is multiplying then dividing by the weight.
Pierre is getting what Klippel is outputting because the formula doesn't alter what Klippel is outputting. Unless I'm not seeing something.

 

[pierre]

I'm not sure what n stands for, and one excerpt I think stated the sample period, which I thought meant frequency.

In any case, the weighted one is what is being used, and I don't understand why it is multiplying then dividing by the weight.
Pierre is getting what Klippel is outputting because the formula doesn't alter what Klippel is outputting. Unless I'm not seeing something.

n is the number of measurements. Rms is not per measurement. it is l^2 norm (weighted). That’s why weights don’t cancel out.

 

[edechamps]

I'm not sure what n stands for, and one excerpt I think stated the sample period, which I thought meant frequency.

In the formula "w" means "weight", not "sample period". n is the number of curves being averaged (which is also the number of weights, of course). And f is just the frequency of interest which is being iterated on - it's basically an irrelevant constant in this context. |H(f)| is the modulus of the transfer function, or in other words, the sound pressure at frequency f (or its square root… I can't remember).

In any case, the weighted one is what is being used, and I don't understand why it is multiplying then dividing by the weight.

It is multiplying by the weight then dividing by the sum of the weights. Which is literally the definition of a weighted average, the only difference here is that it's an RMS weighted average.

Pierre is getting what Klippel is outputting because the formula doesn't alter what Klippel is outputting. Unless I'm not seeing something.

@pierre is doing the same thing you're trying to do: reconstructing the CTA2034 averages using the individual angle data. He's getting what Klippel is outputting because he reached the goal

 

[napilopez]

Thanks for the input @MZKM @edechamps @pierre.

This is definitely more of a curiosity as I worry I might be doing more harm than good if I started publishing a score based on quasi anechoic measurements with limited resolution in the low mids, but it would be nice to see how well my measurements correlate with Amirs by this metric as more of our speakers overlap. I hadn't originally seen MZKM had shared his master sheet when I made that post

@napilopez

If you can parse the graphs with WebPlotDigitizer then most values are automatically computed.
There are tutorial on the internet or member XVX was kind enough to write one dedicated for Spinorama charts https://github.com/pierreaubert/spi...orial/digitalization/Digitization-Tutorial.md
if you are lazy like me, you do not need to do the 2 DI curves, the system will computed them from the others.

You have your datas in a mdat from REWS? correct?

Thanks! REW lets me export the data in various formats, including CSV, and it should be fairly trivial to format my data to match Amir's, though it depends if your software is sensitive to the number of points per octave. I think the Klippel spits out 10 ppo or 20, while the closest options in REW are 12/24. I also can't export multiple measurements in one document like Amir's CTA-2034A file, but again that's easy to fix manually.

 

[MZKM]

Ugh, I thought I had it (listening window), but now it's a 3rd difference:

It's more inaccurate than just simple averaging the curves.

1. Convert to Pascal: 0.00002*10^(Decibel/20)
2. Square
3. Multiply by weight based on angle
4. Sum with other curves (9 total for LW)
5. Divide by sum of all weights used (9 for LW, 1x for On, 4x for 10, 2x for 20, 2x for 30)
6. Sqrt
7. Convert to SPL: 20*log10(Pascal/0.00002)

EDIT: Wait, do you only apply the weights to the Sound Power? I just re-read the 2034 document and it only mentions weights for SP, and I now can get the same as what Klippel outputs for the Listening Window. If so, why are the weights only applied to the Sound Power?

 

[edechamps]

EDIT: Wait, do you only apply the weights to the Sound Power? I just re-read the 2034 document and it only mentions weights for SP, and I now can get the same as what Klippel outputs for the Listening Window. If so, why are the weights only applied to the Sound Power?

My best guess is that's because Sound Power has a well-defined physical meaning and there is only one correct way to calculate it. The same can't be said of other averages like LW which are purely arbitrary. Although in principle I'd agree with you…

 

[MZKM]

My best guess is that's because Sound Power has a well-defined physical meaning and there is only one correct way to calculate it. The same can't be said of other averages like LW which are purely arbitrary. Although in principle I'd agree with you…

Yeah.

I got it working finally (and applying the correct ER calculation). I am out right now so haven’t calculated Sound Power, but it shouldn’t be difficult, just tedious with the 70 measurements and their corresponding weights.

This will make my import process a tad faster, only needing to import the H/V data and not also the CEA-2034 and PIR.

 

[napilopez]

I did the Neumann KH80, Focal Chora 806, Devialet Reactor 900, Q Acoustics 3030i, D&D 8C, and Fluance Ai60 today. I was getting a little paranoid at how close all the scores were but then the Fluance Ai60 ended up with significantly worse performance as expected.

Normal score/ignore LFX/Using listening window

KH80: 5.7/8.1/8.2
Chora 806: 5.7/7.7/8.1
Q Acoustics 3030i: 5.2/7.5/7.8
Devialet Reactor 900: 7.2/7.6/7.8
Dutch 8C: 7.4/8.4/8.4
Fluance Ai60: 3.4/5.4/5.6

Amphion Argon1: 5.3/7.3/7.5
Polk L200: 5.6/7.6/7.7
KEF R3: 6.3/7.8/7.8
Buchardt S400: 6.3/8.1/8.1

Obviously take this all with a grain of salt. Just sharing for the curious and to see if there's any merit in using similar performance metrics from DIY data.

Aside from the obvious fact my measurements aren't truly anechoic and are more subject to human error, I also have to 'cheat' on bass performance for the off-axis curves because I normal only do the single nearfield splice for the on-axis. Vituixcad lets me simulate the other angles, but it's kind of trial and error and guesstimating based on expected results. Still, I think the results are largely in line with what I'd expect from my own interpretations of the measurements, especially the ignore LFX score.

Q acoustics is definitely winning the price to performance ratio. The devialet and dutch 8c are notable for having by far the highest preference score sans subs because bass. It's also encouraging to see that the 8c scores best, which is the expected result, though diminishing returns kick in early.

I think this also shows it's going to be really hard for any speaker to ever get out of the 8s.

Next step is to take some of Amir's klippel generated data and use a 6.5ms gate and see how much the score calculations change to try and estimate how much the lowered resolution affects the score accuracy.

EDIT: Realized that my L200 calculation was only using 12 points instead of 24. Scores went up a bit from 5.5/7.5/7.5 to 5.6/7.6/7.7

 

[bobbooo]

I did the Neumann KH80, Focal Chora 806, Devialet Reactor 900, Q Acoustics 3030i, D&D 8C, and Fluance Ai60 today. I was getting a little paranoid at how close all the scores were but then the Fluance Ai60 ended up with significantly worse performance as expected.

Normal score/ignore LFX/Using listening window

KH80: 5.7/8.1/8.2
Chora 806: 5.7/7.7/8.1
Q Acoustics 3030i: 5.2/7.5/7.8
Devialet Reactor 900: 7.2/7.6/7.8
Dutch 8C: 7.4/8.4/8.4
Fluance Ai60: 3.4/5.4/5.6

Amphion Argon1: 5.3/7.3/7.5
Polk L200: 5.5/7.5/7.5
KEF R3: 6.3/7.8/7.8
Buchardt S400: 6.3/8.1/8.1

Obviously take this all with a grain of salt. Just sharing for the curious and to see if there's any merit in using similar performance metrics from DIY data.

Aside from the obvious fact my measurements aren't truly anechoic and are more subject to human error, I also have to 'cheat' on bass performance for the off-axis curves because I normal only do the single nearfield splice for the on-axis. Vituixcad lets me simulate the other angles, but it's kind of trial and error and guesstimating based on expected results. Still, I think the results are largely in line with what I'd expect from my own interpretations of the measurements.

Q acoustics is definitely winning the price to performance ratio. The devialet and dutch 8c are notable for having by far the highest preference score sans subs because bass. It's also encouraging to see that the 8c scores best, which is the expected result, though diminishing returns kick in early.

I think this also shows it's going to be really hard for any speaker to ever get out of the 8s.

Next step is to take some of Amir's klippel generated data and use a 6.5ms gate and see how much the score calculations change to try and estimate how much the lowered resolution affects the score accuracy.

Thanks for this. Did I see somewhere that you've also measured the Q Acoustics 3020i? Out of curiosity, would you mind calculating the scores for that model as well? I'd be very interested to see how it compares to the 3030i.

 

[napilopez]

Thanks for this. Did I see somewhere that you've also measured the Q Acoustics 3020i? Out of curiosity, would you mind calculating the scores for that model as well? I'd be very interested to see how it compares to the 3030i.

I'm missing a lot of data for the 3020i as it was measured when I performed a lot fewer measurements overall(not even complete front hemisphere vertical data), luckily the front hemisphere has the biggest impact on the shape of the curves, so using what I have and faking the rest, I get a: 3.9/6.8/7.

Edit: I did the concept 20 as I was curious because it basically looks like a better 3020i, and indeed it got a 4/7/7.2

 

[MZKM]

My best guess is that's because Sound Power has a well-defined physical meaning and there is only one correct way to calculate it. The same can't be said of other averages like LW which are purely arbitrary. Although in principle I'd agree with you…

Finally got it all working I think, including the ER fix. However, even ignoring all this, there is still the aspect of the PIR, which I am still getting different from Klippel even with weighting it according to 2034 and checking with and without the ER fix.

I could still have something wrong, but:
F208, PIR, 1st measurement (20.5Hz):
Klippel: 61.5479dB
Manual ER no fix: 61.5178
Manul ER fix: 61.4903

EDIT:I forgot the ER fix is both the +/-90 thru 180 and the average of averages. When I get rid of +/-100 thru +/-170 and change it to the average of all angles, I get what Klippel is outputting.

So, everything should be correct now.

@amirm , the PIR difference was because it wasn't using the fix for the ER curve (and I also didn't know you had to do an rms average of the SPL as Pascal units and thought it was simply the 12/44/44 weights as described in your first measurement post of the JBL 305P, so I blame you ). Sorry you had to contact Klippel for nothing.

I now have to update my master file as well as slowly but surely update all ~50 speaker scores/graphs thus far.

 

[MZKM]

Score: 3.1

Hmm, off by 0.4-0.5 from what I'm getting for the NHT SB2:

1. I still see errors in your +/- range, ~902Hz is ~86.4dB and ~3030Hz is ~76.4, so it should be +/-5dB, not +/-5.7dB
2. SM_PIR: I get the same without the ER fix, I get 0.6 with the ER fix, no big deal
   1. On your main page with the images, it shows 0.78 for Smoothness, not sure where it's pulling that from.
3. NBD_PIR: I get ~0.52 with or without the ER fix, while you get 0.58.
4. NBD_On: I get ~0.66 while you get 0.76.
5. LFX: Just a 1Hz difference, no big deal.

 

[edechamps]

Finally got it all working I think, including the ER fix.

Can you remind me what is this "ER fix" that you are referring to? Is the Klippel data wrong?

 

[MZKM]

Can you remind me what is this "ER fix" that you are referring to? Is the Klippel data wrong?

The Horizontal/Vertical data is correct, but the Early Reflection data is wrong.

Klippel:

1. Rear reflections: +/-90, 180
2. Average all points

Correct:

1. Rear reflections: +/-90 thru 180
2. Average all curves (so an average of averages, meaning the rear reflection now having ~20 points won't dominate)

 

[napilopez]

Can you remind me what is this "ER fix" that you are referring to? Is the Klippel data wrong?

@edechamps see my discussion with Todd Welti from Harman that prompted the fix in this thread. All credit goes to @Dave Zan for first pointing out the problem.

Edit: I did much experimenting with the pre-fix and post-fix method of computing the score. For monopole speakers at least, the average of averages is by far the more consequential change, as it adds much more weight to the vertical reflections. See examples with the Klippel data here.

Going from 3 rear points to 19 barely affects the shape of the curve on any of the speakers I tested because the rear only accounts for 20 percent of the score regardless.

The irony is that I actually think it makes the ER and ERDI curves less useful for a quick look at soundstage performance, though it becomes more relevant for EQ-ability.

 

[edechamps]

The Horizontal/Vertical data is correct, but the Early Reflection data is wrong.

@edechamps see my discussion with Todd Welti from Harman that prompted the fix in this thread. All credit goes to @Dave Zan for first pointing out the problem.

Very interesting, thanks. It take it this issue is present in all datasets @amirm published thus far? Looks like I should add a "recompute ER" item to my TODO list for Loudspeaker Explorer… or at least document the problem.

 

[napilopez]

Very interesting, thanks. It take it this issue is present in all datasets @amirm published thus far? Looks like I should add a "recompute ER" item to my TODO list for Loudspeaker Explorer… or at least document the problem.

I believe so. Amir noted earlier in the thread that Klippel sent him the fix, but it's currently a manual process that has to be applied to speakers individually. So hopefully they apply a patch to make the process easier in the future.

Hopefully it won't affect the score too much. Coaxials should emerge unscathed but speakers with particulalry bad vertical measurements might end up with a worse PIR, though the ER is the only a portion of it.

I think the bigger impact is with the visual assessment of the spinorama. As I noted in the aforementioned thread, for a few speakers it's the difference between a very good ERDI and a practically perfect one, like the D&D 8C:

(Note that the labelling above is outdated and VituixCAD has since implemented the fix).

 

[MZKM]

Looks like I should add a "recompute ER" item to my TODO list for Loudspeaker Explorer… or at least document the problem.

Besides begrudgingly going back and applying the fix to all speakers and correcting it in my preference rating (and changing my posted scores here if it deviates enough to make a 1 decimal point change, though I’m not redoing the Spinormama graphs, I have to screenshot it as downloading the PNG gets wonky), I’m also adding directivity graphs, I’ve already done the horizontal for the NHT (besides the top scorers, I won’t be adding this to previous speakers). What sucks is there is no chart for this, so it’s all conditional formatting (and I can’t auto change the cell borders, so they are visible), and I can’t really display the frequencies.

Google Sheets can do this an easier way, but it’s only 5 colors and you can only change the end and center, the other 2 are automatically chosen. I am using >20 colors.

Main reason I wanted to do these is to give me an excuse not to apply the ER fix, as that is so tedious. Also, I believe Amir’s uses 1dB increments, whereas mine does as well, but the main red color is +/-1dB, so it covers 3 decibels instead of 1. Mine also uses the avg SPL (sensitivity) as the reference, rather than some absolute scale.

 

[napilopez]

Besides begrudgingly going back and applying the fix to all speakers and correcting it in my preference rating (and changing my posted scores here if it deviates enough to make a 1 decimal point change, though I’m not redoing the Spinormama graphs, I have to screenshot it as downloading the PNG gets wonky), I’m also adding directivity graphs, I’ve already done the horizontal for the NHT (besides the top scorers, I won’t be adding this to the previous made speakers). What sucks is there is no chart for this, so it’s all conditional formatting (and I can’t auto change the cell borders, so they are visible), and I can’t really display the frequencies.

What's the biggest change you've seen so far?

 

[MZKM]

What's the biggest change you've seen so far?

Well, I really haven’t done many, so it’s usually not more than a 0.1 change, it’s not substantial (yet).