• WANTED: Happy members who like to discuss audio and other topics related to our interest. Desire to learn and share knowledge of science required. There are many reviews of audio hardware and expert members to help answer your questions. Click here to have your audio equipment measured for free!

Ideas for more meaningful speaker measurements

sigbergaudio

Major Contributor
Audio Company
Forum Donor
Joined
Aug 21, 2020
Messages
2,639
Likes
5,396
Location
Norway
Not sure M-noise should be dismissed, my understanding is that it could be a useful way of better understanding the true dynamic capacity of a speaker. For those who like to play at loud / realistic levels, it's currently not easy to find out or understand a speakers capability of uncompressed playback.
 
OP
Rick Sykora

Rick Sykora

Major Contributor
Forum Donor
Joined
Jan 14, 2020
Messages
3,511
Likes
7,011
Location
Stow, Ohio USA
Got an answer from James that he agreed to being shared:
In my view, I am going to have preconceived notions about a loudspeaker no matter what, so I don’t concern myself with the order in which it is heard. If I look at a design, I will form some inescapable assumptions about it even if I don’t have hard measurements of its performance. If I see its measured performance, that is going to just alter those preconceptions in another way. There is no way around those expectation biases. So I just try to listen with as open mind as possible, regardless of whatever I know or think I know about the system’s performance.

He also mentioned that he and Gene has an interesting youtube video (originally a livestream) that specifically talks about these issues:

While I generally like James body of work and understand his statement, if I had paid for that video, would want a refund! IMO, the video lacks focus and is way longer than it needed to be. Really needs a professional interviewer/producer/editor to hone it down and improve the messaging (which am still unsure what it really is).o_O

My takeaways were that they were trying to make a case for how are very biased by our perceptions and only those who are experienced with measuring and listening together are qualified to judge speaker sound quality. Anyway, as for my original question, the AH response seems to be that there are multiple unavoidable perception biases and not sure whether they have any discipline to try to manage them. If so, would not seem to be an approach that would produce consistency from reviewer to reviewer.

So, really appreciate you asking and hope it goes well. Understand if you reserve your opinion for now, but will be interested to see if others came away with similar takeaways as I did.
 
Last edited:
OP
Rick Sykora

Rick Sykora

Major Contributor
Forum Donor
Joined
Jan 14, 2020
Messages
3,511
Likes
7,011
Location
Stow, Ohio USA
Even if one were to generate a swept sine measurement at 0 dBFS (my own set max reference level) and thereafter play back music at maximum volume... the actual transient peaks acoustically measured esp. in the bass will exceed the level as seen in the former. I may have forgotten the reason for this -- disregarding intersample peaks for now -- hmmn... why is this the case? Is it that some crest factor also needs to be accounted for?

View attachment 199200
Volume Leveling to reduce volume of streamed audio content disabled in JRiver (normal EQ still on)
*Purple traces are individual left and right response. Red trace is both L+R swept sine at 0 dBFS

LZpeak value (likely mostly from bass) is way higher than what I would've expected.

This post requires more information before it can be responded to thoughtfully.

Am interested, but please create another thread as is clearly off topic.

Thanks!

Rick
 

sigbergaudio

Major Contributor
Audio Company
Forum Donor
Joined
Aug 21, 2020
Messages
2,639
Likes
5,396
Location
Norway
While I generally like James work and understand his statement, if I had paid for that video, would want a refund! IMO, the video lacks focus and is way longer than it needed to be. Really needs a professional interviewer/producer/editor to hone it down and improve the messaging (which am still unsure what it really is).o_O

My takeaway was that they were trying to make a case for how are very biased by our perceptions and only those who are experienced with measuring and listening together are qualified to judge speaker sound quality. Anyway, as for my original question, the AH response seems to be that there are multiple unavoidable perception biases and not sure whether they have any discipline to try to manage them. If so, would not seem to be an approach that would produce consistency from reviewer to reviewer.

So, really appreciate you asking and hope it goes well. Understand if you reserve your opinion for now, but will be interested to see if others came away with similar takeaways as I did.

It's recorded as a live stream, so it's streamed live on youtube. Audioholics does those on a regular basis. So it's more of a conversation - and will naturally not be as polished as something that is pre-recorded and then edited. I agree that it would be more effective for someone watching it later with an edited recording. On the other hand, those who actually watch the stream live are able to chat and interact while the conversation unrolls, so I guess there's value in that as well for those participating.

I don't think their point is that only those experienced with measuring and listening are qualified to judge a speaker sound quality. On the contrary, James points out that they're as prone to bias as anyone else. You could even make the point that someone who thinks they are in control of their bias will be affected even more.

With regards to the sequence of listening / measuring, the point (as I understand it) is that this is only one of numerous biases. So the way James see it, it's not a big point to try to control that specific one as he will still be affected by how they look, the brand, his understanding of the construction, etc etc. So rather than try to claim that his subjective review is objective and unbiased, it is what it is. A truthful presentation of his subjective experience of the product.
 
OP
Rick Sykora

Rick Sykora

Major Contributor
Forum Donor
Joined
Jan 14, 2020
Messages
3,511
Likes
7,011
Location
Stow, Ohio USA
It's recorded as a live stream, so it's streamed live on youtube. Audioholics does those on a regular basis. So it's more of a conversation - and will naturally not be as polished as something that is pre-recorded and then edited. I agree that it would be more effective for someone watching it later with an edited recording. On the other hand, those who actually watch the stream live are able to chat and interact while the conversation unrolls, so I guess there's value in that as well for those participating.

I don't think their point is that only those experienced with measuring and listening are qualified to judge a speaker sound quality. On the contrary, James points out that they're as prone to bias as anyone else. You could even make the point that someone who thinks they are in control of their bias will be affected even more.

With regards to the sequence of listening / measuring, the point (as I understand it) is that this is only one of numerous biases, so the way James see it, it's not a big point to try to control that specific one as he will still be affected by how they look, the brand, his understanding of the construction, etc etc. So rather than try to claim that his subjective review is objective and unbiased, it is what it is. A truthful presentation of his subjective experience of the product.
Agree, but at the same time seems to overly discount the objective measurements IMO. Am certainly biased too, but there seemed to be more than a bit of NIH attitude (notably more from Gene).
 

abdo123

Master Contributor
Forum Donor
Joined
Nov 15, 2020
Messages
7,423
Likes
7,940
Location
Brussels, Belgium
I don't understand the argument that there would always be other biases. It's like saying something will always be bad so there is no point in improving it.

Or like removing all restrictions and laws in competitive sport because you can't make any sport 100% fair.

In my opinion it's just the (******) consumer culture of this hobby, literally with any other industry (Computers, Electronics, TVs, Cameras, .etc) products are looked at under an electron microscope.
 

sigbergaudio

Major Contributor
Audio Company
Forum Donor
Joined
Aug 21, 2020
Messages
2,639
Likes
5,396
Location
Norway
I don't think it's fair or useful for me to continue to explain what James thinks about these things. It's not really my place to do so and I will probably not do so accurately. He explains his position quite well in the (long) video I posted, so for those interested that's likely a better source. :)
 

sigbergaudio

Major Contributor
Audio Company
Forum Donor
Joined
Aug 21, 2020
Messages
2,639
Likes
5,396
Location
Norway
I don't understand the argument that there would always be other biases. It's like saying something will always be bad so there is no point in improving it.

But how do you control bias? If it measures poorly, you can be biased to thinking it sounds worse than it does. If it measures good, you can be biased to thinking it sounds better than it does. So it would be good to not measure before listening right? But what if you're biased towards the brand to begin with (negatively or positively). Seeing a measurement before listening might actually negate a prior bias ("Wow, this actually measures quite well, perhaps B&W isn't that bad after all?").

Put another way, the combination of biases filtering what you hear is pretty complex and unpredictable. So we can't conclude that not looking at the measurement before listening will always result in a more objective listening experience.
 

abdo123

Master Contributor
Forum Donor
Joined
Nov 15, 2020
Messages
7,423
Likes
7,940
Location
Brussels, Belgium
But how do you control bias? If it measures poorly, you can be biased to thinking it sounds worse than it does. If it measures good, you can be biased to thinking it sounds better than it does. So it would be good to not measure before listening right? But what if you're biased towards the brand to begin with (negatively or positively). Seeing a measurement before listening might actually negate a prior bias ("Wow, this actually measures quite well, perhaps B&W isn't that bad after all?").

Put another way, the combination of biases filtering what you hear is pretty complex and unpredictable. So we can't conclude that not looking at the measurement before listening will always result in a more objective listening experience.
Lack of information cannot create more bias. I don’t understand your point.
 

sigbergaudio

Major Contributor
Audio Company
Forum Donor
Joined
Aug 21, 2020
Messages
2,639
Likes
5,396
Location
Norway
Lack of information cannot create more bias. I don’t understand your point.

But additional data points can change and potentially reduce existing bias, yes?
 

sigbergaudio

Major Contributor
Audio Company
Forum Donor
Joined
Aug 21, 2020
Messages
2,639
Likes
5,396
Location
Norway
No. Not necessarily. Your second point was that more data can create more bias and I understand / agree with that.

No or not necessarily? That's two different things. :)

If you think that getting more information about something can never do anything but add more bias, I respectfully disagree.
 

pablolie

Major Contributor
Forum Donor
Joined
Jul 8, 2021
Messages
2,022
Likes
3,318
Location
bay area, ca
My take on today's speaker measurements:

They continue to perpetuate the full range speaker (20Hz-20KHz) as the magic ideal. To me that ideal is obsolete. Such speakers are big and hence difficult to position optimally in a real world room.

Personally I much prefer 2+1 setups these days, much easier to set up optimally.

Hence I truly just care how a speaker measures 80Hz-20kHz. Dismissing a speaker for lack of bass these days is rather unimaginative. :cool:
 

Sancus

Major Contributor
Forum Donor
Joined
Nov 30, 2018
Messages
2,923
Likes
7,616
Location
Canada
Not sure M-noise should be dismissed, my understanding is that it could be a useful way of better understanding the true dynamic capacity of a speaker. For those who like to play at loud / realistic levels, it's currently not easy to find out or understand a speakers capability of uncompressed playback.
My understanding of M-Noise is that it's similar to pink noise up to 700hz or so. I don't know of any case in which any normal home/studio speaker would be SPL limited by frequencies above that. I think this is solving for an issue with how the dynamic capacity of extremely large PA/cinema speakers is measured. Not really a problem that we have.

In general, to me, the test signal is not the hard/interesting part of dynamic capacity testing. The hard/interesting part is what are the conditions for failure. We know that THD is totally useless, but what to replace it with? There ARE options, but Amir is opposed to any single metric. Though I'm not convinced that's reason enough not to use a different metric than harmonic distortion graphs which are unreadable and borderline useless.
 
Last edited:
OP
Rick Sykora

Rick Sykora

Major Contributor
Forum Donor
Joined
Jan 14, 2020
Messages
3,511
Likes
7,011
Location
Stow, Ohio USA
I don't understand the argument that there would always be other biases. It's like saying something will always be bad so there is no point in improving it.

Or like removing all restrictions and laws in competitive sport because you can't make any sport 100% fair.

In my opinion it's just the (******) consumer culture of this hobby, literally with any other industry (Computers, Electronics, TVs, Cameras, .etc) products are looked at under an electron microscope.

Yes, some of my thoughts as well…

More specifically, the reviewer is so inhibited by his own perceptions that the best you can expect from him is a really good subjective review? As I mentioned earlier, one way to wash some bias out is for one person to listen and write his review before another makes the measurements and publishes his findings. Some other ways to reduce the individual biases are currently cost or time prohibitive, but possible with enough resources…

Sort of gets back to my OP where there is more manufacturer accountability. If we cannot get them to act, is another answer that may be we get to a day where enough consistent measurement systems are in consumer hands that we can reasonably compare our speakers? If so, maybe need to hear from a visionary from Dirac or Audyssey?
 

markus

Addicted to Fun and Learning
Joined
Sep 8, 2019
Messages
646
Likes
653
Toole writes "more channels" but he means "more speakers".
I'm pretty sure he means "more channels", i.e. controlled and spatially distinct artificial reflections. Of course this also means "more speakers".
The music industry is not giving us more channels.
Atmos does.
 

tuga

Major Contributor
Joined
Feb 5, 2020
Messages
3,984
Likes
4,281
Location
Oxford, England
I'm pretty sure he means "more channels", i.e. controlled and spatially distinct artificial reflections. Of course this also means "more speakers".

Atmos does.

Most of my recordings are 2-channel (sometimes 3-). Are they being remixed into multi-channel for Amos?
 

markus

Addicted to Fun and Learning
Joined
Sep 8, 2019
Messages
646
Likes
653
Most of my recordings are 2-channel (sometimes 3-). Are they being remixed into multi-channel for Amos?
They are. Check Apple Music. Unfortunately the whole thing is more of a marketing thing than an attempt in advancing the art. As long as the music industry isn't implementing any meaningful standards that's mainly all we get.
 

tuga

Major Contributor
Joined
Feb 5, 2020
Messages
3,984
Likes
4,281
Location
Oxford, England
They are. Check Apple Music. Unfortunately the whole thing is more of a marketing thing than an attempt in advancing the art. As long as the music industry isn't implementing any meaningful standards that's mainly all we get.

I'm perfectly happy with my 2-channel rig.
 
Top Bottom