Reviewing Speakers - Measurements and Listening Tests (Video)

[Duke]

A speaker with consistent directivity means that all the details in the recording will be properly transduced into the room at all frequencies.

Imo it may not be that simple, at least from the standpoint of spatial quality.

The line of well-funded scientific inquiry which led to the current state-of-the-art conclusions about loudspeaker directivity DID NOT INCLUDE "the recording's spatial cues dominating perceived spatial quality" as a deliberate priority.

In other words, we have a very clear picture of how to get the tonality right, and I agree that that's the top priority. But we do not have nearly as clear a picture of how to get the spatial quality right, particularly IF "getting the spatial quality right" means that "the recording's spatial cues dominate the perceived spatial quality".

 

[goat76]

If we can’t judge if a speaker is bright, overly bassy, or lacking in tonality in one way or another without comparing it to at least 2 or 3 other speakers at the same time (according to Harman), why is a balanced tonality even that important to begin with?

I really don't have much problem with that, if I use music I'm familiar with and have been listening to for many years it's easy to hear if something is off in one way or another. I'm sure everyone who's demo a pair of speakers, review speakers, share impressions of their own speakers, all use music material they are familiar with and have heard on many occasions.


Until someone tells us exactly what we should look for in the measurements that indicate how a pair of speakers do when it comes to spatial performance, envelopment, and soundstage depth and width, I think we will need all the subjective listening impressions we can come by to give us a more complete picture of the speaker's performance in this regard. Don't put all your trust in one single reviewer's subjective impressions no matter if it's Amir, Mr. Atkinson, or someone else. If many people think a pair of speakers perform particularly well with soundstage, imaging, and spatial clues, it can be a real indication that these particular speakers do better than other speakers with these things.

 

[Thomas_A]

I think judging without relative comparison is rather easy when there are large differences, as usually are with headphones. At least I have some "inner reference" including its bias. Absolute judgement is difficult when the actual differences become smaller, and is much easier when you can compare to a reference.

 

[BullBuchanan]

This was an awesome watch, and very informative. I'm not quite sure I understand the one speaker approach. I get that it helps people "measure" the frequency response of a given speaker much more easily using their ears, but isn't FR just one component of others? If that's all it is, I'm sure you could get a speaker from 50 years ago tuned to the harman response and you'd be set, no?

Amir mentioned in his video about cables that we can't measure them unplugged, because that's not how they're intended to be used, and that's not how we listen to music - a statement I totally agree with. Why does that not matter here? It doesn't seem tome an illusion of a live performance is possible with a single speaker, but does become much more so with a stereo recording and two speakers capable of creating that illusion. Are we saying that's false and that any two speakers with near identical FR will sound near identical when playing music?

 

[Spocko]

Imo it may not be that simple, at least from the standpoint of spatial quality.

The line of well-funded scientific inquiry which led to the current state-of-the-art conclusions about loudspeaker directivity DID NOT INCLUDE "the recording's spatial cues dominating perceived spatial quality" as a deliberate priority.

In other words, we have a very clear picture of how to get the tonality right, and I agree that that's the top priority. But we do not have nearly as clear a picture of how to get the spatial quality right, particularly IF "getting the spatial quality right" means that "the recording's spatial cues dominate the perceived spatial quality".

Good point, and considering that DSP is taking over this very exercise as a way to both address speaker design issues and spatial effects, I'm wondering if this discussion won't be completely moot in 5 years?

 

[Frgirard]

Imo it may not be that simple, at least from the standpoint of spatial quality.

The line of well-funded scientific inquiry which led to the current state-of-the-art conclusions about loudspeaker directivity DID NOT INCLUDE "the recording's spatial cues dominating perceived spatial quality" as a deliberate priority.

In other words, we have a very clear picture of how to get the tonality right, and I agree that that's the top priority. But we do not have nearly as clear a picture of how to get the spatial quality right, particularly IF "getting the spatial quality right" means that "the recording's spatial cues dominate the perceived spatial quality".

Tonality quality is measurable therefore objectivable.
Spacial quality is a brain build not measurable and not objectivable.

A speaker without spacial quality - IMO - doesn't exist.

 

[amirm]

I get that it helps people "measure" the frequency response of a given speaker much more easily using their ears, but isn't FR just one component of others?

It is by far the most important factor that determines preference in controlled testing. Every speaker tested ostensibly has different spatial qualities yet none of that saved them if their frequency response and directivity was not good.

If there is something to spatial qualities beyond what the content and room dictate, it would only matter if two speakers were superb as far as frequency response to begin with and this is rarely the case.

 

[amirm]

But we do not have nearly as clear a picture of how to get the spatial quality right, particularly IF "getting the spatial quality right" means that "the recording's spatial cues dominate the perceived spatial quality".

As I just noted, whatever is there is not a strong property of the speaker or it would screw up the preference tests.

 

[Thomas_A]

Frequency response, dispersion and distortion form the base qualities. However there is probably a limit to those; having good enough quality makes it harder to find differences.

With respect to spatial qualities it is a speaker-room interaction. First there are details in the recording that can be more or less masked due to reflections. Second there are reflections that mask the flaws of stereo and also, add to room ambience. Third, there are reflections giving cues to sound souce distance. As I see it one have measure the ratio of direct: reflected sound of early and late reflections and SPL of the direct:reflected sound to have some index of spatial quality. There are several factors being involved: room, speaker, speaker position, listener position, and angles of reflection.

IMO, very difficult (but not impossible) to standardize conditions for spatial quality.

 

[Deleted member 30699]

It is by far the most important factor that determines preference in controlled testing. Every speaker tested ostensibly has different spatial qualities yet none of that saved them if their frequency response and directivity was not good.

True ... but ... a speaker with good directivity and headroom can be EQ'd (or corrected) to pretty much any tonality.
Isn't this what what active speakers do?

PS. I really liked the comparison in the video with color science.
Audio really is like a video projector. You can calibrate (EQ) it in the dark (nearfield) but you have to control or compensate for ambient light (room) for best results.

 

[Stoutblock]

A very difficult subject explained very well. If we can just get more audiophiles to accept this logic on something as controversial as speaker performance, maybe there is hope for the use of a scientific basis with evaluation of audio equipment in general?

With that said, there is still a fun part of this hobby where we can make up subjective stuff about the synergy of all the equipment in a system because we know increasing the variables would require an exponential increase in objective measurements and a scientist’s head may simply explode In the process.

 

[Deleted member 30699]

A very difficult subject explained very well. If we can just get more audiophiles to accept this logic on something as controversial as speaker performance, maybe there is hope for the use of a scientific basis with evaluation of audio equipment in general?

I think it is a question of boundary.
We would need to decide whether DSP/EQ modifications counts for passive speakers.
Once you put even the most linear speaker in a room, there are room dependent modes and reflections. People that care about this EQ the room. Alternatively, we can EQ room+speaker...

 

[Stoutblock]

I think it is a question of boundary.
We would need to decide whether DSP/EQ modifications counts for passive speakers.
Once you put even the most linear speaker in a room, there are room dependent modes and reflections. People that care about this EQ the room. Alternatively, we can EQ room+speaker...

Agree we can improve (if not fix) system performance in a given room with RTA/DSP/EQ, but that does not help the observers of audio reviews that are trying to decide on the selecting of specific components to purchase. Helping observers make good choices on purchasing of components, I would hope, is the motivation of all audio reviewers?

Unfortunately, I doubt very much this is universally true. IMHO Amir has absolutly no hidden agenda in his reviews which I find very refreshing. For someone who can certainly justify exposing his ego, he is very modest in his approach and his logic is solid.

Right or wrong, reviewers focus on performance of individual components. Amir correctly uses scientific methods to compare “as measured” performance against “accepted” standards that should expose how they would perform in a system. Granted that system may improve (RTA/DSP) or be detrimental (No AC power purification, ha! Could not resist) to the performance of that component.

In my own 2 channel stereo system I have 18 drivers, driven by 8 channels of amplification, controlled by 8 channels of active crossovers and digitally managed by RTA analysis with DSP control. Yes I am a believer in system synergy as the final solution. Ears, room and speakers being the most important elements of the system, speakers being the most important component of a system.

Most all audio reviews are focused on components in order to be useful to the audiophile marketplace, I do wish there were more audio reviews of individual audio systems. Perhaps not as useful, but possibly more interesting to many of us?

 

[PeteL]

Tonality quality is measurable therefore objectivable.
Spacial quality is a brain build not measurable and not objectivable.

A speaker without spacial quality - IMO - doesn't exist.

Do you think it delivers a good quality soundstage?

 

[Duke]

As I just noted, whatever is there is not a strong property of the speaker or it would screw up the preference tests.

How could you know whether something you don't look for is "screwing up" the preference tests? If you only look at preference in mono, how can you know whether that overlooks a spatial property of the speaker which manifests in stereo moreso than in mono?

Fortunately we do have data, albeit not much.

I'm sure you've seen this before, 3rd edition of "Sound Reproduction: The Acoustics and Psychoacoustics of Loudspeakers and Rooms", by Floyd Toole, page 179. In mono, the Rega clearly outscores the KEF in spatial quality; but in stereo, the KEF marginally out scores the Rega:

Obviously spatial quality preference can change when going from mono to stereo. Whether or not that has any significant implication is, I suppose, a matter of opinion.

 

[Thomas_A]

Is there a definition of spatial quality?

Is it the details and reflections recorded?

Or added room reflections?

Or a combination?

 

[Duke]

Is there a definition of spatial quality?

Is it the details and reflections recorded?

Or added room reflections?

Or a combination?

EXCELLENT questions! Imo figuring out "where the goal posts are" is a valuable step, and spatial quality being a multi-faceted thing, there may be more than one viable set of goal posts.

Here is the questionnaire listeners filled out to come up with the scoring shown in post #95 above, questionnaire appearing on page 178 of the 3rd edition of Floyd Toole's book:

Notice that "Definition of Sound Images", "Continuity of the Sound Stage", and "Perspective" are not even evaluated in mono, presumably because they cannot be. They are evaluated in stereo only.

 

[amirm]

How could you know whether something you don't look for is "screwing up" the preference tests? If you only look at preference in mono, how can you know whether that overlooks a spatial property of the speaker which manifests in stereo moreso than in mono?

As you showed below spatial qualities also manifest in mono. No matter what the speaker is or is not doing, it is not able to overcome the power of frequency response.

 

[Thomas_A]

EXCELLENT questions! Imo figuring out "where the goal posts are" is a valuable step, and spatial quality being a multi-faceted thing, there may be more than one viable set of goal posts.

Here is the questionnaire listeners filled out to come up with the scoring shown in post #95 above, questionnaire appearing on page 178 of the 3rd edition of Floyd Toole's book):

View attachment 148492

Notice that "Definition of Sound Images", "Continuity of the Sound Stage", and "Perspective" are not even evaluated in mono, presumably because they cannot be. They are evaluated in stereo only.

Thanks for the protocol. I think that the spatial cues present in the recording can be objectively measured and good retrieval is mainly related to a a very linear frequency response in the listening window (and reasonably low distortion) including reflections occurring within a few ms. A poor response or very early reflections around the speaker position could mask those ambient details. As long as the frequency response is good on-axis without resonances, a narrow dispersion speaker would probably be preferred since other nearby reflections are attenuated. Or an on/in-wall speaker.

It is more complicated regarding the remaining room reflections, and even more so in stereo (envelope, perspective). Some prefer a more "diffuse, wide and big opening of sound", others prefer the opposite with "detailed imaging". This would relate, among other things, to the ratio of direct to reflected sound in the room.

 

[Duke]

As you showed below spatial qualities also manifest in mono. No matter what the speaker is or is not doing, it is not able to overcome the power of frequency response.

I don't think I ever said spatial qualities did not also manifest in mono, nor that frequency response was not the most dominant factor in preference.

Regarding spatial qualities "overcoming the power of frequency response", let's take a close look:

It is clear that the Rega is preferred overall in mono, as it wins in both Sound Quality (by a small margin) and Spatial Quality (by a larger margin). But in stereo, while Rega still wins Sound Quality by a small margin, it now LOSES Spatial Quality by a similar margin. Therefore which "wins" overall in stereo depends on how we weight Sound Quality versus Spatial Quality. At this point let me remind you that Wolfgang Klippel's findings would weight Spatial Quality either EQUAL TO or GREATER THAN Sound Quality, see pages 185-186 of Toole's book, third edition.

Which is the most applicable to how loudspeakers are used out in the real world: The Rega's clear victory in mono, or the virtual tie between Rega and KEF in stereo? The difference being the KEF's improved Spatial Quality in stereo.

Did the KEF's improvement in Spatial Quality in stereo "overcome the power of frequency response"? I think the answer depends on which yardstick is used, so I would say "maybe".

As long as the frequency response is good on-axis without resonances, a narrow dispersion speaker would probably be preferred since other nearby reflections are attenuated.

We used to think that listeners varied a great deal in their perceptions of sound quality, such that individual preference would be the dominant factor in loudspeaker preference. Well, Floyd Toole et al showed us that this is incorrect - that listeners have generally similar preferences when it comes to sound quality.

At this time I THINK we are in that pre-Toolian era, as far as our knowledge of spatial quality preference goes, at least for two-channel. It seems like there is a wide range of individual preference, BUT if we could sit people down in a room where spatial qualities were varied while sound quality stayed the same (sort of the inverse of the single-speaker-away-from-walls listening in Harman's shuffler room), we might find spatial quality preference to be fairly consistent, just like sound quality preference turned out to be. I speculate the hardcore multichannel guys (cough cough @Kal Rubinson cough) may be decades ahead of us two-channel guys in this area.

It is more complicated regarding the remaining room reflections, and even more so in stereo (envelope, perspective). Some prefer a more "diffuse, wide and big opening of sound", others prefer the opposite with "detailed imaging". This would relate, among other things, to the ratio of direct to reflected sound in the room.

Agreed.

All in-room reflections contain spatial information from the recording, AND spatial information about the playback room (among other things). I think our appreciation of the contributions of off-axis sound to preference (in stereo) would only be increased were we to learn more about spatial quality preference.