Speaker Testing: why mono is better

[Blumlein 88]

Correct me if I'm wrong: It sounds to me like you're saying the mono vs stereo spatial quality data Harman isn't showing proves their conclusions, while the data they are showing does not? Please don't take this as an implication of duplicity - I'm merely arriving at a different conclusion from theirs, based on the data they provided. If you see an error in my analysis of their data, please let me know! I have not yet attained infallibility.




In my opinion the acoustic environment was wrong for the Quads. The backwave is ONE HALF of the Quads' output, and they had the backwave firing into medium-weight drapes, which would have selectively attenuated the shorter wavelengths and thus ruined the spectral balance of the in-room reflections (something Toole is normally NOT in favor of). The text which accompanies the diagram mistakenly indicates that the Quads had absorbing pads on the rear half. This would have been true for the original Quads, but was not true for the ESL 63. I have owned both.


View attachment 120396




Thank you for replying, Amir. I can't really argue against the point you make. In your opinion, how much weight should be given to spatial quality in mono versus spatial quality in stereo?

For instance, referring back to the spatial quality scores in your post number 176: Speaker AA wins by a pretty big margin in mono, and speaker E wins by a small margin in stereo. Would you give more weight to AA's score in mono, or to E's score in stereo?

The error is trying to do too much with too little data. In this case only one test of three speakers. BTW, E wins by a small margin in spatial quality while still ranking lower in sound quality in stereo testing. So it appears AA's mono score is more predictive of the overall score in stereo than its stereo spatial quality.

So in the particular case under discussion it follows the general trend. Perceived differences by the listeners are larger with mono, and closer with stereo with overall sound quality being closer, but not changing the ranking one finds in mono testing. So even the data here supports their claim. They are keying on sound quality not spatial quality.

 

[tuga]

And this is BBC engineer H.D. Harwood:

A/B testing

Now the alarming fact is that A /B testing may under certain circumstances give rise to completely wrong results when comparing the sound quality of two loudspeakers.
If pink noise is used as a convenient source, and a deep narrow crevasse produced in it, it has been shown that the effect will be almost inaudible.
If this is listened to for, say, half a minute as if programme were being used to judge a loudspeaker, and then the crevasse is switched out so that a uniform spectrum is produced, the ear will hear a strong colouration at the frequency of the crevasse.
It seems that there are two mechanisms at work; the conscious one ignores the crevasse but the subconscious one detects it clearly.
When the uniform condition is suddenly heard the subconscious mechanism comes forward and points out that there is now a considerable amount more sound energy at the frequency of the crevasse, and as that condition had been accepted as satis-factory the only conclusion to be reached is that there is now an excess in this region and that the sound must now be highly coloured.
Transferring this to loudspeakers it is implied that if one with a crevasse is first listened to then it will probably appear that one with a uniform response is coloured.

 

[Thomas savage]

OK sure, So this comment apply to only the second part my first sentence tough, where I was passing my opinion on your statement, that could have been misunderstood. Note that everything after that first dot, all are punctuated with question marks, of course you don’t have to know or have an opinion on those, but they are simply questions, that I found relevent to the debate, not judgement on what you say, so cool, my post is up there to anyone who have an idea on this. That’s how public forums are. We comment replying to someone, but it’s for anybody to answer.

I'm simply referring to the objective, measurement based testing and attributes known and highlighted via those methods and through the research undertaken by Toole etc where control's and statistical relevance gives weight to subjective assessment.

A Individuals subjective listening tests are something else , the best way to do that is in the room you would be using the speaker , by the person who's looking to aquire them/ it and as intended respecting usage requirements.

If your suggesting a foreign environment, setup and individual would offer something more reliable via stereo assessment vs mono ( subjectively) id tend to think the various contributory factors of setup and room would negate any such advantage. In explicit relation to ASR doing such stereo testing I see only more fog and doubt not more clarity as being the result.

For testing with the ambition of producing as reliable as possible data and direction for people to follow mono seems the only way for us.

Testing speakers and looking for known performance attributes is not the same as ' making science ' however. This is where people get whats done here and what they thinks right somewhat confused imo. In relation to the science it seems mono gives the most reliable data. Iv not seen anything to refute that , id welcome the evidence if it were presented . It would give us something to talk about.

 

[tuga]

The error is trying to do too much with too little data.

There are more errors other than just small speaker and listener samples. And that didn't stop the dogmas from solidifying.
It looks to me as though there was a bit of box-ticking and corner cutting at times in some of the research.

In this case only one test of three speakers. BTW, E wins by a small margin in spatial quality while still ranking lower in sound quality in stereo testing. So it appears AA's mono score is more predictive of the overall score in stereo than its stereo spatial quality.

So in the particular case under discussion it follows the general trend. Perceived differences by the listeners are larger with mono, and closer with stereo with overall sound quality being closer, but not changing the ranking one finds in mono testing. So even the data here supports their claim. They are keying on sound quality not spatial quality.

Which perceived differences are you referring to? Again there's an issue of fitness for purpose.
Blind testing's is obvious, to remove bias.
Pink-noise over a single speaker is fit for assessing tonal balance but as mentioned above Harwood alerts to its unsuitability when it comes to high-Q dips and also peaks, as mentioned in another BBC paper.
Quick AB using music is important to make good use of echoic memory but is manifestly inadequate to encompass all potential speaker performance issues as that would require many tests using varied programme.
Stereo is obviously mandatory do evaluate stereo performance.
The objectivity of single-speaker listening assessments is affected by directivity characteristics.
Both the mono and the stereo shufflers do not take into account speakers' operating requirements such as toe-in or distance to boundaries.
Etc.

 

[Duke]

The error is trying to do too much with too little data. In this case only one test of three speakers.

That's quite possible. Can you be very specific and point out exactly what I stated that was "too much", given the available data?

(Note that the available data I used was apparently good enough for Amir to post it in this thread. Twice.)

BTW, E wins by a small margin in spatial quality while still ranking lower in sound quality in stereo testing. So it appears AA's mono score is more predictive of the overall score in stereo than its stereo spatial quality.

I never stated otherwise.

So in the particular case under discussion it follows the general trend. Perceived differences by the listeners are larger with mono, and closer with stereo with overall sound quality being closer, but not changing the ranking one finds in mono testing. So even the data here supports their claim. They are keying on sound quality not spatial quality.

I never stated otherwise.

But I did state that in this case the spatial quality ranking changed when going from mono to stereo. Did it?

I also stated my opinion that stereo listening is more revealing of loudspeaker spatial quality than single-speaker mono listening. Is it?

 

[DonH56]

We must be able to agree that any reflection is a constant and is therefore undesirable in terms of reproducing the original recorded event in the 3D square around the speakers,
It should simply be like looking into a completely different room from the listening position , reflections will simply destroy Distorts this illusion.

Floyd Toole is a 'researchers' who has tried 'shelf' (reflections) and other errors in the reproduction which until now have been difficult to combat in normal living rooms.
I have never grasped the admiration of Floyd Toole who has written many thick books on everything and absolutely nothing.

HiFi is basically as simple as B&O (Beolab 90) tries to do it, resetting the room so that it does not become part of the equation for hifi,

Dr. Toole is an ASR member; perhaps you should question his credentials and integrity directly in a PM to him.

Personally I abhor the ongoing "audiophile" attacks on anything and anyone technical or "researchers". We've completely flipped from depending upon the science to ignoring and actively denigrating it.

In my personal system I do tend to try to take the room out of the equation, so agree with that premise, though for most of us it is unrealistic.

 

[PeteL]

I'm simply referring to the objective, measurement based testing and attributes known and highlighted via those methods and through the research undertaken by Toole etc where control's and statistical relevance gives weight to subjective assessment.

A Individuals subjective listening tests are something else , the best way to do that is in the room you would be using the speaker , by the person who's looking to aquire them/ it and as intended respecting usage requirements.

If your suggesting a foreign environment, setup and individual would offer something more reliable via stereo assessment vs mono ( subjectively) id tend to think the various contributory factors of setup and room would negate any such advantage. In explicit relation to ASR doing such stereo testing I see only more fog and doubt not more clarity as being the result.

For testing with the ambition of producing as reliable as possible data and direction for people to follow mono seems the only way for us.

Testing speakers and looking for known performance attributes is not the same as ' making science ' however. This is where people get whats done here and what they thinks right somewhat confused imo. In relation to the science it seems mono gives the most reliable data. Iv not seen anything to refute that , id welcome the evidence if it were presented . It would give us something to talk about.

I don't know if I still fully grasp your point, but I agree that there is nothing that can be measured where adding a second speaker wouldn't add data only polluting the results. Intrinsyncly the measured performance tells us all there is to know, or at least to the extent of the current state of the art in measurments. How it "interact" with a second one could never be rigourous enough to be relevant, because the testing conditions will always be arbitrary and will always be a side "symptom" of the individual speaker characteristic. I am only talking about subjective testing. Now, as soon as we put the measurment mic away, I think the main debate that can be had is what is the most rigorous and efficient way to evaluate speakers. We are already moving away from what's the most scientific way to do so in these pages, those would be the types of studies Harman does, not practically achievable.
What, in my opinion, can be debated, is the simple fact that the goal in hifi is, if we want to be objective, should be to evaluate the performance in reproducing, and getting to our ear and brain, a recorded source. In music 99% or more of those are stereo sources. We currently do this now by corrupting those sources before we hear them, Amir listens not what was meant to be heard, but a version of it deprived of a big part of what was artistically expressed. To this I still believe this would be jumping the guns to state that there is a clear rigorous right wrong rigourous answer. Remember, it's a subjective evaluation, no science can remove that fact. As you, I welcome evidence that shows the evaluation of spatial quality for reproducing a stereo image, when removing from the source what constitute a stereo image, is as accurate or better "It would give us something to talk about".

 

[thefsb]

Hopefully next we get a video entitled simply: "Why Mono is Better"

 

[magicscreen]

Now the alarming fact is that A /B testing may under certain circumstances give rise to completely wrong results

Now the alarming fact is that blind testing under every circumstances give rise to completely wrong results

 

[amirm]

In your opinion, how much weight should be given to spatial quality in mono versus spatial quality in stereo?

By whom? A reviewer? If so, in stereo, next to zero. There are incredible number of variables involved in assessment of the spatial quality of a stereo system. How could you possibly rely on any one person's opinion and try to extrapolate to yours and your situation? There are a million ways to position those speakers, adjust their levels, what content is played, where the person sits, etc.

In mono, the assessment can help figure out if you are dealing with a point source or something different.

Really, the best you can hope from a reviewer is correct assessment of the tonality of the speaker. Fortunately that is the dominant factor in speaker preference. What is left is then for you to decide.

 

[amirm]

F.e. one of the BBC papers says that, despite Toole's findings, high-Q resonances may be audible if they hit the fundamental note of an instrument.

Huh? There is no such thing stated by Dr. Toole. His research shows that low Q hits more notes so is more audible in real content. Not that high-Q can't hit any note. This gets us back to listener ability to hear impairments in speakers. The wider the Q, the easier it is for them to hear these artifacts. Same with advantage of mono testing. It allows more artifacts to be heard.

Our auditory filter bandwidth also increases with frequency so at some point, those high Q resonance are simply not heard whether they hit a note or not.

 

[amirm]

In my view and experience AB testing is quite good at determining macro speaker performance differences but requires complementary long-term listening which allows the use of more and varied material, which may highlight previously unnoticed particular issues, and removes the "stress" from the equation.

You don't need long term listening to do all the things you say. In half hour I play tons of different content. Selection of proper content that is more revealing of speaker differences massively helps with this as well.

Your hearing also goes through adaptation if you keep listening over weeks and months, making such assessments harder, than easier.

Ultimately you are expressing typical lay audiophile intuition which in the context of this topic is wrong. It is demonstrated to be wrong using controlled listening tests over decades. If anyone thought different, by now they could have published studies saying otherwise but they have not. For you to substitute what you think is of zero to negative value.

The rest of your post also confused content creation with speaker evaluation.

 

[amirm]

The loudspeakers are placed behind an acoustically transparent but optically opaque curtain, especially if any aspect of the units under test might be visually identifiable.

Didn't read you saying your assessment is done this way. Surely you are not listening for weeks and months in this manner. So what you are quoting is not backing what you are saying.

The paper also says nothing about mono vs stereo testing. And vast majority it has nothing to do with subjective listening of speakers in domestic situations.

 

[Duke]

In your opinion, how much weight should be given to spatial quality in mono versus spatial quality in stereo?

By whom? A reviewer? If so, in stereo, next to zero...

Sorry that my question was unclear; I didn't have reviewers in mind. As an example of the sort of [admittedly hypothetical] situation I had in mind, I wrote:

"For instance, referring back to the spatial quality scores in your post number 176: Speaker AA wins by a pretty big margin in mono, and speaker E wins by a small margin in stereo. Would YOU give more weight to AA's score in mono, or to E's score in stereo? [emphasis added this time]

 

[Duke]

To get to a place where one believes mono testing inadequate vs stereo setup we must then think theres a unknown, ' magical' quality happening when you put two speakers in a stereo configuration. We must think theres a attribute in each we don't understand and can't identify in singular configuration. I see no evidence of this . Has anyone got any ?

If you would be interested, I can describe what I believe to be a stereo setup where something desirable happens to the spatial quality which cannot be evaluated by testing a single speaker. No magic, but some unorthodox thinking from the mind of Earl Geddes. I would be describing a situation which is applicable to a relatively small number of loudspeakers, which are designed to be used a certain way. Let me know if you'd like for me to proceed.

 

[hex168]

Speaking as a bystander to this conversation, I would be interested. In fact, I was thinking about Geddes' cross-firing CD speakers in this context, and was hoping you would chime in.

In all other ways, I agree that the research shows that listening in mono is superior to stereo in discerning differences.

 

[amirm]

"For instance, referring back to the spatial quality scores in your post number 176: Speaker AA wins by a pretty big margin in mono, and speaker E wins by a small margin in stereo. Would YOU give more weight to AA's score in mono, or to E's score in stereo? [emphasis added this time]

I don't give much weight to stereo assessments of anything due to high number of nuisance variables, to borrow a term from Dr. Toole. Also, small margins in context of speaker testing is not that useful due to variables involved.

 

[Jdunk54nl]

Nearly 300 posts of people's feelings yet no one is willing to do the necessary research to prove otherwise. If you don't believe the science that is presented to you, do your own legitimate research and prove it wrong. If you are unwilling, then stop with your feelings and your opinions. Back up what you say with data from legitimate research that can then be independently verified by others.

I am pretty confident that if the National Research Council and Harman Research would have found benefits to multiple speaker testing compared to single speaker testing, it would have been done that way instead of the way it is done now. What would either of these have to lose by doing it that way IF it was better? Harman would then be able to make even better speakers and the National Research Council would have even better data. So ask yourself why are they not doing it?

 

[Duke]

Speaking as a bystander to this conversation, I would be interested. In fact, I was thinking about Geddes' cross-firing CD speakers in this context, and was hoping you would chime in.

Thank you, will do. It might take me a little while to pull together and organize the information I want to include.

I don't give much weight to stereo assessments of anything due to high number of nuisance variables, to borrow a term from Dr. Toole.

Again, thank you for replying. I know you're very busy and appreciate your taking the time.

I would have thought the "nuisance variables" were adequately controlled in the stereo half of the controlled blind test whose data you posted, but maybe not.

So I infer from your reply that you would disregard speaker E's slightly higher spatial quality score in stereo because of the nuisance variables, and you would therefore conclude that speaker AA has superior spatial quality because it scored higher in that category in mono. No reply requested unless I'm misunderstanding you and you want to correct that.

 

[Blumlein 88]

If you would be interested, I can describe what I believe to be a stereo setup where something desirable happens to the spatial quality which cannot be evaluated by testing a single speaker. No magic, but some unorthodox thinking from the mind of Earl Geddes. I would be describing a situation which is applicable to a relatively small number of loudspeakers, which are designed to be used a certain way. Let me know if you'd like for me to proceed.

As mentioned the problem is the variables. Maybe speaker A would also have bettered E in spatial qualities with different stereo positioning. As you well know, you can make considerable trade offs in precise imaging and spatiality with most speakers by how you position them. And it makes sense with arbitrary stereo placement the stereo spatiality scores might not track so straightforwardly as single speaker sound quality scores.