Speaker Testing: why mono is better

[Coach_Kaarlo]

I am using a unicorn of a speaker to make what is essentially a pointless point - we do not know what we do not know.

So how would @amirm review my speaker/s - in mono?

They are handed by design. Where would he sit or measure? Left of centre or right? Or both?

I have measured them - it makes a considerable difference which off axis direction is outside versus inside so to speak. Timbre and frequency response vary widely as one would expect.

Mono may not accurately give an indication of stereo performance in this case.

[Points if you can guess which speaker I have shown above]

 

[richard12511]

Let's clarify the assumptions here so we are on the same page:
Mono Assumption: Because mono testing leads to the same result as stereo testing in terms of ranking the relative performance of speakers, why waste time with stereo testing to get the same result?
Stereo Assumption: There is value in stereo testing because there are instances where the results will contradict the findings from mono testing; this contradiction is significant enough to sway consumer buying decisions.
Resolution: the issue for further study is whether indeed, the above Stereo Assumption is correct: are there instances where the listener would choose the lower ranked mono tested speaker over a more highly ranked speaker in stereo? And if so, what are those conditions? Once those conditions are identified and defined, we can limit stereo testing to just those conditions.

The resolution is to do both many more times and see if the mono assumption holds over a sufficient sample size. Supposedly it already has at Harman, but we don't have access to that data .

I'm planning a blind listening test with my new ABX tool for (hopefully) end of may. I'm definitely gonna try to do both a mono and stereo test, though if time is too limited, it will be mono only.

P.S. If you're near College Station, TX and interested in participating, PM me.

 

[wpwoodjr]

In the video the point is made that a speaker with poor frequency response sounds closer to "better" speakers in stereo than mono, therefore mono testing is better at judging speakers. That doesn't necessarily follow, as that speaker could actually have pretty good stereo imaging which overcomes its frequency response deficiencies enough to raise it's score.

Stereo imaging is quite important to how I judge a speaker, and the differences between speakers in imaging are not subtle to me. Some people may prefer a speaker which images well to one with flatter response that does not image well.

From the video: "...the brain gets so much enjoyment out of that spatial quality..." - exactly!

 

[boselover61]

And whose ears should he trust?

Not sure, but that's like coming to a math forum and say what's the point of multiplication/division we can just use fingers to count up or down

 

[Todd74]

Regarding preference testing isn't it difficult to make these judgements on absolute differences? Preference scaling is relative. Even if you give a pseudo measure it is still relative. It is not like a frequency response graph, I don't think we know the details of the sample size and statistics generated or the confidence intervals. Without that it would be difficult to make a rational judgement on some of these issues raised. It would be great if the papers were available without the paywall. What is clear is that mono testing shows greater differences than if we do stereo testing. It seems reasonable to me from the study as well as common sense that if you test the individual speaker you will also make a rational judgement if the speaker is to be used in stereo. I think the weight of showing that stereo testing is better would need some sort of study rather than just objections with no data. If people are objecting that stereo testing is needed, are there studies from AES to corroborate this?

Fair points. I’m not necessarily suggesting that stereo testing is better, only that mono testing is flawed. Sure, it seems accurate when it comes to speakers that test well in mono— they’re likely to not test poorly in stereo, nor the ear test [especially if you prefer a neutral or clinical sound], but the mono test unfortunately omits all of the speakers that actually do sound as good as or nearly as good in stereo as that better mono performer.

So like I said, the mono data may in fact steer you away from some speakers that would otherwise meet the requirements in stereo data while offering the sonic personality you prefer. I mean honestly, I’m seeing Dynaudio speakers getting bashed here in mono testing, despite consumers buying these same speakers because they like the Dyn sound.

So, yeah, we need to be testing real world imaging and so forth IMO, so that people can make a more educated decision on a speaker based on the personality it’s meant to have, rather than the personality that someone subjectively believes it should have based on their own personal preferences. It’s basically like- well I subjectively don’t like the sonic personality of this speaker as shown in its mono data, so who cares about how it truly images and so forth in stereo tests? But what if some people prefer that speaker for that very same personality, and that speaker happened to perform significantly better in stereo than in mono? Is it still a headless panther?

 

[richard12511]

Resolution: are there instances where the listener would choose the lower ranked mono tested speaker over a more highly ranked speaker in stereo? And if so, what are those conditions?.

I've personally experienced an example of this with one of my previous blind tests. A Revel M105 was strongly preferred in mono, while a JTR 212 was slightly preferred in stereo. Could just be noise, though, as the JTR only slightly won in stereo, and our switch times were nowhere near the Harman shuffler times. People saying it was "hard to remember" was a common complaint. Also, the fact that it was much closer in stereo is actually what Toole's science predicts. The reason the JTR won could have just been chance(think we did about ~20 songs).

My unscientific guess is that it's gonna take a dispersion width edge case like this to find the exception. The two speakers differ massively in their dispersion widths (Revels are 80° wider on average), so definitely far outside the normal 10-20° you usually see. The differences are striking, to the point that the blind was not possible for me. The JTR sounds kinda bad in mono. No spaciousness whatsoever, and the clearer imaging just highlights that your listening to a box right in front of you. It doesn't "disappear" at all. The Revel actually sounds pretty pleasant in mono. Good spaciousness, and the lack of clarity makes the speaker "disappear" better. In stereo it's much closer. Stereo is more like "this one is more enveloping, but this one images better, hmmm". Your brain also loses the ability to pinpoint the source of the sound(tricked into thinking it's coming from the middle), which is the biggest problem I have with the JTR in mono.

 

[pozz]

we listen in stereo not mono

You listen in mono all the time. Phantom center is mono, i.e., a correlated signal sent through both channels to each speaker. What you mean is that you normally listen to two speakers.

Any defects heard by listening to a mono signal through one speaker you will hear in two channel stereo, just not as reliably. But in certain program material they will come out.

I am using a unicorn of a speaker to make what is essentially a pointless point - we do not know what we do not know.

So how would @amirm review my speaker/s - in mono?

They are handed by design. Where would he sit or measure? Left of centre or right? Or both?



View attachment 120183

I have measured them - it makes a considerable difference which off axis direction is outside versus inside so to speak. Timbre and frequency response vary widely as one would expect.

View attachment 120186

Mono may not accurately give an indication of stereo performance in this case.

[Points if you can guess which speaker I have shown above]

Victor/JVC SX-1000 Laboratory (the model name is printed at the bottom of the speaker, so no points).

Every speaker has a point far enough away where the drivers will sum. You have to sit at least that far away and face the speaker. The test procedure doesn't really change to accommodate.

In this case, if measuring the speaker using a full polar plot the "handedness" will force the listener into a very narrow position where the FR is most even, which points to a certain weakness in design. And of course will show up while auditioning just one speaker.

 

[Todd74]

And whose ears should he trust?

Hey, I love having scientific data available, don’t get me wrong.... but when that data is skewed to favor a more neutral speaker like the beloved Revel, then it automatically puts a stain on other speakers with different personalities that may in fact sound exactly as they were engineered to sound and exactly how their consumers want them to sound. Then we get, “Well, it doesn’t have the flat curve that I subjectively prefer and its mono data doesn’t look good, so it’s not a good speaker”... but what if it actually represent its particular sonic personality perfectly and then knocks it out of the park in stereo data?

If not measuring a flat-intended speaker, I think you would have to know what the manufacturer is aiming for, have lots of charts on hand with which you can overlay new data, and then asses how said speaker performs against the personality expectations of said manufacturer. I mean, if I buy a Dynaudio that measures like a Revel, then I probably just bought the wrong speakers. I want my Dynaudio to measure like a Dynaudio, and I’d like the mono data to tell me how well the speaker represents the Dyn sound, not how closely it sounds to a Revel. And then from there, I’d want to know how well it images in real world. Lotta work though!!... likely too much that would make it feasible.

 

[pozz]

I've personally experienced an example of this with one of my previous blind tests. A Revel M105 was strongly preferred in mono, while a JTR 212 was slightly preferred in stereo. Could just be noise, though, as the JTR only slightly won in stereo, and our switch times were nowhere near the Harman shuffler times. People saying it was "hard to remember" was a common complaint. Also, the fact that it was much closer in stereo is actually what Toole's science predicts. The reason the JTR won could have just been chance(think we did about ~20 songs).

My unscientific guess is that it's gonna take a dispersion width edge case like this to find the exception. The two speakers differ massively in their dispersion widths (Revels are 80° wider on average), so definitely far outside the normal 10-20° you usually see. The differences are striking, to the point that the blind was not possible for me. The JTR sounds kinda bad in mono. No spaciousness whatsoever, and the clearer imaging just highlights that your listening to a box right in front of you. It doesn't "disappear" at all. The Revel actually sounds pretty pleasant in mono. Good spaciousness, and the lack of clarity makes the speaker "disappear" better. In stereo it's much closer. Stereo is more like "this one is more enveloping, but this one images better, hmmm". Your brain also loses the ability to pinpoint the source of the sound(tricked into thinking it's coming from the middle), which is the biggest problem I have with the JTR in mono.

That's the skill, right? To distinguish the criteria of evaluation. Tonal balance and spatial qualities have to ranked separately, the first dominating the second in my experience.

Working theory: I believe narrow directivity designs are not significantly different from using room treatment.

 

[amirm]

Are there any scientific reasons for it, though? Like, could these hidden defects be heard in certain material? The studies would have been much more useful by trying to ascertain this, instead of forgetting this "small" philosophical detail being that if you can't hear it it doesn't matter.

You really have walked away thinking in stereo nothing matters as far as fidelity of the speaker??? I sure hope not.

 

[amirm]

I mean honestly, I’m seeing Dynaudio speakers getting bashed here in mono testing, despite consumers buying these same speakers because they like the Dyn sound.

People buy every kind of speaker. What does that have to do with the reality of how they perform? Polk probably sells 100 times more speakers than Dynaudio. You are going to conclude that Polk speakers are better than Dynaudio?

Also, no speaker gets bashed by me for just mono subjective testing. Objective data needs to be bad as well. All my subjective tests are actually in support of objective testing which by itself is in mono.

 

[amirm]

If not measuring a flat-intended speaker, I think you would have to know what the manufacturer is aiming for, have lots of charts on hand with which you can overlay new data, and then asses how said speaker performs against the personality expectations of said manufacturer.

So if a manufacture thought they don't need to put a tweeter in a speaker and it should only play to 5 kHz, you would be assuming that is the right thing to do?

So no, we are not here to listen to manufacturer's aims unless they can show research that what they are doing is more correct than nearly 40 years of research into what is correct.

Really, that is what we do in this forum. We are guided by body of research. Anything else is back to random assumptions and unfounded ideas. There are a million of them when it comes to speakers. No way we should bend in the wind depending on its direction. Manufacturers need to demonstrate the efficacy of their approach with controlled listening tests. If none exists, then they are going to be subjected to science that is based on that.

 

[MerlinGS]

Stereo imaging is quite important to how I judge a speaker, and the differences between speakers in imaging are not subtle to me. Some people may prefer a speaker which images well to one with flatter response that does not image well.

Although I agree with many of the assumptions and conclusions in Toole's (and by extension Amirm's) argument regarding speaker testing, if memory serves, there is a problem with his methodology. Imaging was not a variable listeners were trained to evaluate. In this context, a speaker with excellent frequency response, constant directivity, and wide dispersion would be advantaged against a speaker with excellent frequency response, controlled directivity and somewhat wide dispersion. Assuming no serious performance problems with either speaker, the wide dispersion speaker would be preferred if the emphasis is placed in a sense of "broad space"; however, if the emphasis is placed on imaging, the speaker with controlled directivity would be preferred since it should image better. Yes, there are many assumptions here, but they are being used to highlight the presumed bias against imaging.

 

[amirm]

It’s basically like- well I subjectively don’t like the sonic personality of this speaker as shown in its mono data, so who cares about how it truly images and so forth in stereo tests?

No, no, no. Did you all not watch the video? Or the dozen times I have explained this? What do you mean who cares how it sounds in stereo? We absolutely care how it sounds in stereo because that is the market. It is simply the fact that we have discovered a way to predict performance in stereo that is faster, more accurate and cheaper. All the evidence points to this, and none to the contrary as you are assuming

 

[Todd74]

People buy every kind of speaker. What does that have to do with the reality of how they perform? Polk probably sells 100 times more speakers than Dynaudio. You are going to conclude that Polk speakers are better than Dynaudio?

Also, no speaker gets bashed by me for just mono subjective testing. Objective data needs to be bad as well. All my subjective tests are actually in support of objective testing with by itself is in mono.

No. I’m going to conclude that Polk sells 100x more speakers than Dynaudio bc they’re infinitely more affordable & available, not bc of quality or “sound signature”. Most people who buy Polk have no clue what a Dynaudio speaker is, my aforementioned buddy included who ironically enough has a Polk home theater and was absolutely blown away by how my Dynaudio theater sounded in comparison.

People don’t buy Dynaudio bc they have no other options at their price point. They buy Dynaudio bc that’s the sound signature they prefer.

 

[amirm]

Imaging was not a variable listeners were trained to evaluate.

Of course it was. You all are impossible. It is all show in the video, the papers and my answers. Once more:

The spatial qualities is the formal name for "imaging." It absolutely was analyzed and studied.

 

[richard12511]

That's the skill, right? To distinguish the criteria of evaluation. Tonal balance and spatial qualities have to ranked separately, the first dominating the second in my experience.

Working theory: I believe narrow directivity designs are not significantly different from using room treatment.

Yeah, we didn't separate tonal balance and spatial quality. In fact, we didn't even rate the speakers at all. We just picked preference of A/B/C, which I now regret. The very first blind test I setup, I asked the people to rate the speakers 1-10. People complained that it was too hard to rate a speaker with no reference to what a 1, 5, or 8, etc. sounds like. I eventually relented after the first few songs and just let them pick preference after that. In truth I was struggling to come up with ratings just as much. Have just done preference since then.

I'm gonna try asking them to rate this time, but stress that it's gonna feel hopelessly hard at first, but will get easier as the test goes on. Not sure if this is valid science, but I may just throw away the ratings of the first 10 songs or(just use the preference data), and average the ratings based on the last ~15.

 

[MerlinGS]

I think spatial qualities does not necessarily mean imaging; i.e. sense of space vs placement of instruments in space. In fact I'm pretty sure that has been Geddes stated complaint against Tooles' work.

 

[pozz]

A good write-up on the difficulty of evaluating spatial quality (and rigorously defining what it means) including Toole's work:

https://www.ece.uvic.ca/~peterd/30604/rumseyattributes.pdf

The paper is very good. Thanks for posting it. But I would note that it is about evaluating spatial qualities, not speakers.

He still summarizes the speaker situation quite well.

And then another point for mono:

 

[Todd74]

No, no, no. Did you all not watch the video? Or the dozen times I have explained this? What do you mean who cares how it sounds in stereo? We absolutely care how it sounds in stereo because that is the market. It is simply the fact that we have discovered a way to predict performance in stereo that is faster, more accurate and cheaper. All the evidence points to this, and none to the contrary as you are assuming

I totally get what you said, and I thoroughly appreciate all the self-help videos [they’re awesome!]. The mono testing is a fantastic tool in 1 regard— the speaker that tests well in mono should also test well in stereo.... but unfortunately the same consistency doesn’t hold as true for speakers that test poorly in mono, as some of them actually test well in stereo and occasionally better than their peers, right? So it seems like the mono testing will criticize some speakers that actually sound good in stereo and maybe happen to represent a good value for the money, while also meeting the sound signature the buyer is after. But like I said, and you’ve said, it’s a lotta damn work... and I’m happy to have the level of work that’s already being provided.