• WANTED: Happy members who like to discuss audio and other topics related to our interest. Desire to learn and share knowledge of science required. There are many reviews of audio hardware and expert members to help answer your questions. Click here to have your audio equipment measured for free!

Four Speaker Blind Listening Test Results (Kef, JBL, Revel, OSD)

Bear123

Addicted to Fun and Learning
Joined
Nov 27, 2019
Messages
796
Likes
1,370
My personal listening experience with one and two speakers. It sounds different for both mono and stereo recordings.
So you did double blind tests with multiple speakers and multiple listeners and were able to determine comparing speakers in stereo gave different results than in mono….oh wait….no, you just listened to one speaker, then a pair, concluded the sound was different, and voila, you singlehandedly debunked decades of rigorous scientific research. Well done. What were we all thinking? What were the researchers thinking??? YOU are obviously right!!!! And all this time, we thought mono listening tests gave the same preference results as stereo. If only you had been around decades ago, the research could have been done correctly! :facepalm:
 

ROOSKIE

Major Contributor
Joined
Feb 27, 2020
Messages
1,917
Likes
3,397
Location
Minneapolis
I still don’t get how would you want to achieve that. One is an in wall speaker, the sound is not coming from the same place, of course they will distinguish the two!
That is the point.
Will they?

I am interested to see the results. Both for trained and untrained participants.
The speakers can positioned at equal heights.

Out of curiosity. Did anyone comment during the test that one speaker sounded like an in wall or sounded like it was positioned higher?


So you did double blind tests with multiple speakers and multiple listeners and were able to determine comparing speakers in stereo gave different results than in mono….oh wait….no, you just listened to one speaker, then a pair, concluded the sound was different, and voila, you singlehandedly debunked decades of rigorous scientific research. Well done. What were we all thinking? What were the researchers thinking??? YOU are obviously right!!!! And all this time, we thought mono listening tests gave the same preference results as stereo. If only you had been around decades ago, the research could have been done correctly! :facepalm:
Prolly should be nicer though I understand why you are jazzed.
(And not everyone agrees 100% that mono testing is superior.)
Since many do advocate for mono listening and the person you replied to doesn't understand why maybe be a pal and link to some good articles and resources for them to check into.

For newbs and others here is a video via Audioholics that is a good place to start that is actually slightly pro stereo and very informative on both approaches.
 
Last edited:
OP
M

MatthewS

Member
Forum Donor
Joined
Jul 31, 2020
Messages
94
Likes
859
Location
Greater Seattle
If you'd heard any of the speakers before, then knowing they were among the lineup could have allowed you to identify them (even if subconsciously) and so bias your judgements. What do the results look like if you remove these two (not fully blinded) listeners?

Only one listener had heard all 4 speakers (me). The other listener saw them, but had only heard one of the four 2-3 years prior. You are welcome to exclude my data--I was listener 1. (Though I was actually one of the last folks to take the blind test).

For my test data, I ranked the speakers in this order for every track but one: Revel, Kef, JBL, OSD.

For Just a Little Lovin, I ranked it as: JBL, Revel, Kef, OSD.

My scores were not identical, but they were pretty tightly grouped for each speaker on each track.

What I will say, is that my rating is likely biased, but not because I own the speakers or have listened to them. Trust me, once you blind it and you randomize it, good luck remembering exactly how they sound, especially when you normally listen in stereo.

It is because I knew what the spin data for the speakers looked like. As I was listening, if something stood out as off or different, I was immediately thinking, those highs sound terrible, that's the OSD. But that's the extreme case with a really terrible speaker. Between the Kef and the Revel it wasn't at all obvious. I even liked the JBL best on one track.
 

amirm

Founder/Admin
Staff Member
CFO (Chief Fun Officer)
Joined
Feb 13, 2016
Messages
44,376
Likes
234,526
Location
Seattle Area
For newbs and others here is a video via Audioholics that is a good place to start.
That's not a good place to start. My video is! :)


The counter is quite weak. No way you can draw conclusions of imaging from anyone's home setup.
 
OP
M

MatthewS

Member
Forum Donor
Joined
Jul 31, 2020
Messages
94
Likes
859
Location
Greater Seattle
Out of curiosity. Did anyone comment during the test that one speaker sounded like an in wall or sounded like it was positioned higher?

No.

The test subjects were almost all sitting off-axis from the speakers. Once you move off axis at 10' our ability to accurately pinpoint a single speaker in that tight cluster of 4 (behind a black screen) is basically zero. We intentionally grouped them closely, and the reason we had them at the back edge of the table was to have as equal a playing field as possible.

We all think we have golden ears, but trust me when I say that once you truly blind it, a lot of what you think you know goes out the window.
 
OP
M

MatthewS

Member
Forum Donor
Joined
Jul 31, 2020
Messages
94
Likes
859
Location
Greater Seattle
Anyone reading this thread should really stop and go watch @amirm's latest video. It couldn't be a more apropos review to follow this test. When we conducted this experiment, we tried to follow as closely the examples provided by Toole in his book given the limitations we had.

 

Chromatischism

Major Contributor
Forum Donor
Joined
Jun 5, 2020
Messages
4,767
Likes
3,706
Like the OP, I also compare speakers in mono.

I started out by downmixing to mono. I hated the way it sounded and couldn't continue the test.

I now just use one channel only.

Is there a consensus on which method is better? Or am I not supposed to use my typical music to downmix to mono?

If downmixing is really the way, I will have another go at it at my next opportunity.
 

amirm

Founder/Admin
Staff Member
CFO (Chief Fun Officer)
Joined
Feb 13, 2016
Messages
44,376
Likes
234,526
Location
Seattle Area
Is there a consensus on which method is better? Or am I not supposed to use my typical music to downmix to mono?
I almost always use one channel of stereo.

The only time I use converted mono music is when I am testing two different speakers on my desk and want to have instant switching using my control software.
 

Chromatischism

Major Contributor
Forum Donor
Joined
Jun 5, 2020
Messages
4,767
Likes
3,706
I almost always use one channel of stereo.

The only time I use converted mono music is when I am testing two different speakers on my desk and want to have instant switching using my control software.
I downmixed to mono through a foobar2000 DSP option and just faded left/right. I couldn't find a way to set a key to go 100% left or right so I used the slider. That was another issue.

Good to hear I'm not the only one who just uses one channel.
 

Stokdoof

Active Member
Forum Donor
Joined
May 14, 2019
Messages
141
Likes
288
Location
The Netherlands

thewas

Master Contributor
Forum Donor
Joined
Jan 15, 2020
Messages
6,757
Likes
16,219
Like the OP, I also compare speakers in mono.

I started out by downmixing to mono. I hated the way it sounded and couldn't continue the test.

I now just use one channel only.

Is there a consensus on which method is better? Or am I not supposed to use my typical music to downmix to mono?

If downmixing is really the way, I will have another go at it at my next opportunity.
Only intensity stereophony is compatible to mono downmix (jn the past most popular music as to be transmitted via radio and listened at small mono radios), time-of-arrival stereophony (usually high audio quality recordings like some Jazz) is not compatible and can sound poor/weird when downmixed.
 

Thomas_A

Major Contributor
Forum Donor
Joined
Jun 20, 2019
Messages
3,423
Likes
2,407
Location
Sweden
Nice experiment. It would be interesting to repeat such an experiment with one speaker and applying various EQ to mimic the different speakers measured on-axis response.
 

fordiebianco

Senior Member
Forum Donor
Joined
Mar 11, 2020
Messages
355
Likes
752
Location
British Isles
2 of the 12 listeners did. The two of us that organized it knew the speakers going in--but we were careful to randomize it so neither of us knew what was what. I connected them all up and did the virtual routing and then covered them behind the blind. My partner then used a random number generator to randomize everything. Inside the software they were just labeled A, B, C, D--but he didn't know what was what. "Speaker 1" for Fast Car and "Speaker 1" for Just a Little Lovin were randomized, they may have been the same and they may have been different.

Many thanks for taking the time and effort to perform the tests and posting the protocol and results. As this is the internet, there are the usual naysayers being naysayers, but I for one appreciate this undoubtedly helpful initiative. It has sparked a really interesting debate on testing and statistics and I think it's definitely frontpage material. I hope others get inspired by your efforts and post something similarly enlightening.
 

uwotm8

Senior Member
Joined
Jul 14, 2020
Messages
401
Likes
457
we thought
I guess "we" should sound more convincing than "me", show more authority etc. Well, maybe it will work next time, keep trying.
Also if decades of research gave "us" such a perfect reference as AKG K371 sound then I have very bad news for Korean investors:facepalm:
 

Semla

Active Member
Joined
Feb 8, 2021
Messages
170
Likes
328
For all the criticism here, this is large, well-executed study that managed to show that untrained, blinded listeners are able to assign different scores to different speakers. Of course one could argue that it isn't perfect, but every experiment is a compromise. The sample size is large enough (>200 observations!) to detect effects given the variability of the data. There was randomization and blinding. And all of this in a hobby project - I've encountered much worse work from professionals.

Given the effort required it doesn't come as a surprise you don't see this type of speaker assessment more often. I do hope this setup serves as inspiration for other members to run and report their own comparisons. Congratulations, and thanks @MatthewS!
 

Yorkshire Mouth

Major Contributor
Joined
Apr 22, 2020
Messages
1,254
Likes
1,154
Location
God's County - Yorkshire
Okay, this is a great experiment, but I feel there’s something more to be done.

It appears that speakers which went lower did better. No surprise there. But what would be interesting would be to throw in a relatively poor speaker, which went lower.

In other words, how much if what we like about a speaker is how low it goes, and how much is made up of how good it is otherwise.

Once again, great experiment, many thanks.
 

Yorkshire Mouth

Major Contributor
Joined
Apr 22, 2020
Messages
1,254
Likes
1,154
Location
God's County - Yorkshire
That's a good point as far as getting louder and lower means much for many people including Amir:)

"Lower means better" for me, too, within certain limits, of course.

My current stereo pairs (B&W DM600 and Monitor Audio Bronze BX1) both sound far better with a sub.

But here's the thing. I wonder how a pair of excellent, but small, bass-lite speakers would fare next to a pair of semi-decent/budget speakers of the same frequency range, but with the latter having a sub added.

I'm going to stick my neck out and guess that the latter would win out every time, as long as they weren't horrendously offensive.

But that's just a guess, I'd love to see it tested.
 

Bear123

Addicted to Fun and Learning
Joined
Nov 27, 2019
Messages
796
Likes
1,370
I guess "we" should sound more convincing than "me", show more authority etc. Well, maybe it will work next time, keep trying.
Also if decades of research gave "us" such a perfect reference as AKG K371 sound then I have very bad news for Korean investors:facepalm:
My tone was snarky, apologies. After proclaiming that comparing speakers in mono is "just not right", you defended your position by saying that listening to one speaker and listening in stereo sounds different. My point is that this has nothing to do with the fact that when multiple speakers were compared in double blind listening tests, the preference results were always the same in mono and stereo, but stereo made things much more difficult and time consuming. I'm not discussing headphones here. It's clear that you hold the research in disdain because it doesn't match with your opinion, but it takes a much more valid argument than "nuh-uh" to negate well established research, whether it matches with ones opinion or not.

Also for the record, I haven't stated my personal opinion on the validity of testing in mono, but your statements like "zealots who promote it like it's a Holy Grail and The Only Way of True Sound" certainly reveals your view on the research and anyone who does adhere to it.
 
Top Bottom