Binaural blind comparison test of 4 loudspeakers

[PierreV]

Do you know if these come with a calibration curve?

I'll report back when I get them. I actually have no idea what the result will be. I was really surprised either by the low quality of the recordings used in this test or by the speakers themselves (if the recording is OK). There are people on youtube who appear to get very decent results. I am quite sure I won't be able to achieve anything close.

https://www.youtube.com/user/gettingrobbed/videos comes to mind, with the caveat that I heard him a long time ago and can't listen to his videos right now as I am in the office with Logitech Z-10 speakers

The goal is a comparison anyway, not accuracy. As we have just seen, even a poor recording of so-so tracks seems to give consistent results.

 

[Thomas_A]

I'll report back when I get them. I actually have no idea what the result will be. I was really surprised either by the low quality of the recordings used in this test or by the speakers themselves (if the recording is OK). There are people on youtube who appear to get very decent results. I am quite sure I won't be able to achieve anything close.



https://www.youtube.com/user/gettingrobbed/videos comes to mind, with the caveat that I heard him a long time ago and can't listen to his videos right now as I am in the office with Logitech Z-10 speakers

The goal is a comparison anyway, not accuracy. As we have just seen, even a poor recording of so-so tracks seems to give consistent results.

Ok thanks. Is it the VPM-10 model? It says to be used together with the Go Pro cameras.

 

[charleski]

I'm not so sure about this. IME one just adapts to the sound of speakers one listens to regularly, and this sound signature becomes one's personal reference. Therefore it makes more sense for me to get neutral speakers and adapt to its neutral sound (to overcome one's own circle of confusion).

The primary reference to which our ears adapt is direct human speech. Every other reference is swamped by the amount of speech we hear every day. And frankly the notion that you should devote effort to retraining your preferences just to meet a standard of neutrality strikes me as bizarre. Concerns about a ‘circle of confusion’ may be relevant for engineers involved in the production of recordings, but are irrelevant for those of us who just want to enjoy some pleasant music.

I couldn’t disagree more strongly with the idea that domestic listeners should force themselves to adapt to a balance that meets a notional standard just for the sake of aligning with the norm. And frankly it’s utterly unnecessary. The whole beauty of DSP is that it allows the system to be tailored to fit the user, rather than forcing the user to adapt to the system.

 

[Triliza]

Interesting indeed. We should run those types of tests more often. I am toying with the idea of trying with my pairs.
What would be a decent minimal setup to record speakers in stereo? Rode NT4? But I see it is out of stock everywhere right now.
Superlux S502 seems to get good reviews, is in stock and much cheaper but is rated to 40Hz only.
Zoom H series?

@joentell has some binaural tests in his youtube channel. Maybe he can offer some advice on this topic.

 

[gags11]

Thank you for this informative test!
My guesses were really tied to what loudspeakers I heard before and what they reminded me of. I am glad I can tell what a B&W signature is, lol.

I have never heard the Klipschhorn or the ESL, but I had older Klipsch RF7ii, and the ESL reminded me of that.

 

[PierreV]

Ok thanks. Is it the VPM-10 model? It says to be used together with the Go Pro cameras.

CS10-EM

 

[oivavoi]

The primary reference to which our ears adapt is direct human speech. Every other reference is swamped by the amount of speech we hear every day. And frankly the notion that you should devote effort to retraining your preferences just to meet a standard of neutrality strikes me as bizarre. Concerns about a ‘circle of confusion’ may be relevant for engineers involved in the production of recordings, but are irrelevant for those of us who just want to enjoy some pleasant music.

I couldn’t disagree more strongly with the idea that domestic listeners should force themselves to adapt to a balance that meets a notional standard just for the sake of aligning with the norm. And frankly it’s utterly unnecessary. The whole beauty of DSP is that it allows the system to be tailored to fit the user, rather than forcing the user to adapt to the system.

I'd say yes and no. I agree that there's no point in forcing oneself to listen to one kind of speaker over the other, if one's preference goes in another direction. But at the same time, it rings intuitively true to me that one develops a preference for certain kinds of sound reproduction - beyond the human voice in natura. Note that I'm not aware of any research which backs this, it's my gut feeling. If that is the case, isn't it somewhat logical at least to train's one preference to speaker systems which are likely to sound good with a variety of material?

I don't follow that advice religiously myself, though. My own preference is heavily influenced by the fact that I'm exposed to acoustic music almost on a daily basis, just as often as an amateur performer as a listener. I'm therefore sensitive to what I perceive as the artificiality of phantom sources, and I tend to like omnis and dipoles, which to my ears sound more like real acoustic instruments than conventional speakers. But I'm not sure I would recommend such a speaker setup to someone who primarily listens to RnB or heavy metal, for example.

DSP and EQ is always a good idea in any case!

 

[LTig]

I'd say yes and no. I agree that there's no point in forcing oneself to listen to one kind of speaker over the other, if one's preference goes in another direction. But at the same time, it rings intuitively true to me that one develops a preference for certain kinds of sound reproduction - beyond the human voice in natura. Note that I'm not aware of any research which backs this, it's my gut feeling. If that is the case, isn't it somewhat logical at least to train's one preference to speaker systems which are likely to sound good with a variety of material?

I don't follow that advice religiously myself, though. My own preference is heavily influenced by the fact that I'm exposed to acoustic music almost on a daily basis, just as often as an amateur performer as a listener. I'm therefore sensitive to what I perceive as the artificiality of phantom sources, and I tend to like omnis and dipoles, which to my ears sound more like real acoustic instruments than conventional speakers. But I'm not sure I would recommend such a speaker setup to someone who primarily listens to RnB or heavy metal, for example.

I can only speak about my own personal experience. Each time I changed speakers (cheap 3-way Heco -> DIY 3-way with KEF/Audax -> Magnepan MG 1.6 -> Klein&Hummel O300D) I needed some time to really discover the virtues of the new speaker because the frequency response changed. Especially with the first change I was somewhat discontent with the sound of the DIY speakers (recessed mids and highs) while a colleague who came and listened them praised their sound. After I had adapted to them I hated the Hecos for their brigthness and graininess.

 

[LTig]

The primary reference to which our ears adapt is direct human speech. Every other reference is swamped by the amount of speech we hear every day. And frankly the notion that you should devote effort to retraining your preferences just to meet a standard of neutrality strikes me as bizarre. Concerns about a ‘circle of confusion’ may be relevant for engineers involved in the production of recordings, but are irrelevant for those of us who just want to enjoy some pleasant music.

I couldn’t disagree more strongly with the idea that domestic listeners should force themselves to adapt to a balance that meets a notional standard just for the sake of aligning with the norm. And frankly it’s utterly unnecessary. The whole beauty of DSP is that it allows the system to be tailored to fit the user, rather than forcing the user to adapt to the system.

There is no effort required to adapt to the sound of new speakers. It just happens on its own and audiophools name it "break in period".

As I understand hifi it means accurate reproduction of the source. Most members of ASR complain if a DAC or amp has deviations from flat FR by 0.5 dB or more. How then can one accept much higher deviations in very expensive speakers when cheaper ones are more flat? My goal is to have a system as accurate as possible, based on measurements and using room EQ. Then use tone controls to change the FR of bad recordings (or for any other reason) to my personal taste. So far I couldn't be happier with what I have today.

 

[TimVG]

@thewas could you update the opening post with the results? Easier to reference..

 

[LTig]

@thewas could you update the opening post with the results? Easier to reference..

I'd rather prefer a link to the opening post behind a spoiler, so that a new reader still feels the suspense.

 

[charleski]

isn't it somewhat logical at least to train's one preference to speaker systems which are likely to sound good with a variety of material?

If you're 14 and just starting to listen to music seriously, then sure. If you're 50+ and have already spent decades listening, then I don't see the point. We're not talking about differences in material here. A good speaker should be good for metal as well as chamber music no matter where you lie on the distribution. The point is that individual preferences for tonal balance display a degree of diversity, a diversity that is on display in Olive's results as well. The auditory system definitely has a large capacity for plasticity and it's certainly possible to alter one's preferences, but that's putting the cart before the horse: the purpose of your audio gear is to serve your personal needs, you shouldn't have to adapt those to suit the gear.

 

[ctrl]

A great comparison! Thanks @thewas.

Most interesting I found speaker D (Quad ESL 2805), it made a pretty good impression when first listening to the music samples, but gradually it became clear that certain frequency ranges were not present enough.
This was easier with speaker A (B&W 802D2), compared to speaker B (Revel F52).

Hope that this first, rather just fun and limited amateurish attempt might lead to more and better such tests in the future which as shown in existing research can work quite well and you had some fun.

Binaural listening comparisons, done "correctly", certainly have great potential.

It is even sufficient to measure a binaural "impulse" in the listening room. This can then later be convolved with any music signal.
Imagine a database filled with binaural "signatures" of loudspeakers, which can be compared with each other, using arbitrarily selectable pieces of music.

 

[Biblob]

@thewas can you expand a bit more on the mic setup? What kind of mics, distances, room etc.

 

[q3cpma]

If you're 14 and just starting to listen to music seriously, then sure. If you're 50+ and have already spent decades listening, then I don't see the point. We're not talking about differences in material here. A good speaker should be good for metal as well as chamber music no matter where you lie on the distribution. The point is that individual preferences for tonal balance display a degree of diversity, a diversity that is on display in Olive's results as well. The auditory system definitely has a large capacity for plasticity and it's certainly possible to alter one's preferences, but that's putting the cart before the horse: the purpose of your audio gear is to serve your personal needs, you shouldn't have to adapt those to suit the gear.

The assumption here is that innate preference does exist. I wouldn't be surprised in the least if listening habits with non neutral equipement played a big role.

 

[thewas]

I'd rather prefer a link to the opening post behind a spoiler, so that a new reader still feels the suspense.

That's what I immediately added to the original post when I uploaded the results today.

 

[thewas]

@thewas can you expand a bit more on the mic setup? What kind of mics, distances, room etc.

Sure, see following links/posts

Binaural blind comparison test of 4 loudspeakers

I listened on my EQ'd headphones. Right out of the gate, loudspeaker C wasn't in the race. On track 2 it sounded like I was listening through a cardboard tube, something was clearly broken here. The choice quickly boiled down to A vs B. Both had some issues, especially on track 2 (I suspect...

www.audiosciencereview.com

and

Binaural blind comparison test of 4 loudspeakers

Just wait until the owners of B&W and Klipschhorn stumble over this thread ... :confused: Maybe @amirm can promote this thread to the main page and give it some exposure then :)

www.audiosciencereview.com

 

[charleski]

The assumption here is that innate preference does exist. I wouldn't be surprised in the least if listening habits with non neutral equipement played a big role.

I don't really know what 'innate' preference means I don't think any of this is congenital. But I'm pretty sure everyone has a preferred tonal balance that's developed over their entire life. If you spent hours every day listening to one particular badly unbalanced speaker then that might have an effect, but it would be balanced out by exposure to other sources. Direct acoustic speech is the primary determinant of auditory plasticity - listening to a bad speaker in the background isn't going to change things, though blasting your eardrums with some tinny IEMs at high volume will.

But the fact is none of that matters. The system needs to serve the needs of the user, and a system that fails to do that is, at best, the wrong system for that user.

 

[Kain]

Are the recordings still available? When I click on the download link in the first post it states "File Not Found".

Nothing on the above? If anyone still has the recordings/file, will it be possible for you to reupload it if it isn't too much of an issue?

 

[thewas]

Nothing on the above? If anyone still has the recordings/file, will it be possible for you to reupload it if it isn't too much of an issue?

Just uploaded it again and updated the link, please keep in mind though that it will also expire in 30 days.