Binaural blind comparison test of 4 loudspeakers

[charleski]

I listened on my EQ'd headphones.

Right out of the gate, loudspeaker C wasn't in the race. On track 2 it sounded like I was listening through a cardboard tube, something was clearly broken here.
The choice quickly boiled down to A vs B. Both had some issues, especially on track 2 (I suspect issues with the recording of this track), but B clearly had a bump in its frequency response. I estimated an error around 1-2kHz, it turns out it had a bump around 2-4kHz; red:speaker A, green: speaker B on track 1:

The net result of this was that the timbral balance became disjointed, especially for instruments at higher registers. This felt like a case of 'hearing something new' that shouldn't have been there. On speaker B the structure of the orchestra fell apart and while speaker A wasn't perfect, it retained that structure far better.

[Note that I made my choice before doing any analysis on the files.]

 

[GXAlan]

It's not blind anymore once you open up the spectrogram.

I didn't do the spectrogram until today.

(I will make it a spoiler tag.)

 

[Thomas_A]

So now I made recording of what I think is the same version as the test tracks. The first file below is a raw recording with my OM1 microphones taped to the ears pointing up to the roof. Severe limitations compared to a true binaural recording, but at least my whole body is there. The OM1 spec here: http://www.lineaudio.se/OM1.html

Because of the raw recording in "binaural mode" any listening should be done with a headphone whose target curve comply with binaural target curve, whatever that curve is:

sing raw.wav

Shared with Dropbox

www.dropbox.com

The second file has a simple and approximate EQ applied to linearise for the room response (as I've measured previously) so a shelf boost above about 2 kHz of 4 dB, a bit more 10-20 kHz and reduction of bass below 100 Hz by 3 dB. This file should then be listened using a headphone that includes the "room response" (Harman target or similar) or through speakers placed in a room.

sing corr.wav

Shared with Dropbox

www.dropbox.com

Overall I think my files sound very different from the test files, which may be due to my simple recording method.

EDIT; I still don't think its the same version in my files....

 

[BostonJack]

I agree with these examples sounded complete off. I thought this was going to be a test of something with a massive HF rolloff as a red herring to see how many people complained about the recordings versus who could hear a difference and rank them. Using AKG K3003 with a Cambridge Audio DacMagic XS. (great IEMs by the way; sold for $1300 10 years ago, now just $400)

I waited until Monday before analyzing the tracks, relying just on my "this sounds off" post as my proof.

Ran the soundprint on the files today for Sample 1
A B
C D

Spoiler

View attachment 155918


In B and D, you can clearly see that missing band around 10 khz which might be what I thought I was hearing. It's not that high of a signal though.

B was the winner in this group and what you can see is that B and D had the highest content in 6K and down range, but D has a broader hole between 5-6kHz.

Really interesting. For me, you might just as well draw a line at 8kHz and ignore everything above. (soon to be 67, def. lost high freq. hearing).
So it makes sense that I wouldn't hear difference between B and D. I must have a preference for that hole at 6-8Khz. Maybe I'm just listening more in the mid-range? My preferences make sense. Congrats to everyone with intact hearing range!

 

[richard12511]

I listened on speakers (JTR 212RT and Genelec 8351b) and one pair of headphones (AKG K371). I also listened over 2 sessions to see if my preferences changed over time(which they did, slightly). On all three systems I had basically the same preference. For the vocal track, I started with slightly preferring speaker D, but transitioned over time to a slight preference for speaker B then D > A > C. Unfortunately, I really didn't like D on the other track. For the other track, I strongly preferred speaker B, then A, then D, then C. C was the worst on both tracks for me, and I feel quite happy to see that no one else likes it either .

 

[GXAlan]

Really interesting. For me, you might just as well draw a line at 8kHz and ignore everything above. (soon to be 67, def. lost high freq. hearing).
So it makes sense that I wouldn't hear difference between B and D. I must have a preference for that hole at 6-8Khz. Maybe I'm just listening more in the mid-range? My preferences make sense. Congrats to everyone with intact hearing range!

I can still hear 18 kHz sine wave with Apple Earbuds but I have to angle it in exactly the right way. I'm not sure if listening to high frequencies at high volumes is dangerous or not for hearing since I'm not sure what SPL it is reaching at full blast.

I probably listen at 65-75 dB and when I go to a non-THX movie theater, I'll bring foam earbuds.

 

[RHO]

Is it 4x the same speaker with different EQ applied or distortion added?

...or a phase shift?

 

[Thomas_A]

Before revealing the speakers it could be useful to know, if possible, what the conditions for the recording was. Room size, furniture, recording distance?

 

[thewas]

Before revealing the speakers it could be useful to know, if possible, what the conditions for the recording was. Room size, furniture, recording distance?

Typical "audiophile" listening, large room and too large listening distance (3+ meters), not very dry acoustics, thus quite more reflected sound than what people like many of us who try to follow the studio recommendations studio are used to.

 

[Thomas_A]

Typical "audiophile" listening, large room and too large listening distance (3+ meters), not very dry acoustics, thus quite more reflected sound than what people like many of us who try to follow the studio recommendations studio are used to.

Ok thanks. One general impression is that the sound is a bit honky or nasal. As the sound is coming from a megaphone.

Is it possible, if known, to disclose what the music is? I don’t think I’ve found the correct version of the Bach yet.

 

[thewas]

Ok thanks. One general impression is that the sound is a bit honky or nasal. As the sound is coming from a megaphone.

Is it possible, if known, to disclose what the music is? I don’t think I’ve found the correct version of the Bach yet.

I am expecting to get some more detailed documentation about the recordings in the next few days, will post it when I get it.

 

[Zvu]

Listening preference from high to low song 1: D>B>A>C
Listening preference from high to low song 2: B>D>A>C

Sennheiser, HD515.

 

[BostonJack]

I can still hear 18 kHz sine wave with Apple Earbuds but I have to angle it in exactly the right way. I'm not sure if listening to high frequencies at high volumes is dangerous or not for hearing since I'm not sure what SPL it is reaching at full blast.

I probably listen at 65-75 dB and when I go to a non-THX movie theater, I'll bring foam earbuds.

Well, I used to think my fellow employees at Lexicon (early 80's) were somewhat affected in protecting their hearing, now I respect their efforts. Between age, some high power rifle competition (with earplugs and muffs), and intermittent use of power tools (generally without ear protection), my hearing loss is no great mystery. Keep protecting your ears!

Do we have a speaker reveal coming soon?

 

[thewas]

Do we have a speaker reveal coming soon?

Sure, when we see that the current discussions are subsided.

 

[Thomas_A]

I think it is interesting to note that despite various headphones or speakers, a majority preferred speaker B and C was the poorest. My big question is the potential room-in-room effect that we get listening on a binaural recording without having binaural target curves. If there is a typical room response at the recording session we would double that by listening through the next room or using headphones with a Harman curve. So if these are raw binaural recordings these speakers should sound brighter than we hear in most of our setups.

 

[oivavoi]

I think it is interesting to note that despite various headphones or speakers, a majority preferred speaker B and C was the poorest. My big question is the potential room-in-room effect that we get listening on a binaural recording without having binaural target curves. If there is a typical room response at the recording session we would double that by listening through the next room or using headphones with a Harman curve. So if these are raw binaural recordings these speakers should sound brighter than we hear in most of our setups.

I'll just mention that it's a fairly common procedure for speaker testing these days to do it exactly like this: Make binaural recordings of loudspeakers, play them for people through headphones, ask them to rate what they hear. I remember at least one such test which was done with Sennheiser 650, without any equalization. These preference tests seems to correlate well with preferences for listening to actual loudspeakers, according to this study: https://www.aes.org/e-lib/online/browse.cfm?elib=16086

 

[Thomas_A]

I'll just mention that it's a fairly common procedure for speaker testing these days to do it exactly like this: Make binaural recordings of loudspeakers, play them for people through headphones, ask them to rate what they hear. I remember at least one such test which was done with Sennheiser 650, without any equalization. These preference tests seems to correlate well with preferences for listening to actual loudspeakers, according to this study: https://www.aes.org/e-lib/online/browse.cfm?elib=16086

Ok but it surely depends on the acoustics in the recording room. Listening to my examples above from a recording in a typical small living room and evaluating using headphones with Harman target curves will not translate to how that speaker sounds in that room. You would need binaural target curves.

 

[oivavoi]

Ok but it surely depends on the acoustics in the recording room. Listening to my examples above from a recording in a typical small living room and evaluating using headphones with Harman target curves will not translate to how that speaker sounds in that room. You would need binaural target curves.

You may be right about that. But if you record speakers A and B in a room, there is high chance that if people will prefer a recording of A over B, they will also actually prefer loudspeaker A over B in reality. The question is not if binaural recordings sound exactly like the loudspeaker-in-room (they don't), but whether the relative ranking of loudspeakers changes because they get recorded. Probably not.

 

[BostonJack]

You may be right about that. But if you record speakers A and B in a room, there is high chance that if people will prefer a recording of A over B, they will also actually prefer loudspeaker A over B in reality. The question is not if binaural recordings sound exactly like the loudspeaker-in-room (they don't), but whether the relative ranking of loudspeakers changes because they get recorded. Probably not.

How much of this pertains to the efficacy of Youtube reviews/comparisons of speakers? I've heard some reviews where the mic'ing seems to be giving a sense of the "speaker in a room". Is there any utility to this type of comparison?

 

[muslhead]

Listening preference from high to low song 1: D>B>A>C
Listening preference from high to low song 2: B>D>A>C

Sennheiser, HD515.

This matched my preference exactly.
I was listening via my desktop setup neumann kh120a & speedwoofer sub.