Blind Listening Test 2: Neumann KH 80 vs JBL 305p MkII vs Edifier R1280T vs RCF Arya Pro5

[MatthewS]

@Talisman

I'd argue room treatment isn't particularly necessary--it's a living space and to fix bass with room treatment means living in a padded room with no space to move.

I have measurements from my small room with multiple subwoofers and bass management:

Revel In-Wall and Multi-Sub Optimized (MSO) Listening Room Build

A while back I set out to put the best sound possible (at resonable cost) into our small den. Being only 11’ by 11’, there really isn’t room for a bunch of home theater, audio equipment, or speakers. But I still wanted great sound. Design Targets: Slim mount 65’ TV In-wall speakers 2 hidden...

www.audiosciencereview.com

Check out the measured response before and after.

 

[Blumlein 88]

Listening blind test with a single speaker?

Read this post about that.

Blind Listening Test 2: Neumann KH 80 vs JBL 305p MkII vs Edifier R1280T vs RCF Arya Pro5

I didn't explain this in the video--but it was actually the musical selection that sounded terrible. We wanted something classical and this piece was on Harman's list. But it wasn't recorded well and was extremely hard to pick out differences. It was more the recording sounded terrible, not...

www.audiosciencereview.com

 

[hex168]

From the fact that by comparing the anechoic responses to those in the room, it is very clear how the footprint of the room, especially for the medium and medium-bass frequencies, substantially aligns all the speakers making the small frequency response improvements typical of a better speaker, essentially irrelevant. For many, it was the amount of bass perceived, not the response, that made the difference.
View attachment 275585
From around 600hz down the responses are virtually identical with the peaks and troughs of the room. only the reinforcement of the low frequencies in the jbl is noticed (which in fact has been noticed and preferred by some)

My understanding is that in-room steady state response is not what we hear.

The response we hear is more the direct arrival response (same as anechoic or gated), plus early arrival reflections (less than ~6msec), and then to a lesser degree later reflections. The frequency response of later reflections is perceived separately, and it detracts if it is not smooth, but the ideal rolloff of later reflections is not known and probably depends on the directivity of the speaker, distance to the sidewalls and their reflectivity.

All of this is mixed up inseparably in the steady state response, which therefore is not what we hear. Hence the recommendation to EQ the speaker's anechoic response above Schroeder, but not the steady-state response.

 

[Guerilla]

Why should you do that? It's about the complete loudspeaker. A pleasant bass reproduction is part of the sound impression if you want to use the speakers alone, without a sub. (This is how I do it)

If you ask you wont get it

 

[Guerilla]

Absolutely. As sweetchaos mentions Put a low cut so amount of bass wont affect result. Pretty useless as is

 

[Newman]

Congratulations, I commend your effort and intent.

I kept thinking, as I read your report of your struggles, that this really highlights the magnitude of the achievement of pinnacle researchers in this field in eliminating possible sources of error, achieving statistical confidence and, even more importantly, training listeners and designing test scorecards and criteria of assessment that allow meaningful discrimination between sounds.

Thank you.

 

[ROOSKIE]

What would I change in future tests of this kind? Probably I would not sit with a hard reflective wall a foot behind my head.

That was my 1st reaction, but I am not used to a wall right behind me and when I experience it elsewhere I think everything sounds pretty bad.
Very cool that OP did this testing and critically I am surprised that the seating position was chosen.
It is an unfortunate spot.
Maybe it is what those folks are used to? @MatthewS would you mind if I ask, why that spot?

However, decades of double-blind tests show that listeners hear problems in loudspeakers more readily when listening in mono. The superior sound quality was less clearly reflected in scores in stereo comparison tests, and even less in multichannel evaluations. Monophonic components exist in stereo and multichannel programs, so designing loudspeakers to meet the most stringent (mono) test was considered worthwhile. But,the question remains; why? Were the audible defects more clearly revealed because the spatial complexity and inherent amplitude/phase (linear) distortions of stereo were absent? Is this why headphone listening has such an almost magical clarity? One sound to each ear, not two. It does seem reasonable.

This is the background within which the question is being asked. That there are people who think stereo is somehow a naturally superior form of reproduction is a testament to human tolerance and adaptability.

Preface, I completely understand the rationale for a mono-cetric testing approach.

That said, since my interest is fixed in 2 channel music, this is actually why I prefer doing my subjective testing in stereo. If flaws are hidden when the speakers are only ever used in a particular way, are they flaws( in that scenario)? I understand when designing for multiple use senarios & certainly from a technical perfection mindset that ever exposing the flaws is very cool. In reality if say a speaker has a cabinet resonance that is audible from 1" aways but impossible to hear with 85db of content being reproduced and sitting 10feet away would I ultimately care? If stereo masks flaws in a similar fashion as listening distance does to a cabinet resonance- then it seems,' it is what it is' = gone.

Of course I still want manufacturers to do what needs to be done to make speakers that excel under lab conditions and in mono. As a buyer if real room conditions & stereo, hide/mask certain issues and then the price and saving are right it seems approriate to concede the issues are not relevant(in that application).

For me in terms of enjoyment when I have a listening session, stereo is fastly superior to mono. I love having a soundstage in front of me that helps create the illusion of a live perfromance or at least a landscape in front of me. I am sure multichannel could ultimately do this better, though for now(last 25 years) the elegance of of only 2 speakers handling this is very rich to me and it is quite magical even.
If it is just background listening then whatever, but for sitting in the 'audio arena' by myself or better yet with my GF I just love the stereo effect. I would not be able to tolerate it's loss for a modest gain in tonality.

If 3channel recordings somehow become more commonplace I could see that being an upgrade for me, I seriously doubt mono would be a upgrade over stereo for me(for enjoyment sessions) though I have never actually considered it beyond occasional testing (and I am not suggesting you are suggesting doing so).

I deffinetly appreciate playback via speakers far more than any headphones I have ever tried. Obvioulsy not apples to apples but I can't think of a time where I was ever stunned by the expereince of headphones like I am/&have been with speakers. I use them from time to time and it is fine and sometimes a wonderfull moment but never as truly enthralling nor as realistic sounding to me. Talk about having to tollerate and to adapt to an 'unatural' only in the head type of sound space with near zero bodily involvment. Yikes & I realize for some it is even the full opposite preferance so who knows???

 

[Ralph_Cramden]

Just checked my Amazon order history. Yes, I did pay $99 each for my lovely little JBL 305p MKII's!

 

[LTig]

We may be thinking about this differently. I was thinking you use a microphone to match SPL levels. (Yes, even as I often say for other components use a voltmeter at the speaker terminals.) So most impedance differences would not matter done that way though it could matter in some cases and I don't know the best way to take care of that.

If you compare power amplifiers feeding a speaker and you measure the level of bandlimited pink noise (regardless whether at the output terminals or using a mic) then the poweramp with higher output impedance will lead to more bass and treble compared to the other power amp, giving the former an advantage (loudness effect), because your test signal has neither lows nor highs and hence does not reveal the different frequency responses. In those cases it might be better to measure pink noise (not limited) at the speaker terminal.

 

[Blumlein 88]

If you compare power amplifiers feeding a speaker and you measure the level of bandlimited pink noise (regardless whether at the output terminals or using a mic) then the poweramp with higher output impedance will lead to more bass and treble compared to the other power amp, giving the former an advantage (loudness effect), because your test signal has neither lows nor highs and hence does not reveal the different frequency responses. In those cases it might be better to measure pink noise (not limited) at the speaker terminal.

Maybe.....
Those differences in output impedance interaction are usually smaller than the difference in one speaker having much deeper response or having some hump in response at either end of the spectrum. Or having a sloping treble or a hot treble etc. I think that is why Toole suggested using the more limited pink noise. I've done it both ways and the bandwidth limited way seems to work better. I fear with speakers there may be no perfect way to match levels because unless FR is matched you really cannot fully match levels.

An example with a more precise device. Comparing microphone pres once, I think they had 9 of them. Anyway, one of them had a slope starting about 8 khz and was down 3 db at 20 khz. It also rolled off below 30 hz. Another had reduced treble though less than the previous one. All the others were flat enough to 20 khz not to matter. Those two were the only one's detected reliably as different unsighted. If using pink noise, you'd slightly boost the one with roll offs at both ends to get a match on voltage. So through the midrange where most music is it would be louder. It still sounds different, but probably would get preferred due to the louder midrange. In this test they matched a tone at 1 khz which I think was the right call. Speakers are more complex, but I think matching the core midrange of the music makes sense.

Now this is my opinion (at least somewhat informed), I'm in no position to say it is best or only way.

 

[MatthewS]

Maybe it is what those folks are used to? @MatthewS would you mind if I ask, why that spot?

Short answer: Balancing limited time with variables to control.

Long answer:

It's our living room, I have small kids, and the listening test was conducted over multiple days because our listeners weren't avaiable to all come at once. We had to take everything down and put it all back together each day. Trying to ensure that folding chairs got in the exact same position each time was one too many variables to control and frankly we were still rushing to get the turntable tuned, keep the levels matched, etc.

If you look at our planning post, we intended to have someone seated away from the wall:

Speaker Blind Listening Test Part 2 Planning

@Inverse_Laplace and I will be conducting a second blind listening test. If you missed the original results you can read about the blind listening test here: https://www.audiosciencereview.com/forum/index.php?threads/four-speaker-blind-listening-test-results-kef-jbl-revel-osd.25818/ We are...

www.audiosciencereview.com

 

[Floyd Toole]

Maybe.....
Those differences in output impedance interaction are usually smaller than the difference in one speaker having much deeper response or having some hump in response at either end of the spectrum. Or having a sloping treble or a hot treble etc. I think that is why Toole suggested using the more limited pink noise. I've done it both ways and the bandwidth limited way seems to work better. I fear with speakers there may be no perfect way to match levels because unless FR is matched you really cannot fully match levels.

Absolutely true!

 

[Floyd Toole]

@Floyd Toole

I’ve read your book. An earlier edition. Apologies if I missed this in a later edition.
In developing the Preference score, did you (or Sean) move the speakers into a room and retest and see what happens?
Did you retest in stereo see what happens to the preference score?

You said: "In developing the Preference score, did you (or Sean) move the speakers into a room and retest and see what happens?" Of course we did, that is where the double-blind subjective evaluations were performed. Measurements in the room - steady-state "Room curves" are not greatly informative above the transition frequency - one needs the anechoic data to interpret them.

The preference score was an academic exercise to test an hypothesis - namely that there was a correlation between spinorama data and double-blind listener evaluations. It served its purpose, but in real life I know of nobody in Harman who uses it - including me - we use two eyes and a brain to examine the family of curves and arrive at conclusions that embrace trade-offs. For example, simple bass extension - a limited bass extension can result in a lower calculated score, but in the real world a properly integrated subwoofer system can elevate it to an even higher level. A single significant resonance or a spectral tilt can be equalized to attenuate its audibility, but many resonances would present a serious problem, yet both could yield similar scores. And so on. Look at the curves, learn to interpret them, forget about the all-embracing "score".

Every time we did a stereo test, following a mono test, the same loudspeakers got the highest ratings. Why? Stereo includes two mono sources, left and right hard pans which happen in most recordings, including the classics. Besides, if a resonance is strong enough, even it penetrates the masking effect of stereo listening.

The 3rd edition has more information.

 

[jhaider]

…

[*]Motorized turntable: 1.75s switch time between any two speakers

By far the most significant improvement was the motorized turntable. We were able to rotate to any speaker in 1.75 seconds and keep the tweeter in the same location for each speaker. The control board also randomized the speakers for each track automatically and was controllable remotely from an iPad.

View attachment 275371
View attachment 275372
…

That is so cool. Very impressive all around.

 

[ehabheikal]

The score of the neumann and the jbl seem to have swapped between the text at the start of the post and the graph of preference rating

 

[DavidMcRoy]

In "live" experiences there is only one direct sound arriving at each ear from a single sound source. In stereo all sound images between the loudspeakers are phantoms, created from identical sounds radiated by each loudspeaker – double-mono - with inter-channel amplitude or time differences to provide location cues for image position. Each ear receives two versions of the same sound separated by a delay and modified by head diffraction. The only exceptions are the hard-panned sounds emerging from the left and right of the soundstage; these are monophonic components and are timbrally and spatially distinctive. So stereo listening is a hybrid experience, partly mono but mostly double-mono with the inherent corruptions.

In developing my own tri-amplified DIY "controlled directivity dipole" speakers, DSP control of each individual driver's EQ, polarity, level and delay has allowed me to explore how those parameters affect stereo imaging performance. I have found that the quickest, most efficient way to get a pair of my speakers to nail good stereo imaging is to start by measuring and listening to just one speaker. A single speaker, operating on its own, has its own audible "stereo imaging characteristics." While making changes to those parameters in real time, you can easily hear various parts of the spectrum shift in their apparent phantom locations about the space in close proximity to the speaker. Getting a single, subjectively "point source" image that has no "spread" or "spaciousness" when evaluating a single speaker corresponds with good stereo imaging when two or more of those identically processed speakers are used to create a convincing stereo sound field. Directivity is at play here.

Delay between drivers plays a huge role, in that inter-driver delay affects how the energy from individual drivers "combines" in the movement of air. (Time-alignment, anyone?) I'm not convinced that the timing is at play hear in and of itself, so much as how we react to the ways in which the timing causes reinforcements and cancellations at various frequencies in the crossover region. (Or, is that pretty much the same thing?) In any event, I believe that time-alignment among drivers in a multi-driver system corresponds with better imaging. There. I said it.

Naturally, closely matching the in situ acoustical properties of multiple speakers with each other (symmetry) in a stereo array likewise results in more-focused imaging.

 

[617]

As an amateur speaker designer I think they big takeaway here is how similar, and broadly performant small woofer speakers are. The directivity mismatch between drivers is minimal with speakers of this size and so I would expect competitive performance from all, but not the degree of uniformity in preference shown here.

Moving up to 8" woofers would start to exaggerate directivity issues - some devices manage this well, some do nothing, assuming that near-field use will proportionally demphasize off-axis response.

This is really nice work and might be the nerdiest thing ASR inspired anyone to do, with the possible exception of a guy buying an NFS.

 

[audiotron]

Nicely done.

 

[Hart]

Great job, one thing I wonder are there some speakers that work in stereo better than others? Image better?

 

[Roland301]

Very interesting, thanks!

The room is a limiting factor, though. My experience is that my own KH310 sound really bad without room treatment, but the more treatment i use, the more even subtle nuances become important.
You can see that room influences are dominating the FR of every speaker.
So I wonder if the results were clearer and differences between speakers more important in a more treated room? Still an interesting experiment, of course, because all speakers have the same conditions.

I will say, having listened to speakers in a full anechoic chamber the performance of the speaker really starts to come into play more. I found myself comparing the experience more to headphone listening, whereas in a normal room I really notice the acoustics of the room more than the more subtle characteristics of the speaker. With nearfield listening it's easier to pick things apart, but room treatment still helps.

By the way, if anyone has a chance to do some anechoic listening, it's worth it. The stereo image is uncannily good, even with relatively mediocre speakers. But, the sweet spot is only about 5 inches wide before the entire stereo image collapses, so you can't enjoy it with friends