Blind Listening Test 2: Neumann KH 80 vs JBL 305p MkII vs Edifier R1280T vs RCF Arya Pro5

[computer-audiophile]

Very nice work.

Would be fun if you had the opportunity to do a binaural recording with in-ear microphones of a test session, and link the file here.

These would introduce additional sound distortions. From a distance, through the mediation of digital media, one cannot reproduce such a listening test, imo.

 

[Thomas_A]

These would introduce additional sources of error. From a distance, through the mediation of digital media, one cannot make such a listening test, imo.

Yes the method will introduce errors, but I do not agree that they are useless:

Binaural blind comparison test of 4 loudspeakers

Lets see how the preferences of our community are when listening to binaural recordings of four loudspeakers. Of course this is not a test according to scientific standards and has also significant methodological limitations like for example how well can binaural cues capture and recreate the...

www.audiosciencereview.com

Binaural blind comparison test of 4 loudspeakers - II

Since the discussion on the last vote about the sound quality of speakers via binaural recordings was so harmonious, I thought it would be a good idea to repeat the whole thing with other binaural recordings of speakers. And that's why I just plagiarized the thread title (and more) from @thewas...

www.audiosciencereview.com

 

[computer-audiophile]

No problem!

Let’s agree to disagree.

 

[MediumRare]

Tremendous effort! Also great to hear you’re working on refinements.

I have a background in consumer preference testing and consistently found more complex tests require minimum 75 to 100 participants to get statistically reliable differences. Since that is almost certainly impossible for you I’d suggest you focus on more sensitive approaches where a smaller "n" will give reliable (repeatable) and valid results.

Suggestion: start with what you believe are the best and worst candidates and do an ABX Triangle test using just one of the tracks consistently.
1. Which of the "3" speakers is different?
2. Which of the "3" is preferred?
3. Score Raters ability to detect the different speaker.

Once you get valid results with a reliable cohort of testers who can detect differences, only then do a "shootout" tournament structure (same ABX triangle test) to do the actual speaker testing. You can accumulate scores as you do 1 v 5, 1 v 4, 1 v 3, etc.

Suggestion: do the shootout at both of Amir’s loudness levels, as research shows 1. Participants are more discriminating at normal volume but 2. Speaker problems and differences become more pronounced at higher volumes.

Suggestion: Only after using one of the tracks for a full ABX, then use a different track and see if the results are consistent. Eventually you can score the tracks’ ability to enable the participants to be more discriminating.

Happy to help otherwise if you like.

 

[D700]

You tested studio quality speakers without Mark Knopfler and Steely Dan? Is that legal? I'm pretty sure that's on the warning label...

 

[tktran303]

Very interesting.

I suspect the room is masking the differences.

Consider testing again outdoors.

 

[dasdoing]

imo Adam TV5 should have been included.
but a great test

 

[Multicore]

What is the Preference Score in parens in the listing of speakers?

 

[Multicore]

An interesting observation: for one group of listeners, we had to level match the speakers again and in our haste, we used pink noise instead of the actual material. This excites all frequencies equally which isn’t necessarily representative of the musical selections. The Neumann KH80 was a full 3db lower (ITU R 1770) when using the music tracks than most of the other speakers (we measured after the test and we clearly could hear differences in the volume of each speaker.) We threw out this data for our analysis, but the speaker with the lowest level was universally given awful ratings by each listener.

The good news is that all you gotta do to make your speakers sound better is Pump Up The Jam.

 

[bmwr75]

If I understood correctly, you did these blind listening tests using one speaker. Seems like the use of speaker pairs would yield more useful subjective results. I've never though one speaker of any make and model sounded good.

 

[FrankW]

Freaking awesome. Might I be so bold as to suggest only testing/comparing 2 pairs at a time, one directly behind other, 180 degree rotations? It might be for naught, but I wonder if having the speakers even so slightly to the sides would create a touch of side reflections?
Bravo to you MatthewS.

 

[Thomas_A]

If I understood correctly, you did these blind listening tests using one speaker. Seems like the use of speaker pairs would yield more useful subjective results. I've never though one speaker of any make and model sounded good.

The one-speaker approach is most sensitive to errors in frequency response.

 

[brandonhall]

I don't have any insight to add or knowledgeable contributions to make. I just wanted to say that stuff like this is why I love ASR. Brilliant work all-around!

 

[DavidMcRoy]

imo Adam TV5 should have been included.
but a great test

I'm using five Adam Audio T5Vs as surround bed speakers and four JBL 305s as height speakers. The T5Vs are more neutral, but Adam needs to revisit that fragile, rear-mounted volume control. Under very delicate use they have an alarming failure rate in my experience, breaking free of the circuit board. I broke three of them out of seven speakers.

 

[sejarzo]

Very interesting.

I suspect the room is masking the differences.

Consider testing again outdoors.

How many people listen to these speakers outdoors on a regular basis?

Assume that you participated in both an outdoor and an indoor test. You, and most others, found that $1000 speaker significantly outperforms a $150 speaker when outdoors, but then you couldn't discriminate the difference indoors--both sounded equally good or bad, depending on one's perspective. Which would you buy?

Seems to me the goal was to determine if there was a statistically significant preference among a mixed group of trained and untrained listeners, in a reasonably typical "recreational listening" indoor environment.

 

[dasdoing]

I broke three of them out of seven speakers.

wow. that's bad

 

[dasdoing]

JBL has the most neutral midrange in the in-room meassurement.

I would love to see the unsmoothed measurments

 

[kemmler3D]

This is an amazing test, definitely fighting the good fight here. I think the results and the in-room measurements are a great illustration of (reminder of) how much the room matters. A lot of the differences in the speakers are swamped by the effects of the room.

I think the bass question, and the level matching questions, are pretty interesting. If you do any more of these tests I will follow with interest.

 

[martijn86]

ITU R 1770 loudness instead of C weighting

Are the songs individually normalised or just the loudspeaker levels? Wat is the target loudness in LUFS?

 

[MatthewS]

A couple of things I found interesting :
-Speakers were difficult to separate from each other
-Even with a fairly good frequency response, "They all sound terrible" ( comment at the end )

I didn't explain this in the video--but it was actually the musical selection that sounded terrible. We wanted something classical and this piece was on Harman's list. But it wasn't recorded well and was extremely hard to pick out differences. It was more the recording sounded terrible, not the speakers.

The Neumann on "I Can See Clearly Now" was mindbogglingly good.