• WANTED: Happy members who like to discuss audio and other topics related to our interest. Desire to learn and share knowledge of science required as is 20 years of participation in forums (not all true). There are daily reviews of audio hardware and expert members to help answer your questions. Click here to have your audio equipment measured for free!

Speaker Blind Listening Test Part 2 Planning

MatthewS

Member
Forum Donor
Joined
Jul 31, 2020
Messages
56
Likes
293
Location
Greater Seattle
@Inverse_Laplace and I will be conducting a second blind listening test. If you missed the original results you can read about the blind listening test here:

https://www.audiosciencereview.com/...stening-test-results-kef-jbl-revel-osd.25818/

We are considering tests to attempt to answer the following questions:
  • Are speakers with good preference scores preferred over speakers with poor preference scores?
  • What happens to ratings when bass is rolled off to equalize differences in bass response between speakers?
  • How effective is EQ on fixing a speaker with poor tonality, but good directivity?
  • ABX testing a stereo speaker pair with Dirac room correction (bass only) vs REW & PEQ correction
  • ABX testing a stereo speaker pair with Dirac bass only correction vs full range correction
Changes from last round:
  • Motorized turntable
  • ITU R 1770 loudness instead of C weighting
  • Significantly larger listening room
  • Attempt to bring in more listeners that skew trained
  • 5 powered bookshelf/monitors (preference ratings from 1.9 to 6.2)
  • Take room measurements of each speaker at prime listening position
  • 20 second clips from Harman’s list of recommended tracks
  • Assistance from a statistician on experimental design and statistical tests
We will use the Motu M2/M4 as our DAC again. The biggest difference in our method this time around will be the use of a motorized turntable that will automatically switch between any speaker in 1.75 seconds. @Inverse_Laplace completed the design and engineering and construction is underway.

Turntable Assy Wireframe.png


The listen space will look something like this:
roomfloorplan copy.jpg

We will conduct the blind listening tests on 3/28/22 and 3/29/22 in the Puget Sound region.


Two primary questions for ASR members:
  • What other hypotheses might you want to see investigated?
  • Any suggestions for ways to improve the rigor of the test?
 

sweetsounds

Member
Joined
Apr 24, 2019
Messages
55
Likes
98
Wow, this is impressive.

My recommendation to note down for each test subject to analyze results
- listening position (as room equalization will have different effects)
- own classification: musician, hifi enthusiast, trained listener, amateur etc.
- for each vote: did subject hear a difference or is he guessing? (Even guessing is OK)
- age

Subject should keep poker faces and avoid to involuntarily influence their peers.

The sequence of the speakers will be always the same, right? This will be a challenge for ABX to not be able to guess from the time it took to get to speaker X.

There should be reasonable absorption in the room, e.g a carpet in front of the speakers.

If speakers have huge differences in height, ensure tweeter axis is comparable by using supports under the smaller speakers.

Use pink noise for testing as well. For me this works best, I detect equalization changes with it much better than with music.
 
OP
M

MatthewS

Member
Forum Donor
Joined
Jul 31, 2020
Messages
56
Likes
293
Location
Greater Seattle
I forgot to mention that we will conduct ABX testing of listeners to qualify them. I don't expect we will have issues as speakers sound rather different. Once that happens the first three tests happen on the turn table with 5 speakers. Participants will be asked to rate on a scale of 1-10 to represent audio fidelity of the speaker. The order of the speakers in each test (by song) will be randomized. So, speaker A on song 1 will (probably) not be the same speaker as speaker A on song 2.

The remaining tests will happen with a matched set of speakers playing in stereo. That will be sighted since we aren't trying to assess the performance of the speaker--we are measuring the performance of room correction.

The listening positions will be fixed. We did track test results to individuals in the previous test and will again.

We had been on the fence about including pink noise. We elected to exclude it previously--but we might add it again.
 

sdiver68

Active Member
Joined
Dec 14, 2021
Messages
140
Likes
68
My only request..some with 1db difference. So Speaker A is a test subject, Speaker A + 1db is another, Speaker B, Speaker B +1db, etc...

Ultimately answers the question, can I make up for perceived shortcomings simply by advancing the volume 1 notch?
 

Pdxwayne

Major Contributor
Joined
Sep 15, 2020
Messages
3,249
Likes
1,162
What kind of abx testing to qualify listener?

It would be interesting to know more about listeners' hearing sensitivities by doing some online distortion tests, dB different tests, etc.

Also, are they primary speakers listeners or headphones listeners? It would be interesting to know about their favorite speakers and headphones.
 
OP
M

MatthewS

Member
Forum Donor
Joined
Jul 31, 2020
Messages
56
Likes
293
Location
Greater Seattle
What kind of abx testing to qualify listener?

It would be interesting to know more about listeners' hearing sensitivities by doing some online distortion tests, dB different tests, etc.

Also, are they primary speakers listeners or headphones listeners? It would be interesting to know about their favorite speakers and headphones.

We will likely take two speakers of reasonably similar behavior and test that the listener can correctly identify the X. Do you have other suggestions?

I recently completed an audiogram at my audiologist that included tests all the way to 20khz, I'll published it when we get there. We've thought about using Harman's How to Listen application to evaluate participants.

I think most people are speaker listeners--but a brief survey beforehand is a great idea. We can definitely put that together.
 

fricccolodics

Member
Joined
Apr 1, 2021
Messages
28
Likes
30
Are you using the Harman list the link is suggesting (spotify)?
Or do you use lossless?
I would suggest to use less pop material and more dynamic recordings of acoustic music (jazz / classical),
preferably live material without much studio editing done.
 
Top Bottom