• WANTED: Happy members who like to discuss audio and other topics related to our interest. Desire to learn and share knowledge of science required. There are many reviews of audio hardware and expert members to help answer your questions. Click here to have your audio equipment measured for free!

Dissecting a listening comparison situation - and ways of doing it better

fas42

Major Contributor
Joined
Mar 21, 2016
Messages
2,818
Likes
191
Location
Australia
This is going to look at http://matrixhifi.com/ENG_contenedor_ppec.htm, and point out the issues not addressed in the testing methodology - and how I would approach it, including the way the source material is used.

TBD ...

BTW, as a first go, what can people pick that is 'wrong' with the experiment, as presented in that link?
 

Blumlein 88

Grand Contributor
Forum Donor
Joined
Feb 23, 2016
Messages
20,521
Likes
37,050
Well, the human switchers are an area of possible concern. Plus the time needed to switch. Plus listening to far too long a sequence to choose. Though as is often the case it was people who wanted it that way. You can tell people their results are better with short selections and quick switching all you want they damn well will not believe you. I also wonder how they kept the CD's in synch though I suppose more similar to conventional audiophile auditioning it wouldn't matter. Plus picking for best sounding is a different animal than picking for hearing a difference. Rather than 38 people picking what they think sounds best would they have gotten a different result from one person picking 38 times. Maybe one person's sound best is opposite of another persons.

I also am guessing none of these are what you see wrong with it.
 

Blumlein 88

Grand Contributor
Forum Donor
Joined
Feb 23, 2016
Messages
20,521
Likes
37,050
Having read one of your replies in Fight Club, I surmise you would say they didn't stress test the two systems to see once they started to fail how they differed?
 
OP
fas42

fas42

Major Contributor
Joined
Mar 21, 2016
Messages
2,818
Likes
191
Location
Australia
Well, the human switchers are an area of possible concern. Plus the time needed to switch. Plus listening to far too long a sequence to choose. Though as is often the case it was people who wanted it that way. You can tell people their results are better with short selections and quick switching all you want they damn well will not believe you. I also wonder how they kept the CD's in synch though I suppose more similar to conventional audiophile auditioning it wouldn't matter. Plus picking for best sounding is a different animal than picking for hearing a difference. Rather than 38 people picking what they think sounds best would they have gotten a different result from one person picking 38 times. Maybe one person's sound best is opposite of another persons.

I also am guessing none of these are what you see wrong with it.
Pretty well. As a first step, there is no mention of whether the "premium" system was working at a standard that people who were familiar with the gear that made it up considered acceptable. IOW, it was not mentioned whether people were allowed to listen to that system at their leisure, using any recordings they were very familiar with, to assess whether it showed the qualities they would have expected of it - this is part of the, call it say, 'calibration' of the test.
 

tomelex

Addicted to Fun and Learning
Forum Donor
Joined
Feb 29, 2016
Messages
990
Likes
572
Location
So called Midwest, USA
perhaps surprisingly, I see nothing worth complaining about, simple test, predictable AND accurate results IMO.
 
OP
fas42

fas42

Major Contributor
Joined
Mar 21, 2016
Messages
2,818
Likes
191
Location
Australia
perhaps surprisingly, I see nothing worth complaining about, simple test, predictable AND accurate results IMO.
Which is the main failing, in fact.
 
OP
fas42

fas42

Major Contributor
Joined
Mar 21, 2016
Messages
2,818
Likes
191
Location
Australia
Having read one of your replies in Fight Club, I surmise you would say they didn't stress test the two systems to see once they started to fail how they differed?
More that I would bring recordings to the table which I know most systems struggle with, and raise the volume until one system was showing severe anomalies, as compared to the other. This does not have to be anywhere near loud, just enough to highlight where they vary. As an example, I would play Status Quo, and listen to the quality of the cymbals - years ago, most systems failed to get this right at headphone levels, let alone something decent in volume.
 

watchnerd

Grand Contributor
Joined
Dec 8, 2016
Messages
12,449
Likes
10,408
Location
Seattle Area, USA
Because the speakers are the same, the stress test will primarily be one of the amps.

The YBA Alpha is rated for 70 W into 8 ohms.

The Behringer is rated for 300 W into 4 ohms.

It wouldn't be shocking for the pricier amp to succumb to stress first.
 
OP
fas42

fas42

Major Contributor
Joined
Mar 21, 2016
Messages
2,818
Likes
191
Location
Australia
Because the speakers are the same, the stress test will primarily be one of the amps.

The YBA Alpha is rated for 70 W into 8 ohms.

The Behringer is rated for 300 W into 4 ohms.

It wouldn't be shocking for the pricier amp to succumb to stress first.
The ATCs are a little lower in sensitivity than usual, 85dB rating; the Behringer is really a 125W into 8 ohms unit, if you look at the specs; there's barely a slip of paper between 70 and 125W in acoustic terms - they're reasonably closely matched, nominally. The key would be, which amp can produce at close to clipping more cleanly.
 
OP
fas42

fas42

Major Contributor
Joined
Mar 21, 2016
Messages
2,818
Likes
191
Location
Australia
Just looking at the test again, another thing sticks out: the set of testers included some termed objectivists - IOW, those whose "mantra" is that reasonable equipment will sound identical. For them, it would in their interest to say they heard no difference, the natural inclination.

If you're attempting to assess whether people who claim better gear sounds better can actually pick the difference, then including people who believe otherwise, or who have no interest in such in the testing, and having them part of the statistical summary of the results - then you're not testing what you claim to be testing! First principle of science: understand how to test ...
 
OP
fas42

fas42

Major Contributor
Joined
Mar 21, 2016
Messages
2,818
Likes
191
Location
Australia
Sorry, I haven't forgotten this ... will get it happenin' soon ...
 
Top Bottom