• WANTED: Happy members who like to discuss audio and other topics related to our interest. Desire to learn and share knowledge of science required as is 20 years of participation in forums (not all true). Come here to have fun, be ready to be teased and not take online life too seriously. We now measure and review equipment for free! Click here for details.

Steve Guttenberg compares subjective and objective reviews

amirm

Founder/Admin
Staff Member
CFO (Chief Fun Officer)
Joined
Feb 13, 2016
Messages
17,436
Likes
14,921
Location
Seattle Area
#41
Where are the audiophiles in this graph (the ones that trust their ears) ?
I suppose they would see themselves as 'better than trained'.
Noticing that most regurgitate what reviewers say chances are they are closer to the reviewers ?
Here you go: https://www.audiosciencereview.com/...s/are-our-preferences-different-in-audio.284/



Notice how the Harman trained listeners rated speaker M in green extremely low (close to zero) yet the reviewers rated it at a 3 to 4.
 

RayDunzl

Major Contributor
Central Scrutinizer
Joined
Mar 9, 2016
Messages
6,251
Likes
2,298
Location
Riverview, FL
#43
Ask Sean if maybe the chart was printed upside down in the beginning and he never issued a correction.
 

RayDunzl

Major Contributor
Central Scrutinizer
Joined
Mar 9, 2016
Messages
6,251
Likes
2,298
Location
Riverview, FL
#44
Are the green ones Ray’s?
A different (newer, slightly lesser) model, but still a panel and a woofer.

I've noticed you are still trying this, trying that...

I've been with these for over twenty years.

They must do something right...
 

amirm

Founder/Admin
Staff Member
CFO (Chief Fun Officer)
Joined
Feb 13, 2016
Messages
17,436
Likes
14,921
Location
Seattle Area
#46
Here is one of the setups when I took the harman blind test (after the reveal/curtain opening):

harman speaker test blind.jpg


You can see the Martin Logan, B&W and JBL speaker that is frequently used in the tests.
 

RayDunzl

Major Contributor
Central Scrutinizer
Joined
Mar 9, 2016
Messages
6,251
Likes
2,298
Location
Riverview, FL
#47
*waits for the "Caption the Photo" contest to start

"And this, well, nevermind..."
 

Dialectic

Addicted to Fun and Learning
Patreon Donor
Joined
Sep 26, 2017
Messages
806
Likes
475
Location
New York
#48
Where are the Zus, the Audio Notes, and the Linn Kans?
 

Blumlein 88

Major Contributor
Joined
Feb 23, 2016
Messages
5,131
Likes
3,797
#49
*waits for the "Caption the Photo" contest to start

"And this, well, nevermind..."
And this needs a better amplifier.

And this sounds better in Florida.

And this is so different people don't realize how good it sounds.

And this takes room correction very well.

With apologies to Ray from another electrostatic fan.
 

amirm

Founder/Admin
Staff Member
CFO (Chief Fun Officer)
Joined
Feb 13, 2016
Messages
17,436
Likes
14,921
Location
Seattle Area
#50
And this needs a better amplifier.
I had a ton of subjectivists dismiss the tests on the basis of the amplifier used in Harman tests!
 

Dialectic

Addicted to Fun and Learning
Patreon Donor
Joined
Sep 26, 2017
Messages
806
Likes
475
Location
New York
#51
I had a ton of subjectivists dismiss the tests on the basis of the amplifier used in Harman tests!
And what was that amplifier?
 

amirm

Founder/Admin
Staff Member
CFO (Chief Fun Officer)
Joined
Feb 13, 2016
Messages
17,436
Likes
14,921
Location
Seattle Area
#53
Joined
Dec 7, 2018
Messages
77
Likes
10
#54
The other myth here is that we all have different preferences (hence his comment on who to listen to). That is just not so when we test speakers blind. Here again is Harman's research and preference for four speakers among different groups of listeners:

View attachment 19866

Note that the relative scoring of each speaker remains the same no matter which group is selected.

HARman trained listeners are more picky but have the same order preference that reviewers and academics had as I have highlighted.
If everyone have the same preference then the lines should be perfectly horizontal.
Just by looking at the graph, it shows that we do have individual preference.
 

Blumlein 88

Major Contributor
Joined
Feb 23, 2016
Messages
5,131
Likes
3,797
#55
If everyone have the same preference then the lines should be perfectly horizontal.
Just by looking at the graph, it shows that we do have individual preference.
They key thing is all groups rated the speakers in the same order. Trained listeners were more picky and the ratings varied by a larger amount which is what one would expect.

So untrained, trained, musicians, salespeople, reviewers, all, all groups picked the same kind of speaker as best. This is what you would expect. They all picked the same as worst (unfortunately for Ray and myself). So everyone has the same ordered preference for the same qualities, but trained and experienced people are more picky about it and indicate larger differences in their evaluation of the sound.

All seems perfectly reasonable to me.
 
Joined
Dec 7, 2018
Messages
77
Likes
10
#56
They key thing is all groups rated the speakers in the same order. Trained listeners were more picky and the ratings varied by a larger amount which is what one would expect.

So untrained, trained, musicians, salespeople, reviewers, all, all groups picked the same kind of speaker as best. This is what you would expect. They all picked the same as worst (unfortunately for Ray and myself). So everyone has the same ordered preference for the same qualities, but trained and experienced people are more picky about it and indicate larger differences in their evaluation of the sound.

All seems perfectly reasonable to me.
I understand, but my point still stand, the graph shows that we do have individual preference.
Even the order of the preference on the graphic is not absolute. If you look at the graph, it shows that for Acad10, loudspeaker I & loudspeaker P is reversed compared to everyone else. (".......So everyone has the same ordered preference for the same qualities......"<--- incorrect)
Which proves my point that we do have individual preference.
 
Joined
Dec 7, 2018
Messages
77
Likes
10
#57
The other myth here is that we all have different preferences (hence his comment on who to listen to). That is just not so when we test speakers blind. Here again is Harman's research and preference for four speakers among different groups of listeners:

View attachment 19866

Note that the relative scoring of each speaker remains the same no matter which group is selected.

HARman trained listeners are more picky but have the same order preference that reviewers and academics had as I have highlighted.
Actually the relative scoring of each speaker is different. (does not remain exactly the same) I measured the relative difference using a ruler. This is the graph.
3.1cm , 1cm , 3.7cm

Harman Preferences(mod).png
 

SoundAndMotion

Active Member
Joined
Mar 23, 2016
Messages
101
Likes
40
Location
Germany
#58
I understand, but my point still stand, the graph shows that we do have individual preference.
Even the order of the preference on the graphic is not absolute. If you look at the graph, it shows that for Acad10, loudspeaker I & loudspeaker P is reversed compared to everyone else. (".......So everyone has the same ordered preference for the same qualities......"<--- incorrect)
Which proves my point that we do have individual preference.
Actually the relative scoring of each speaker is different. (does not remain exactly the same) I measured the relative difference using a ruler.
Hi Dragon,

@Blumlein 88 and @amirm may have been a little careless with some word choices, but the thrust of what they are saying is that "relative" choices are generally preserved between groups. "Relative" means the rank is preserved, but not the absolute value or absolute difference. The P - I distinction deserves special consideration.

One can see from the graph the significant result:
P,I >* B >** M
* for 15/16 individual groups, and the combined group (p<0.0001)
** for all individual groups, and the combined group (p<0.0001)

Even though individual scores and differences between scores vary, the rank is well preserved.

The difference between P and I is less clear. Since the tabular data equivalent to the graph is not in the paper, one has to eyeball it. But it is clear that for 9/16 groups there is not a statistically significant difference. For 4/16 groups there is clearly a statistically significant difference, with 3 preferring P and 1 I. For 3/16 the significance is unclear from the graph (since I don't trust pixel measurements). When all the groups are combined the mean difference between P and I is 0.336 (from the paper), with a significance of p=0.0214.

Some will argue that is significant, but from my experience with this type of data, since the p=0.05 cutoff is arbitrary, other ways of viewing the data are relevant, and I would prefer further tests to draw a conclusion about P vs. I.

Cheers, SAM
 
Last edited:

andreasmaaan

Major Contributor
Patreon Donor
Joined
Jun 19, 2018
Messages
2,628
Likes
1,719
#59
I think what the graph shows is that all the groups showed similar preferences.

I don't have a copy of Dr Toole's book and in any case I'm not sure how much is explained by the research in it, but unlike @dragonspit4 my concern is not so much about the relative scores varying (sure, the do, but not very significantly, at least in the case of B vs M vs P/I). My concern is that the tests all took place in the same room.

I'd like to see how different rooms affected preferences, and to what extent the results hold when the room is varied.

Perhaps there is more on this in the book, or in Harman's published research?
 

flipflop

Active Member
Joined
Feb 22, 2018
Messages
229
Likes
187
#60
Perhaps there is more on this in the book, or in Harman's published research?
Probably not as specifically as you would like. Chapters 13 and 17 do touch on the subject.
 
Last edited by a moderator:
Top Bottom