• WANTED: Happy members who like to discuss audio and other topics related to our interest. Desire to learn and share knowledge of science required. There are many reviews of audio hardware and expert members to help answer your questions. Click here to have your audio equipment measured for free!

The frailty of Sighted Listening Tests

whazzup

Addicted to Fun and Learning
Joined
Feb 19, 2020
Messages
575
Likes
486
The How to Listen software was developed to train Harman employees, according to its site. Amir has previously noted the level he was able to reach in the training. So certainly if one can reach that level or exceed it, they can claim they are at least an equal listener.

Yes, if they can show reliable proof of attaining certain levels, they can definitely claim that.
Whether they have the experience / knowledge to propose electronic / physical / mechanical solutions to say, improve the speaker's sound to target a certain market though....
 

MattHooper

Master Contributor
Forum Donor
Joined
Jan 27, 2019
Messages
7,194
Likes
11,806
That's all you've got? This ridiculous insinuation? That a modest donation to the site constitutes sufficient skin in the game for bias? I am biased, towards people who expend a lot of effort to add tangible value, and against cynical critics who don't seem to add any value at all.

I don't agree with everything Amir publishes. I've made a couple of requests of Amir in review threads to consider changing his measurement strategy, and he's declined both and stayed his course. That's his prerogative; he's the one with the investment and the drive to create the reviews, and his preferences out-weigh mine. I'm not bent out of shape about it, nor do I see fit to argue about them. For speaker reviews, I get about 90% of the value from the measurements, and perhaps the balance out of Amir's subjective impressions, but I don't consider his inclusion of subjective impressions as detracting from the value of a review at all. I interpret them in context.

I am just so tired of the critics who contribute nothing but their criticism, and go on and on about it should Amir not agree.

The tone of your reply suggests you took my reply too seriously.

You jokingly or half-jokingly suggested: "I think you should have a new ASR posting rule. You have to be a Forum Donor at the $100 level or higher to have a running argument with you about review efficacy. I've noticed there is a negative correlation between donors and complainers."

How serious where you?

I simply pointed out, specifically matching your level of seriousness, an alternative conceivable consequence of the logic. Contributing or not contributing to the website does not guarantee quality thinking/critique/feeback. And biases can run in many directions.

The point being: anyone should be able to present a critique of any claims made here by anyone, and the evaluation of that contribution ought to be based on the quality of the critique, not on whether the individual has donated or not. Do you actually disagree?

And doesn't that half-hearted suggestion, if taken seriously: "We'll only respond to a critique of our methods if you have already paid us some money or lent us gear" sound just a bit..."off" to you?

Again: please notice the context - batting around half-joking ideas - and also that I have happily endorsed Amir's testing including the sighted tests.
 
Last edited:

Wes

Major Contributor
Forum Donor
Joined
Dec 5, 2019
Messages
3,843
Likes
3,788
Michael Fremer, for example, boasts how he cannot listen to any digital source without becoming physically ill.

And this is because of his vast experience with many systems, making him experienced in detecting certain things?

Interesting - I cannot listen to Michael Fremer without becoming physically ill.
 
D

Deleted member 17820

Guest
Since speakers are enjoyed by also seeing them, I don't see why people would want to hide them while listening. It's well known that visual stimuli change neurotransmitters and cognitive function. Think of a man seeing a good looking girl, everything changes. So visual stimulation affects brain function that also affects how were here and listen.
 
Last edited by a moderator:

aarons915

Addicted to Fun and Learning
Forum Donor
Joined
Oct 20, 2019
Messages
685
Likes
1,140
Location
Chicago, IL
Since speakers are enjoyed by also seeing them, I don't see why people would want to hide them while listening. It's well known that visual stimuli change neurtransimters and cognitive funtion. Think of a man seeing a good looking girl, everything changes. So visual stimulation affects brain function that also affects how were here and listen.

I like my speakers to look and sound good as well but if I'm comparing them to a different brand and want to know which one has the better sound quality, I'll do the comparison blind. I haven't personally had different results blind vs sighted but it's just a good way to ensure you aren't being biased.

You bring up a good point though, if the mind thinks you're listening to a live event that just sounds perfect, does it really matter how your speakers actually sound? In that way Bias could actually help the subjective experience by fooling your mind into thinking the sound quality is better than it actually is.
 

SoundAndMotion

Active Member
Joined
Mar 23, 2016
Messages
144
Likes
111
Location
Germany
Since speakers are enjoyed by also seeing them, I don't see why people would want to hide them while listening. It's well known that visual stimuli change neurtransimters and cognitive funtion. Think of a man seeing a good looking girl, everything changes. So visual stimulation affects brain function that also affects how were here and listen.
If someone produced a speaker that looked just like a good looking girl, I might buy them regardless how they sound...
 
D

Deleted member 17820

Guest
I like my speakers to look and sound good as well but if I'm comparing them to a different brand and want to know which one has the better sound quality, I'll do the comparison blind. I haven't personally had different results blind vs sighted but it's just a good way to ensure you aren't being biased.

You bring up a good point though, if the mind thinks you're listening to a live event that just sounds perfect, does it really matter how your speakers actually sound? In that way Bias could actually help the subjective experience by fooling your mind into thinking the sound quality is better than it actually is.

I use to run live sound for events and there was always times where I would be adjusting the EQ on one channel and it sounded WAY better, then I noticed the EQ wasn't engaged, haha, so I know I can't trust myself, but if I think it sounds better then that's also good?
 

richard12511

Major Contributor
Forum Donor
Joined
Jan 23, 2020
Messages
4,335
Likes
6,700
Difference and preference are separate animals. To me it is clear that a speaker with score x but a large dispersion disruption where woofer hands of to tweeter will sound worse than a speaker with the same score to whatever decimal places but correct geometry and a well-engineered crossover.
"

What do you think about the comparison between the SVS Ultra and the Revel M106 and M105, which have an almost identical(same magnitude and same frequency) dispersion disruption at the crossover(shown by the overlays that @edechamps and @bobbooo), very similar tonality (within 1db most frequencies), and very similar dispersion width(M106 slightly less, M105 slightly more)?. The SVS crossover mismatch looks *slightly worse(imo) when you really zoom in, but it's a tiny difference. Both speakers still have basically the same directivity error, dispersion, and tonality, and yet one sounds excellent, while the other sounds bad. I've been trying to figure out why that is in the other thread, so I'm curious what you see. Amir's subjective impressions of the SVS are inline with most of the other subjective impressions I've read(though I've never heard them personally). I do own the M105, which to my ears, sounds excellent.

Attached my original comparison(with directivity error circled in red) as well as the overlays the others posted. To me they look very close(almost within of limits sample to sample variation of the same speaker). I'm honestly not seeing huge differences in the spins. Someone else posted an interesting note about the distortion measurements, though that's not a part of the spinorama(which should be sufficient, according to Toole). The M106 has higher H2 and overall distortion than the SVS, but the SVS has more H3 distortion. The M105 has more distortion overall than the M106. I'm curious of your thoughts, given that you've seen a ton of these.

I don't put all that much faith into the Olive score anymore, so I prefer to just compare the spins directly to try and find answers.
 

Attachments

  • Screen Shot 2020-08-04 at 12.32.21 PM.png
    Screen Shot 2020-08-04 at 12.32.21 PM.png
    995.7 KB · Views: 75
  • Loudspeaker Explorer chart(31).png
    Loudspeaker Explorer chart(31).png
    98.3 KB · Views: 95
  • SVSvsM106 .png
    SVSvsM106 .png
    483.7 KB · Views: 79
Last edited:

solderdude

Grand Contributor
Joined
Jul 21, 2018
Messages
15,891
Likes
35,912
Location
The Neitherlands
However, the fact remains that it’s not incumbent upon anyone to travel to Amir to prove they can hear differences. It is not about “trust.” No one person is the ultimate authority, and there certainly are other people who can claim as much expertise. One might as well say Amir must travel to some other person’s house to prove he can reliably distinguish between the Revel and SVS. Then if another person doesn’t trust that result, Amir must travel to that person’s house, and on and on.

Funny ... wiggeling a way out of this. Someone thinks they can easily show Amir he has cloth ears, put him on his number, make him look like an idiot in the process and get a lot of money. Remember ... it is easy to hear the differences between DAC's/amps whatever. Easy money right there.
No takers ? why ? because Amir should come to their homes ?

Come-on if one could really tell similarish measuring gear apart reliably under lab conditions Amir would not offer money for this extraordinary feat .

No takers because deep inside they fear there is a chance they might not be able to tell 'night and day' apart when they are in a room with no windows.
 

richard12511

Major Contributor
Forum Donor
Joined
Jan 23, 2020
Messages
4,335
Likes
6,700
Since speakers are enjoyed by also seeing them, I don't see why people would want to hide them while listening. It's well known that visual stimuli change neurotransmitters and cognitive function. Think of a man seeing a good looking girl, everything changes. So visual stimulation affects brain function that also affects how were here and listen.

This opens up a whole nother can of worms. Blind testing is the best way to determine the quality of sound entering one's ears, but may be flawed for determining the quality of sound one's brain interprets in sighted listening.
 

tuga

Major Contributor
Joined
Feb 5, 2020
Messages
3,984
Likes
4,281
Location
Oxford, England
Where did I say I don’t believe listeners can be trained? I’ve posted the links to such training. I’ve just pointed out that Amir isn’t the only trained listener.



If Amir, who’s previously said speaker blind tests aren’t as complicated or costly as people claim, can’t set up a blind tests for speakers, it’s certainly unreasonable to expect someone who wants to show they can hear a difference between amps or DACs to travel with their equipment to Amir to take a blind test. He cannot be the only person capable of using a multimeter and a passive switch.

Someone posted the Fremer comment on his blind tests above. Likewise, people (with industry credentials!) on other sites have set up and passed blind tests. It’s simply untenable IMHO to dismiss such tests while at the same time allowing Amir to claim that, in sighted listening weeks apart, he can reliably differentiate between a 5.70 and 5.79 rated speaker.

You put too much faith in the ratings... There you have it.

Spins are about frequency response and dispersion and how that relates with a preference poll taken from a small sample of listeners some of which were untrained.
 
Last edited:

tuga

Major Contributor
Joined
Feb 5, 2020
Messages
3,984
Likes
4,281
Location
Oxford, England
Rusty, if you read Kevin Voecks' comments on listening tests for the Salon 2, I think you might agree that he seems to feel that some things are better determined by listening, than by measurements. Similarly, Floyd Toole has commented in his book, "A recent listening test proved its worth when it revealed that a loudspeaker having excellent looking spinorama data (Section 5.3), which normally is sufficient to describe sound quality, was not rated highly as expected. The problem was found to be intermodulation distortion, an extremely rare event, associated with the ways sounds from a woofer and tweeter combined in a concentric arrangement--so constant vigilance and listening are essential." FLOYD TOOLE. AND KEVIN VOECKS. FROM HARMAN.

In other words, a Spinorama "normally is sufficient to describe sound quality" ... except when it isn't.
 
Last edited:

blueone

Major Contributor
Forum Donor
Joined
May 11, 2019
Messages
1,179
Likes
1,494
Location
USA
The tone of your reply suggests you took my reply too seriously.

Perhaps, but the wording of your reply struck me more as sarcasm than humor. That's probably what spurred a misinterpretation.

You jokingly or half-jokingly suggested: "I think you should have a new ASR posting rule. You have to be a Forum Donor at the $100 level or higher to have a running argument with you about review efficacy. I've noticed there is a negative correlation between donors and complainers."

How serious where you?

At the time I posted it, about 50% serious. This thread keeps going on and on, so I think now I'm up to about 95%. Not joking.

I simply pointed out, specifically matching your level of seriousness, an alternative conceivable consequence of the logic. Contributing or not contributing to the website does not guarantee quality thinking/critique/feeback. And biases can run in many directions.

The point being: anyone should be able to present a critique of any claims made here by anyone, and the evaluation of that contribution ought to be based on the quality of the critique, not on whether the individual has donated or not. Do you actually disagree?

I don't agree with the structure of your question. I agree that being a donor does not imply higher quality of contributions to discourse like this. My point was, as I mentioned, I've noticed donors tend to have more polite discussions and fewer philosophical arguments. My initial impression would be that the opposite would be the case, that donors would have a sense of entitlement and be more inclined to think they can set expectations with Amir. That non-donors seem to feel entitlement actually strikes me as rather weird.

Again: please notice the context - batting around half-joking ideas - and also that I have happily endorsed Amir's testing including the sighted tests.

Noted.
 

MattHooper

Master Contributor
Forum Donor
Joined
Jan 27, 2019
Messages
7,194
Likes
11,806
This opens up a whole nother can of worms. Blind testing is the best way to determine the quality of sound entering one's ears, but may be flawed for determining the quality of sound one's brain interprets in sighted listening.

Yes, that's an issue I kept bringing up in other threads.

I work by manipulating the impression of sounds and picture all day long (sound design for film and tv). We very often use substitute sounds rather than real sounds (or the sounds recorded by the on-set microphone), and the combination of what you are seeing and what you are hearing changes your perception. If I play you the sound and let you know it is celery being twisted and snapping, that's what you will hear. If I simply play it with the accompanying visual of a character having his finger twisted and snapped in two, you'll perceive it as being a bone twisting and breaking.
I can take the sound of a vacuum cleaner and that's what you'll hear. Add the visual of a hovercraft or spaceship, and then you'll perceive it as the engine of that craft. Ben Berrt of Star Wars sound design fame took an elephant roar and when placed with the visual stimuli, everyone perceived it as the sound of a Tie Figher's engine.

Blind testing is great for weeding out variables, but if those variables snap back together, are almost always found comingled in normal situations, then that too is worthy of consideration.
 

krabapple

Major Contributor
Forum Donor
Joined
Apr 15, 2016
Messages
3,168
Likes
3,712
MQA2 ? :D


------------
Very interesting this thread, I read it carefully.
My opinion is that what the AP555 and Klippel measures is replicable, what Amir listens to with his training, training and knowledge is not replicable.
So, posting subjective opinions gives snake sellers, golden ears, etc. the opportunity to question ASR. :(


I see where you are going, but I would not say this is the core problem.

I would say that Amir asserting without careful qualification that he would take the sighted evals of 'trained listeners' over blinded evals of untrained, gives cover to the wild subjectivism of the audio hobby, unwittingly or not. Because *as we should all be well aware of by now*, every 'audiophile' bloviating in Stereophile et al, from Mikey Fremer on down to some anonymous punter on a web board, proceeds from the stance that they are a 'trained listener'. That's in the everyday world, not the lab.

So, first, define what a 'trained listener' is, and talk about where and when one might encounter their expertise.

Then , at least, we can all talk about the same thing.
 

Coach_Kaarlo

Active Member
Joined
Apr 27, 2020
Messages
196
Likes
222
Location
Sydney
Enjoying the read, unusual to find so many informed and similarly aligned views on a forum - one big happy ASR family.

Seriously though, so easy to criticise, so hard to step up and make the difference you believe in. I am grateful for the time and effort some make to help others on here, and equally for @amirm who is trying to effect change in an industry which still worships the sacrificing of virgins (or similar belief orientated dogma).

Apologies if someone has already asked this question - I am half way through the reading of this thread.


But I wonder if anyone had thought critically about perhaps the clearest and most common source of difficulties in listening for differences between speakers - the testers hearing loss!! Fairly linear with age, and certain activities make it worse, and not to mention the male bias towards significant hearing loss generally.

Subjective interpretations of audio from the limited range of hearing an individual has, does not seem insignificant, easy to have tested to confirm tho I guess.

How well does @amirm hear these days?(joking)
 

Rusty Shackleford

Active Member
Joined
May 16, 2018
Messages
255
Likes
550
Funny ... wiggeling a way out of this. Someone thinks they can easily show Amir he has cloth ears, put him on his number, make him look like an idiot in the process and get a lot of money. Remember ... it is easy to hear the differences between DAC's/amps whatever. Easy money right there.
No takers ? why ? because Amir should come to their homes ?

Come-on if one could really tell similarish measuring gear apart reliably under lab conditions Amir would not offer money for this extraordinary feat .

No takers because deep inside they fear there is a chance they might not be able to tell 'night and day' apart when they are in a room with no windows.

There’s no “wiggling out.” Science cannot only occur at Amir’s house. Amir has made clear that he self-trained, then used How to Listen. He’s also stated that he can hear differences between DACs, even without level matching, because “once you are trained, you can easily look past most if not all volume differences.” Holding others to a different standard isn’t acceptable.
 

NDC

Member
Joined
Jul 18, 2020
Messages
86
Likes
115
Location
Sydney, Australia
There’s no “wiggling out.” Science cannot only occur at Amir’s house. Amir has made clear that he self-trained, then used How to Listen. He’s also stated that he can hear differences between DACs, even without level matching, because “once you are trained, you can easily look past most if not all volume differences.” Holding others to a different standard isn’t acceptable.

Rusty, seems a bit intellectually dishonest to quote @amirm from 2009 - 11 years ago* on another forum in support of an argument you’re having today in a very different context. That’s a lot of time for learning and views to change... just seems a little below the belt, no?

*AVSforum posts
 

Rusty Shackleford

Active Member
Joined
May 16, 2018
Messages
255
Likes
550
Rusty, seems a bit intellectually dishonest to quote @amirm from 2009 - 11 years ago* on another forum in support of an argument you’re having today in a very different context. That’s a lot of time for learning and views to change... just seems a little below the belt, no?

*AVSforum posts

How on earth is it “intellectually dishonest” or “below the belt” to quote someone’s public posts? Moreover, what does 2009 have to do with anything? This forum is full of citations, including from Amir, that are far older than 11 years!
 
Top Bottom