saw your update and replied.You probably missed my updated post. But no, I don't see your wish to open that door as helpful.
saw your update and replied.You probably missed my updated post. But no, I don't see your wish to open that door as helpful.
I don't think Fremer was saying he took all the Harman training. His tossed results were at another public blind test. He scored 5 for 5 and was labeled a lucky toss expected out of all the public results. I think Clark offered to let him try and see if he got 5 for 5 again or maybe Fremer offered. In any case Fremer took exception and believed his results were not respected properly.You probably missed my updated post. But no, Amir's words has been proven by 3rd party data (like the articles) time and time again. And his words have a clear logic, unlike people who opposes in very general terms and then fails in providing backing data.
If he actually scored a job at Harman, trained there for 8 months as a listener and then willing to stake his professional certificate / reputation on the line for saying what he says, sure. I probably won't be the first to call him out, his peers probably will.
So anyone else can verify that 'his results are tossed'? That he took the harman tests? That he did the 8 months of training?
You tell him that we are talking about different speakers, not amps.What are you going to do when Michael Fremer pipes up and says "I did that test. Now you all have to defer to my (sighted) views." Hmmm? He already writes things like this:
"I've taken many blind tests including at Harman's test facility and demonstrated the reliability of my listening, and both JA and I took part in a 5 amplifier blind test at an AES after I agreed to a challenge made by former Audio writer Dave Clark who claimed all amps that measure the same sound the same. I got all identifications correct and then though it wasn't part of the test, once we knew the identities of all five amps, I correctly identified ("blind") which was which. Nonetheless, though I scored 100% and JA scored 80%, because the population did no better than chance, we were both declared "outliers" and our results were tossed! What a joke. Interestingly, Clark included a VTL 300 tube amp in his test, an amp that measured and sounded very different than the others, which were solid state. Yet most of the AES crowd couldn't hear that one either! Why? Because under double blind conditions inexperienced test takers usually screw up and them (sic) did."
You want to open that door?
I don't think Fremer was saying he took all the Harman training. His tossed results were at another public blind test. He scored 5 for 5 and was labeled a lucky toss expected out of all the public results. I think Clark offered to let him try and see if he got 5 for 5 again or maybe Fremer offered. In any case Fremer took exception and believed his results were not respected properly.
Having heard the amps used (VTL 300) I too believe one might hear a difference. But I've not blindly tested myself.
...and, just so that no-one has to infer what my position is on this rather overheated topic, this is it:
Ergo, I attend to Amir's sighted reports with interest and respect. I don't agree with those who diss him relentlessly over them.
- DBT is the gold standard for testing for a perceptual preference or difference that lies solely in the sound waves.
- A highly trained listener's sighted report would be a bronze standard. Somewhere between gold and nothing at all.
- Simple sighted listening is no standard at all, and proven so.
Also ergo, when Amir concludes with something like, "my sighted listening differed from the measurements, therefore more research is needed into better measurements", it raises a flag for me. If the measurement system was derived and confirmed using the gold standard, which I believe is the case, then you can't disprove it using a bronze test method. Yes, you can raise a query based on a bronze standard test, but the obvious next step needed, before reaching conclusions, is to validate the bronze test result with a gold standard test, i.e. DBT. Once that confirmation is made, then we can start to say "a better measurement suite is needed".
Other than that proviso, I am very happy with the test reports and with the subjective component, and respectful.
cheers
I think a lot of people participating in this thread is losing sight of what this website is.
A few previous comments have allured to the connotation of this site's name: Audio SCIENCE REVIEW. But unless I am mistaken (and correct me if I am wrong), this is not a scientific research / peer review site. This is, first and foremost, an audio enthusiast site with a strong emphasis on the science of high fidelity gears. All other discussions are really subordinate to that purpose.
A majority of ASR members (I am among them) are amateurs with no no scientific knowledge / experience in the field. We are here primary to discuss the merits of commercially available audio equipment, and I am very grateful for those members who are experts in this field contributing their knowledge to further our understanding. We should be thankful for Amir's time and effort in testing gears because he is doing this out of courtesy, not obligation. This is his site, after all, and it doesn't have to behave as a democracy buy clear he has choose to.
I want to point out that amateurs quoting other's research is not a scientific review, because I see nobody offering a counter-hypnosis on the issues raised thus far on blind vs sighted testing, trained vs. experienced vs. inexperienced listeners. A true scientific discussion will involve coming up with an alternative take on the 'flawed' theory / methodology, and coming up with ways and then proceed on testing / verifying that hypothesis. After 11 pages of debate I see no evidence of that.
Amir is carrying out audio testing on equipment using what he understands as best practice / techniques currently available. I think a healthy discussion on Amir's methodology is welcomed, but again the agenda here is to test and evaluate audio equipment, and not necessarily on reviewing the methodology itself, let alone the credential of the reviewer. Once again - this is not a science peer review site, this is an audio equipment review site.
I think debating on the testing methodology is good, but not when members start attacking the validity of the very purpose of this site.
One idea I can suggest is perhaps ASR has grown sufficiently large and influential now, that we need to establish a Charter, which will clearly state the principles and purpose of the site. It will act as a Caveat emptor sign for visitors who enters this site.
Careful. This makes you vulnerable to the reply: Maybe it's the ones without skin in the game, sunk cost bias, who can be more objective about the site.
[Only kidding. A little...]
Note that while you do not want to 'open the door' to trained listeners being able to hear things others can't, there are specific companies like Dolby, Microsoft and Harman who actually opened their doors, trained and paid them. Or are you taking the position that such people, and job positions, do not actually exist?
Update:
Did a quick google search and this came up:
http://www.stroudaudio.com/skilled-listener-training.htm
Is this considered a real thing from your pov? That someone can be trained and go on to work as a skilled / trained listener?
Update2:
Is Jodie Lampert a real person with a real job (among '40 other trained listeners' as mentioned in the article)?
https://www.hourdetroit.com/community/harman-international-car-audio-tech-test-listeners/
The problem is that people who claim to hear differences that do not seem to be very likely probably won't volunteer.
Not talking about transducers but amps, pre-amps, DAC's etc. There are multiple ways to test.
Power amps, of course, are load dependent and could in certain circumstances differ from lab conditions. Might even reach audible thresholds.
That doesn't mean that you can hear things that are not measurable, demonstrable.
Amir has offered money for those that can demonstrate this ability.
That should be like taking candy from a baby. The offered reward by Amir is enough to compensate for expenses made to visit Amir.
Why hasn't anyone taken this opportunity ?
The test HAS to be administered by someone who knows how to test correctly. We all know very, very few people compare or test correctly.
In fact the very very vast majority that makes claims of audibility does so based on obviously sighted and incorrectly performed 'tests'.
Valid to them, not valid to act as definitive proof in court.
I think the way you look at Amir and ASR and its goals differ from the one set out by Amir.... The way you look at it seems to stem more from an 'haters' or 'sceptics' standpoint towards measurements.
I believe Amir's goal is to 'peer review' manufacturers specs, to weed out stuff that doesn't perform well. Also to inform people that are like-minded. Maybe to point out flaws to manufacturers. Its goal isn't to please or ridicule people that prefer subjective and sighted testing.
What ASR can do is point out why certain testing is SUSPECT (and thus not very valid).
of course... my p.o.v. not necesarilly yours or others. It's why I like it here. There are 'like minded' folks here and those that want to learn a bit.
Also those who oppose ASR views who like to put a magnifying glass on everything and every word but refuse to test properly.
That's life. Everyone should be free. Life isn't always fair either.
Sure, you use 'However, IF we open the door...', forgive my incorrect paraphrasing.Where did I say I don’t want to or don’t
Sure, you use 'However, IF we open the door...', forgive my incorrect paraphrasing.
The rest of the information I posted is still pretty straightforward:
Trained listeners are 'a thing' and their work is to (not being one, I can only guess) catch the nuances of audio systems/speakers (that untrained listeners cannot hear, by simple definition), presumably for the betterment of their company products.
Update:
Your comment was '...If we open the door to trained listeners being able to hear things others can't...'
Since we're talking about 'trained' listeners, then it is natural that the 'others' you mentioned are untrained. With their training, that is precisely the case: they are able to hear things 'others' can't....
The door is already open, there's no IF.
If Amir, who’s previously said speaker blind tests aren’t as complicated or costly as people claim, can’t set up a blind tests for speakers, it’s certainly unreasonable to expect someone who wants to show they can hear a difference between amps or DACs to travel with their equipment to Amir to take a blind test. He cannot he the only person capable of using a multimeter and a passive switch.
Someone posted the Fremer comment on his blind tests above. Likewise, people (with industry credentials!) on other sites have set up and passed blind tests. It’s simply untenable IMHO to dismiss such tests while at the same time allowing Amir to claim that, in sighted listening weeks apart, he can reliably differentiate between a 5.70 and 5.79 rated speaker.
It’s simply untenable IMHO to dismiss such tests while at the same time allowing Amir to claim that, in sighted listening weeks apart, he can reliably differentiate between a 5.70 and 5.79 rated speaker.
I’m saying that there’s no reason anyone can’t say, “I’ve trained on How to Listen, and I detect _____, even if untrained listeners can’t.” The “if we” wasn’t about the wider world, it was about ASR, where claims that some people can hear things that others (and even measurements) cannot detect have been roundly mocked.
Setting up a blind test with speakers is complicated. It involves at least a helper. 2 speakers close together, both need to be connected to an amplifier (or the not used one shorted), levels must be matched (as far as possible) etc.
I still don't understand why people that are so convinced they can so clearly hear differences and live at a reasonable distance from Amir do not take him up on his offer .. I mean.. its like taking candy from a baby..
Not having trust in Amir setting up a decent blind test is a straw man. Besides when he is not even capable of doing that more power to the one aking the test.. it should be even more audible. No.. that's a lame excuse.
Blind test are easy to pass when the differences are so big they are audible or the test was flawed and there was trickery in play to enforce a certain outcome.
When the 5.70 and 5.79 rated speakers have certain (small) distinctive traits... why not ?
I mean they can be close in tonal balance but differ in some aspects.
Its like saying... well I have 2 DACs that both have a SINAD of 85. It may e very possible to tell them apart with 100% certainty.
When one is rolled off and the other is not or one is noisy and the other one not but has higher distortion.
When you really want to put the wringer on Amir... take the blind test. Or make a bet he can't reliably tell those rated speakers apart and witness it.
Sure, thanks for the clarification. Yes, it's true that people can learn from the online listening tests and eventually can make such statements too. That's the nature of the democratization of knowledge.
A simple analogy:
Certified accountants have all gone through the proper accounting training, so they can say that they can do amazing things with spreadsheets that others can't. And they won't be mocked.
Of course, there have been certified accountants who threw out professional ethics, broke the law and got locked up. And there will be new upcoming accountants who will one day break the law (some of them). But rogue accountants are a totally separate issue from the training.
If Amir is a certified, paid-by-corporations-to-work, trained listener, then it's within his professional right to say that he can hear things others cannot.
At the same time, there is a clear distinction between saying, 'I took a course online and passed', versus, 'I was trained and worked for years to test speakers'. The onus is now on the reader and his ability to see the distinction.
Amir to claim that, in sighted listening weeks apart, he can reliably differentiate between a 5.70 and 5.79 rated speaker.
One should expect any careful listener to be able to hear a difference between two speakers of any given "Harman Score," including Speaker A scored x and speaker B also scored x, unless every measurement that went into those scores was also identical. Remember that we're talking about transducers radiating into 3D acoustic space, not an electronics box that takes an input over a cable and has an output over a cable.
Difference and preference are separate animals. To me it is clear that a speaker with score x but a large dispersion disruption where woofer hands of to tweeter will sound worse than a speaker with the same score to whatever decimal places but correct geometry and a well-engineered crossover.
It still blows my mind that all this brouhaha started because some vocal people were miffed Amir didn't like a speaker of common but very flawed geometry, which resulted in predictably poor measured performance. IMO the more interesting question isn't "why did this bad speaker sound bad" but "why was Amir so nice to the GRR DIY speaker of similarly terrible design?"