Post research here that casts doubt on ASR objectivism

[Inner Space]

A perfectly linear radiating loudspeaker results in a perfect line when measured with a microphone (see blue curve).
However, something completely different arrives at your eardrum due to the HRTF (see red curve). The HRTF "changes" the sound pressure curve of the ideal loudspeaker considerably.
View attachment 132280
Source: Lowbeats.de
The red curve shows the sound pressure curve of an ideal loudspeaker directly at your eardrum.

Really useful graphic, thanks. The red curve is like an upside-down Fletcher-Munson curve, which makes sense, I suppose - the world comes at us with flat sound power, and we have evolved to mechanically amplify select parts of it.

 

[SubOjectivist]

I am. You sent me a non-experimental review/editorial written by non-doctorate published in a lower-tier journal in the health sciences, and I'm still not sure what point you were trying to make. The Harman papers are experimental papers published by Ph.D.'s in a top engineering journal.



Well for starters, do you know what an impact score is? Because if you did, you probably wouldn't have asked "what's wrong with open access?"



Okay, at least this paper directly measured perception of benefit vs. actual benefit. But for the last time, what's your point? We already know that humans can perceive a benefit when none exists. It's the placebo effect. This study just further support that concept, in the same way that cable believers perceive a strong sound quality benefit from snake oil cables, when no such difference in perceptible sound quality actually exists, as demonstrated in a blinded, controlled, listening test.



I hope you realize that some of the Harman experiments included diverse human populations, including listeners living in different countries, different genders, and different ages.

Yeah, many women, 22% actually. In your country or on Earth, as I said before, it's definitely slightly more than 50%.
Pfizer or Moderna, they used exclusively their own employees to conduct the RCTs of Covid-19 vaccines candidates?
Do you also assume that a 13 y.o. kid will respond exactly like a 65 y.o. adult? Ask drug companies what they think. Compare their athletic performance, or cognitive abilities (like the ability to learn new things).

When you mix a heterogenous population and try to get an average, you end up getting the typical European adult, 50% female, 50% male, who needs 2000 kcal daily according to the E.U. . Very wise, indeed.
Just in case you believe energy requirements are the same for both sexes : https://academic.oup.com/ajcn/article/99/1/71/4577371

 

[garbulky]

The thread was an invitation to contribute research that refuted several common hypothesis on ASR. The hypotheses are summarized in the OP, with minor edits made to incorporate research that was referenced in the replies. As you might imagine, there hasn't been a whole lot of research to refute these hypotheses. Perhaps you know of some?

No, sorry I don't know of any, at least ones that pass scientific rigor. Would be interested in reading...

 

[BrEpBrEpBrEpBrEp]

But then, why not utilizing their source of inspiration more broadly, for instance, by declaring the source of funding or the potential conflicts of interests? How good is science that can't be trusted?
Also, why not (ASR) use vendor neutral approaches for reviewing hardware? AKG Acoustics is a part of Harman (now Samsung). So, AKG headphones are reviewed using methodology / research from Harman, sic. They also used an AKG K712 to simulate other headphones. How can you be certain results would have not been different using another brand? Can you prove it? I guess no ...

It says Harman International on the AES page, as well as in the paper right below the researchers' names. Not to mention that anyone who would read the paper would know already that Dr. Olive works for Harman. The source of funding definitely isn't obscured in any way.

ASR uses the Harman curve because it was developed using measurements to replicate the tonality of good speakers, and then it was validated using many subjective tests. It's the best approximation we have of what the mixing and mastering engineers were hearing through their studio monitors.

Would you trust Klippel for speakers measurements if they also developed their own range of speakers? Would you evaluate such speakers with a Klippel? Or Audio Prevision if they manufactured DACs? You would also review their DACs with an AP?

Yes, if they published a series of studies demonstrating the validity of their methodology. Amir could review a cheaper AP analyzer model using his APx555, I have no problem with that. Why would it matter?

 

[SubOjectivist]

It says Harman International on the AES page, as well as in the paper right below the researchers' names. Not to mention that anyone who would read the paper would know already that Dr. Olive works for Harman. The source of funding definitely isn't obscured in any way.

ASR uses the Harman curve because it was developed using measurements to replicate the tonality of good speakers, and then it was validated using many subjective tests. It's the best approximation we have of what the mixing and mastering engineers were hearing through their studio monitors.



Yes, if they published a series of studies demonstrating the validity of their methodology. Amir could review a cheaper AP analyzer model using his APx555, I have no problem with that. Why would it matter?

Still, it's not clearly stated.
And what about conflicts of interest that were not disclosed, if there is any?

The problem, is that the instrument could be calibrated to give better results for the same brand of equipment that is measured. Don't be so naive.
Or perhaps, it's well established that all companies are honest, right? They never lie, cheat or try to mislead consumers. MQA is totally transparent, because after all, they said so ('MQA files are delivered losslessly and reconstruct exactly the sound that an artist, studio or label approves. ').
https://bobtalks.co.uk/a-deeper-look/all-that-glitters-is-not-golden/


Maybe it's the best study on the subject so far, so what? Does that necessarily mean that ASR should rely on it?
@preload Double standard for studies?

 

[pozz]

Exactly. While people might "reason" that older listeners might prefer more treble (and hey, it makes sense), the research demonstrates that they do NOT. Older listeners have essentially the same treble preferences as younger listeners. And @pozz, you provide a very cogent explanation as to why.

And this is why it's important to do empiric research - it's because it turns out, "reasoning" things out doesn't always give you the correct answer. It might impress people at a party, but it doesn't mean you arrived at the correct answer. What DOES affect treble preference (in headphones at least), according to published Harman research, is gender and country of residence. Is that something anyone here could have predicted with certainty?

I remember reading that pro listeners tended to prefer a deeper tilt (less treble, more bass), although not significantly more. Can't remember where. Bob McCarthy of Meyer Sound Group mentioned it in his book, but without citation. Might be worth looking at Francis Rumsey's work too.

 

[weasels]

I just read it diagonally, fair enough. But I have another one for you, don't worry :
'Furthermore, in sleep-deprived individuals treated with Methylphenidate, results have shown that people overestimated their own performance, and experienced a subjective sensation of stimulation with only a mediocre improvement in attention. The perception of methylphenidate as an effective cognitive enhancer may be more related to the subjective effects experienced by the user than its realistic implications.'

Full study link: https://link.springer.com/article/10.1186/1472-6939-15-20

Did I get this part wrong, too?

First, a reference to subjectivity in a field of study other than psychoacoustics is irrelevant to the OP. So I'm not sure what point there is in referencing this article at all.

Second, the very passage you quote states that the perception of the drug is related to subjective effects. That is as axiomatic a statement as can be possibly made, and does nothing to refute any of the scientific or analytical points raised elsewhere in the thread.

Even if you get an objective metric from a sample of subjective preferences, in the end of the day, it will be subjectively based, that you want it or not

Are you arguing that it is impossible to objectively analyze subjective preferences? That would certainly be a unique position. I'm sure the firms paying 6 figure salaries to quantitative analysts to analyze social media data they paid millions for will be thrilled to know they can shut down all that nonsense and save some money.

 

[BrEpBrEpBrEpBrEp]

Still, it's not clearly stated.
And what about conflicts of interest that were not disclosed, if there is any?

The problem, is that the instrument could be calibrated to give better results for the same brand of equipment that is measured. Don't be so naive.
Or perhaps, it's well established that all companies are honest, right? They never lie, cheat or try to mislead consumers. MQA is totally transparent, because after all, they said so ('MQA files are delivered losslessly and reconstruct exactly the sound that an artist, studio or label approves. ').
https://bobtalks.co.uk/a-deeper-look/all-that-glitters-is-not-golden/

I literally says on the first page of the study that they work for Harman International. How could it possibly be more clearly stated? I promise you that absolutely not one person at an AES conference would be misled to believe that they didn't work for Harman.

And, by the way, their headphones didn't score well by this research. They only started developing good-scoring headphones afterwards. You know who's headphones happened to be close to the Harman curve, even before it was developed? Sennheiser.

And it is literally impossible to calibrate an audio analyzer to bias for a given company's analog equipment. How could it even tell through analog connections?

 

[preload]

Yeah, many women, 22% actually. In your country or on Earth, as I said before, it's definitely slightly more than 50%.
Pfizer or Moderna, they used exclusively their own employees to conduct the RCTs of Covid-19 vaccines candidates?
Do you also assume that a 13 y.o. kid will respond exactly like a 65 y.o. adult? Ask drug companies what they think. Compare their athletic performance, or cognitive abilities (like the ability to learn new things).

When you mix a heterogenous population and try to get an average, you end up getting the typical European adult, 50% female, 50% male, who needs 2000 kcal daily according to the E.U. . Very wise, indeed.
Just in case you believe energy requirements are the same for both sexes : https://academic.oup.com/ajcn/article/99/1/71/4577371

I'm sorry, but did you just compare the billion-dollar research effort to find a vaccine for a disease that has killed 3.5 million people to a privately-funded study looking at headphone audio quality?

Yeah, I think I'm done here. I can tell that you have some basic knowledge of experimental research (as in textbook/coursework knowledge), but it's really not translating into a coherent conversation, and certainly not one at a professional level. I tried, and I'm going to cut my losses and exit this dialogue.

 

[raistlin65]

Still, it's not clearly stated.
And what about conflicts of interest that were not disclosed, if there is any?

Now you need to eliminate one of your assumptions.

This research article was not written for you. You are not the intended audience.

It's an AES conference proceeding paper written for a specific scientific/engineering discipline, and as such relies on disciplinary knowledge.

It is disclosed that this is written by Sean Olive, and he is associated with Harmon International. Everyone in the discipline knows what that means. Scientific research articles do not have to disclose things such that it is meaningful to a lay person, or someone from another scientific discipline.

 

[SubOjectivist]

I literally says on the first page of the study that they work for Harman International. How could it possibly be more clearly stated? I promise you that absolutely not one person at an AES conference would be misled to believe that they didn't work for Harman.

And, by the way, their headphones didn't score well by this research. They only started developing good-scoring headphones afterwards. You know who's headphones happened to be close to the Harman curve, even before it was developed? Sennheiser.

And it is literally impossible to calibrate an audio analyzer to bias for a given company's analog equipment. How could it even tell through analog connections?

Working for Harman doesn't make it de facto the (sole) source of funding. They could have gotten a grant or have a kind of private / state funded partnership or whatnot.

The audio analyzer could store the pre-measured electrical features of a measured equipment in its internal memory, and recognize it. Why would that be impossible? Even if there is some sample to sample variation, that probably could be account for with a small margin. The more characteristics are memorized, it easier it should be.
How could Bosch detect testing for cars' particles / gas emissions during tests? They somehow managed to do it.

 

[Xulonn]

...A tool doesn't have to be extremely accurate to be useful.

Excellent point and related to countering the whining of AGW/Climate change denialists who try to challenge climate models.

I always think of George Box on models:

Remember that all models are wrong; the practical question is how wrong do they have to be to not be useful.

 

[BrEpBrEpBrEpBrEp]

Working for Harman doesn't make it de facto the (sole) source of funding. They could have gotten a grant or have a kind of private / state funded partnership or whatnot.

The audio analyzer could store the pre-measured electrical features of a measured equipment in its internal memory, and recognize it. Why would that be impossible? Even if there is some sample to sample variation, that probably could be account for with a small margin. The more characteristics are memorized, it easier it should be.
How could Bosch detect testing for cars' particles / gas emissions during tests? They somehow managed to do it.

I'm not even going to address how you're mistaken on a technical level. I believe this forum generally avoids engaging in baseless conspiracy theories and, in that spirit, I'm no longer going to engage with you in this thread. Let me know when there's evidence of wrongdoing by Dr. Olive.

 

[SubOjectivist]

First, a reference to subjectivity in a field of study other than psychoacoustics is irrelevant to the OP. So I'm not sure what point there is in referencing this article at all.

Second, the very passage you quote states that the perception of the drug is related to subjective effects. That is as axiomatic a statement as can be possibly made, and does nothing to refute any of the scientific or analytical points raised elsewhere in the thread.



Are you arguing that it is impossible to objectively analyze subjective preferences? That would certainly be a unique position. I'm sure the firms paying 6 figure salaries to quantitative analysts to analyze social media data they paid millions for will be thrilled to know they can shut down all that nonsense and save some money.

Nope. But it just gonna be that: an objective analysis of ... subjective preferences.
But as I stated before, my remarks were made only in the light of this study. I provided some specific examples.




Apart from a minority who also recognized explicitly that in their opinion, the study was far from being perfect (thanks to them), the majority (well, at least the more vocal ones) tended to disagree with me, fair enough.
I could just ignore the Harman curve if I read the FR graph for a pair of headphones. I'm totally fine with that. Incidentally, I don't need new headphones atm.

Anyway, thanks also to those who disagreed, sometimes it's even more interesting to discuss with people who have a different opinion than our own. If we totally agree with each other, then there is not much to talk about, right?

Last but not least, even if I no fan of the harman curve to say the least, I enjoyed reading DAC or amplifiers reviews here.

 

[SubOjectivist]

I'm not even going to address how you're mistaken on a technical level. I believe this forum generally avoids engaging in baseless conspiracy theories and, in that spirit, I'm no longer going to engage with you in this thread. Let me know when there's evidence of wrongdoing by Dr. Olive.

Ok, so know you call skeptics conspiracy theorists? I had the view that, in order to remain objective, scientists must remain skeptical. But whatever.

 

[pkane]

Here's some research that casts doubt on everything objective and therefore, by inference, on ASR:

https://www.technologyreview.com/20...ts-theres-no-such-thing-as-objective-reality/

 

[Jimbob54]

Here's some research that casts doubt on everything objective and therefore, by inference, on ASR:

https://www.technologyreview.com/20...ts-theres-no-such-thing-as-objective-reality/

Oh no, we had a reality denier pass through these parts recently. It didnt go well.

 

[BrEpBrEpBrEpBrEp]

Honestly I doubt these quantum effects are going to suddenly mean that an inverted Harman curve is just as faithful a reproduction of the studio as the regular one. But I guess the research will show if that's the case.

 

[pkane]

Oh no, we had a reality denier pass through these parts recently. It didnt go well.

No reason to deny reality, it exists. But it can be different for everyone, if you extrapolate the result from the experiment to objects larger than elementary particles

 

[Wes]

Is it time for this yet?