Message to golden-eared audiophiles posting at ASR for the first time...

[ahofer]

I think it might be useful to sticky a thread that compiles blind test studies and results for various electronics.

not for lack of trying:

https://www.audiosciencereview.com/forum/index.php?threads/catalogue-of-blind-tests.8675/

 

[ahofer]

Bias has different meanings and Googling turns up very specific examples like McGurk which may or may not often apply to HiFi.
Here we use it much more broadly.
But, to laymen, it is pejorative.
Bias and prejudice are what the other guy has. Good intuition and insight are what I have.

So I agree with Hon that it is not the best term to throw at newbies on their first visit.

Perhaps. Personally, I see it as used very widely in many different ways. Yes, there is racial bias, but there is unconscious bias and the very broad field of cognitive biases, where Nobel prizes have been awarded etc (and there is this sort of thing as well, and this right here on ASR). This is a problem with users who haven't read much. If they come in here with guns blazing and they don't have even this level of worldliness...

OK, I will try to use "cognitive bias", so the cyber-research works a little better. Thank you.

 

[richard12511]

Too much Groupthink and Too Little Critical Thinking

The two most obvious characteristics of posts in far too many threads on this forum, with the approaching one thousand posts in this thread being a singularly-good example are:

- A massive concentration of Groupthink. New members are drawn to the group and repeat the group speak even though "G" is small, the group is huge.

The subject of a thread, even where the technology is quite clear, such a cables, is soon taken over by clones who demonstrate little technical knowledge, but parrot what they have read on the thread. They berate anyone unfortunate to offer any questions or alternative opinions. It is hard to see any advantages to boutique cables or similar nonsense, but given the way a large number of members approach this question it is rather embarrassing to agree with the obvious science and be on the same side as these all too numerous individuals.

The continued references to DBT is a painful example of this groupthink combined with ignorance. Valid DBT's of overall audio questions are almost never run, but the single answer to many questions often refers to DBT. It is a way for those who know little to believe they are spouting some universal truth.

IMO Amir has begun his educational video series partly (mainly?) in response to the demonstrated ignorance of a huge huge number of the members here.

- A miniscule quantity of Critical Thinking. Even measuring the critical thinking changes its value.

I kinda get what you're saying, and somewhat agree. I think it would be useful to have a stickied thread that compiles various electronic blind tests and their results. There is one test Amir posted in another area that demonstrates people successfully differentiating amplifiers under blind conditions.

I really like this test, and often link to. It's an easy test that anyone can take right away of a not even SOTA DAC, it will test to see if that DAC is perfect to beyond the limits of one's own hearing. Also, it's just $9, so for those claiming to hear improvements from high end SOTA DACs, it's a fun reality check to see if they can even hear any improvements beyond $9.

 

[richard12511]

not for lack of trying:

https://www.audiosciencereview.com/forum/index.php?threads/catalogue-of-blind-tests.8675/

Nice! Exactly what I was talking about, but would be nice to have it stickied.

 

[whazzup]

How would you suggest we go about determining if it's 1 or 2 when someone posts here? I would say that most of those subjectivists jumping into ASR, except for an occasional manufacturer, belong to your truth #2. Does this make a difference? And how would you approach someone from group 2 differently if they come here with an intention of showing to themselves and to others why their beliefs are sacred and can't be questioned? A few do come to learn, but these are often obvious from the first few posts. These are far and few in between, but yes, unfortunately they may also get attacked and turned away as their intentions are misinterpreted. So, how do we determine someone's intent?

Here's a suggestion:

If it comes to a point where someone is suspected of being a troll, or has no intention of holding discussion in good faith, the moderator can request the poster to answer some mandatory yes/ no questions.

For example:
1. Bass and treble controls have existed in all sorts of Class A/AB/D/H/etc amps for at least the last 100 years, does he agree with this?

2. Does he agree that increasing the treble tone control of an amp results in a brighter sound? And that it's an audible effect for him?

3. Does he agree that increasing the bass tone control of an amp results in a stronger bass? And that it's an audible effect for him?

4. If he says yes to the above, does he then think that whatever audio equipment he's arguing about, it is affecting the sound output in an audible manner, YET it is different from the effects shown by the tone controls?

Or if no, why and how?

I might not have worded them well, but if he simply refuses to answer the above in a straight forward fashion, then the moderator can decide the next course of action.

Because there's no way to move a discussion forward if someone already refuses to accept the above questions as facts.

 

[sergeauckland]

Let's be concerned about all three:

Consider TIM -- transient intermodulation distortion. Is it concerning? I encountered it years ago when it was brought to light by Marshall Leach when he published a DIY project for building a "low TIM amplifier" in a popular magazine. Parasound seems to still recognize its potential significance, mentioning it in some amp specs. This article describes it in the context of designers back in the day noting "a lack of correlation between conventional amplifier distortion measurements and listening tests...": Audiophile Review

I asked Google about "perceptual bias." Lots of junk, of course. The only substantial piece I found-- a genuine research project based on the perception of colors--concluded that bias quickly fades or even disappears completely--over a short time. I don't think "bias" is a helpful term to throw around in the context of friendly audiofiley discussions; it's too easily weaponized and has unhappy moral overtones.

In a word, No! An amplifier that can output full power at 20kHz at low enough distortion will be fine. Provided the source doesn't send a higher voltage at a higher frequency, (which it can't on speech or music, whatever sampling rate is used) then no TIM is possible. Good amplifiers have a wide open-loop bandwidth and a restricted closed loop bandwidth to avoid any such problems.
The simplest way is just to have a low pass filter at an amplifier's input, then TIM just isn't possible.

TIM was a 'thing' back in the 1980s, and proved back then to be a nonsense for any half-decent amplifier. If anyone drags up TIM now, they're creating a non-existant problem for them to solve.

S.

 

[ahofer]

Nice! Exactly what I was talking about, but would be nice to have it stickied.

My call in that thread was for a group-sourced effort, a Wiki (Amir does not like Wikis, unfortunately). The main problem with other catalogues is that a) they have immense link-rot and b) they are not well organized by type of electronic/test. I though that if the members here chipped in, we could move the highest quality tests to the top, archive the paper/website to avoid link rot, and organize by type of equipment and methodology.

Lots of work for us casual hobbyists. I'd still like to do it.

 

[richard12511]

but it annoys me that when interesting counterclaims are made (I very much feel bigguyca's post does that) that they are often not fully considered.

Why though? Given we have a mountain of evidence showing what they say is impossible, why should we "fully consider" claims that provide zero evidence other than personal anecdote? You haven't really given a good answer for that, imo.

 

[Jimbob54]

Buster Chestnut said:
And almost to the minute, someone posts something asinine on a ”science” forum implying:
(1) we know and can measure all there is about sound and human hearing
Raistlin65 said
I don't think that's what most of the regular members here think.

We know that sound can be measured.

We don't always know how human physical hearing/the brain will interpret it.

> unfortunately Buster is correct. I can go back and provide specific quotes if necessary but some people have posted that unless you can provide measurements for what you are saying, your posts are basically worthless and not welcome

Depending on what was under discussion in the threads and specific responses this surprises me . The line I am more familiar with seeing (and probably saying) is unless you can provide evidence to support what you are saying then the post may not be of much if any value. Usually when a claim is made for a preference or hearing a difference in components where that would be unlikely. Asking for measurements would only make any sense where the poster is referring to measurements themselves but not sharing them.

You watch, it will be one of my posts you drag up

 

[richard12511]

There's a difference between teasing and lambasting. and I would expect the eagerness of others to join in of a different crowd. Didn't expect it here

There is, though I personally don't like either. I think it's possible to disagree or dismiss claims without teasing or lambasting.

I just disagree with folks saying we should take every extraordinary claim with no controlled evidence seriously.

 

[pkane]

Why though? Given we have a mountain of evidence showing what they say is impossible, why should we "fully consider" claims that provide zero evidence other than personal anecdote? You haven't really given a good answer for that, imo.

I think @BDWoody is doing a great job of trying to get the newcomers to understand what kind of evidence would be acceptable here. It's important for someone who wants to be considered seriously to come here with more than the "I hear it therefore it must be true" attitude. Not everything is cut and dry in audio, there are still plenty of areas where more study is needed, but these don't include basic electronics or Physics. Someone claiming that their sighted hearing preferences negate basic science is not only silly but arguing against these claims gets really tiring as it is repeated so often. More tolerance and respect might be appropriate, but we are all only humans... at least I think most of us here are

 

[ahofer]

One interesting technique that we might consider is a "steelman" thread (opposite of Strawman), where we go through what we think are the *highest quality* objections to our approach, instead of piling on low quality objections. It may be revealing of some *actually interesting* ideas, as well as how people behave if the objections are stated in a more qualified way.

But you guys first . Honestly, I don't think I understand the psycho-acoustic research well enough to formulate a good steel man. But I am past the point of having some humility about my abilities, I hope.

 

[Jimbob54]

One interesting technique that we might consider is a "steelman" thread (opposite of Strawman), where we go through what we think are the *highest quality* objections to our approach, instead of piling on low quality objections. It may be revealing of some *actually interesting* ideas, as well as how people behave if the objections are stated in a more qualified way.

But you guys first . Honestly, I don't think I understand the psycho-acoustic research well enough to formulate a good steel man. But I am past the point of having some humility about my abilities, I hope.

Still wont stop myriad people rocking up in the middle of the review thread for the latest Topping DAC saying that the Gustard has better highs or the SMSL handles transients better (always bloody transients!)

 

[SIY]

I agree with you that in the wild, there are far more people who believe their senses than charlatans trying to scam. I do not know the ratio for those posting here, but I sense there are more of #2.

I think it is a mistake to assume that #2 (sense believers) would not exist unless they had been bamboozled by charlatans.

Our perceptions being influenced by biases caused by previous experiences, stories, tribal beliefs is how life works. We could not function if every judgement we made had to be thought through as if it was the first time we had ever encountered it. The hungry bear would catch us before we had thought things through.

So, assuming that your NAD amp sounds better because of your previous encounters with the brand, the technology, the function first styling etc. is no different from enjoying your wine more because you know its a sought after vintage from a boutique vineyard in France. Somehow it is acceptable to say you think a wine exceptional without someone in a wine forum insisting on a blind test. I wonder what percentage of oenophiles could pass such a test? If they could not, would their enjoyment of wine be improved if from that time forward they drank exclusively the boxed stuff?

I agree with Hon (above) that bias is a loaded word. Kahneman calls it thinking fast. Thinking fast cannot be cured, but our autonomic responses can be changed. Its complicated, we all have a bit of Dunning Kruger I think. A first step may be (to paraphrase a current political figure) treat everyone with respect and do not fast think that you know their motivations until you actually do.

The oft-used wine analogy is a poor one. Technical wine evaluation is always done double blind. Any certification requires blind tasting. All significant wine competitions are conducted double blind. Many AOC/DOCG qualifications require a blind tasting. And so on.

 

[ahofer]

The oft-used wine analogy is a poor one. Technical wine evaluation is always done double blind. Any certification requires blind tasting. All significant wine competitions are conducted double blind. Many AOC/DOCG qualifications require a blind tasting. And so on.

Perhaps this actually extends the analogy, but the chemistry of what we taste is pretty well understood, hence the rise of Frankenwines.

Have a look at this interaction between the "audio doctor" (a NJ dealer, who is very nice in person) and me on that topic, starting here, where he makes a hilariously wrong claim about how wines are indistinguishable by chemical analysis yet taste different.

In fact, this thread is a little like our newbies here at ASR. I was getting the heavily subjective vibe of the place, and starting to ask pointed questions about the internal logic of everyone's assumptions. Partly prompted by the Audio Doctor himself, since he always tried to predict how different combinations of equipment would sound, I asked:

When speaking of system "synergies", do you consider these to be chaotic? or are they a predictable sum of the character of the components? I'm surprised at people who think they can predict the sound of a system from their perceptions of the components (derived, in turn, from other system combinations), and even more surprised and suspicious of the 'tone control' approach to purchasing cables and amplifiers suggested by another forum member (who does happen to be a dealer).

I think these two views are contradictory. If we think that components have 'magical' synergies beyond our ability to measure, then it seems unlikely that we also can predict how combinations of components will sound.

 

[GGroch]

Why though? Given we have a mountain of evidence showing what they say is impossible, why should we "fully consider" claims that provide zero evidence other than personal anecdote?......

I am confused now. In your post here you somewhat agree with what bigguyca was saying. When I posted my annoyance no one had either liked his post or responded to it. I hoped his post and others like it that question assumptions to be considered. You did. Thank you!

 

[BDWoody]

One interesting technique that we might consider is a "steelman" thread (opposite of Strawman), where we go through what we think are the *highest quality* objections to our approach, instead of piling on low quality objections. It may be revealing of some *actually interesting* ideas, as well as how people behave if the objections are stated in a more qualified way.

But you guys first . Honestly, I don't think I understand the psycho-acoustic research well enough to formulate a good steel man. But I am past the point of having some humility about my abilities, I hope.

I think something along this line could work...

Maybe we can come up with a list of the most common (straw man) arguments we see, and for each of them ask a qualified/willing member to write up the 'Steel Man' response in a way that is not insulting or patronizing, but that allows those who *actually* want answers to get them, without trying to sort through the search history.

We can create a new "Start Here to begin your debunking" forum, with sticky threads for each of the Steelman (science based) positions.

Maybe 8-10 common ones. That would let us point people there, rather than try to figure out new ways to say the same thing that won't bore the shit out of everyone.

Thoughts on some version of that?

 

[raistlin65]

Measurements made on this site completely validated my initial claim. It was only the personal attacks , the unwillingness of anyone to even ask to or see a picture of setup , help me work through the steps after I indicated my willingness to do so, and to give many alternate reasons suggesting I was delusional or a complete moron for believing I heard a difference instead of even considering I actually did that surprised me.

A lot of people have tried to discuss with you why perceived differences in DACs in sighted, non-volume leveled listening often will not reflect the measurements of DACs due to perceptual biases.

So rather than focusing on how you felt slighted by a few people, why not engage in those conversations and learn more about why this is true? As well as why your assertion that there are things that people hear that cannot be measured can be problematic?

You have said you are here to learn, so do that. Just because you didn't like what a few people said, that doesn't prevent you from taking advantage of those who are willing to discuss these topics with you. For example, I replied to you above, but you did not respond, instead choosing to make the focus of your comments since then on how others behave.

That being said, if you continuously restate a claim that people have already explained to you is contrary to the shared understanding of science held by this community, you may feel like people are piling on. That's because this is an open forum, and people are not responding just to and for you, but rather to others who might be reading. They are casting their vote for other readers to see. It is what it is.

At the same time, what people are asking you to do here is embrace a paradigm shift away from relying on personal anecdotal experience about audio for understanding it. And that is often difficult for people, since it's a different way of understanding of how knowledge is made. If you want to better grasp how much perceptual biases influence all of us, I would suggest reading Daniel Kahneman's Thinking Fast and Slow, which is about his groundbreaking work in cognitive biases, and for which he won the Nobel prize in Economics. It's a fantastic book. Once you understand his work, you'll totally grok what people are saying here about perceptual biases.

 

[GGroch]

The oft-used wine analogy is a poor one. Technical wine evaluation is always done double blind....

Of course it is done DB.
My point was not that DBTs are invalid or never used in competitions/business etc. but that they are not generally required of diners who post their opinions on a wine blog. There are hundreds of examples of interest areas where subjective opinions and reviews by consumers can be posted on forums or Yelp without demands for double blind proof.

Enjoying audio gear is I think more like enjoying wine than doing cancer research where the standards must be very high. If you are a manufacturer creating audio gear, or running a "Big Shot" comparison test like Tyll used to publish, DBT is the way to go. I guess my wording was not clear about that.

I recognize that this is Audio Science Review, not Audio Fancy, so the perspective is different here. It should not surprise us when newbies are caught off guard by this difference. As we consider a welcome letter to Golden-Eared Audiophiles, describing that perspective should be part of it.

 

[ahofer]

I think something along this line could work...

Maybe we can come up with a list of the most common (straw man) arguments we see, and for each of them ask a qualified/willing member to write up the 'Steel Man' response in a way that is not insulting or patronizing, but that allows those who *actually* want answers to get them, without trying to sort through the search history.

We can create a new "Start Here to begin your debunking" forum, with sticky threads for each of the Steelman (science based) positions.

Maybe 8-10 common ones. That would let us point people there, rather than try to figure out new ways to say the same thing that won't bore the shit out of everyone.

Thoughts on some version of that?

That sounds worthwhile - although a different project. My thinking was could a few of the engineers here make the strongest case possible for some part of the subjectivist belief system, whether it is the value of subjective and time-distant comparisons, or potential for differences we can't measure right now, etc. It is only in understanding the strongest elements of the opposing argument that one can claim to have dealt with it fairly.