watchnerd
Grand Contributor
You seem to have missed the forest for the trees, the meta point of the satire.
-
WANTED: Happy members who like to discuss audio and other topics related to our interest. Desire to learn and share knowledge of science required. There are many reviews of audio hardware and expert members to help answer your questions. Click here to have your audio equipment measured for free!
NORMS AND STANDARDS FOR DISCOURSE ON ASR
- Thread starter svart-hvitt
- Start date
You seem to have missed the forest for the trees, the meta point of the satire.
dc655321
Major Contributor
- Joined
- Mar 4, 2018
- Messages
- 1,597
- Likes
- 2,236
@svart-hvitt, for somebody that has multiply and mistakenly complained of others failing to acknowledge prior work, the "math problem" you presented does suffer that same ill. This does present somewhat interesting food for thought, so thanks for that. Just be honest where you got it from...
- Joined
- Aug 14, 2018
- Messages
- 2,773
- Likes
- 8,155
1) My impression is that "gold standard" gets used around here as a loose synonym for "state of the art" aka "highest-performing among units tested so far." For example, the SINAD of the Oppo BDP-205's DAC was the "gold standard" until Amir measured one, and then two, other DACs with slightly higher SINAD. But sure, if "gold standard for SINAD" too easily gets interpreted as "gold standard in all respects," then I have no problem with you proposal that we stop using "gold standard" since it could lead some to believe that a device that measures the best on one metric is the best on all metrics. However, it does seem ultimately to be a semantic issue since I very much doubt that anyone here automatically equates a single measurement with overall quality in such a one-to-one comparative way; and since comparative measurements by definition produce units that perform better and worse, with one being the best (or tied for best).
2) Another point that has some merit but in practice is IMHO a semantic one given the limitations of a web forum and hobbyist site. One could perhaps call this site "Audio Measurement Review" instead of "Audio Science Review." But I would say that Audio Science Review actually is fitting because in addition to focusing on electrical measurements, this site is also about a rigorous, scientific approach to other aspects of audio science. So while this site might not focus as much on the science of psychoacoustics or the mechanics of human hearing, this site is very much science-based in that it opposes and is critical of points of view that use those "soft" sciences to make unwarranted claims. So this site promotes a scientific approach to audio, and so IMHO there is no need for this to be "acknowledged" more than it is already.
3) This is irrelevant and has no practical implications or meaning for this site. All scientific inquiry "borrows" approaches, methodologies, and tools from other branches. Biology experiments are impossible as scientifically valid endeavors without statistics, for example.
4) Yes, so-called "soft" sciences related to audio could benefit from innovation. However, your claim that 1970s speakers sound "perfectly fine" compared to modern speakers is a flabby comparison that conflates measurable performance with user preference - equates "preference" with "truth," which is something you say we should NOT be doing (see item 6). There's been a well-documented set of refinements and changes in speaker design and performance since the 1970s, especially as one goes farther down the price-point scale. One does not have to insist that everyone should prefer the sound of modern speakers in order to acknowledge that computer-aided design, improved materials science, and other innovations have resulted in, for example, $150-$200/pair low-end bookshelf speakers available today that objectively (and to many folks subjectively) kick the crap out of similarly priced 1970s and '80s offerings that were well-regarded in their time.
5) Irrelevant to this forum, since this forum would clearly support that already, and so this is already part of the culture here.
6) Again, this is foundational to the culture of this forum already - no one here thinks preference equals truth. What you are missing in your repeated references to psychology and "soft" sciences is that there is a difference between individual listening preferences on the one hand and fidelity on the other. A 1970s Marantz receiver paired with 1970s Pioneer speakers might sound super-pleasing to many people, but that setup is demonstrably lower in fidelity than many modern combinations. If there is a psychological or medical reason that a majority of humans making such a comparison might prefer the lower-fidelity setup, that is indeed worth exploring - but no one has actually demonstrated that a majority of humans do prefer the lower-fi setup. And whether or not one's equipment should color all music that gets played through it is not a matter for any science, "hard" or "soft" - it's a philosophical and personal-preference question, or perhaps a social-norm question: "is fidelity the goal of audio reproduction?" is not a question that can be answered scientifically under any conventionally understood meaning of science.
7) Your "one man does a test is still a sample size of one" is nonsense. It is indeed crucial to have repeatability of experiments - so a single experiment should indeed be questioned unless or until a second experiment achieves the same results. But that's not sample size - a single experiment on 5,000 people can easily be more statistically significant than three experiments on 10 people each, because the sample size is far larger. The benefit of having two additional experimenters in that scenario has to be weighed against the benefit of having a total of 4.700 more subjects in the sample in the single large experiment,
Last edited:
A purists view then. The Circle of confusion is what it is, and we sometimes use tone controls or EQs for adjust what we think sounds natural to us. Also it means stereo recordings should be reproduced in stereo, and any use of matrix decoders to create stereo to multichannel will not be a match to what the engineers heard. The third point; despite the inherent flaws of stereo we still go for a target response in our reproduction that may well be wrong given that the recording engineers do not apply consistent corrections for those errors. Unlike the Circle of confusion where errors can be all over the place, systematic errors from stereo reproduction is a different topic. We cannot fully repair the flaws of stereo, but we can do some adjustment. Thus we cannot assume that a speaker should be perfectly linear in frequency response. It depends IMO.
OP
svart-hvitt
Major Contributor
- Joined
- Aug 31, 2017
- Messages
- 2,375
- Likes
- 1,253
- Thread Starter
- #185
I have been very open on my usage of vox populi, i.e. the origin of that concept with regard to what we call “wisdom of crowds” today. See these two links for reference on my being open on where I get inspiration from:
https://www.audiosciencereview.com/...-and-smooth-off-axis.8090/page-19#post-202858
https://www.audiosciencereview.com/...-for-discourse-on-asr.8212/page-6#post-206584
My example of the median bias is a more formal “evidence”, that can be backed up with real life data across two decades and large data sets.
dc655321
Major Contributor
- Joined
- Mar 4, 2018
- Messages
- 1,597
- Likes
- 2,236
That's fair. I honestly missed your earliest reference to Galton. My apologies.
But your example "problem" is/was clearly adapted from that paper without acknowledgement. A bit out of character
watchnerd
Grand Contributor
The missing factor in all of this is commercial interest to finance R&D.
Much of the boutique high-end cottage industry seems to be making a living selling nice audio jewelry to the well-heeled middle-aged crowd. Anecdotally, this wing of the business seems to do just fine iterating on the past, albeit at a small scale.
Even the "innovative" end of the high end spectrum, with companies like Kii Three, are mostly doing applied engineering, as opposed to primary research, and it's pretty small scale. Nothing really new going on there, just building better mouse traps.
I can only think of a few other audio segments that actually care about and spend on primary R&D:
1. Car audio, much of it OEM
2. Smart speakers.....say what you will about whether the Apple Home Pod has audiophile street cred, there is a fair bit of engineering cleverness attempted.
3. Virtual reality
We can wish all we want for "better science", but, let's face it, "home audio" is a mature industry at this point with very limited upside growth potential, which limits capital investment.
Much of the boutique high-end cottage industry seems to be making a living selling nice audio jewelry to the well-heeled middle-aged crowd. Anecdotally, this wing of the business seems to do just fine iterating on the past, albeit at a small scale.
Even the "innovative" end of the high end spectrum, with companies like Kii Three, are mostly doing applied engineering, as opposed to primary research, and it's pretty small scale. Nothing really new going on there, just building better mouse traps.
I can only think of a few other audio segments that actually care about and spend on primary R&D:
1. Car audio, much of it OEM
2. Smart speakers.....say what you will about whether the Apple Home Pod has audiophile street cred, there is a fair bit of engineering cleverness attempted.
3. Virtual reality
We can wish all we want for "better science", but, let's face it, "home audio" is a mature industry at this point with very limited upside growth potential, which limits capital investment.
Last edited:
Absolutely. And add in "hearing aids" as #4.
OP
svart-hvitt
Major Contributor
- Joined
- Aug 31, 2017
- Messages
- 2,375
- Likes
- 1,253
- Thread Starter
- #189
That's fair. I honestly missed your earliest reference to Galton. My apologies.
But your example "problem" is/was clearly adapted from that paper without acknowledgement. A bit out of character
What Galton showed, and on which “the wisdom of crowds” concept is based, is that the median is powerful in digging out the truth.
What wasn’t so clearly understood (?) by Galton and proponents of “the wisdom of crowds” concept, is the fact that the median will continue to produce the best estimates in a world of confusion and nonsense too.
This nuance is important, I believe, and it constitutes a problem for the “wisdom of crowds” proponents in inquiries where we don’t have a reference that is truth, i.e. situations where truth will reveal itself.
I discussed the math with a well-known professor (I think he’s been mentioned in this thread or another recent thread), and it was obvious he hadn’t thought through the problem before.
OP
svart-hvitt
Major Contributor
- Joined
- Aug 31, 2017
- Messages
- 2,375
- Likes
- 1,253
- Thread Starter
- #190
Thanks for good answer!
However, I cannot leave my position which is the observation that Hard is High Status, something which seems to be a prevalent attitude on ASR as well. Elsewhere on ASR, we’ve had people deriding social sciences due to social sciences’ lack of Hard.
Akerlof (2019) briefly discussed the same Hard=High Status problem:
“4. Reasons for Bias Towards Hard
The question remains: why do economists have Hardness bias? I suggest three possible
reasons, which also, at least partially, explain why this bias has become stronger over time. Reason 1: Place in the Scientific Hierarchy. In their article “The Superiority of
Economists,” Fourcade, Ollion and Algan (2015) argue that economists "see themselves at or near the top of the pecking order among the social scientists."5 Economists take great pride in their view of their discipline as "the most scientific of the social sciences," and they look down upon sociologists and political scientists for their "less powerful analytical tools."6 This desire for place in the pecking order, I would argue, is a leading motive for Hardness bias”.
From Fourcade et al. (2015) quote by Akerlof (2019):
“The intellectual trajectories of the social-science disciplines diverged substantially over the course of the twentieth century. Economics has left behind the historical emphasis of its continental youth in an effort to emulate paradigmatic natural sciences, such as physics (Mirowski 1989)”.
In other words, I think audio science has a Hard bias too, as well as a bias towards blind tests as its preferred method of inquiry (blind tests are good but are they the ultimate revealer of truth?).
Your last point, where you accuse me of nonsense, is utter nonsense. Or maybe we talked past each other?
Say I have a million data points that indicate with a high degree of statistical significance that x explains y, or that x and y are related. What I didn’t tell you, is that all the datapoints are taken from two days worth of a high frequency data set. So the data set of one million data points is heavily biased toward a short period - maybe a weekend or a national holiday - that is not representative of the general trend.
My one man show could also be based on a statistical model that is not suited for the task.
My one man could also be exposed to biases, say incentives.
Etc.
Independent verification is key. The developers of the Delphi method saw this, and the idea behind independent observations is also the reason we have meta studies.
OP
svart-hvitt
Major Contributor
- Joined
- Aug 31, 2017
- Messages
- 2,375
- Likes
- 1,253
- Thread Starter
- #191
WHY ARE AUDIO PEOPLE SO CONFUSED?
- - - - - - - WARNING LONG POST! SEE LAST SENTENCE OF THIS COMMENT FOR PROVOKING QUESTION - - - - - - -
Talking to audio interested people is like diving into a see of confusion. People on ASR see this easily when they look down at the camp of “subjectivists”.
“Subjectivists are like a cult”, is a common characteristic by people on ASR. Let me add flavor to that observation, one social group accusing the other for bering religious, put to paper by late professor Robert H. Nelson (1944-2018):
“Economics and environmentalism are belief systems that shape their adherents’ way of thinking about the world. We might just as accurately characterize them as secular religions, which most theologians count as real religions (see, for example, Tillich 1963), but many people prefer to regard them as competing belief systems. Many (not all) economists and environmentalists thus function in the world as advocates for their belief systems and associated values, albeit often more implicitly than explicitly (Nelson 1991, 2001, 2010).
This view admittedly is not the usual understanding of the social sciences and ecological science, which have long professed to seek “value neutrality.” Depending on the audience, however, people often agree to a surprising extent that economics and environmentalism are actually religions. When the subject comes up informally in conversations with economists (and with policy analysts, if perhaps less predictably), I find little disagreement with the idea that environmentalism is a religion—to most economists, the claim seems fairly obvious. Environmentalists often react similarly, but the other way around: economics, for most environmentalists, is a religion. Neither group, however, is comfortable with the characterization of their own thinking as religious (and the economists are more uncomfortable with it than the environmentalists)”.
Source: https://pdfs.semanticscholar.org/64ef/666d456f2416e274b6d5cc27bbb1256e6a44.pdf
In other words, it’s quite common to meet educated people of diverging opinions accusing one another of being religious and deluded.
Prominent audio scientist, @Floyd Toole , invented the concept of “circle of confusion” in audio (in other words, gentleman Toole doesn’t accuse anyone of being religious or deluded, just “confused”). His point on the circle of confusion is a fascinating one and I recommend everyone reading Dr. Toole’s idea (https://secure.aes.org/forum/pubs/journal/?ID=524).
I will, however, argue that what some have called the “gold standard” for inquiry into audio - and a method preferred by Dr. Toole - can lead to its own circle of confusion.
Blind tests of preferences in audio will only reach an equilibrium if preferences are stable or equal Truth. Truth is per definition stable and could therefore yield an equilibrium in audio. However, as marketeers know, preferences are hardly stable and a marketeer would never propose that preferences are truth seeking.
The point is, as preferences are drifting, blind tests of preferences will yield drifting equilibria.
Ironically, a speaker which is Truth, may not be the preferred one in a world of drifting preferences. So even if audio researchers came up with a Truth Speaker, we would still be guaranteed “innovations” in a world where blind tests of drifting preferences are one’s guide.
This provoking idea may be illustrated by looking at the product portfolio of a science oriented loudspeaker producer, Genelec, who produce active speakers only since 1978. Take a look at their Master series, say 1234. The first iteration of the 1234 goes back to a least 1989. Since then, there have been two new versions of the “same” monitor, in 1998 and 2015. The most obvious changes to the 1234 have been amplifier size and DSP. The speaker per se looks the same as 30 years ago! However, today’s 1234 amplifier weighs only 11 kg, while the 1989 edition was 71 kg. Today’s amplifier specifications are better than in 1989. In addition, the speaker has DSP to be adjusted to each room’s individual, unique characteristic instead of continuing making a one-size-fits all speaker that practices “The Bed of Procrustes”, i.e. one bed (speaker) to fit all guests (rooms) instead of customization per software (https://en.m.wikipedia.org/wiki/The_Bed_of_Procrustes).
The naive observer could accuse Genelec of stagnation, lack of innovation because the speaker looks the same as decades ago. On the other hand, the same observer would celebrate all the new speakers of the speaker industry, where the wheel seems to be reinvented by every cycle. The naive observer would be right in his assessments if innovation were truth seeking, right? But what if the “innovations” where preference seeking against a drifting goal instead, missing Truth by every cycle.
What if Truth - or a close approximation to Truth - in audio were revealed many years ago? What kind of innovation would you expect in this case? Would downsizing of electronics and DSP qualify as real innovations? What is the general trend in high end audio; smaller electronics plus DSP, or same-shit-new-wrapping speakers where form and colour are adjusted to newer preferences?
Another example: ATC has been accused by ASR regular @Purité Audio of being stuck in the past. However, couldn’t it be the other way around? Could it be that ATC discovered a good approximation to Truth decades ago, a discovery which explains ATC’s conservative design and a “desperate dealer’s” frustration with a Truth seeking business model?
Isn’t it obvious that dealers seek Preference Speakers, as opposed to Truth speakers?
My point is: Could we argue that a model of inquiry that focuses on preferences is ideally suited for an industry which is marketing oriented as opposed to a field which is Truth oriented?
- - - - - - - WARNING LONG POST! SEE LAST SENTENCE OF THIS COMMENT FOR PROVOKING QUESTION - - - - - - -
Talking to audio interested people is like diving into a see of confusion. People on ASR see this easily when they look down at the camp of “subjectivists”.
“Subjectivists are like a cult”, is a common characteristic by people on ASR. Let me add flavor to that observation, one social group accusing the other for bering religious, put to paper by late professor Robert H. Nelson (1944-2018):
“Economics and environmentalism are belief systems that shape their adherents’ way of thinking about the world. We might just as accurately characterize them as secular religions, which most theologians count as real religions (see, for example, Tillich 1963), but many people prefer to regard them as competing belief systems. Many (not all) economists and environmentalists thus function in the world as advocates for their belief systems and associated values, albeit often more implicitly than explicitly (Nelson 1991, 2001, 2010).
This view admittedly is not the usual understanding of the social sciences and ecological science, which have long professed to seek “value neutrality.” Depending on the audience, however, people often agree to a surprising extent that economics and environmentalism are actually religions. When the subject comes up informally in conversations with economists (and with policy analysts, if perhaps less predictably), I find little disagreement with the idea that environmentalism is a religion—to most economists, the claim seems fairly obvious. Environmentalists often react similarly, but the other way around: economics, for most environmentalists, is a religion. Neither group, however, is comfortable with the characterization of their own thinking as religious (and the economists are more uncomfortable with it than the environmentalists)”.
Source: https://pdfs.semanticscholar.org/64ef/666d456f2416e274b6d5cc27bbb1256e6a44.pdf
In other words, it’s quite common to meet educated people of diverging opinions accusing one another of being religious and deluded.
Prominent audio scientist, @Floyd Toole , invented the concept of “circle of confusion” in audio (in other words, gentleman Toole doesn’t accuse anyone of being religious or deluded, just “confused”). His point on the circle of confusion is a fascinating one and I recommend everyone reading Dr. Toole’s idea (https://secure.aes.org/forum/pubs/journal/?ID=524).
I will, however, argue that what some have called the “gold standard” for inquiry into audio - and a method preferred by Dr. Toole - can lead to its own circle of confusion.
Blind tests of preferences in audio will only reach an equilibrium if preferences are stable or equal Truth. Truth is per definition stable and could therefore yield an equilibrium in audio. However, as marketeers know, preferences are hardly stable and a marketeer would never propose that preferences are truth seeking.
The point is, as preferences are drifting, blind tests of preferences will yield drifting equilibria.
Ironically, a speaker which is Truth, may not be the preferred one in a world of drifting preferences. So even if audio researchers came up with a Truth Speaker, we would still be guaranteed “innovations” in a world where blind tests of drifting preferences are one’s guide.
This provoking idea may be illustrated by looking at the product portfolio of a science oriented loudspeaker producer, Genelec, who produce active speakers only since 1978. Take a look at their Master series, say 1234. The first iteration of the 1234 goes back to a least 1989. Since then, there have been two new versions of the “same” monitor, in 1998 and 2015. The most obvious changes to the 1234 have been amplifier size and DSP. The speaker per se looks the same as 30 years ago! However, today’s 1234 amplifier weighs only 11 kg, while the 1989 edition was 71 kg. Today’s amplifier specifications are better than in 1989. In addition, the speaker has DSP to be adjusted to each room’s individual, unique characteristic instead of continuing making a one-size-fits all speaker that practices “The Bed of Procrustes”, i.e. one bed (speaker) to fit all guests (rooms) instead of customization per software (https://en.m.wikipedia.org/wiki/The_Bed_of_Procrustes).
The naive observer could accuse Genelec of stagnation, lack of innovation because the speaker looks the same as decades ago. On the other hand, the same observer would celebrate all the new speakers of the speaker industry, where the wheel seems to be reinvented by every cycle. The naive observer would be right in his assessments if innovation were truth seeking, right? But what if the “innovations” where preference seeking against a drifting goal instead, missing Truth by every cycle.
What if Truth - or a close approximation to Truth - in audio were revealed many years ago? What kind of innovation would you expect in this case? Would downsizing of electronics and DSP qualify as real innovations? What is the general trend in high end audio; smaller electronics plus DSP, or same-shit-new-wrapping speakers where form and colour are adjusted to newer preferences?
Another example: ATC has been accused by ASR regular @Purité Audio of being stuck in the past. However, couldn’t it be the other way around? Could it be that ATC discovered a good approximation to Truth decades ago, a discovery which explains ATC’s conservative design and a “desperate dealer’s” frustration with a Truth seeking business model?
Isn’t it obvious that dealers seek Preference Speakers, as opposed to Truth speakers?
My point is: Could we argue that a model of inquiry that focuses on preferences is ideally suited for an industry which is marketing oriented as opposed to a field which is Truth oriented?
Last edited:
In other news, physicists calibrate their equipment and use controls in their experiments. Crazy, no?
I disagree. Some of its measurements may be demonstrably worse than the modern combination, but in terms of overall fidelity, simply having a wider baffle gives it superior characteristics in some ways. Also, it may be sealed which is demonstrably better than the modern version. Ditto it may be three-way as opposed to the modern two-way.
The point is that 'demonstrably lower in fidelity' is too ambitious a claim when it comes to the field of speakers and acoustics. If all else was equal, and every other measurement and specification was better, I think it would be true. But as long as the 70s speaker has some different characteristics that can be argued to be superior (they can), then nothing has been demonstrated unequivocally.
2nd verse, same as the first...
except...384th verse, same as the first...
I hear this as one side saying 2+2 is 4. And you say...well, doesn't that depend on whether the Vox Pop really want it to be something else, or would prefer it to be...lets say...4.2.
The other thing is...I still think I'm missing the 'point.' Is your goal to get Amir to change fundamentally how he does things?
To try to get the forum to see it the 'right' way, where what ML hears in his barn has the same validity as what is measured and presented here? To just kinda sit around and have a chat about your latest musings?
Maybe I haven't smoked enough...
watchnerd
Grand Contributor
I'm trying to understand what, exactly, you're proposing.
Are you suggesting that ASR engage in a regime of DBT for every product measured?
If so, how, in a way that is affordable, scalable, and repeatable?
If that's not what you're suggesting, what is the test harness you're proposing?
Lastly, as a marketer myself (not in audio), the job of marketing is best left up to the vendors; it's their itch to scratch, not ASR's.
watchnerd
Grand Contributor
You're applying the wrong lens.
These are business decisions involving product development and marketing. A business realizes that continuity of design elements is important to appeal to existing fans / target markets, with necessary evolution of the look and features to encourage the existing customer base to upgrade to new, "better" versions, periodically.
You also see this in cars, motorcycles, watches, laptops, smart phones, etc, etc.
The decisions aren't made to seek "Truth" (whatever that is). These decisions are made to make money and grow revenue.
Applying epistemological or even scientific models is the wrong framing for the decision trees involved.
- Joined
- Feb 23, 2016
- Messages
- 20,771
- Likes
- 37,635
Galton in motion.
OP
svart-hvitt
Major Contributor
- Joined
- Aug 31, 2017
- Messages
- 2,375
- Likes
- 1,253
- Thread Starter
- #198
What if I presented to you a scientific framework that suited your business plan too? Sounds like having your cake and eating it too.
Please note, vox populi has its pros and cons. I am now highlighting the cons.
watchnerd
Grand Contributor
Who are "I" and "you", in this context?
- Joined
- Aug 14, 2018
- Messages
- 2,773
- Likes
- 8,155
I have a suggestion: Instead of trying to convince folks here that they have a bias towards "hard" science regardless of whether or not "hard" science is the most relevant or the most able to describe what we actually perceive as good sound, why not start a thread in which you link to studies or helpful summary articles covering the "soft"/psychological/medical/etc research that you think would be helpful to add to the normal discussion here at ASR?
In other words, instead of continuing to try to convince people that the culture here is a certain thing, why not move on to changing or supplementing that culture with the kinds of information and contributions you feel are underrepresented? Surely a thread on psychoacoustics is going to find at least some takers here, who will comment and participate in it.
Last edited:
Similar threads
- Poll
- Poll
