The Audio Science Review Manifesto???

[garbulky]

Is there anything specific I have written that you would you say is contradicted by that? There is nothing you say here that I disagree with in particular.

Your points aren't incorrect. There is such a thing as subjective bias and also when things aren't blind it may (but not will) make things prone to a visual and perceptive bias leading to different conclusions.

I think a re-wording would be better. A lot of is and will should be changed to "may" or "might". Overall the piece is written heavily implying that the listener was victim or will be victim to subjective bias and therein get it "wrong" when they declare a difference. Yet we have no idea if they were or would be victim to a subjective bias that then made them "wrong" when they declare a difference. Though this could have been. On the other hand, they also may not have been and got it correct. We are making an assumption here and telling the reader what they experienced was due to subjective bias without actually running any test on the person and gear in question to do so.

It is also testing how it “makes you feel”. While that is interesting psychologically, it does not justify claims about how the device sounds because audio quality is no longer the only variable.

Though it's understandable what you are saying here. The wording sound in "how the device sounds" implies the perception of the human brain which uses multiple senses for its perception including audio quality. The "bias" we are talking about is normal for us. It's not incorrect for somebody to see something look shiny that they like and hear it sounding better. Their perception was working fine and wasn't wrong.

Since we are talking about perception it does justify claims how the device sounds because, something attractive "enhancing the sound" is part of the perception. But if you said something like "does not justify claims about the sound output by the speaker/device" vs "how the device sounds would be more accurate" because now you are not talking about perception which is subjective vs the actual sound output which is objective.

If the listener is expecting to hear more detail through a particular device, they usually will; their brain will add detail that never existed. This confounds the ability of an unblinded test to objectively determine how the device really sounds.

We don't know if they usually will and this is heavily implying that the user will also do this. My problem is the wording "will". Changing it to "may" would be more appropriate. As well as "may confound". And also at the end "how the device really sounds" could be edited to "how the device really sounds without additional perceptive bias".

◦ Your significant other is just as likely to be picking up on social cues as they are to be hearing a real improvement. It is possible that you could get the same response by rustling around behind your A/V setup for half an hour without changing anything and then asking what they thought.In addition to the above, your significant other is just as susceptible to session-to-session test-variance as you are.

Instead of your significant other IS just as likely. Change to "may be just as likely". Because we don't know if they are for sure just as likely.

*minor nitpick Instead of it IS possible. Change to "it may be possible" though technically even "it is possible" would pass muster but it's not guaranteed to be possible for that particular person. But if you don't want to change that, that's fine.*

If they say it sounds better, that doesn’t mean they are lying; usually they want to support you, so they really do hear the improvement that they expect.

We don't know anything about if the wife wants to support me in my audio testing. My wife is usually annoyed when I ask her feedback on audio (you know how it is!). She does not cut corners when telling me something sounds good or bad. Sometimes I think it sounds bad and she thinks it sounds great. So by pointing to spousal support being the reason your wife hears things better is not scientific. It is an assumption and possibly shortchanging the wife's ability to make independent decisions.

You could say "Maybe they are picking up on your excitement or enthusiasm and become biased to hear the improvement that you expect."

 

[nintendoeats]

I would like to see some discussion before softening those points. I can concede off the bat that " is just as likely to be picking up on social cues " is not literally true, so that should be tweaked.

 

[nintendoeats]

@garbulky Sorry for the short response earlier, I was at work. I really appreciate your taking the time to find specific examples that concern you and present reasoning.

I would like to see what the community at large thinks, since at this level of granularity (about the tone of the document) individual sentiment can vary quite a bit. I tend to err on the side of simple, solid statements, put forth with as much conviction as I believe the argument merits. In general, that means taking a statement that "might be" and finding the more specific statement that just "is". The more you have to hedge, the less forceful your argument is (even if it applies to more situations).

Also, I don't really want to get into "if a tree falls in a forest" arguments about what sound is, so I will remove the mention of "how things really sound". What I wrote fits my definition of sound perfectly, but I can see how that is an endless debate it is better to stay out of.

 

[nintendoeats]

You know, I went to implement the stuff I mentioned and I wound up addressing most of your comments (sometimes with overall structural changes). I think the writing is better for it.

 

[Deleted member 65]

I’m not sure that ASR needs or is by any means strengthened by a Manifesto. ASR will become what it’s members, contributors and moderators jointly decides what it will be. This guided by a high level, straight forward About Document. Keep things simple, we’ll make ASR to what we want it to be by our actions.

 

[nintendoeats]

I’m not sure that ASR needs or is by any means strengthened by a Manifesto. ASR will become what it’s members, contributors and moderators jointly decides what it will be. This guided by a high level, straight forward About Document. Keep things simple, we’ll make ASR to what we want it to be by our actions.

I understand that position. What about the FAQ concept?

 

[LuckyLuke575]

While the intention is in the right place, I fear that putting something like this in black & white could become a straw man for subjectivists, naysayers, and trolls to pull down.

 

[LTig]

I would like to see some discussion before softening those points. I can concede off the bat that " is just as likely to be picking up on social cues " is not literally true, so that should be tweaked.

Replace "is" by "chances are high that...". "Might be" is not strong enough.

 

[garbulky]

Replace "is" by "chances are high that...". "Might be" is not strong enough.

So what are those chances, then and based on what tests? Who and what gear is being tested here? Let's see and link data. The better the data, the more accurately we can deliver authoritative meaning.

 

[oivavoi]

I understand that position. What about the FAQ concept?

I really like the idea of a FAQ! That seems more sciency to me, somehow, than a manifesto, which seems more like an ideological thing.

Also, a FAQ seems more open, addressing Garbulky`s points above. That may make people feel more at home here even if they don`t agree with everything in the manifesto.

I, for one, am not very concerned with anything that happens more than 80 db down from the signal. If I get a dedicated music room with much lower noise floor in the future - then, perhaps, it might make sense to me to chase that kind of perfection in gear. But not now, with my current room. So that's not why I'm here. I simply don't adhere to the "I need to have absolutely transparent gear" ethos, I'm more in the "transparent enough for my own ears" camp. I'm interested in the technical side of things, and thus enjoy reading Amir's measurement-driven reviews, but mostly I just enjoy learning about audio in general from all the people here who know much more than I do.

I would therefore like a FAQ which somehow accommodates "my" kind of ASR membership also - that's it's not only about measurements here, but also about general audio discussions? But then a FAQ can of course also respond to questions such as "Why do reviews here care about noise levels beyond which what can reasonably be heard"? "My wife told me she likes my new diamond-coated usb cable before she knew it was diamond-coated, do you really think she's wrong?" Etc.

 

[nintendoeats]

While the intention is in the right place, I fear that putting something like this in black & white could become a straw man for subjectivists, naysayers, and trolls to pull down.

I can understand that, and its the kind of thing that is always scary when putting ideas in writing (Socrates was opposed to writing for related reasons, so everything we know about him comes from contemporaneous writers). On the other hand, it seems potentially powerful to come up with something really solid and difficult to attack that we can point at for common cases.

I know that I'm not perfectly careful every time I reiterate an idea that I am very familiar with. At work, often when we are debating how something functions I will personally need to go back and look at something I wrote to determine the exact behavior of the thing in question. There is a risk and a reward in committing to a single, well-formed description of something.

If you come up with a good argument and don't express it publicly, then nobody can attack it but also nobody can learn from it. If somebody does attack it, and does so successfully, then the argument gets better through iteration. If the argument is attacked unsuccessfully by a counter-argument that is without merit, the attacker looks a fool in the eyes of the learned.

That got a bit hoity-toity at the end, so here is a picture of cat wearing headphones:

 

[oivavoi]

Thinking a bit more, I think I would like something like this in the Faq, which starts with for example:

1. "What is ASR?" Yadi yada yada, forum of people who like to discuss audio etc
2. "Who runs ASR"? Yadi yada yada, present Amir (and Thomas), but many members here also contribute a whole lot, etc
3. "Why measurements/reviews"? etc etc, maybe have Amir fill in something here about the rationale for that
....
and then perhaps go on to questions about blind testing, subjectivity, and all that.
And perhaps have one point about recommendations about audio system? "Most members here would agree that speakers are the most important by far, that's why we hope to get into speaker measurements in the future".
And one question about rules for membership - when can people get banned? Etc

The point is to make into a more general faq about the site, what goes on here, so that people can have these questions in one place.

 

[nintendoeats]

The point is to make into a more general faq about the site, what goes on here, so that people can have these questions in one place.

I can see this working really nicely. It might also soften the blow of "here is all this stuff you might believe, and I'm going to tell you why its wrong", since it will be couched in "hey, cool stuff! Science yo!".

 

[LuckyLuke575]

Thinking a bit more, I think I would like something like this in the Faq, which starts with for example:

1. "What is ASR?" Yadi yada yada, forum of people who like to discuss audio etc
2. "Who runs ASR"? Yadi yada yada, present Amir (and Thomas), but many members here also contribute a whole lot, etc
3. "Why measurements/reviews"? etc etc, maybe have Amir fill in something here about the rationale for that
....
and then perhaps go on to questions about blind testing, subjectivity, and all that.
And perhaps have one point about recommendations about audio system? "Most members here would agree that speakers are the most important by far, that's why we hope to get into speaker measurements in the future".
And one question about rules for membership - when can people get banned? Etc

The point is to make into a more general faq about the site, what goes on here, so that people can have these questions in one place.

Yes, this kind of FAQ format would work well, so that it's more informative than trying to be a gospel truth or manifesto

 

[nintendoeats]

I just wanted to update, this is not dead. It was a busy weekend, and now a very hectic week at work (release coming up on Monday). I'm a bit zonked.

 

[pozz]

Nice writing. I would modify the following slightly:

As an introduction for new users, we have created this outline to help you understand the philosophy at the heart of the ASR community. If it does not align perfectly with your views, that is entirely fair; you are free to challenge it. Just know that [...] arguments about judging audio equipment by measurement have been going on for at least a century, and more aggressively in the last few decades. Many users come here because they have grown weary of this debate.

Just be prepared to present evidence. Arguments...

And perhaps also add a line somewhere with the following (feel free to modify):

Many debates tend to be debates of principle. While everything done at ASR does point to a set of loosely-organized principles and a philosophy, it's more worthwhile to address the practicalities, details and context. Put it down as principle that listening will reveal differences. Sure: but what kind of differences and under what kind of conditions? Put it down as principle that a power cable will change the sound signature of a device. Sure: but which aspects? Put it down as principle that a person can hear more than can be measured. Sure: but what can be measured and what can't? What's on each list? The issue is really one of precise description. Given enough precision, your ability to compare your experience to that of others increases.

This needs to be modified:

If the listener is expecting to hear more detail through a particular device, they usually will; their brain can add detail that never existed [also...]

The hearing system has an active filter/stream selection/averaging mechanism which allows you to voluntarily select from several concurrent sound sources and block others (known as the "cocktail party" effect). In short: you will legitimately hear differences in unchanged content, particularly if you are paying close attention. It's not just imagination at work.

And the Wireworld review you quoted showed differences in noise, not jitter (the test signal was a 1kHz tone, not a J-test). The loopback differences disappeared when a well-designed DAC was inserted into the chain.

One thing I'd like to emphasize is that when new members join their concerns and so on have to be treated as legitimate. So the FAQ and overall response should be direct but polite. There's value in cultivating both self-awareness (in other words knowledge of your own ability to hear, to discriminate, to judge) and technical knowledge of how these devices operate.

 

[pozz]

In short: you will legitimately hear differences in unchanged content, particularly if you are paying close attention. It's not just imagination at work.

A good example: musicians, when tuning instruments, or engineers, when mixing, can listen to specific parts of a sound's spectrum. This ability is called "isolating partials".

 

[nintendoeats]

@pozz Really nice comments, I will definitely incorporate them. I share your general philosophy of the thing, as I think do many people who have already posted.

You are correct about the Wireworld article, I conflated it with another article I was reading at the same time about jitter with different cables. I am now desperately curious about why the cable made a difference though.

 

[pozz]

@pozz Really nice comments, I will definitely incorporate them. I share your general philosophy of the thing, as I think do many people who have already posted.

You are correct about the Wireworld article, I conflated it with another article I was reading at the same time about jitter with different cables. I am now desperately curious about why the cable made a difference though.

It's the separation of signal and power leads. The Wireworld cables are incredibly inflexible because of it though.

 

[tomchr]

I like it! "Et cetera" is two words. You could also just use "etc." like everybody else.

It may be worthwhile to mention that there are many well-established psychological effects that would allow someone to perceive a difference between two identical stimuli (as in your example with the recording engineer tweaking the settings on a disabled channel). It may be worthwhile to note that such perceptions are not indicating that a person is stupid or otherwise inferior. Just that they're human.

I encourage those further interested in reading about human decision-making to read Dan Ariely, "Predictably Irrational". He approaches the subject from a behavioural economics perspective. It's a fascinating read (backed by science!)

Tom