nintendoeats
Member
In the comments for the first Ciúnas Audio ISO DAC semi-review, @AndrovichIV and @LTig suggested having a FAQ or introduction to address common anti-measurement sentiments that people bring up. Since I am a professional technical writer and quite a fan of ASR, I thought I would try my hand at it. I came up with the following draft, which can of course be expanded or revised ad nauseum.
Note that I have included this in spoiler tags to make it entirely clear that it does not officially represent Audio Science Review in any way.
I welcome any and all feedback (including "this is a bad idea, why did you bring this up").
After some discussion, there was general agreement that some kind of FAQ that addressed common arguments would be useful. Below is my working version of something like this.
Note that I have included this in spoiler tags to make it entirely clear that it does not officially represent Audio Science Review in any way.
Hello! Welcome to Audio Science Review. This site is intended as a place to discuss audio under the umbrella of scientific rigor through measurement of equipment. Our friendly community is learning-driven; regardless of your current knowledge-level about audio science, we hope that you will find other users that are excited to teach you what they know and explore new ideas about what they don't.
As an introduction for new users, we have created this outline to help you understand the philosophy at the heart of the ASR community. If it does not align perfectly with your views, that is entirely fair; you are free to challenge it. Just be prepared to present evidence. Arguments about judging audio equipment by measurement have been going on for at least a century, and more aggressively in the last few decades. Many users come here because they have grown weary of this debate.
The basic philosophy of ASR is as follows:
Thank you for taking the time to read this document, and welcome to the community!
As an introduction for new users, we have created this outline to help you understand the philosophy at the heart of the ASR community. If it does not align perfectly with your views, that is entirely fair; you are free to challenge it. Just be prepared to present evidence. Arguments about judging audio equipment by measurement have been going on for at least a century, and more aggressively in the last few decades. Many users come here because they have grown weary of this debate.
The basic philosophy of ASR is as follows:
- High quality consumer audio equipment must have high fidelity. Once a recording is published, the transmission medium (radio waves, record grooves, digital files) and devices it passes through (DACs, amplifiers, cables, speakers) should change that recording as little as possible. If a device does not change the sound passing through it in a way that humans can perceive, the device is called audibly transparent.
- The mechanics and limits of human hearing have been understood fully for decades. We therefore have a complete definition of the characteristics a device must have to be audibly transparent.
- The only way to determine if audio equipment is audibly transparent is to take measurements. You must do this using equipment with better characteristics than the equipment you are trying to measure. With modern devices of passable quality, the human auditory system rarely qualifies.
- Because your perception of sound changes across listening sessions, and even moment to moment, it is unreliable in identifying subtle differences in audio equipment.
- If you know what equipment is being used in a listening test, your brain will fill in what you expect to hear. If you don’t know what is being used, your brain will just guess. If somebody tells you that they think an expensive device sounds better than a cheap one, the person probably really does hear an improvement in quality. Put both devices behind a curtain however, and the story might be very different. Because of this, repeated double-blinded tests are the only human experiments that matter (unless the difference reported is truly obvious, such as blatant distortion).
Thank you for taking the time to read this document, and welcome to the community!
I welcome any and all feedback (including "this is a bad idea, why did you bring this up").
After some discussion, there was general agreement that some kind of FAQ that addressed common arguments would be useful. Below is my working version of something like this.
1. I hear a difference between two things, therefore they must really be different somehow.
2. Double-blind testing is unfair, because...
4. My significant other (who is not interested in audio) says that they can hear a difference when I change these “audibly transparent” devices/settings/files/cables. Therefore, they must be different.
5. Expensive cables can improve a signal.
◦ Every person who has an interest in audio has said this about some device, file, or setting at some point. Every audio engineer has adjusted the EQ on a channel until they thought it was perfect, only to realize that the channel was disabled the whole time.
Do an ABX test with proper blinding and a good number of iterations (say 20 for a low-importance test). If you can choose correctly 75% of the time, then there is a very good chance that you really are hearing a difference. If not, remember that you are not alone and the human perceptual system likes to lie to you.
2. Double-blind testing is unfair, because...
◦ Unblinded subjective testing of audio equipment is inherently unscientific. Here are some common endings to the sentence above, and the standard rebuttal.
a) ...the way a device looks and feels is part of the experience.
• If the way a device looks affects the listener’s experience, then an unblinded test is by definition no longer just testing the audio quality of the device. It is also testing how it “makes you feel”. While that is interesting psychologically, it does not justify claims about how the device sounds because audio quality is no longer the only variable.
b) ...knowing what device is being tested helps the listener know what to listen for.
• The human perceptual system makes a lot of guesses based on what a person expects to see/smell/hear. This makes it highly suggestible, and this applies particularly to hearing. If the listener is expecting to hear more detail through a particular device, they usually will; their brain can add detail that never existed. This confounds the ability of an unblinded test to objectively determine the fidelity (or even pleasing infidelity) of the device's audio output.
3. Companies wouldn’t sell expensive equipment that makes bold claims about performance if it didn’t make a difference.◦ As a general rule, companies are profit-motivated, not truth-motivated. They will sell anything they can convince you to buy.
a) But company X is run by somebody I trust!
• One possibility is that they are not worthy of your trust; a good liar must appear to be speaking in earnest.
The second possibility is that they are drinking their own kool-aid. Nobody is more motivated to perceive a quality improvement from a piece of equipment than the manufacturer. If they are not testing their equipment using proper measurements and/or double-blind studies, then they are likely to be very susceptible to the psychological effects outlined elsewhere in this FAQ. Even if they are performing scientific tests, they might not be giving the results of those tests appropriate weight.
4. My significant other (who is not interested in audio) says that they can hear a difference when I change these “audibly transparent” devices/settings/files/cables. Therefore, they must be different.
◦ Your significant other could be picking up on social cues, rather than hearing a real improvement. It is possible that you could get the same response by rustling around behind your A/V setup for half an hour without changing anything and then asking what they thought. If they say it sounds better, that doesn’t mean they are lying; they might really hear a difference, even if the device output hasn't changed.
Your significant other is just as susceptible to session-to-session test-variance as you are. Additionally, your personal excitement about changed might be infectious. If they know that you are interested in audio, and that you have been playing with your system, they are probably also expecting to hear a change.
5. Expensive cables can improve a signal.
◦ Everything in your audio chain (with some specific exceptions) reduces fidelity of the signal. This includes your cables. A good cable will at best degrade the signal as little as possible and keep out external noise.
It is possible for a cable to modify a signal in a way other than adding noise; for example, if the cable has high capacitance it will tend to lose high-frequency information. This is a reduction in fidelity, by definition. If you like that effect, you would be better served by using EQ to get the exact amount of high-end rolloff you want instead of buying an expensive cable that acts as a non-adjustable low-pass filter.
a) Expensive power cables can improve a signal.
• The socket in your wall is connected to an electrical substation by hundreds of meters of cheap copper wire. This cable picks up RF interference, switching noise from other devices, and who knows what else. Amplifier manufacturers know this, so they put capacitors on the AC input to smooth out this noise. Even if 50 cm of additional copper wire could somehow filter this noise, it would be entirely redundant.
The entire purpose of having capacitors in the power circuit of an audio device is to ensure that the power system can’t affect the audio signal, positively or negatively.
b) Expensive digital cables can improve a signal.
• Because digital signals operate at very high frequencies, it is possible to see reduced jitter between cables. However, this can only go so far; the noise that can creep into a system when using even the worst digital cable is almost always inaudible. The reasoning in 5) still applies, because you cannot get a signal that is better than "correct".
6. Analog audio formats are higher quality than digital audio formats.◦ A record cutter receives a voltage level and etches a groove on some vinyl in a way which represents that level.
A tape recorder receives a voltage level and aligns some iron filings on some tape in a way which represents that level.
A digital recorder receives a voltage level and writes a number in some RAM in a way which represents that level.
Note the constancy in those 3 scenarios: each type of recorder stores a representation of a voltage level. There is nothing else for a recorder of any kind to store. The only other things available to be recorded are accidental modifications to that voltage level by the recording process: errors such as distortion, noise, and flutter. Ergo, the only thing a particular recording medium can do is reduce the fidelity of the stored signal. That reduction in fidelity might be pleasing, but it is something that a digital file is fully capable of storing because the errors are still just changes in the recorded voltage level.
If you record playback of a reel-to-reel tape using a decent quality digital audio interface, the resulting file will be audibly identical to the live playback. The reel-to-reel outputs a voltage level, and the audio interface records that voltage level. There is nothing else traveling down the wire to be recorded.
7. Lots of people believe that something makes a difference in audio, therefore it must be true.◦ Lots of people believe lots of things, and lots of those people and things are mistaken.
Think of something lots of people believe that you think is wrong. Now imagine that you were arguing with somebody who believe this thing and they said "I know lots of people who believe this, so it must be true". It is unlikely that you would find this argument compelling.
In the audio industry specifically, there are many people and companies with a vested interest in spreading misinformation and encouraging anti-scientific thinking (knowingly or accidentally). Once this misinformation spreads to enough people, it can become accepted fact regardless of whether it makes sense. Therefore, the fact that lots of people think that something makes a difference to audio output is not a good argument.
Last edited: