• WANTED: Happy members who like to discuss audio and other topics related to our interest. Desire to learn and share knowledge of science required as is 20 years of participation in forums (not all true). Come here to have fun, be ready to be teased and not take online life too seriously. We now measure and review equipment for free! Click here for details.

AES paper: The Role of Critical Listening in Evaluating Audio Equipment Quality

audiophile

Active Member
Joined
Oct 7, 2019
Messages
132
Likes
76
#1
The Role of Critical Listening in Evaluating Audio Equipment Quality
Presented at 91st AES Convention 1991 October 4-8 New York
by Robert Harley, Stereophile Magazine (now the editor at The Absolute Sound)

Subjective critical listening can reveal aspects of audio equipment quality not exposed by traditional objective methods. Subjective listening impressions, however, are often unfairly dismissed as mysticism, even when conducted by conscientious, technically oriented practitioners.

This paper outlines the methods and underlying philosophy of professional critical listening, explores the reasons why subjective listening is rejected by the scientific audio community, and draws the distinction between serious listening and pseudoscientific claims.


Let's read and discuss
 

Attachments

Fluffy

Addicted to Fun and Learning
Joined
Sep 14, 2019
Messages
510
Likes
736
#2
At a glance, I don't see any graphs or data in this paper. Is there any test done to validate the ability of " technically oriented practitioners" to discern subtle differences that measuring devices can detect, that lie beyond the hearing capability of non-technically oriented listeners?
 

audiophile

Active Member
Joined
Oct 7, 2019
Messages
132
Likes
76
#3
At a glance, I don't see any graphs or data in this paper. Is there any test done to validate the ability of " technically oriented practitioners" to discern subtle differences that measuring devices can detect, that lie beyond the hearing capability of non-technically oriented listeners?
Don't know yet, I have only read the first two pages which seemed quite intriguing. Will need some more free time to read it all.
 

FrantzM

Major Contributor
Forum Donor
Joined
Mar 12, 2016
Messages
1,351
Likes
1,470
#4
From the thing (Was that an article? An essay.. a ... ? :rolleyes:)
Finally, "bloom" is even more unintelligible and abstract to most people because it describes a
phenomenon not readily encountered, recognized, or discerned in all audio system. The ability of a
playback system to reveal "bloom" is a subtle refinement rarely differentiated by the vast majority of
the music-listening public. The word is absolutely meaningless to those who haven't experienced
"bloom" in a music playback system, yet is highly descriptive and full of meaning to those who have
experienced the phenomenon.
I suppose the AES has been excessively kind to allow a person to present this "essay". The copious amount of the verbiage one would expect in any High End magazine and nothing scientific to even try or present an attempt to back it up ...

If you have time to waste .. then .. else ...
 

audiophile

Active Member
Joined
Oct 7, 2019
Messages
132
Likes
76
#5
I suppose the AES has been excessively kind to allow a person to present this "essay".
The theme of their convention was "Audio Fact and Fantasy: Reckoning With the Realities". I guess, from the organizers' perspective Harley was representing the "fantasy" side of the "great debate"
 

Juhazi

Addicted to Fun and Learning
Joined
Sep 15, 2018
Messages
861
Likes
938
Location
Finland
#6
Well, this was a presentation at AES Congress in 1991. It is not a scientific article. Obviously it was orded by organizers in order to open debate and amuse the scientifically oriented participants. Those happen in every Congress!

The paper itself is self-explanatory and actually quite revealing of the thinking of subjectivists. Same arguments live happily now in 2020!
 

Juhazi

Addicted to Fun and Learning
Joined
Sep 15, 2018
Messages
861
Likes
938
Location
Finland
#7
In another thread there is discussion of how to listen, untrained, trained and slow. This slow is new term presented by Thomas Lund who works for Genelec. In my opinion that was just an opening for more research and discussion, the benefit and structure of slow listening is still open and unverified.

Reference list of Lund tells a lot how vaque slow listening is (just opinions and questions)

REFERENCES [1] T. Lund and A. Makivirta, “On Human Perceptual ¨ Bandwidth and Slow Listening,” Proceedings of Tonmeistertagung, AES reviewed paper, Cologne (2018). ISBN 978-3-9812830-9-9. [2] D. Oakley and P. Halligan, “Chasing the Rainbow: The Non-Conscious Nature of Being,” Frontiers in Psychology, vol. 14, no. 8 (2017 Nov.). https://doi.org/10.3389/ fpsyg.2017.01924 [3] S. Hameroff and R. Penrose, “Consciousness in the Universe. A Review of the ‘Orch OR’ Theory,” Physics of Life Reviews, vol. 11, no. 1 (2014). https://doi.org/ 10.1016/j.plrev.2013.08.002 [4] P. Jedlicka “Revisiting the Quantum Brain Hypothesis: Toward Quantum (Neuro)biology?” Frontiers in Molecular Neuroscience vol. 10, no. 366 (2017). https://doi.org/10.3389/fnmol.2017.00366 [5] H. v. Helmholtz, “Treatise of Physiological Optics: Concerning the Perceptions in General,” in T. Shipley,Classics in Psychology (1925, original book published 1856). [6] K. Wilber, Sex, Ecology and Spirituality. The Spirit of Evolution (Shambhala Books, 1995). [7] J. Tønnesvang et al., The Four Quadrant Model (Klim Publishing Books, 2015). [8] M. Kleineberg, “The Blind Men and the Elephant. Towards an Organization of Epistemic Contexts,” Knowledge Organization, vol. 40, no. 5, pp. 340– 62 (2013). https://doi.org/10.5771/0943-7444-2013-5- 340 [9] W. F. Boron and E. L. Boulpaep, Medical Physiology, 2nd ed. (Elsevier, 2011). [10] K. Kupfm ¨ uller, “Nachrichtenverarbeitung im Men- ¨ schen,” Springer Verlag, Taschenbuch der Informatik No. 3, pp. 429–454 (1974). https://doi.org/10.1007/978-3- 642-65588-3 [11] R. S. Siegler, “Emerging Minds: The Process of Change in Children’s Thinking (Oxford University Press, 1996). [12] J. D. Reiss, “A Meta-Analysis of High Resolution Audio Perceptual Evaluation,” J. Audio Eng. Soc., vol. 64, pp. 364–379 (2016 Jun.). https://doi.org/10.17743/ jaes.2016.0015 [13] S. Bech and N. Zacharov, Perceptual Audio Evaluation–Theory, Method and Application (John Wiley & Sons, 2006). https://doi.org/10.1002/9780470869253 [14] T. C. Koopmans, “Measurement without Theory,” Review Econ Stat., vol. 29, no. 3, pp. 161–72 (1947). https://doi.org/10.2307/1928627
 

SIY

Technical Expert
Technical Expert
Joined
Apr 6, 2018
Messages
3,614
Likes
6,785
Location
Phoenix, AZ
#8
As usual, Harley starts with a deep misunderstanding and does not comprehend what "subjective" means, re-defining it as "utterly lacking in even the most minimal controls."

I.e., this is up to his usual standards. Worthless.
 

FrantzM

Major Contributor
Forum Donor
Joined
Mar 12, 2016
Messages
1,351
Likes
1,470
#9
As usual, Harley starts with a deep misunderstanding and does not comprehend what "subjective" means, re-defining it as "utterly lacking in even the most minimal controls."

I.e., this is up to his usual standards. Worthless.
He understands. The FUD is intentional. It aligns with his interests.
 

audiophile

Active Member
Joined
Oct 7, 2019
Messages
132
Likes
76
#10
As usual, Harley starts with a deep misunderstanding and does not comprehend what "subjective" means, re-defining it as "utterly lacking in even the most minimal controls."
I cannot find this quote in the PDF. What page #?
 

anmpr1

Major Contributor
Forum Donor
Joined
Oct 11, 2018
Messages
1,091
Likes
1,511
#14
Although some A/B comparisons with other known or comparably priced components are made, the single presentation method is the best way to determine the long-term quality of the component in question.

This is an example of question begging. That is, presuming true what in fact remains an open question, and still under consideration.

The experimenter's agenda is often to prove that no audible differences exist rather than to discover if differences do exist.

No. The purpose of a blind test is to take the experimenter's agenda and the subjects bias out of the picture. It immediately controls for bias/agenda.

There is an adversarial relationship between subject and experimenter, and the subject is aware that he will be exposed to ridicule if he "fails."

Alternately, if the subject is able to prove his claims, the experimenter is aware that he in turn will be exposed to ridicule. It works both ways, Bob.

The playback system, music, room, and other conditions are all foreign to the subject.

Not true. What about hi-fi man Steve Zipser who, in his own house with his own gear and with his own CDs couldn't tell the difference between his expensive Nelson Pass class A amps and a 10 year old Yamaha integrated?

The experimenter controls all aspects of the test, including the music used, playback level, how long tile subject can hear each presentation, how many times the subject call hear each presentation, the rapidity of switching between presentations, and in which musical passage the switching occurs.

Again, not true. In an ABX, the subject can listen as long as he likes. See above. This is an old JAES paper. Does Harley still argue this way? Will JAES publish anything that's handed to them?
 

Blumlein 88

Major Contributor
Forum Donor
Joined
Feb 23, 2016
Messages
7,869
Likes
9,478
#15
snippage........
This is an old JAES paper. Does Harley still argue this way? Will JAES publish anything that's handed to them?
This is an old JAES paper.

Does Harley still argue this way? Yes.

Will JAES publish anything that's handed to them? It sometimes looks that way.
 

audiophile

Active Member
Joined
Oct 7, 2019
Messages
132
Likes
76
#16
As usual, Harley starts with a deep misunderstanding and does not comprehend what "subjective" means, re-defining it as "utterly lacking in even the most minimal controls."
It's in the conclusion on page 17.
Call me dumb but I can't find that quote on page 17, after reading it from top to bottom. Here's what I see Harley saying about subjective critical listening, starting at the end of page 2:

Contrary to the objectivists' misconceptions, much subjective critical listening as practiced by magazine reviewers is conducted under carefully controlled conditions - more controlled, in fact, than the conditions present during many other stages in the music recording and reproduction process.
...
Careful controls are also maintained during subjective critical listening. Levels between components under audition are matched to 0.1dB or less. Linear differences, such as whether the unit is polarity-inverting or not, are accounted for. Listening sessions are conducted virtually daily for weeks or even months before the review is written.


And here's what in the Conclusion on page 17:

The responsible subjective approach, which combines careful controls with technical understanding, is far more objective than the method's critics realize.
 

SIY

Technical Expert
Technical Expert
Joined
Apr 6, 2018
Messages
3,614
Likes
6,785
Location
Phoenix, AZ
#17
Yes, it’s delightful that he has no idea of what a control is.
 

SIY

Technical Expert
Technical Expert
Joined
Apr 6, 2018
Messages
3,614
Likes
6,785
Location
Phoenix, AZ
#18
This is an old JAES paper.

Does Harley still argue this way? Yes.

Will JAES publish anything that's handed to them? It sometimes looks that way.
To be fair to JAES, this is a conference paper, not a journal paper, so the standards are... relaxed. No actual refereeing.
 

TimF

Member
Forum Donor
Joined
Dec 15, 2019
Messages
27
Likes
16
#19
Can we use experiments with sight and taste to help understand: 1) the limits of perception, 2) the normative sensitivity to subsets of the perceptive range, 3) the normative preferences for subsets of the perceptive range and particular harmonies--i.e., focus or attention triggering frequencies, and 4) common misperceptions and illusions we are likely to fall into (such as a slight emphasis on certain mid range frequencies). We seek to repeat specific experiences, and often their inverse or contrary experiences (horror movies). Can I smell the iron in blood? Some people cannot eat off aluminum plates or use aluminum forks because they think they can taste the metal. Prescott, AZ, has an unusual number of people with so-called environmental disease that are sickened by nearly everything in the environment and cannot tolerate the smell of cosmetics, and who are regularly poisoned by god knows what. If getting the sound perfect and just right becomes too big of a thing for someone, then maybe the activity of getting it right is focused (obsessive) activity for its own sake that is unconsciously occupying us with a safe endeavor that is for the most part the least significant thing in our life--the activity is a flight from the real which is often troubling. My ProAc speakers have a cavity that can hold sand, and various levels of sand affects the quality of the bass output. The sand I drained out and put in plastic bags was very special sand just for highest quality audio speakers (just kidding).
 

Blumlein 88

Major Contributor
Forum Donor
Joined
Feb 23, 2016
Messages
7,869
Likes
9,478
#20
Can we use experiments with sight and taste to help understand: 1) the limits of perception, 2) the normative sensitivity to subsets of the perceptive range, 3) the normative preferences for subsets of the perceptive range and particular harmonies--i.e., focus or attention triggering frequencies, and 4) common misperceptions and illusions we are likely to fall into (such as a slight emphasis on certain mid range frequencies). We seek to repeat specific experiences, and often their inverse or contrary experiences (horror movies). Can I smell the iron in blood? Some people cannot eat off aluminum plates or use aluminum forks because they think they can taste the metal. Prescott, AZ, has an unusual number of people with so-called environmental disease that are sickened by nearly everything in the environment and cannot tolerate the smell of cosmetics, and who are regularly poisoned by god knows what. If getting the sound perfect and just right becomes too big of a thing for someone, then maybe the activity of getting it right is focused (obsessive) activity for its own sake that is unconsciously occupying us with a safe endeavor that is for the most part the least significant thing in our life--the activity is a flight from the real which is often troubling. My ProAc speakers have a cavity that can hold sand, and various levels of sand affects the quality of the bass output. The sand I drained out and put in plastic bags was very special sand just for highest quality audio speakers (just kidding).
Weber's law maybe.
https://en.wikipedia.org/wiki/Weber–Fechner_law
 
Top Bottom