Is Audio Science Review going about it all wrong? Or partly wrong? Or all right?

[Ron Texas]

I tried to read this thread, but it is too all over the place. One thing is certain, hanging out around here has changed my perspective on playing recorded music for the better.

 

[jsrtheta]

As an avid DIY-er having dsp -> hypex amps -> linkwitz Lx521, I sometimes think there oughtta be a law against this. Whenever I hear something clipping/distorting/hissing, my first thought is never "oh, cool, a vintage feel," my neuroticism always makes me worry a speaker is breaking up/a cap is about to go/I'm getting EMI from somewhere/something is wrong. I have to stop whatever I'm doing and double-check it's supposed to be like that through my reference headphone setup before I can relax. So many modern pop recordings are clipping or distorting heavily on some channel pre-mixdown, it really makes me wonder what the engineer was going for. A recent Sia recording comes to mind- can't remember which one.

Am I the only one who fears audio component apocalypse when he hears signs of distortion/clipping/etc? Thankfully, I've only ever been right once- it was the night after a guest almost put their elbow through a midrange driver.

You are not alone. You are far from being alone.

 

[jsrtheta]

To determine audibility thresholds for various forms of distortion, for example.

IIRC, it has been used to test for the ability to distinguish polarity from reverse polarity. Didn't shut Clark Johnsen up, though.

 

[andreasmaaan]

IIRC, it has been used to test for the ability to distinguish polarity from reverse polarity. Didn't shut Clark Johnsen up, though.

That too. Also things like:

• frequency range of human hearing
• masking thresholds, including post-masking and pre-masking
• perceived loudness vs frequency vs SPL (loudness contours)
• audibility of resonances
• audibility of changes in SPL
• group delay audibility

And just about everything else that tells us what an audio system is supposed to do to be useful for humans

 

[Cosmik]

That too. Also things like:

• frequency range of human hearing
• masking thresholds, including post-masking and pre-masking
• perceived loudness vs frequency vs SPL (loudness contours)
• audibility of resonances
• audibility of changes in SPL
• group delay audibility

And just about everything else that tells us what an audio system is supposed to do to be useful for humans

But it can only compare two existing things, and in order to know what aspect of the sound it is that you are comparing, you have to know that the equipment is transparent except for the item in the above list you think you are testing for in isolation. If the equipment is transparent, as an audiophile your job is done, anyway.

But how do you know the equipment is transparent? Because a human can't hear the difference between it and some other equipment? Maybe neither is transparent.

In an earlier post you said that a difference-based listening test could reveal whether 'SOTA' is needed and this could be useful to save money. But who can say which piece of equipment is SOTA? Isn't that just a value judgement? Just because one of them is more expensive does not make it better.

Let me show you how such confusion would occur. This is from the brochure of a brand of amplifier that many people would lust after and, in comparison with a Quad 405, would regard as closer to SOTA:

Amplifiers with similar measurements are not equal, and products with higher power, wider bandwidth, and lower distortion do not necessarily sound better. Historically, that amplifier offering the most power, or the lowest IM distortion, or the lowest THD, or the highest slew rate, or the lowest noise, has not become a classic or even been more than a modest success. For a long time there has been faith in the technical community that eventually some objective analysis would reconcile critical listener's subjective experience with laboratory measurement. Perhaps this will occur, but in the meantime, audiophiles largely reject bench specifications as an indicator of audio quality. This is appropriate. Appreciation of audio is a completely subjective human experience. We should no more let numbers define audio quality than we would let chemical analysis be the arbiter of fine wines. Measurements can provide a measure of insight, but are no substitute for human judgment.

Why are we looking to reduce a subjective experience to objective criteria anyway? The subtleties of music and audio reproduction are for those who appreciate it. Differentiation by numbers is for those who do not.

As in art, classic audio components are the results of individual efforts and reflect a coherent underlying philosophy. They make a subjective and an objective statement of quality which is meant to be appreciated. It is essential that the circuitry of an audio component reflects a philosophy which address the subjective nature of its performance first and foremost.

Given this 'philosophy' (that gives the amplifier a free pass with respect to measurements), I think it highly likely that this brand may well sound different from a Quad 405. Does this then indicate that the Pass is SOTA and the Quad not? Or the other way round? If the Quad sounds the same as an AV receiver and they both sound different from the Pass what does that tell you about the necessity for SOTA? And which is SOTA?

I think the difference-based listening test has told you nothing about what is SOTA, and nor has it told you where the thresholds of transparency lie unless your equipment is much more transparent than is necessary. But you cant know that unless you make assumptions - that you could have made without the listening tests.

 

[andreasmaaan]

In an earlier post you said that a difference-based listening test could reveal whether 'SOTA' is needed and this could be useful to save money. But who can say which piece of equipment is SOTA? Isn't that just a value judgement? Just because one of them is more expensive does not make it better.

By SOTA, I meant “best measuring” (notwithstanding that I’d argue some experimentally derived knowledge of the human auditory system is required to even know what this term means in the first place).

SOTA has nothing to do with cost IMO.

But it can only compare two existing things, and in order to know what aspect of the sound it is that you are comparing, you have to know that the equipment is transparent except for the item in the above list you think you are testing for in isolation. If the equipment is transparent, as an audiophile your job is done, anyway.

But how do you know the equipment is transparent? Because a human can't hear the difference between it and some other equipment? Maybe neither is transparent.

We’re covering old ground, but I’ve argued many times that the main purpose of this research is to test human hearing, not equipment.

But let me give you a concrete example.

We can produce a masker tone and a maskee tone trivially with everyday audio gear and a computer or tone generator. Both masker and maskee will sit tens of dB above any distortion produced by the system. Hundreds of subjects exhibit remarkably similar thresholds of detection in quiet. The experiment is repeated hundreds or thousands of times on different subjects using multiple tones, different kinds of noise, different time delays, etc. Results obtained are quite consistent across systems and subject groups.

What are we prohibited from concluding on the basis that we don’t know the test systems are transparent?

 

[Cosmik]

We’re covering old ground, but I’ve argued many times that the main purpose of this research is to test human hearing, not equipment.

OK, but yesterday you were saying it was useful to tell us whether we could get away with cheaper equipment.

(Btw, can you point to any current discussions on ASR that aren't a re-hash of something that's been discussed before? )

 

[sergeauckland]

When hearing thresholds were first established, going back to the 1930s and 40s, commercially produced items were nowhere near good enough, and what was used were laboratory equipments, which were good enough, but impossible to reproduce commercially, both on the grounds of cost and complexity. Those criteria have been refined in later measurements, and we're now at the situation where everyday equipment that can be bought anywhere for very little is far better than audible thresholds. Now, it's mostly not necessary to concern oneself with the minutiae of sound reproduction, just buy whatever product suits in terms of facilities and appearance.

As to listening tests, bit rate compression algorithms such as the MP series, were developed using listening tests to determine masking, audibility and annoyance thresholds, and these sorts of developments continue, although now with greater bandwidths available even on mobile devices, most research is on compressing images rather than audio, or as in the case of MQA, on further monetising what already exists.

It's, I think, pointless using listening tests to choose between two similarly transparent amplifiers, say. That is best done sighted, as facilities and looks are what matters once the issue of sound quality has gone away.

S

 

[Shadrach]

We seem to be suffering from a touch of looking down the wrong end of the telescope.
The debate regarding ABX is often centered around the equipment. This tends to produce these long threads where various philosophical viewpoints and technical discussions regarding the merits of one design, or another, surface.
Try turning the telescope around and look at the observer, forget about the hardware.
For me the great thing about ABX is it demonstrates the fallibility of the observer, the listener. It's pretty meaningless as a guide to choosing equipment because of all the value judgements the prospective purchaser will apply to their choice. What it does show is a truely fascinating insight into our perception. It's also a lot of fun when conducted in circumstances where 'reputations' are not at stake.

 

[Cosmik]

It's, I think, pointless using listening tests to choose between two similarly transparent amplifiers, say. That is best done sighted, as facilities and looks are what matters once the issue of sound quality has gone away.

Of course, but the example of the Pass amplifier illustrates the problem in this. Measurements are ambiguous and incomplete (does the same THD figure represent the same type of distortion, etc.) and so it is not possible to state unambiguously that two amplifiers are similarly transparent from measurements alone. If it were, the Quad 405 would have been the end of the matter in 1975.

The listening test is (misguidedly IMO) seen as the way of breaking out of the impasse. Preference-based listening tests are riddled with problems (practical and 'philosophical'), so the difference-based listening test is promoted as the harder 'science'. But it is a purely relative measure and its results are meaningless unless tied back to an absolute reference. How do you make your reference absolute? Umm... measurements..? And so the circle continues.

 

[RayDunzl]

I tried to read this thread, but it is too all over the place. One thing is certain, hanging out around here has changed my perspective on playing recorded music for the better.

To paraphrase the end of Deckard's meeting with Bryant:

"I wasn't worried about my gear when I came in here, and I'm twice as not worried now."

 

[andreasmaaan]

OK, but yesterday you were saying it was useful to tell us whether we could get away with cheaper equipment.

Well it has that benefit too

But seriously, if we understand masking thresholds we can infer the type and degree of nonlinear distortion that is likely to audible. Add a margin of error and you're good to go IMO.

(Btw, can you point to any current discussions on ASR that aren't a re-hash of something that's been discussed before? )

Very true

 

[sergeauckland]

Of course, but the example of the Pass amplifier illustrates the problem in this. Measurements are ambiguous and incomplete (does the same THD figure represent the same type of distortion, etc.) and so it is not possible to state unambiguously that two amplifiers are similarly transparent from measurements alone. If it were, the Quad 405 would have been the end of the matter in 1975.

The listening test is (misguidedly IMO) seen as the way of breaking out of the impasse. Preference-based listening tests are riddled with problems (practical and 'philosophical'), so the difference-based listening test is promoted as the harder 'science'. But it is a purely relative measure and its results are meaningless unless tied back to an absolute reference. How do you make your reference absolute? Umm... measurements..? And so the circle continues.

As far as (transparent) performance went, the 303 before the 405 was the end of the matter, albeit for a limited range of loads and loudspeaker sensitivities. This was improved in the 405, then 405-2, then again in terms of power output in the 606. I think the reason we have so many competing products is the same reason why we have 30 different shampoos or 20 different toothpastes in every supermarket...because we can...and manufacturers all want a slice of the same pie. There's certainly no technical reason for one amplifier over another, and hasn't been since the late 1970s except for power output and possibly load tolerance...not much will drive Apogee Scintillas.

I dont understand why most amplifiers can't be deemed transparent from measurements alone, given that most amplifiers' errors are so far below audibility. There are a few, like PASS, that don't measure that well, and so may or may not be transparent, and if not transparent, may have characteristics that people like, even if not accurate. It's also an easier sell if an amplifier is not transparent, as the salesman can congratulate the purchaser on their 'Golden Ears' for picking that one out on demo.

S.

 

[Blumlein 88]

Of course, but the example of the Pass amplifier illustrates the problem in this. Measurements are ambiguous and incomplete (does the same THD figure represent the same type of distortion, etc.) and so it is not possible to state unambiguously that two amplifiers are similarly transparent from measurements alone. If it were, the Quad 405 would have been the end of the matter in 1975.

The listening test is (misguidedly IMO) seen as the way of breaking out of the impasse. Preference-based listening tests are riddled with problems (practical and 'philosophical'), so the difference-based listening test is promoted as the harder 'science'. But it is a purely relative measure and its results are meaningless unless tied back to an absolute reference. How do you make your reference absolute? Umm... measurements..? And so the circle continues.

The series connected amplifier test might help in some respects. This is where you connect an amplifier and feed the result to another amplifier. So you can compare the effects of the amp vs a piece of wire. Your reference is the wire. A pretty good absolute reference. Similar testing can be done for ADC/DAC passes.

 

[Blumlein 88]

As far as (transparent) performance went, the 303 before the 405 was the end of the matter, albeit for a limited range of loads and loudspeaker sensitivities. This was improved in the 405, then 405-2, then again in terms of power output in the 606. I think the reason we have so many competing products is the same reason why we have 30 different shampoos or 20 different toothpastes in every supermarket...because we can...and manufacturers all want a slice of the same pie. There's certainly no technical reason for one amplifier over another, and hasn't been since the late 1970s except for power output and possibly load tolerance...not much will drive Apogee Scintillas.

I dont understand why most amplifiers can't be deemed transparent from measurements alone, given that most amplifiers' errors are so far below audibility. There are a few, like PASS, that don't measure that well, and so may or may not be transparent, and if not transparent, may have characteristics that people like, even if not accurate. It's also an easier sell if an amplifier is not transparent, as the salesman can congratulate the purchaser on their 'Golden Ears' for picking that one out on demo.

S.

I want to believe all this. The big thing is amplifiers are tested usually with resistive loads. Not speakers. There is less good info on that, but some reason to think transparency isn't available with all loads for all amps.

The Audiograph Load box (Power cube), combined with some good modeling of some speakers could give us much better info than is usually available on the matter.

 

[Cosmik]

The series connected amplifier test might help in some respects. This is where you connect an amplifier and feed the result to another amplifier. So you can compare the effects of the amp vs a piece of wire. Your reference is the wire. A pretty good absolute reference. Similar testing can be done for ADC/DAC passes.

I think someone alluded to a possible problem with that a few pages back, though: if you start with something distorted in some way (for the sake of argument, let's say 50% THD) and you then add an extra 0.5% of distortion and can't hear the difference, you can't then say that the 0.5% is below the threshold of audibility.

Of course you're talking about less extreme examples than that, but the principle applies: if you are talking about determining absolute transparency, you can't do it using a relative comparison without assuming that you already have transparency in one of the cases. I don't see anything wrong with making such an assumption - but it then prompts the question: if you already know what is transparent and possess it in the form of a 45-year old amplifier available for £100 on eBay and £25 in 'tribute' kit form, why even bother with the test?

 

[Zog]

Why do coins appear and disappear when I see Penn & Teller?

Because you paid to get in. You took the coin out of your pocket at the ticket booth ...

 

[andreasmaaan]

Of course you're talking about less extreme examples than that, but the principle applies: if you are talking about determining absolute transparency, you can't do it using a relative comparison without assuming that you already have transparency in one of the cases. I don't see anything wrong with making such an assumption - but it then prompts the question: if you already know what is transparent and possess it in the form of a 45-year old amplifier available for £100 on eBay and £25 in 'tribute' kit form, why even bother with the test?

I agree, but this is where designing tests that get around individual components' inherent distortion comes in, for example by testing masking thresholds using masker and maskee tones that are far far higher in level than any distortions. If the results of these masking studies seem to agree with the results of nonlinear distortion studies, we have a very solid basis to make inferences about nonlinear distortion audibility thresholds IMHO.

 

[FrantzM]

As far as (transparent) performance went, the 303 before the 405 was the end of the matter, albeit for a limited range of loads and loudspeaker sensitivities. This was improved in the 405, then 405-2, then again in terms of power output in the 606. I think the reason we have so many competing products is the same reason why we have 30 different shampoos or 20 different toothpastes in every supermarket...because we can...and manufacturers all want a slice of the same pie. There's certainly no technical reason for one amplifier over another, and hasn't been since the late 1970s except for power output and possibly load tolerance...not much will drive Apogee Scintillas.

I dont understand why most amplifiers can't be deemed transparent from measurements alone, given that most amplifiers' errors are so far below audibility. There are a few, like PASS, that don't measure that well, and so may or may not be transparent, and if not transparent, may have characteristics that people like, even if not accurate. It's also an easier sell if an amplifier is not transparent, as the salesman can congratulate the purchaser on their 'Golden Ears' for picking that one out on demo.

S.

I agree wholeheartedly. Lost in the discussion before your post was the profit motive: To sell something, there has to be a story, a justification. Back 20~30 years ago, the High End magazines lead the way. They were opinion-makers. I can vividly remember the influence TAS or Stereophile had on the hobby, on audiophiles, on the High End Industry. If you remember, for the longest time HP from TAS would never be seen on a photograph to preserve the mystique of H .. P!! (What was that? ) (Truth to be told he introduces a big portion of audiophiles to Floyd E. Toole, to the notion of room treatment and other room related items...)... Back to the present discussion. Most manufacturer don't declare end game and stick on the same product. There are a few exceptions in the world of High End: Burmester has had the same Top of The Line preamp 808 since the 1980!!! Yes the year 1980 .. and the same TOL of the line Amp 909 since the year 1990!!! FM Acoustics has a few models dating that far back too, aside from these few brands... most High End Audio change components regularly touting progress in the parts and new insights on musicality and PRAT or other BS that sells a lot.. Audio Research changes components every week . They can have Ref 5 ref 50, Ref 500, we're waiting for reference 5000.. with new tubes. , new metallurgy, New Faceplates except for the rack handles for components that never go in an EIA rack ... So the goal of transparency for amplifier may have been attained a while back. I have a Yamaha CA2010 (late 70's , early 80's) and it drives almost anything presented to it. Most recently we tried with a ML Sequel and it delivered ... Of course that is an anecdote and I have no data ...

I would like to see how some amps fare in the SINAD department things like the Old School Krell that could go as high as 1 MHz while delivering tons of power at ease in any loads, sane or insane down to a short circuit ... or amps like Spectral or Boulder.
An old-school Krell on most speakers doesn't seem to sound like most other amps. They seem to deliver something in the bass that most amps don't ... This is not limited to Krell, The old Mark Levinson amplifers (27, 28 etc) would do the same on most speakers. What i am remembering date back to when I was full-card carrying subjectivist, rejecting most Audio measurements .. while being a practicing EE ... I have not heard the nenewer Krell or Mark Levison ...

I have read about the Cube box and it seems to be so far the best way to anticipate how an amp might sound. There are out there some speakers with weird impedance curves, phase behavior and low sensitivity. I am not sure all amplifers can drive those adequately... How does a person goes acquiring a Cube Box? Does ASR need one?

 

[DonH56]

The Power Cube (or Active Load Box for AP analyzers) is several thousand dollars last I checked (few years ago). I am sure AP adds to that with their control SW and resale markup. @amirm probably knows. They have a cool (ahem) water-cooled version for high-power amp testing.

AudioGraph site (Power Cube): https://www.audiograph.se/