General debate thread about audio measurements

[amirm]

That said, for my own purposes, I find that the problem of listening bias can often be circumvented by other means. For example, if I notice something, and then I learn that other listeners had the same impression independently, I tend to believe that what I heard was real.

Ten wrongs don't make it right. Subjective impressions are so wrong and so frequently that by that metric, you would believe everything including water going upstream by itself.

 

[amirm]

'There seems to be some loose consensus that the Sabre-brand DAC chipsets have a "presence range" somewhere in the upper midrange to lower treble that sounds bright, despite measuring ruler-flat.

I own and have tested more DACs than anyone. There is no such phenomena. It is easily imagined to be there alright but no reality behind it.

 

[josh358]

It's worth keeping in mind that this amp was created not only to reduce noise and distortion but also to avoid the euphonic qualities that would make for inaccurate monitoring in studios.

A really "good" sounding amp like the First Watt SIT3 (besides clipping because of its low output) would be totally unsuitable. Same goes for the euphonic qualities of certain DACs—but not for the rest of the signal chain. The Lavry AD122 96 has a strong reputation for the saturation processing it uses on clipping signals during recording (kind of like tape), for example, and then there are loads of things said about compressors, EQs, filters, mixers, etc.

But then the question becomes which is accurate. Unless we're talking something blatantly euphonic like high second harmonic distortion I don't always find it easy to tell. And sometimes you're just shuffling the deck chairs on the Titanic, e.g., reconstruction filters can sound different but technically, they're all wrong in different ways -- this one has preringning, that one doesn't but is minimum rather than linear phase, etc.

 

[josh358]

Ten wrongs don't make it right. Subjective impressions are so wrong and so frequently that by that metric, you would believe everything including water going upstream by itself.

The probability of two people hearing something very specific independently is seriously low, unless they're influenced by some kind of stereotype (tube amplifiers sound euphonic, etc.). And I rule those things out.

 

[amirm]

But then the question becomes which is accurate. Unless we're talking something blatantly euphonic like high second harmonic distortion I don't always find it easy to tell. And sometimes you're just shuffling the deck chairs on the Titanic, e.g., reconstruction filters can sound different but technically, they're all wrong in different ways -- this one has preringning, that one doesn't but is minimum rather than linear phase, etc.

That is the madness in audio subjectivism which we avoid here. Cleanse your mind of all of that. I go through hours of measurements to show accuracy. I combine that with psychoacoustics. This absolutely instructs us as to transparency and accuracy. Everything else is folklore and has no proper basis.

 

[josh358]

I own and have tested more DACs than anyone. There is no such phenomena. It is easily imagined to be there alright but no reality behind it.

You and Pur1n can have a food fight.

Seriously, I would have to wonder why not. What music did you listen to, and on what speakers?

 

[amirm]

The probability of two people hearing something very specific independently is seriously low, unless they're influenced by some kind of stereotype (tube amplifiers sound euphonic, etc.). And I rule those things out.

Not really when they all parrot the same thing. Oh there was more micro dynamics. The background was blacker. The soundstage wider. I could go on. All the reviews read like each other.

I have tested products that are supposedly euphonic but they absolutely are not. They just distort the sound, making it brighter. Audiophiles are lucky that they don't hear the impairments and instead, substitute improper testing to arrive at positives.

 

[amirm]

Seriously, I would have to wonder why not. What music did you listen to, and on what speakers?

I listen to every genre except country music. My system is listed here: https://www.audiosciencereview.com/...measurements-of-lyngdorf-roomperfect-eq.6799/

 

[amirm]

You and Pur1n can have a food fight.

The day he can present a single peer-reviewed research behind his subjective impressions, we will have a fight. Until then, what he or another confused subjectivist thinks if of no concern to me or our membership here.

In this forum we are guided by what we can prove, not what we think, read or imagine. We do that with objective measurements, discussion of science, psychoacoustics, engineering design, etc. That way, we have very high confidence in our opinions, standing on shoulders of giants.

Stay with us and you will find a much happier life in the future, one where you can count on your opinions being true. That is why we are here.

 

[josh358]

That is the madness in audio subjectivism which we avoid here. Cleanse your mind of all of that. I go through hours of measurements to show accuracy. I combine that with psychoacoustics. The absolutely instructs us as to transparency and accuracy. Everything else is folklore and has no proper basis.

Far from it. I would point out for example that much of this stuff has been successfully ABX'd, not to mention identified in blind tests. And often, it does correlate with measurements. But I don't know of anyone who can completely predict the sound of a component from measurements. For example, Nelson Pass discovered a few years ago that the addition of third harmonic increases the sense of depth, but only if it's added out of phase. Now who the hell would have guess that? And even if you knew, how would you infer it from a measurement?

Really, if measurements were telling us the full story about DAC's they all would sound pretty much the same, so much of what's in the graphs is below the threshold of hearing!

 

[josh358]

The day he can present a single peer-reviewed research behind his subjective impressions, we will have a fight. Until then, what he or another confused subjectivist thinks if of no concern to me or our membership here.

In this forum we are guided by what we can prove, not what we think, read or imagine. We do that with objective measurements, discussion of science, psychoacoustics, engineering design, etc. That way, we have very high confidence in our opinions, standing on shoulders of giants.

Stay with us and you will find a much happier life in the future, one where you can count on your opinions being true. That is why we are here.

I can never count on what I say being true! But I worked most of my life as an audio engineer and I have a pretty good idea of what we can hear and what we can't. I don't personally know of any engineers in pro audio who would say you can't hear the difference between DAC's and amplifiers. Though I imagine you could scrounge one up if you really tried. Hell, you can ABX two op amps in series -- why wouldn't you be able to hear these differences? Even if all DACs were the same, the output stages alone would sound different.

 

[March Audio]

But then the question becomes which is accurate. Unless we're talking something blatantly euphonic like high second harmonic distortion I don't always find it easy to tell. And sometimes you're just shuffling the deck chairs on the Titanic, e.g., reconstruction filters can sound different but technically, they're all wrong in different ways -- this one has preringning, that one doesn't but is minimum rather than linear phase, etc.

Filter ringing is a big red herring. It's in the transition band at nyquist. In other words it's too high frequency to hear. The plots you see of ringing are by definition illegal signals that are bandwidth limited in real systems.

https://troll-audio.com/articles/filter-ringing/

In my experience FWIW, the filters that sound different are those that impact the frequency response below 20kHz.

 

[Veri]

Of course, there are differences in soundstaging. The Gungnir for example has a much deeper soundstage on my system than the e38. Schiit, rightly or not, attributes that to the closed form reconstruction filter.

You mean this, VERY simple/modest FR filter schiit calls its "mega combo burrito" filter...?
https://www.audiosciencereview.com/...eview-battle-of-schiit-audio-dacs.5487/page-4

Nice system, but by revealing I mean something like stats or ribbons.

Ahh yes, the ultimate reverse UNO card. My system is more revealing.

 

[josh358]

That is over generalising in my experience and opinion.

And there are way too many factors involved to say for certain which might be the dominant one.

I have listened to different generations of the ESS chips, albeit all with different makes, and output stage designs.

ES9018, ES9028Q2M, ES9038Q2m, ES9038PRO.

Subjectively, with blind volume equalisation (since I do not have the right equipment to equalise them), the ES9018 sounds somewhat different from the other chips on the list. And if anyone wishes to talk about "glare", the only one on the list that sounds like it does have the so called "glare" is the 9018. So, even from a completely subjective point of view, this "ESS Glare" thing is completely outdated. Those forum parrots need to update their listening experiences.

Now, going back to a little more logic but still subjective experience, the intrinsic output design stage of the different DAC's can play a part too.

The ES9018 DAC I used had a separate RCA output option. Direct from its headphone output, it sounded "airy" and "well separated" and a little thin in its bass presentation and was very obviously tonality wise, different from the other ESS chips on the list. (I have actually spent a full 3 hour session comparing all of them back to back. ) But then I plug the RCA output to the JDS Atom and listened to it, the differences between it and the ES9038Q2M and the ES9038PRO were much , much less discernible. For that matter, the ES9038Q2M was from a Khadas Tone Board.

So, whatever people say about the idea of "ESS Glare", its at best outdated. At worst, its their inherent bias as a result of listening to outputs with poor treble acuity prior to it or worse, using systems that have poor implementations high on distortion, which our ears are much more sensitive to from the treble range onwards than the bass frequencies.

Edit and Addendum:

All the above comparisons were done using A HD800, HD6XX and HE4XX, with all 3 headphones being used repeatedly between identical tracks for comparison.

My current round of DAC comparisons actually began this with a comparison of an ES9018 DAC to an ES9038Pro DAC of similar design (Exasound e28 Mk II vs. Exasound e38 Mk II). The latter had among other things better highs, but since this is a different model that wasn't necessarily due to the chip -- as Exasound's George Klissarov put it to me, "We've learned a lot about DAC design since [we designed the 9018]." So I think the only way you can really tell what to attribute to the chip is to experiment with the chips in similar circuits, and eliminate other differences such as choice of reconstruction filter.

An example of the importance of implementation might be the fact that in Amirm's measurements some ESS implementations show an IM hump and some don't. (What causes the hump in the first place, the distortion cancellation circuit?)

But we do know that some ESS chips have unique electrical characteristics, such as filter overshoot and the aforesaid IM hump, in at least some implementations. Much of what I heard in my comparison was consistent with the overshoot that we know is there, and with an IM hump as well -- though "consistent" is of course far from the same as "due to".

I'm not sure what to make of the issue of high frequency distortion from other components in the chain. Certainly, the AHB2 isn't prone to high frequency distortion, and neither are ribbon drivers or the headphones I tried. But many recordings, unfortunately, do have serious high frequency distortion and it's always possible that a more accurate DAC would mask it less. Or, alternatively, that a DAC that emphasizes certain components (e.g., the overshoot) would exaggerate it. So that's one of the many questions I have about what I heard. Subjectively, when there was high frequency distortion or emphasis in a recording (e.g., violins miked from above), the distortion or emphasis was more apparent on the e38 than on the Gungnir, and on the AHB2 than the A21.

 

[josh358]

Filter ringing is a big red herring. It's in the transition band at nyquist. In other words it's too high frequency to hear. The plots you see of ringing are by definition illegal signals that are bandwidth limited in real systems.

https://troll-audio.com/articles/filter-ringing/

In my experience FWIW, the filters that sound different are those that impact the frequency response below 20kHz.

That would be true in a perfectly linear system. Neither amplifiers nor loudspeakers nor ears nor the air are perfectly linear and it's well known that the nonlinearity of air alone can make ultrasonic sounds audible.

Personally, I have no opinion on whether the ultrasonic ringing has audible effects, as I'm not aware of any experiments. But I can't rule it out simply because the ringing occurs at inaudible frequencies.

As to limiting the bandwidth of the signal to avoid ringing, that is well known, but in practice, I am not sure how you would do that, as the whole purpose of the filters is to limit the bandwidth of said signals.

 

[Purité Audio]

Are you really an engineer Josh, there are many forums , nearly all of them in fact, that would value your subjective contribution.
Keith

 

[GrimSurfer]

Really, if measurements were telling us the full story about DAC's they all would sound pretty much the same, so much of what's in the graphs is below the threshold of hearing!

In a response to one of my posts, you said that not everything of significance to the listening experience is measured. Now you're saying that which is measured should add up to pretty much the same thing.

I see these statements as the basis for the kinds of circular arguments and ludicrous statements that persist in the audio world. Accepting these lays the groundwork for accepting all kinds of subjectivist nonsense.

Some measured phenomenon lie outside the audible range but others do not. Some things we struggle to measure and portray (such as phase) but can perceive. But this in no way means that we should throw out methods of measurement at our disposal. It means we should continue to advance these methods while trying our best to add reasoned meaning to the results we do have.

 

[josh358]

Are you really an engineer Josh, there are many forums , nearly all of them in fact, that would value your subjective contribution.
Keith

My God, Keith, this is degenerating. I will say this: as someone who worked in pro audio most of his life, and known and worked with many other audio engineers, both on the production/post and design sides, I don't think I've ever met *any* who share the naive belief that all equipment sounds the same, or that a limited suite of measurements can completely characterize the sound of equipment. Most would chuckle at this kind of conversation.

We use measurements heavily as part of the design process, but measurements are a tool: we are ultimately designing for the human ear, and I don't know of anyone who knows how to do that optimally with measurements alone, because the psychoacoustics are just too complex, with too many unknowns. Audio is littered with the bones of equipment that measured great and sounded awful because at the time the measurements were made, we didn't know enough to measure the right things. Indeed, many of the routine measurements that we used today, e.g., jitter rejection, were developed precisely because the measurements we were using proved inadequate.

 

[edechamps]

@josh358: I see a lot of claims in your last few posts, but no references to studies (e.g. properly conducted double-blind tests) to back them up. You might have more success convincing people here if you could provide some.

 

[josh358]

In a response to one of my posts, you said that not everything of significance to the listening experience is measured. Now you're saying that which is measured should add up to pretty much the same thing.

I see these statements as the basis for the kinds of circular arguments and ludicrous statements that persist in the audio world. Accepting these lays the groundwork for accepting all kinds of subjectivist nonsense.

Some measured phenomenon lie outside the audible range but others do not. Some things we struggle to measure and portray (such as phase) but can perceive. But this in no way means that we should throw out methods of measurement at our disposal. It means we should continue to advance these methods while trying our best to add reasoned meaning to the results we do have.

I'm not sure what you're saying here. The quote "Really, if measurements were telling us the full story about DAC's they all would sound pretty much the same, so much of what's in the graphs is below the threshold of hearing!" implies exactly the same thing. So let me try to clarify my point. I'm saying that when I apply what I know of psychoacoustics to Amirm's measurements, most of them would appear to be below the threshold of hearing, which is about -10 dB SPL in the midrange and in practice somewhat higher since a quiet room might have ambient noise of 40 dB SPL and research suggests that we can hear about 10 dB below the noise floor.

That being the case, one hypothesis is that not everything of significance is being measured. In some cases, I know this is true. For example, I've mentioned the underdamped filters in ESS DAC's, and it's easy to measure transient response. We know as well what underdamping, overdamping, and critical damping sound like, since we've had a lot of experience. So that would have to be measured.

In any case, in no case have I suggested throwing out the measurements that we have! I wouldn't read this site if I weren't interested in the measurements, and their correlation with what we hear. But I've seen Amrin wonder whether a certain measurement was audible because it was of such low magnitude, and I have the same question. I have to suspect that when we measure something that's 90 or 115 dB down it's audible contribution is insignificant and that when we do hear differences, we're hearing something else. But in many other cases, I just don't know. Is that ESS IM hump audible, for example, on music and with real world transducers, and under what circumstances?

I've found that establishing a correlation between measurement and audibility is a lot harder than some here seem to assume. If you talk to a designer, someone who has spent time in the lab measuring and listening, you'll learn a lot, at least I do. I had a very interesting conversation with John Siau at an AES convention a couple of years ago, for example, in which he discussed the effect of non-linear distortion on the reproduction of piano. I think you'd be surprised at how much there is to learn about what matters, and what does not.