• Welcome to ASR. There are many reviews of audio hardware and expert members to help answer your questions. Click here to have your audio equipment measured for free!

Master Thread: Are Measurements Everything or Nothing?

A question for the engineer types basically hopefully without activating any Luddite triggers.

So if my two DAC's measure exactly the same I.E. basically flat as a pancake and -120db+ distortion why can I easily discern a difference. The premise is that something is not being measured? I take as an example a Topping D90SE and a Gustard X26III. The D90SE is vary harsh in the treble compared to the X26 III. What would be a way of a measured test to confirm that? Thanks for your responses. Chhers.
We've been conducting blind tests for over 25 years, investigating phenomena like this.
We've often invited guests who claim to hear such differences between DACs with good and similar measurements.
So far, none of these guests have been able to discern these differences in a controlled blind test environment.

I've listened to both the Topping D90SE and the Gustav X26III several times, but I've never noticed such a difference.

It would certainly be interesting if you could provide these recordings; a Windows PC isn't even required.
 
Indeed. That pre-supposes you are actually measuring the right thing. Where you statement fails is you are assuming no audible difference. So berrying your head in the sand is not really what the OP was aimed at.

There is no sand and this forum has a lot of great resources to learn about the (very thorough) measurement regime
 
Known differences are the harshness in the D90SE compared to the X26III.

You haven't proven that with double-blind AND level-matched listening comparisons. Nor have you tested your particular unit to make sure it is functioning to spec.
 
There is no sand and this forum has a lot of great resources to learn about the (very thorough) measurement regime
Indeed. But science is a funny thing. It tends to be black and white until someone comes along with scientific explanation and test results to show the difference. At this moment in time I don't think we are quite there yet. Hence the OP. I thought the scientist guru's here would be able to conclude the same findings?
 
Firstly compare with very basic controls, level matched and unsighted.
Keith
 
You haven't proven that with double-blind AND level-matched listening comparisons. Nor have you tested your particular unit to make sure it is functioning to spec.
The get out clause :) So basically what you are saying is there "Is no difference". Well that was very helpful. Thanks.
 
In countless threads here on this forum, it has been explained why using acoustic measurement devices is insufficiently precise when attempting to level-match electronic devices. Level-matching electronic devices must be done electronically.

Using acoustic devices (dB meter and REW), the best you can hope for is level matching within about 1dB. That's not nearly sufficiently precise. Levels need to be matched to within 0.1dB from the DAC outputs to the amp.
 
Considering the claim by the OP I'm sure he/she is using the less harsh DAC. Send the other one back to the factory, or to ASR, and have it checked. There is either a difference or there is not. Things break, and incorrect testing is done all the time. If it were me, I would definitely want to find out what was wrong with my DAC or my testing procedure.

You certainly wouldn't be the first one to hear a difference and find out you are not testing it correctly or that there is truly a difference because of a pre-test requirement, filter, or an actual problem, BUT if you don't test, test, test, it is like pissing into the wind. How is that for a proper answer? If you hear a difference, then figure out why.

I actually made my living by being able to hear a problem in very expensive machinery. I've never not been able to figure out why, either by testing with equipment or by simply swapping parts. Remember to thine own self be true. Either you are willing to figure out why or use the DAC you like and forget about the "WHY."

Lots of luck, though.

Regards
 
Indeed. But science is a funny thing. It tends to be black and white until someone comes along with scientific explanation and test results to show the difference. At this moment in time I don't think we are quite there yet. Hence the OP. I thought the scientist guru's here would be able to conclude the same findings?
May I be a little blunt with you?
If people want support from science, then they also have to be willing to support science, right?
Can you understand the frustration of many people, especially on ASR, when someone keeps coming up with listening impressions that completely contradict the measurements, but is absolutely unwilling to prove it in any way?

I'm definitely not saying that the differences you hear aren't real, but there could be at least three reasons for them.
- One of your devices isn't working properly or has a defect.
- There are dependencies related to cables and/or connected devices that can lead to actual audible differences due to capacitive and inductive differences/influences.

I'm just going to assume that the source material and the source are absolutely identical.
Same amplifier and same speakers?
 
Considering the claim by the OP I'm sure he/she is using the less harsh DAC. Send the other one back to the factory, or to ASR, and have it checked. There is either a difference or there is not. Things break, and incorrect testing is done all the time. If it were me, I would definitely want to find out what was wrong with my DAC or my testing procedure.

You certainly wouldn't be the first one to hear a difference and find out you are not testing it correctly or that there is truly a difference because of a pre-test requirement, filter, or an actual problem, BUT if you don't test, test, test, it is like pissing into the wind. How is that for a proper answer? If you hear a difference, then figure out why.

I actually made my living by being able to hear a problem in very expensive machinery. I've never not been able to figure out why, either by testing with equipment or by simply swapping parts. Remember to thine own self be true. Either you are willing to figure out why or use the DAC you like and forget about the "WHY."

Lots of luck, though.

Regards
Indeed. I'm and engineer by trade. The WHY bit is very intriguing. Hence the original post to a Science forum. Just because there isn't a scientific method of measuring something it doesn't mean it doesn't count or can be ignored. Cheers.
 
May I be a little blunt with you?
If people want support from science, then they also have to be willing to support science, right?
Can you understand the frustration of many people, especially on ASR, when someone keeps coming up with listening impressions that completely contradict the measurements, but is absolutely unwilling to prove it in any way?

I'm definitely not saying that the differences you hear aren't real, but there could be at least three reasons for them.
- One of your devices isn't working properly or has a defect.
- There are dependencies related to cables and/or connected devices that can lead to actual audible differences due to capacitive and inductive differences/influences.

I'm just going to assume that the source material and the source are absolutely identical.
Same amplifier and same speakers?

Everything is the same bar the DAC's. I know where you are coming from. But alas it can't be proven unless you can measure it. Even though you can detect a difference. I've read and re-read lots of reports that the D90SE is a bit harsh in the treble. So at the moment I've not reason to believe mine is any different. So the feedback from this forum is simply: "They will sound identical unless there is a fault as our current testing shows" Which makes me think we are not quite there yet with scientific methods. In the design industry it's always prudent to do listening tests. As you can't guarantee that all the testing in the world might have missed something. Cheers.
 
Which makes me think we are not quite there yet with scientific methods. In the design industry it's always prudent to do listening tests. As you can't guarantee that all the testing in the world might have missed something. Cheers.
On most "audiophile" forums, you'll find lots of agreement! ;) And they'll make excuses about why controlled blind listening tests are "invalid" or unnecessary. But this is audio SCIENCE review.

Listening tests can be scientific if they are blind, controlled, and statistically repeatable:
What is a blind ABX test?
Controlled Audio Blind Listening Tests (video)

Amir, our host here is a trained listener. When he reviews products he listens (non-controlled listening tests) AND measures. He backs-up what he hears (if anything) with the measurements. He understands his own human limitations and admits to fooling himself on a few occasions.

...There are some things that are difficult to define and measure but they don't apply to DACs or electronics. i.e. Some acoustic characteristics or the characteristics of lossy file compression (MP3, etc.).
 
Everything is the same bar the DAC's. I know where you are coming from. But alas it can't be proven unless you can measure it. Even though you can detect a difference. I've read and re-read lots of reports that the D90SE is a bit harsh in the treble. So at the moment I've not reason to believe mine is any different. So the feedback from this forum is simply: "They will sound identical unless there is a fault as our current testing shows" Which makes me think we are not quite there yet with scientific methods. In the design industry it's always prudent to do listening tests. As you can't guarantee that all the testing in the world might have missed something. Cheers.

We just established that you didn't level match the DACs properly. And you want to keep making claims that you can hear differences while at the same time making excuses for why you can't make a brief recording of each DAC for analysis.

So you've read the D90SE is a "bit harsh in the treble." You take that as evidence that it is so. We see it as evidence that your brain is now expecting to hear that, and so you do.

Do a proper controlled test. If you can still hear differences, document the test and show us. Then send in the DAC for Amir to analyze.

One of two things will happen: 1) You won't be able to hear differences in controlled testing. 2) You will be able to hear differences, and then Amir's testing will reveal obvious measurable reasons why.
 
Everything is the same bar the DAC's. I know where you are coming from. But alas it can't be proven unless you can measure it. Even though you can detect a difference. I've read and re-read lots of reports that the D90SE is a bit harsh in the treble. So at the moment I've not reason to believe mine is any different. So the feedback from this forum is simply: "They will sound identical unless there is a fault as our current testing shows" Which makes me think we are not quite there yet with scientific methods. In the design industry it's always prudent to do listening tests. As you can't guarantee that all the testing in the world might have missed something. Cheers.
In our opinion, you are drawing the wrong conclusions from your observations.

Start here:

Yes I know this article is about amps, not DACs, but the same concerns apply. The whole article is good, but pay special attention to the right-hand column of page 5. Sighted listening is diabolical, even when you don’t think there should be any difference.

Even people who believed that differences in well-measuring amps were not audible could not help but perceive significant differences between amps. However, once the comparisons were done blind, those differences vanished.

This is an important thing to understand if you’re going to get anywhere with understanding audio. It has far-reaching implications. It’s human nature to detect illusory differences under sighted conditions. You should expect it, and certainly not draw any conclusions about differences from it.

In this case, you have noticed a difference in two DACs with samples that that have both measured transparently here. You have not measured your own devices, you have not properly level-matched, and you have not conducted your test under blind conditions.

Yet you deduce that since you hear a difference, there must be something that ASR does not measure that accounts for these differences. And you express disappointment that we don’t join you in your deduction, considering such attitude “unscientific.”

Yet why do you think that you are any different than the people who heard significant differences in amps under sighted conditions, but were helpless under blind conditions? That’s by far the most parsimonious explanation.

You are not the first to come to us with subjective reports of this kind. For many of us, including me, we have heard differences in our own equipment that went away under blind conditions. You are not coming down from the mountain with a bold and prophetic vision to challenge our orthodoxy. We’ve all been there before. Yet, people here gave you all kinds of advice for how you can record the files, compare the files, and share the files to be analyzed here. Because it’s fun and who knows, maybe someday we’ll get stumped.
 
So you've read the D90SE is a "bit harsh in the treble."
I’ve noticed that once one review makes such a statement, all the others follow suite, but NONE provide any measurements to back up the claim.

The Astell and Kern HC-4 dongle DAC is a case in point. One early reviewer claimed it had harsh trebles. The others, plus customer reviewers on Amazon (etc), all followed suite.

I tested it - sounded the same as my other DACs. It did have the high output volume that all the reviews also notes. But not the harsh trebles.

The explanation? People read that review and then heard what the reviewer said they’d hear.

That’s why a properly conducted level matched truly blind test is the only way to confirm you can hear audible differences.

And such a test is not trivial to set up. Ideally, both the listener and the one switching between devices should be unaware of which device is the active one. Otherwise ‘tells’ can subtly creep in and influence the result.
 
Everything is the same bar the DAC's. I know where you are coming from. But alas it can't be proven unless you can measure it. Even though you can detect a difference. I've read and re-read lots of reports that the D90SE is a bit harsh in the treble. So at the moment I've not reason to believe mine is any different. So the feedback from this forum is simply: "They will sound identical unless there is a fault as our current testing shows" Which makes me think we are not quite there yet with scientific methods. In the design industry it's always prudent to do listening tests. As you can't guarantee that all the testing in the world might have missed something. Cheers.
And that's precisely the problem. You read that the D90SE sounds a bit shrill in the high frequencies.
Perceptual distortions of this magnitude have been scientifically proven hundreds, even thousands, of times.
No one wants to admit that they are so easily and so strongly manipulated.

If you have even a basic understanding of measurements and the corresponding hearing thresholds, you should know that clearly audible differences are very evident in both measurements and recordings.

The fact that these very people who claim to hear such differences fight tooth and nail against contributing to the investigation of these perceived differences is, unfortunately, a prime example of what happens time and again.
Unfortunately, your behavior further undermines the credibility of such statements.
 
Known differences are the harshness in the D90SE compared to the X26III.

These are not known, they are claimed by you without any clear facts to back it up. You could relatively easily record the output of both via RCA to your laptop/PC to a .wav file and then run it through an ABX test like foobar2000 ABX plugin.

Then present findings here.

That something is missing in the current measurement suite for DACs is currently the least likely reason for what you are experiencing.
 
Back
Top Bottom