General question about a DAC

[Jakob1863]

The measurements that Amir had made on Topping products (D30, D50, etc) have shown that these products are scientifically very sound (pardon the pun).
Help me out here: so if a D30 which sells for around $130.00 with superb numbers, does that mean that if a $10,000 DAC had the same great measurements that the 2 units would sound very much alike?

It´s still hard to say.
Although i might sound like a broken record (because i´ve posted it so often), it is important to remember/realize that it is all about the models that we are using instead of the far more complex reality.
Measurements - or more specific a set of measurements - is done due to a certain model from which we conclude what measurements might be sufficient to cover the performance of an audio device.

To conclude about the audibility we use a model of our hearing sense from which we conclude, based on the measured numbers, if a difference might be audible.

So if you want to conclude from measurements, or ask others to conclude, then you must keep in mind that any conclusion is based on a set of premises:
-) the measurements cover everything about the measured device that might be of relevance
-) it is ensured that in another combination of audio devices (like in your personal reproduction environment) the device will not behave differently
-) your listening sense (more precise your hearing mechanism/brain combination) works exactly like the model

If these premises are correct, then at least logically correct conclusions are warranted.
However, it seems that sometimes (?often?) the premises aren´t correct, so any conclusion might be wrong.

P.S: imo it is quite important too, to remember that the price of a device isn´t always a useful indicator of sound quality/differences, as it depends on to many variables especially in a luxury market segment

 

[House de Kris]

The measurements that Amir had made on Topping products (D30, D50, etc) have shown that these products are scientifically very sound (pardon the pun).
Help me out here: so if a D30 which sells for around $130.00 with superb numbers, does that mean that if a $10,000 DAC had the same great measurements that the 2 units would sound very much alike?

Yes, I'm just a newb here, but I'm having difficulty really grasping the question at hand. I don't think I've ever seen two different products have the same great measurements. It is my premise that if two objects measure differently, then it really shouldn't be a surprise if they sound different. Even Spirit84's question hits on this (it would appear), with the final question being "that the 2 units would sound very much alike?" 'Very much alike' is not the same as 'sound exactly the same.' And, if they don't sound exactly alike, then by definition, they must sound different. It would appear, then, the question comes down to a $130 unit that measures the same as a $10,000 unit sounding different from each other. I would think that is quite probable. Perhaps the DAC that is 100x in price satisfies an audio need of the purchaser that the lower cost unit doesn't fulfill and is worth it to the purchaser. Not sure where any phoolery plays into this at all.

I'm sure I've missed something here, so fill in the blanks for me, please. Oh, and be kind to the newb.

 

[RayDunzl]

I don't think I've ever seen two different products have the same great measurements. It is my premise that if two objects measure differently, then it really shouldn't be a surprise if they sound different.

The levels at which measurements can be taken and differrentiated has become exceedingly small.

Wanna do the math?

 

[amirm]

I'm sure I've missed something here, so fill in the blanks for me, please. Oh, and be kind to the newb.

The first point to note is that our instrumentation is exceptionally sensitive. I can measure distortions down to -160 dB. Human hearing system has about 110 dB or so of dynamic range.

Additionally, our hearing system has blind spots. It is that which allows us to throw way 90% of the music in an MP3, and the other 10% sound remarkably close to the original. Clearly massive distortions there are not heard. Electronic measurements of distortions in lossy compression would be horrific!

So while it is good when there is no measured difference, in which case the odds are hugely against audible differences, when there are differences we need to interpret them according to models of our hearing system. You frequently see me do that in my measurement reviews.

As to the $10,000 DAC, some of those DACs actually underperform much cheaper ones due to poor engineering, lack of design verification or just plain bad design ideas.

 

[RayDunzl]

Sample Math:

I can measure distortions down to -160 dB

If the scale is Volts, and the maximum ( 0dB ) is 2 volts, the level measured is 0.00000002 volts.

If the scale is a 1500kg car, the measurement is down at 0.015 grams

If the scale is distance, say LA to New York (2451 miles), the measured level is 1.55 inches.

If the scale is time, say a 90 year life, the measured level is 28.3 seconds.

 

[Sal1950]

As to the $10,000 DAC, some of those DACs actually underperform much cheaper ones due to poor engineering, lack of design verification or just plain bad design ideas.

But they do come in fully machined and custom finished cases that cost the manufacturer $2k each to produce. And the manufacturer has spent a small fortune on marketing and advertisement that must be divided between what is in reality a very small number of production units that must be recouped. High End HiFi production has enormous hidden costs.

 

[Spirit84]

Allow me to quote form Roger Sanders (sanderssoundsystems.com).
This is taken from his white paper: "Audio Equipment Testing White Paper" which is available on his site.
I will leave the ASR scientific experts to debate what is written below.

"Because ABX testing takes a lot of time and effort, I always subject the equipment to instrument tests first to assure it meets the basic quality criteria (BQC) for high fidelity sound. I find that a significant amount of equipment fails to meet BQC on instrument testing.

The BQC are:
1) Inaudible noise levels (a S/N of 86 dB or better is required
2) Inaudible wow and flutter (less than 0.01%)
3) Linear frequency response across the audio bandwidth (20 Hz - 20 KHz +/- 0.1 dB)
4) Harmonic distortion of less than 1%

If components fail the BQC, they will sound different on an ABX test. But if this is so, why bother to go to all the trouble of doing an ABX test on them? After all, you will already know the cause of the differences you will hear because you found it using instrument testing.

Specifically, if a component has a poor S/N, you will hear hiss on an ABX test that will cause the component to sound different from one that has a good S/N and is silent. If the frequency response isn't linear, the sound will be different from one that has linear response -- and the instrument measurement will tell you which one is accurate. If high levels of distortion is present, you will hear that as lack of clarity, muddy sound, a sense of strain, poor imaging, and most of the other subjective comments audiophiles use to describe the sound they hear. If wow and flutter is high on one component, you will not need to ABX test it to know that.

The results of ABX testing usually are quite surprising to most audiophiles. They quickly discover that components that meet the BQC always sound identical to each other. Only if components fail the BQC (and many do), will they sound different.

Now I understand that many audiophiles will find that hard to believe. But don't shoot me, I'm just the messenger. If you don't believe that components that meet the BQC sound identical, then you need to do some well-controlled listening test and prove it to yourself.

Let me stress a very important point. Many audiophiles immediately become defensive and think that what I just said is that all audio equipment sounds identical. NOTHING COULD BE FURTHER FROM THE TRUTH! Of course many components sound different from each other.

But they think I said that all components sound the same (which is untrue). They have heard differences between such components, so immediately disregard the whole idea of ABX testing. This is a tragedy.

I said that components "that meet the BQC sound identical." This is true. They do. But a great many components do not meet the BQC, so do NOT sound identical.

The point of doing controlled testing is to find out what is causing the differences in sound that is heard. I am not saying that audiophiles are deaf. I am trying to help them understand how to do testing that will show the true causes of the differences they hear between components.

Valid testing requires that you apply some basic scientific principles to the task. Science is not incompatible with the audiophile world. In fact, it is an essential and very helpful tool in finding out the facts and determining what is causing the differences we all hear between components.

So don't dismiss science. After all, it is science and engineering that provided you with the components you now enjoy. There is no magic and magicians don't design audio equipment -- engineers do."

 

[Wombat]

Allow me to quote form Roger Sanders (sanderssoundsystems.com).
This is taken from his white paper: "Audio Equipment Testing White Paper" which is available on his site.
I will leave the ASR scientific experts to debate what is written below.

"Because ABX testing takes a lot of time and effort, I always subject the equipment to instrument tests first to assure it meets the basic quality criteria (BQC) for high fidelity sound. I find that a significant amount of equipment fails to meet BQC on instrument testing.

The BQC are:
1) Inaudible noise levels (a S/N of 86 dB or better is required
2) Inaudible wow and flutter (less than 0.01%)
3) Linear frequency response across the audio bandwidth (20 Hz - 20 KHz +/- 0.1 dB)
4) Harmonic distortion of less than 1%

If components fail the BQC, they will sound different on an ABX test. But if this is so, why bother to go to all the trouble of doing an ABX test on them? After all, you will already know the cause of the differences you will hear because you found it using instrument testing.

Specifically, if a component has a poor S/N, you will hear hiss on an ABX test that will cause the component to sound different from one that has a good S/N and is silent. If the frequency response isn't linear, the sound will be different from one that has linear response -- and the instrument measurement will tell you which one is accurate. If high levels of distortion is present, you will hear that as lack of clarity, muddy sound, a sense of strain, poor imaging, and most of the other subjective comments audiophiles use to describe the sound they hear. If wow and flutter is high on one component, you will not need to ABX test it to know that.

The results of ABX testing usually are quite surprising to most audiophiles. They quickly discover that components that meet the BQC always sound identical to each other. Only if components fail the BQC (and many do), will they sound different.

Now I understand that many audiophiles will find that hard to believe. But don't shoot me, I'm just the messenger. If you don't believe that components that meet the BQC sound identical, then you need to do some well-controlled listening test and prove it to yourself.

Let me stress a very important point. Many audiophiles immediately become defensive and think that what I just said is that all audio equipment sounds identical. NOTHING COULD BE FURTHER FROM THE TRUTH! Of course many components sound different from each other.

But they think I said that all components sound the same (which is untrue). They have heard differences between such components, so immediately disregard the whole idea of ABX testing. This is a tragedy.

I said that components "that meet the BQC sound identical." This is true. They do. But a great many components do not meet the BQC, so do NOT sound identical.

The point of doing controlled testing is to find out what is causing the differences in sound that is heard. I am not saying that audiophiles are deaf. I am trying to help them understand how to do testing that will show the true causes of the differences they hear between components.

Valid testing requires that you apply some basic scientific principles to the task. Science is not incompatible with the audiophile world. In fact, it is an essential and very helpful tool in finding out the facts and determining what is causing the differences we all hear between components.

So don't dismiss science. After all, it is science and engineering that provided you with the components you now enjoy. There is no magic and magicians don't design audio equipment -- engineers do."

Individuals will perceive differences between items that meet the BQC . They will dismiss the BQC. They will dismiss DBT. They will dismiss psychoacoustics. They will dismiss measurements. They will dismiss scientific research. Etc.

All they can offer as an alternative is "I know what I hear", ad nauseum, and no credible, repeatable verification process to back-up the opinion. I know where the 'air' is and it is usually hot.

 

[House de Kris]

The first point to note is that our instrumentation is exceptionally sensitive. I can measure distortions down to -160 dB. Human hearing system has about 110 dB or so of dynamic range.

Additionally, our hearing system has blind spots. It is that which allows us to throw way 90% of the music in an MP3, and the other 10% sound remarkably close to the original. Clearly massive distortions there are not heard. Electronic measurements of distortions in lossy compression would be horrific!

So while it is good when there is no measured difference, in which case the odds are hugely against audible differences, when there are differences we need to interpret them according to models of our hearing system. You frequently see me do that in my measurement reviews.

As to the $10,000 DAC, some of those DACs actually underperform much cheaper ones due to poor engineering, lack of design verification or just plain bad design ideas.

It's nice that you can measure distortions down to -160dB. I found this great site by searching for info on a Topping D50. I have not been able to catch up on my reading and digest all reviews written and shared on this site. Have you ever found any device that has a sole distortion component that is at -160dB?

I always thought that MP3 threw away 90% of the data (i.e., when comparing 128k MP3 to RBCD rates), not 90% of the music. By saying electronic measurements of distortions in lossy compression 'would be' horrific, kinda implies to me that you haven't made such measurements. Any plan to go that route? It may still satisfy Roger Sanders' BQC threshold of acceptability.

Like I wrote a couple paragraphs previous, I haven't read all reviews here. Are there any two devices tested that do indeed have no measured difference? If so, I certainly agree that it would be highly unlikely that there'd be any audible differences. And this really is the thrust of my question earlier, are there any two devices that measure the same?

 

[House de Kris]

Sample Math:


If the scale is Volts, and the maximum ( 0dB ) is 2 volts, the level measured is 0.00000002 volts.

If the scale is a 1500kg car, the measurement is down at 0.015 grams

If the scale is distance, say LA to New York (2451 miles), the measured level is 1.55 inches.

If the scale is time, say a 90 year life, the measured level is 28.3 seconds.

I suppose these are all examples intended to trivialize the significance of something 160dB down from a reference. If I were driving at 65MPH and get distracted for a few 10s of milliseconds, make an inappropriate decision after the distraction, and end up doing a head-on into an oncoming semi doing 70MPH, I could end up with a severed limb or massive brain damage in much less than 2.8 seconds. 2.8 seconds is 180dB below 90 years (if I am to trust your math). Out of my whole 90 year life, that 2.8 seconds would be incredibly significant.

 

[Wombat]

Not trivialise. Put into perspective.

Your analogy is false in terms of risk and consequence, let alone direct relevancy.

 

[Blumlein 88]

It's nice that you can measure distortions down to -160dB. I found this great site by searching for info on a Topping D50. I have not been able to catch up on my reading and digest all reviews written and shared on this site. Have you ever found any device that has a sole distortion component that is at -160dB?

I always thought that MP3 threw away 90% of the data (i.e., when comparing 128k MP3 to RBCD rates), not 90% of the music. By saying electronic measurements of distortions in lossy compression 'would be' horrific, kinda implies to me that you haven't made such measurements. Any plan to go that route? It may still satisfy Roger Sanders' BQC threshold of acceptability.

Like I wrote a couple paragraphs previous, I haven't read all reviews here. Are there any two devices tested that do indeed have no measured difference? If so, I certainly agree that it would be highly unlikely that there'd be any audible differences. And this really is the thrust of my question earlier, are there any two devices that measure the same?

Well your deeply into degree of differences aren't you? Even with two identical devices there will be some low level differences that are measurable. Yet when we read people's opinion about sound quality of a given model of a given make we don't usually question if one sample sounds better than another. Only when some QC issue has arisen.

So differences of degree. How much difference is an audible difference? Two devices can measure differently and sound indistinguishable from one another. Or the type or level of difference may be audible.

So to answer your question there are no two devices that would measure exactly the same to the level of measurements possible today. Not even two devices sliding off an assembly line one after another. Yet if you think no two devices can possibly sound the same, I would say you are a long way from making that point. You also have the issue of random noise which means the same exact device may measure ever so slightly differently now than it did an hour ago or 10 minutes ago.

 

[amirm]

I always thought that MP3 threw away 90% of the data (i.e., when comparing 128k MP3 to RBCD rates), not 90% of the music.

??? The data is what becomes music once it goes through your DAC. But yes, it does throw away real music. Simultaneous tones for example can be masked and what is eliminated. Others are lowered in resolution below masking/hearing thresholds.

Yes, some data compression also happens if you have say, nothing but digital zeros but that is not going to get you much compression with real music.

By saying electronic measurements of distortions in lossy compression 'would be' horrific, kinda implies to me that you haven't made such measurements. Any plan to go that route? It may still satisfy Roger Sanders' BQC threshold of acceptability.

The distortions in lossy compression go from zero to huge values on audio frame by frame basis (milliseconds). The encoder makes judgements relative to its bit budget as to how much it will compress/discard data. Simple signals like sine waves are trivial to compress so no distortion takes place. As such, classical audio measurements do not work. As a result, Roger's BQC thresholds do not apply here.

As to my use of the word "would" you can take that as will. I know how the system works down to minute level as my team at Microsoft developed lossy compression algorithms and this is one of my areas of professional expertise. When you take out a chunk of content, traditional distortion measures go through the roof. They have no choice but to do so.

We don't normally show this because such distortion measures are psychoacoustically blind so we don't measure them using classical hardware measurement techniques. That said, I have absolutely measured and found bugs in some encoders when it comes to classical signals. Some for example change frequency response. Or cause overflow.

 

[amirm]

It's nice that you can measure distortions down to -160dB. I found this great site by searching for info on a Topping D50. I have not been able to catch up on my reading and digest all reviews written and shared on this site. Have you ever found any device that has a sole distortion component that is at -160dB?

Because that is well below threshold of audibility, in my measurements I usually go down to -130 to -140 dB or so. And yes, in some tests devices do have distortions below that. Take a look at this frm RME ADI-2 DAC review:

There are some spikes here and there but they are below -130 dB. So even though we see them, I can't with any straight face say they point to audible problems.

 

[House de Kris]

Well your deeply into degree of differences aren't you? Even with two identical devices there will be some low level differences that are measurable. Yet when we read people's opinion about sound quality of a given model of a given make we don't usually question if one sample sounds better than another. Only when some QC issue has arisen.

So differences of degree. How much difference is an audible difference? Two devices can measure differently and sound indistinguishable from one another. Or the type or level of difference may be audible.

So to answer your question there are no two devices that would measure exactly the same to the level of measurements possible today. Not even two devices sliding off an assembly line one after another. Yet if you think no two devices can possibly sound the same, I would say you are a long way from making that point. You also have the issue of random noise which means the same exact device may measure ever so slightly differently now than it did an hour ago or 10 minutes ago.

Yes, very deeply into degrees of differences. But, we may be losing sight of my point. That being, if things measure differently, should we be surprised if they are determined to sound different? Note, I'm not claiming that I necessarily can differentiate between two devices that measure very closely to each other. But, if someone claims that they could hear differences between two identical models off the assembly line, who am I to say they can't?

We all have different sensitivities to detection of erroneous signals. For example, a friend gave me a CD he'd made from one of those big black things with music on it. I found it unlistenable due to the tracking distortions. He couldn't hear it. Even listening to it on the same CD player, on the same system, in the same room, at the same time, sitting side by side, he noticed nothing wrong. I was being tortured by all the distortion. In post #27 amirm alludes to human dynamic range of 110dB being linked to audibility of distortion. Does this apply to my friend, and does it imply he has something like only 30dB of dynamic range since he couldn't hear severe distortion? That was a gross example, I know. Still other times, I can hear distortion that my audio fiends cannot. Then again, I've sat and listened to systems where someone goes on and on about a sound they're hearing that isn't right, I can't hear it at all. My point in all these examples is that perhaps we shouldn't make a one-size-fits-all blanket statement about the audibility below some arbitrary threshold of the detection of audible distortion to the individual. Coming back to, if it measures different, should we be surprised if someone says they can hear a difference?

I'm not so sure it all comes down to the human mechanics of hearing. This may be a phenomena of listening instead. Going back to my friend with the mistracking errors on CD. The same sounds were entering both of our ears. We are both human. Our hearing functions quite similarly. Yet he could not detect any problem. In my opinion, the difference most likely is chalked up to different degrees of listening development, not hearing mechanics.

 

[Sal1950]

But, if someone claims that they could hear differences between two identical models off the assembly line, who am I to say they can't?

But delusional BS is just that, BS.
You could claim that you could fly like Superman, to which I can only reply, prove it.
That's what Bias Controlled Double Blind testing is for, to separate the illusion from reality.

 

[House de Kris]

But delusional BS is just that, BS.
You could claim that you could fly like Superman, to which I can only reply, prove it.
That's what Bias Controlled Double Blind testing is for, to separate the illusion from reality.

If I understand your position, as I was sitting next to my friend and claiming to hear gobs of tracking distortion and he couldn't, he should have claimed I was spewing delusional BS. If stuff measures differently, why is it so hard to believe it may actually sound different? Your prejudices control your behavior into knee-jerk reactions of claiming anyone who can hear differences is a delusional BSer. Perhaps those that can hear a difference between two identical models off the assembly line actually did participate in a double blind test with bias control to verify the differences. Thus, the basis for their claim.

 

[Sal1950]

Perhaps those that can hear a difference between two identical models off the assembly line actually did participate in a double blind test with bias control to verify the differences. Thus, the basis for their claim.

And perhaps they can fly like Superman.
If someone claims to hear the difference between two components whose difference in measurement is 30db below what is generally considered audible, then evidence of proof is required.

 

[Blumlein 88]

Yes, very deeply into degrees of differences. But, we may be losing sight of my point. That being, if things measure differently, should we be surprised if they are determined to sound different? Note, I'm not claiming that I necessarily can differentiate between two devices that measure very closely to each other. But, if someone claims that they could hear differences between two identical models off the assembly line, who am I to say they can't?

We all have different sensitivities to detection of erroneous signals. For example, a friend gave me a CD he'd made from one of those big black things with music on it. I found it unlistenable due to the tracking distortions. He couldn't hear it. Even listening to it on the same CD player, on the same system, in the same room, at the same time, sitting side by side, he noticed nothing wrong. I was being tortured by all the distortion. In post #27 amirm alludes to human dynamic range of 110dB being linked to audibility of distortion. Does this apply to my friend, and does it imply he has something like only 30dB of dynamic range since he couldn't hear severe distortion? That was a gross example, I know. Still other times, I can hear distortion that my audio fiends cannot. Then again, I've sat and listened to systems where someone goes on and on about a sound they're hearing that isn't right, I can't hear it at all. My point in all these examples is that perhaps we shouldn't make a one-size-fits-all blanket statement about the audibility below some arbitrary threshold of the detection of audible distortion to the individual. Coming back to, if it measures different, should we be surprised if someone says they can hear a difference?

I'm not so sure it all comes down to the human mechanics of hearing. This may be a phenomena of listening instead. Going back to my friend with the mistracking errors on CD. The same sounds were entering both of our ears. We are both human. Our hearing functions quite similarly. Yet he could not detect any problem. In my opinion, the difference most likely is chalked up to different degrees of listening development, not hearing mechanics.

We all have different abilities. We all run different times in the 40 yard dash. I can do it in 5 seconds. Some can do it in less time. I knew this guy who couldn't walk, couldn't do 40 yards at all. If someone says they can do the 40 yard dash in 2 seconds, who am I to say they can't?

The above is essentially what you were saying. If someone says two bits of gear sound different and why not they have different distortion at the -130 or -140 db level, who am I to say different? Someone who understands that such distortions would be below the physical point of being heard. So again degrees of difference is what we are discussing. If we are talking tracking distortion in an LP, then sure that is audible. If your friend didn't hear it he has become acclimated to it or has deficient hearing.