Serious Question: How can DAC's have a SOUND SIGNATURE if they measure as transparent? Are that many confused?

[solderdude]

I can get that. Especially as a lawyer I have a hint of knowledge about burden of proof

But it seems that when some people give proof with numbers and measurements, it is "Here is proof".
And when some other people come up with measurements and numbers stating the opposite, it is "Your measurement procedure is biased => your equipment is not enough => you have no knowledge".

Under these circumstances, it is really difficult to tell who "really knows it's stuff" and depending on results of measures given by one or the other is highly questionable.

So much for science...

It is really simple.

Either one measures several aspects with sufficiently good equipment (your average computer soundcard is not good enough) and have knowledge on how to measure properly ... or you use your ears.
When measuring you have to understand measuring limits and the used equipment and how to ensure signals are within required levels.

In the latter case (using the ears as instrument), when it is for truth-finding, you need to level match within 0.1dB (1%) under load and the one doing the testing must never know which devices you are comparing. On top of that you need to rule out chance of guessing correctly. This means statistically enough 'attempts' have to be made. It is also essential that each 'attempt' is random.

It is not that people doubt your ears or gears nor that you have to have knowledge about electronics to do a proper comparison.
What's been questioned is the method used to arrive at conclusions.
There are quite a few veterans here that have fallen in the trap (and still do) when not all 'rules', or should I say 'laws' are used.
Omit one of them and the conclusions drawn are bound to be incorrect.

Leave out level matching within 0.1dB (requires measurement equipment) and people WILL select the slightly louder one as preferred.

Leave out the 'under load' requirement and level differences may become bigger than the required 0.1dB.

Leave out the 'not knowing part' and your brain will hear things that aren't there. I know this is the hardest pill to swallow afterall you hear it right ?
Fact is, and many have tested this, even when one is told or thinks something is different/has changed/is playing one can actually hear differences even when in reality nothing actually changed. It is a real phenomenon and merely thinking you know this doesn't make it disappear.
Experienced testers know this.

Leave out the random part and you will again 'know' what is playing. During the test one must never know whether one listens to A or B and must use the ears only.

Leave out the statistically valid' part and you have a high percentage of guessing correctly. You need at least 10 to 20 times to determine this.
opponents say that this induces stress. It does strain you because you are trying very hard to listen out for 'something' you don't know what.
Here it helps IF you know what to listen for. Experience is the game.
Then opponents say... well the time is too short, you need long term tests. The best part is you can do the 'blind' test over days, weeks, months if needed. And that works great.

Break 1 or more 'laws' and your comparison is usually flawed and you pick your preference incorrectly when you want to do 'truth finding'.
When you just want to 'select something' you feel works for you then you can break the rules.
In this case it isn't about really wanting to know but to select something for yourself.

This we see here all day long. People telling they can hear differences that are so clear they don't even have to apply all of the rules or cut the rules slack. It is painfully obvious they are just venting an opinion and inquiries always show they tested incorrectly. Call that experience.

It is fine to have a preference, we all do, but that doesn't make it a truth. people regard it as truth.

 

[Wombat]

Hi,

I can get that. Especially as a lawyer I have a hint of knowledge about burden of proof

But it seems that when some people give proof with numbers and measurements, it is "Here is proof".
And when some other people come up with measurements and numbers stating the opposite, it is "Your measurement procedure is biased => your equipment is not enough => you have no knowledge".

Under these circumstances, it is really difficult to tell who "really knows it's stuff" and depending on results of measures given by one or the other is highly questionable.

So much for science...

Science and law have differences in this regard.

Knowing who knows their stuff usually comes down to peer review rather than lay opinion - I think this also applies in law and other professions, too.

Science: The onus is usually on the the person making the initial claim to justify it(unless they have established credibility in this regard). This typically means supporting it with existing credible supporting scientific truth(consensus). i.e. the truth is usually available for reference.


Law: The defendant is assumed to be guilty until proved innocent and this makes the process one of establishing credible evidence for a trial result.

Some of the evidence is not as solid as in science, e.g. witness recollection, physical appearance descriptions, speed estimation, weapon or shadow. Assumptions, vagaries, prejudices, legal precedents and more are involved. Is it not a duty of a lawyer to blur or hold back truth if this benefits the client?

Wading through legal interpretations on this topic is a nightmare. FWIW here is a simplification.

https://effectiviology.com/burden-of-proof/

 

[LTig]

It is really simple.

Either one measures several aspects with sufficiently good equipment (your average computer soundcard is not good enough) or you use your ears.

In the latter case, when it is for truth-finding, you need to level match within 0.1dB (1%) under load and the one doing the testing must never know which devices you are comparing. On top of that you need to rule out chance of guessing correctly. This means statistically enough 'attempts' have to be made. It is also essential that each 'attempt' is random.

It is not that people doubt your ears or gears nor that you have to have knowledge about electronics to do a proper comparison.
What's been questioned is the method used to arrive at conclusions.
There are quite a few veterans here that have fallen in the trap (and still do) when not all 'rules', or should I say 'laws' are used.
Omit one of them and the conclusions drawn are bound to be incorrect.

Leave out level matching within 0.1dB (requires measurement equipment) and people WILL select the slightly louder one as preferred.

Leave out the 'under load' requirement and level differences may become bigger than the required 0.1dB.

Leave out the 'not knowing part' and your brain will hear things that aren't there. I know this is the hardest pill to swallow afterall you hear it right ?
Fact is, and many have tested this, even when one is told or thinks something is different/has changed/is playing one can actually hear differences even when in reality nothing actually changed. It is a real phenomenon and merely thinking you know this doesn't make it disappear.
Experienced testers know this.

Leave out the random part and you will again 'know' what is playing. During the test one must never know whether one listens to A or B and must use the ears only.

Leave out the statistically valid' part and you have a high percentage of guessing correctly. You need at least 10 to 20 times to determine this.
opponents say that this induces stress. It does strain you because you are trying very hard to listen out for 'something' you don't know what.
Here it helps IF you know what to listen for. Experience is the game.
Then opponents say... well the time is too short, you need long term tests. The best part is you can do the 'blind' test over days, weeks, months if needed. And that works great.

Break 1 or more 'laws' and your comparison is usually flawed and you pick your preference incorrectly when you want to do 'truth finding'.
When you just want to 'select something' you feel works for you then you can break the rules.
In this case it isn't about really wanting to know but to select something for yourself.

This we see here all day long. People telling they can hear differences that are so clear they don't even have to apply all of the rules or cut the rules slack. It is painfully obvious they are just venting an opinion and inquiries always show they tested incorrectly. Call that experience.

It is fine to have a preference, we all do, but that doesn't make it a truth. people regard it as truth.

One posting more which deserves to be sticky.

 

[mansr]

But it seems that when some people give proof with numbers and measurements, it is "Here is proof".
And when some other people come up with measurements and numbers stating the opposite, it is "Your measurement procedure is biased => your equipment is not enough => you have no knowledge".

If you're referring to the graphs posted in this thread yesterday, there is obviously something wrong with the guy's equipment or methods. Failure to recognise this suggests a lack of experience/competence. Still, his biggest mistake was not producing these bad results, it was the flat-out refusal to consider the possibility that something might be wrong.

 

[mansr]

OK. So it is not equipment than everyone can afford..
But it seems a lot of people in here have one or have access to one.

An Audio Precision analyser is very expensive, and only a few people here have (access to) one. I have only a $500 Tascam ADC. It gives a reasonable idea of what's going on up to a SINAD of 100 dB or so. Above that its own distortion starts taking over, though it is still useful for detecting non-harmonic frequency spikes.

 

[PenguinMusic]

If you're referring to the graphs posted in this thread yesterday, there is obviously something wrong with the guy's equipment or methods. Failure to recognise this suggests a lack of experience/competence. Still, his biggest mistake was not producing these bad results, it was the flat-out refusal to consider the possibility that something might be wrong.

Hi,

Perfectly understood.

Now let's assume the guy has the requested skills, and super high quality equipment and follows the methods by the book and still has the same results ?

What then ?

Regards.

 

[mansr]

Now let's assume the guy has the requested skills, and super high quality equipment and follows the methods by the book and still has the same results ?

What then ?

Then the DAC he's testing is broken. We know the D30 isn't that bad.

 

[LTig]

Hi,

Perfectly understood.

Now let's assume the guy has the requested skills, and super high quality equipment and follows the methods by the book and still has the same results ?

What then ?

Regards.

Then he should send the DUT to @amirm for a check how it measures there. If the results are similar the DUT is probably broken.

 

[PenguinMusic]

Then the DAC he's testing is broken. We know the D30 isn't that bad.

Hi,

OK. That is all I wanted to know... Perfectly clear.

Regards.

 

[cjm2077]

Hi,

Perfectly understood.

Now let's assume the guy has the requested skills, and super high quality equipment and follows the methods by the book and still has the same results ?

What then ?

Regards.

If the results are similar to other results doing the same test, then you just accept them. If they are considerably different from results other people have produced, then the guy testing may want to repeat the test, and a second person with a similar setup will be asked to repeat the test with the same conditions. You also may share data files so that people can check for issues in processing. Replication is an essential part of peer review.

 

[solderdude]

Basically I believe it’s because they don’t like me.

That's not it (at least for me). It is the test method(s) I think needs to be addressed.
I would urge you to set aside your convictions and just start testing using proper controls.
It will (or at least should) bring you insights you never new existed.

I have been on both 'sides'. I made my choice based on what I was able to hear with bias eliminated as much as possible.
Learning audibility levels of myself (and occasionally others) was eye opening.

 

[chris0202]

That's not it (at least for me). It is the test method(s) I think needs to be addressed.
I would urge you to set aside your convictions and just start testing using proper controls.
It will (or at least should) bring you insights you never new existed.

I have been on both 'sides'. I made my choice based on what I was able to hear with bias eliminated as much as possible.
Learning audibility levels of myself (and occasionally others) was eye opening.

Interesting, but I cannot learn anything from you like I did from Jan Didden I am afraid. I have been using Neurochrome HP2 to compare DACs (2 input, 1 button and source switched). I know they can be different.

In the end of the day, I can spend hours doing the same thing over and over, and always some quick key strokes turned up new "errors" that pushed everything back to ground zero. Unequal amount of investment in energy, I burn out first

I was just curious when would someone said I changed setting deep within software menus between tests to manipulate result. I got a "broken" ADC is the closest by far.

 

[mansr]

Interesting, but I cannot learn anything from you like I did from Jan Didden I am afraid. I have been using Neurochrome HP2 to compare DACs (2 input, 1 button and source switched). I know they can be different.

In the end of the day, I can spend hours doing the same thing over and over, and always some quick key strokes turned up new "errors" that pushed everything back to ground zero. Unequal amount of investment in energy, I burn out first

I was just curious when would someone said I changed setting deep within software menus between tests to manipulate result. I got a "broken" ADC is the closest by far.

If you still have the recordings, could you post them here? Then people could analyse them to their heart's content without bothering you.

 

[mansr]

No, I am busying deleting all my posting from the forum.

Not so confident after all.

 

[Jimbob54]

No, I am busying deleting all my posting from the forum. Maybe I will leave that standing out for solderdude.

Screenshot them all, dude.

 

[solderdude]

Interesting, but I cannot learn anything from you like I did from Jan Didden I am afraid.

You don't have to learn anything from me.
All I hope is that you consider using a more scientific way of testing.
It is really eye opening and sobering when you do.

I have no idea what Jan Didden has to do with it though.

 

[solderdude]

Just decided not to get along with the forum but in a passive way

Why not engage in conversations with not so like-minded persons in order to expand every-ones understanding.
Tell us WHY you test sighted and choose to ignore certain controls that are deemed essential to prevent forms of bias.

 

[LTig]

Basically I believe it’s because they don’t like me, so they keep picking on me not getting the same result as Amir’s and ignored those differences.

Certainly not. ASR is a science based community which means that measurement results of a DAC should be equal (within tolerances) regardless of who obtained the results. If this is not the case we must search for the cause: different conditions, different measurement instruments, broken DAC, broken instruments, what ever. You've got quite a few hints what could be wrong with your measurment (here, here, here) but unfortunately you chose to ignore those helpful postings.

Let me try once more. What you showed here is certainly a clipped signal, either on the playback chain or the record chain side or somewhere in between. To get rid of clipping you can repeat the test using a test signal with 10 dBFS less amplitude than used before. If this removes most of the higher harmonic distortion components you can start to check where clipping occurs.

And you should always run the ADC with 24 Bit sampledepth and preferably at a higher samplerate than the DAC under test. Only then you are able to see out of band noise and get meaningful results for SINAD (the ADC needs to have 10 dB higher SINAD than the DAC, as I have learned when I posteds my first measurements). This means when @amirm measures components at -130 dB for this DAC your ADC must be equal or better than -140 dB, because the distortion components of DAC and ADC add up.

 

[chris0202]

Why not engage in conversations with not so like-minded persons in order to expand every-ones understanding.
Tell us WHY you test sighted and choose to ignore certain controls that are deemed essential to prevent forms of bias.

I have problems with your way of communication and working things out. That's about it.

ASR is actually a fun place. You guys have 180-degree thinkings with people in SBAF/Head-FI, but 360-degree doings. I am here mostly just to observe how people react. That concludes. I won't ask for ban though, I need the account alive for comunication in private.

 

[chris0202]

Certainly not. ASR is a science based community which means that measurement results of a DAC should be equal (within tolerances) regardless of who obtained the results. If this is not the case we must search for the cause: different conditions, different measurement instruments, broken DAC, broken instruments, what ever. You've got quite a few hints what could be wrong with your measurment (here, here, here) but unfortunately you chose to ignore those helpful postings.

Let me try once more. What you showed here is certainly a clipped signal, either on the playback chain or the record chain side or somewhere in between. To get rid of clipping you can repeat the test using a test signal with 10 dBFS less amplitude than used before. If this removes most of the higher harmonic distortion components you can start to check where clipping occurs.

And you should always run the ADC with 24 Bit sampledepth and preferably at a higher samplerate than the DAC under test. Only then you are able to see out of band noise and get meaningful results for SINAD (the ADC needs to have 10 dB higher SINAD than the DAC, as I have learned when I posteds my first measurements). This means when @amirm measures components at -130 dB for this DAC your ADC must be equal or better than -140 dB, because the distortion components of DAC and ADC add up.

Thanks for the input. But I am tired. I will get the thing shared elsewhere if I have the mood some other day.

That frequency response analysis is meaningless for me on this to begin with. I just wanted to show 2 DACs can have difference. Not that I can get the same results with Amir.