You couldn’t hear the difference - Proof! Beautiful video of David Mellor (Audio Masterclass) about sound differences between DAC

[krabapple]

The smaller the differences (DACs, cables), the more heated the discussion

A nonexistent difference is a very small one, indeed.

 

[Deleted member 90227]

Hello. I cam across this experiment especially the man claiming DACs are scam. That is funny. I wrote to David my response that his experiment and conclusion are invalid. Below are some insights:

Briefly speaking, the test is about switching (inserting cuts) from one source DAC to another and playing the entire sound file in one sequence checking if people can spot the changes as a measure of whether people can hear the difference. As the results did not show people are able to identify the number of cuts, the study concluded that DACs sound the same. I can easily say the data collection method is invalid and below are some supporting insights. I will not show you references but please feel free to explore and validate what I am writing yourself.

There is an entire field of study about psychoacoustics. Let's begin with:

Sequential Perception in Music: We do not hear sound in isolation of what came before. For example, when playing music notes like C and then G notes that keep repeating on a piano keyboard we hear it as falling motion and that is relaxing. However, when we start with G and then C repeating we hear the notes as rising and that brings tension. So, this known order creates psychological effects. Basically the two are the same so if someone misses the first note, that person will hear something else from a person starting listening from the beginning.

Therefore, in music there is auditory context or sequential dependency. The notes we hear color the way we experience the next note. Another example why the same music chord can feel “happy” in one context and “sad” in another — because the preceding material frames it.

The above are proven scientific theories. You can find many papers proving what I am saying. So, your testing method of switching makes people hear the parts of the entire sound file all together. The brain is not designed in this context for some people to spot when a change happens because the change falls within the same time window segment depending when they started listening. The window segment for people is between 20-40ms. This is why in DSP, they process an entire file in segments and then when all played, we hear it all together well,

I am not designing a test here, but knowing the above, the AB test should play a part. Before switching, you need to wait more than 40ms like preferably an entire second or two and then play the other DAC with the same music. Also you need to do it the other way around as people still get impacted with the thing they heard before. This is known by sales people and this is why they play the speaker they want to sell you to be the last one. Also, we need to play complex music as here where the brain excels in decoding complex audio signals especially with vocals and many instruments' depth involving harmonics.

People researching DSP and MIR (music information retrieval) are very much aware of the above. I hope this is enough.

 

[Deleted member 90227]

Hello. I cam across this experimant especially the

David Mellor from audio masterclass and Alain Pauls are YouTubers and Audio engineers.
They wanted to find out, if Differences between DAC are audible.

So both of them started polls in which they asked the audience if it is possible to hear, how often they switched between an original audio file and it looped signal with an extra conversion.

They also provided the original wave files, because it’s known that the YouTube soundqualtiy is compressed.

The results were telling: No audible differences at all!


David Mellor has the results now: 505 Participants!
And I have left a comment, as Davis suggested: Beginn your comment with Your test is invalid because……
So I wrote: „Your test is invalid because you have crushed all my audible Illusions!“

Check also out Alain Pauls (AP Masterings) video (same test, same result, 300 Participants)

Let me make my point clear. From one side you are absolutely correct that in the music industry soft scam is so widely spread and most or all DACs in the picture of the original post are in the group. So, a $100K R2R DAC is a scam and when you know how it is designed and what is inside it will be a disaster. Your are buying for $100K 20KG of aluminum with a normal $200 R2R DAC using upsampling and optical conversion which creates jitter. However, this does not mean DACs do not sound different. The sweet spot for a good DAC is around $1000 and if you want to buy Chord Dave for $15K that is fine if you know what is inside.

 

[totti1965]

Hello. I cam across this experimant especially the

Let me make my point clear. From one side you are absolutely correct that in the music industry soft scam is so widely spread and most or all DACs in the picture of the original post are in the group. So, a $100K R2R DAC is a scam and when you know how it is designed and what is inside it will be a disaster. Your are buying for $100K 20KG of aluminum with a normal $200 R2R DAC using upsampling and optical conversion which creates jitter. However, this does not mean DACs do not sound different. The sweet spot for a good DAC is around $1000 and if you want to buy Chord Dave for $15K that is fine if you know what is inside.

Hmmmm, maybe you have a point, but even if I would call myself a just "90 % Objectivist", I would never ever claim, that I could hear the differences between a US$ 200 Dac and a US$ 100.000 DAC in a real randomized and absolutely propper Doubleblind Test!
But: the 10 % of a subjectivist inside myself tells me that a the AKM Chip inside the WiiM Pro Plus sound much better than the SMSL D6s, digitally feeded by the same WiiM Pro Plus. And because of an extremly tiny tiny click between two Tracks -it using the D6s and Tidal - I would perform easy 20 out of 20 even in a real blind test!
But @amirm would call that a little bit of cheating there!
And he would be right

 

[Svend P]

However, this does not mean DACs do not sound different.

How so? If they measure identical, how can they sound differently?

'How can DAC's have a SOUND SIGNATURE if they measure as transparent?'

Oct 4, 2019

I know there are people out there that think cables affect sound, which is much worse, but there really is no response to something like that, but just to smile and nod. But what about people who talk about DACS as if they were headphone drivers or speakers, and talk about the SOUNDSTAGE, IMAGING, and MIDRANGE of a DAC? I actually don't know what to say to people to not be rude. If you try explaining that a DAC isn't something that actually changes the sound, they accuse you of having "a hard-on for measurements", as if it were the measurements themselves that tell you that DACs don't...

• Willhelm_Scream
• Replies: 10,921
• Forum: DACs, Streamers, Servers, Players, Audio Interface

• 

 

[totti1965]

The above are proven scientific theories. You can find many papers proving what I am saying. So, your testing method of switching makes people hear the parts of the entire sound file all together. The brain is not designed in this context for some people to spot when a change happens because the change falls within the same time window segment depending when they started listening. The window segment for people is between 20-40ms. This is why in DSP, they process an entire file in segments and then when all played, we hear it all together well,

I am not designing a test here, but knowing the above, the AB test should play a part. Before switching, you need to wait more than 40ms like preferably an entire second or two and then play the other DAC with the same music. Also you need to do it the other way around as people still get impacted with the thing they heard before. This is known by sales people and this is why they play the speaker they want to sell you to be the last one. Also, we need to play complex music as here where the brain excels in decoding complex audio signals especially with vocals and many instruments' depth involving harmonics.

People researching DSP and MIR (music information retrieval) are very much aware of the above. I hope this is enough.

First I want to say welcome to the audiosciencereview forum!
Can you give us a link to one or two of these papers?
I thought until now, that, as shorter as the time between the switch as better it would be in regard of perceiving differences - and that a difference of 0.00 s would be the optimum....

And: It seems that you have some knowledge....
Are you working in the audio / audio engeneering business?

Curious greetings,

Thorsten

 

[Deleted member 90227]

How so? If they measure identical, how can they sound differently?

'How can DAC's have a SOUND SIGNATURE if they measure as transparent?'

Oct 4, 2019

I know there are people out there that think cables affect sound, which is much worse, but there really is no response to something like that, but just to smile and nod. But what about people who talk about DACS as if they were headphone drivers or speakers, and talk about the SOUNDSTAGE, IMAGING, and MIDRANGE of a DAC? I actually don't know what to say to people to not be rude. If you try explaining that a DAC isn't something that actually changes the sound, they accuse you of having "a hard-on for measurements", as if it were the measurements themselves that tell you that DACs don't...

• Willhelm_Scream
• Replies: 10,921
• Forum: DACs, Streamers, Servers, Players, Audio Interface

• 

The measurement only gives an indication because as we know most measurements focus on measuring THD at 1KHZ. So two DACs can produce the same measurements and they sound very different.

Music and human hearing is complex as the brain is a complex organ. Read about the complexity theory is interested. Such complexities open the door for scammers filling the audio industry by producing products that are in many cases haing inferior performance and people buy them for many reasons.

So, it is always true, to stop spending once you do not hear the difference. And enjoy the music. At the end we hear the music through our brain and we can enjoy it on a small portable speaker as well as on a massive 1 million dollar sound system. Sometimes a small speaker is more pleasant.

 

[Deleted member 90227]

First I want to say welcome to the audiosciencereview forum!
Can you give us a link to one or two of these papers?
I thought until now, that, as shorter as the time between the switch as better it would be in regard of perceiving differences - and that a difference of 0.00 s would be the optimum....

And: It seems that you have some knowledge....
Are you working in the audio / audio engeneering business?

Curious greetings,

Thorsten

I have a PhD and doing some work on MIR which is music information retrieved. I am originally a computer scientist with MS in computer science. I will send youb some research material on the subject. Also, feel free to explore how human hearing works. It is fascinating.

 

[RandomEar]

The measurement only gives an indication because as we know most measurements focus on measuring THD at 1KHZ. So two DACs can produce the same measurements and they sound very different.

A typical measurement on ASR incorporates much more:

• 1 kHz THD+N
• Dynamic range
• Frequency reponse
• Intermodulation distortion
• Jitter
• 32 tone test
• Linearity
• Filter response
• Wideband distortion

This set of tests characterizes a DAC very well. If two devices perform identical in these measurements (within measurement precision), what could cause them to still perform / sound different using the same source material, level and filter settings?

There have also been multiple tests where two DAC outputs were captured using a high end audio interface and the resulting traces were compared using DeltaWave. As far as I remember, the results turned out as boring as expected. I'm not saying the test in this thread is a good test. But in general, audio DACs are a solved problem and with the exception of some rather niche manufacturers producing obscure or outdated designs (R2R, DACs with tubes and such), the vast majority of DACs past 100$ should be audibly transparent. Suggesting that you need to invest 1000$ to reach that level seems really odd to me.

 

[dominikz]

The measurement only gives an indication because as we know most measurements focus on measuring THD at 1KHZ.

Please look at any DAC review on ASR and you will see much, much more is measured than just THD at 1kHz. It is similar with other reviewers that focus on data (e.g. Archimago, L7Audiolab etc).

So a question to you, if e.g. two DACs show good results across a wide spectrum of tests, including a perfectly flat 20Hz-20kHz frequency response and ENOB >16bit in the 20Hz-20kHz multitone test, would you really expect them to sound different in a bias-controlled and level-matched listening test?

If so, what exactly would be different?

So two DACs can produce the same measurements and they sound very different.

I hope you agree that if something sounds "very different" we can find some measurement that demonstrates a difference. Do you have any references of formal listening tests supporting your claim with any statistical relevance? As a researcher yourself you must be aware that without data the community here can't take this claim as fact purely on your stated academic credentials.

The above are proven scientific theories. You can find many papers proving what I am saying. So, your testing method of switching makes people hear the parts of the entire sound file all together. The brain is not designed in this context for some people to spot when a change happens because the change falls within the same time window segment depending when they started listening. The window segment for people is between 20-40ms. This is why in DSP, they process an entire file in segments and then when all played, we hear it all together well,

I am not designing a test here, but knowing the above, the AB test should play a part. Before switching, you need to wait more than 40ms like preferably an entire second or two and then play the other DAC with the same music. Also you need to do it the other way around as people still get impacted with the thing they heard before. This is known by sales people and this is why they play the speaker they want to sell you to be the last one. Also, we need to play complex music as here where the brain excels in decoding complex audio signals especially with vocals and many instruments' depth involving harmonics.

Personally, I'm not convinced that you can directly apply findings from MIR to DAC comparisons. First of all, DAC are not musical instruments nor musical pieces - they are sound reproduction devices.
So when you are comparing two DACs you are comparing the exact same music, what changes may be the amount of noise, as well as linear and non-linear distortion.
And there's research demonstrating that fast, gapless switching makes people better at detecting noise and distortion - not worse.

But even if you disagree with this - in an typical ABX test (i.e. not the one in the video in OP) no one forces the listeners to switch quickly. Test participants are typically free to listen to each sample from beginning to end, and make their choice then, or switch quickly.

Regardless of the speed of switching I have yet to see one example of well-executed listening test where two well-measuring DACs were found to sound different with any statistical significance. If such an example exists, I'm of course happy to give it a read!

 

[Deleted member 90227]

A typical measurement on ASR incorporates much more:

• 1 kHz THD+N
• Dynamic range
• Frequency reponse
• Intermodulation distortion
• Jitter
• 32 tone test
• Linearity
• Filter response
• Wideband distortion

This set of tests characterizes a DAC very well. If two devices perform identical in these measurements (within measurement precision), what could cause them to still perform / sound different using the same source material, level and filter settings?

There have also been multiple tests where two DAC outputs were captured using a high end audio interface and the resulting traces were compared using DeltaWave. As far as I remember, the results turned out as boring as expected. I'm not saying the test in this thread is a good test. But in general, audio DACs are a solved problem and with the exception of some rather niche manufacturers producing obscure or outdated designs (R2R, DACs with tubes and such), the vast majority of DACs past 100$ should be audibly transparent. Suggesting that you need to invest 1000$ to reach that level seems really odd to me.

Your writing is one of the most impressive text I read recently on the subject. . I would love to continue the communication on the testing part knowing that I am not into this area as a profession. The last experiment seems interesting because if we are able to go analog and back to digital and produce the same PCM data points in the original file, this will be an end game. So, I would appreciate if you share more highlights on that experience it will be great. I am sure the results will show some errors worth investigating.

I own some equipment like Chord Dave + Mscaler, Chord Hugo 1, Topping D900 and Fiio K17 AND k11 R2R. Each of those DACs sound different and it is easy for me to spot that with some accuracy. The difference is clearer when I use my Focal Utopia or speakers. When using a planner magnetic headphones the differences are trimmed. So, this is about subjective listening. As a scientist I prefer to take myself outside the equation as we people are perhaps the biggest sources of bias.

Eventhough there are differences, however the music is basically also very similar except the R2R which does some coloring. The rest are very close. It seems as you mentioned it is to do more about digital filtration. And hence we bring them closer when we use external digital upsampling and filters. I conclude that a $1000 is sweet spot to buy a good DAC. Spending more is a personal choice but it will not be about batter sound but a preference. Those who put $100k on MSB DAC is a soft scam. I heard one and it sounds worse than all good R2R DACs in the 2 to 5K range.

 

[Purité Audio]

As a ‘scientist’ have you considered unsighted comparison?
Keith

 

[Deleted member 90227]

Please look at any DAC review on ASR and you will see much, much more is measured than just THD at 1kHz. It is similar with other reviewers that focus on data (e.g. Archimago, L7Audiolab etc).

So a question to you, if e.g. two DACs show good results across a wide spectrum of tests, including a perfectly flat 20Hz-20kHz frequency response and ENOB >16bit in the 20Hz-20kHz multitone test, would you really expect them to sound different in a bias-controlled and level-matched listening test?

If so, what exactly would be different?


I hope you agree that if something sounds "very different" we can find some measurement that demonstrates a difference. Do you have any references of formal listening tests supporting your claim with any statistical relevance? As a researcher yourself you must be aware that without data the community here can't take this claim as fact purely on your stated academic credentials.


Personally, I'm not convinced that you can directly apply findings from MIR to DAC comparisons. First of all, DAC are not musical instruments nor musical pieces - they are sound reproduction devices.
So when you are comparing two DACs you are comparing the exact same music, what changes may be the amount of noise, as well as linear and non-linear distortion.
And there's research demonstrating that fast, gapless switching makes people better at detecting noise and distortion - not worse.

But even if you disagree with this - in an typical ABX test (i.e. not the one in the video in OP) no one forces the listeners to switch quickly. Test participants are typically free to listen to each sample from beginning to end, and make their choice then, or switch quickly.

Regardless of the speed of switching I have yet to see one example of well-executed listening test where two well-measuring DACs were found to sound different with any statistical significance. If such an example exists, I'm of course happy to give it a read!

Your points are very interesting and I agree with almost all of them. My feedback was the testing experiment done by cutting chuncks of sound and merging them together is perhaps not accurate. I shared in my previous notes how DAC technology is converging towards accuraatly producing similar outputs. Subjectively speaking, I may assume most of the differences in well design DACs are either due to digital filtration or the DAC topology like R2R, ESS, AKM, FPGAs, ... However many audiophiles still buy exotic equipments at crazy prices because they like distortion which is not wrong. What is interesting is they do not know that

 

[dominikz]

I own some equipment like Chord Dave + Mscaler, Chord Hugo 1, Topping D900 and Fiio K17 AND k11 R2R. Each of those DACs sound different and it is easy for me to spot that with some accuracy. The difference is clearer when I use my Focal Utopia or speakers. When using a planner magnetic headphones the differences are trimmed. So, this is about subjective listening. As a scientist I prefer to take myself outside the equation as we people are perhaps the biggest sources of bias.

Exactly.

From personal experience: a while ago I thought I could hear a clear difference between the DACs of my RME Babyface and Topping E50 when I used them informally.

I knew both measured reasonably well, so to make sure I wasn't imagining things, I level-matched the output of both devices to <0,1dB accuracy and used a switch box to be able to switch between them easily.

Unsurprisingly, all of the difference I thought I heard went away when levels were identical and I didn't know which device was playing. Suddenly both sounded the same.

 

[dominikz]

My feedback was the testing experiment done by cutting chuncks of sound and merging them together is perhaps not accurate.

It is not ideal, for sure.

But I do see some value, as it can help people get some idea of how our mind can easily trick us. Most people won't have the resources or the competence to facilitate a real-life bias-controlled listening test with level-matching so this may be the closest they can get to it.

If a less-than-ideal online listening test triggers skepticism towards audio-voodoo in a few people, and especially if it fuels some scientific curiosity in some of them, then it has IMHO served its purpose well.

 

[Deleted member 90227]

As a ‘scientist’ have you considered unsighted comparison?
Keith

I am loving this community. Thank you guys for bringing such insights.

I appreciate if you please read my other comments as I believe I did address your valuable notes.

One aspect I believe we all agree on is adapting scientific means as guiding principles of our discussios which is awesome. In research we include a theoretical or conceptual frameworks to be the basis through which we do our study. Here, with my little search I came across the following: physiological (psychoacoustics), perceptual (Gestalt/ASA), cognitive (information theory, schema-based), and embodied (movement-based) perspectives—giving a multi-layered view of how humans experience sound and music.

It will be great to explore the above together and align the same with your knowledge and technical expertise. Thanks again.

 

[dominikz]

I am loving this community. Thank you guys for bringing such insights.

I appreciate if you please read my other comments as I believe I did address your valuable notes.

One aspect I believe we all agree on is adapting scientific means as guiding principles of our discussios which is awesome. In research we include a theoretical or conceptual frameworks to be the basis through which we do our study. Here, with my little search I came across the following: physiological (psychoacoustics), perceptual (Gestalt/ASA), cognitive (information theory, schema-based), and embodied (movement-based) perspectives—giving a multi-layered view of how humans experience sound and music.

It will be great to explore the above together and align the same with your knowledge and technical expertise. Thanks again.

It is always great to see new members willing to share knowledge/experiences and engage in good faith. Welcome to the community!

 

[totti1965]

I am loving this community. Thank you guys for bringing such insights.

I appreciate if you please read my other comments as I believe I did address your valuable notes.

One aspect I believe we all agree on is adapting scientific means as guiding principles of our discussios which is awesome. In research we include a theoretical or conceptual frameworks to be the basis through which we do our study. Here, with my little search I came across the following: physiological (psychoacoustics), perceptual (Gestalt/ASA), cognitive (information theory, schema-based), and embodied (movement-based) perspectives—giving a multi-layered view of how humans experience sound and music.

It will be great to explore the above together and align the same with your knowledge and technical expertise. Thanks again.

What always puzzled me about many hardcore objectivists is the belief that this boundary is sharp.
Yet it is evident that even with a 100% placebo effect, where "actually" no differences exist in the physical world, in the brain of the listener who has a certain expectation, whose attention is drawn to certain details in the music that they paid less attention to in the previous listening session, different neurons simply fire, and different neural connections are strengthened or modulated, than in an imaginary placebo-immune listener.
So, kind of in the "real world," namely in the brains and precisely during listening, the aforementioned difference between the $100,000 DAC and the $200 DAC actually exists!!!
Although, if the well-heeled music lover were told the exact opposite while switching channels, namely that he was listening to the $100,000 DAC when the $200 DAC was actually running, that same music lover would actually "hear" these very differences: this time in favor of the supposed €100,000 and the actual €200 DAC.
An electroencephalogram could easily prove this!

I chose the past tense ("made me suspicious") because I realized that this forum is indeed decidedly only about the devices and not about the underlying psychology.
So the boundary is kind of sharp.
And it is not easy to convince my audiophile friends here in Germany.
But thanks god there is also the 10% subjectivist in me, so there is much understanding!

 

[dominikz]

What always puzzled me about many hardcore objectivists is the belief that this boundary is sharp.
Yet it is evident that even with a 100% placebo effect, where "actually" no differences exist in the physical world, in the brain of the listener who has a certain expectation, whose attention is drawn to certain details in the music that they paid less attention to in the previous listening session, different neurons simply fire, and different neural connections are strengthened or modulated, than in an imaginary placebo-immune listener.
So, kind of in the "real world," namely in the brains and precisely during listening, the aforementioned difference between the $100,000 DAC and the $200 DAC actually exists!!!
Although, if the well-heeled music lover were told the exact opposite while switching channels, namely that he was listening to the $100,000 DAC when the $200 DAC was actually running, that same music lover would actually "hear" these very differences: this time in favor of the supposed €100,000 and the actual €200 DAC.
An electroencephalogram could easily prove this!

I chose the past tense ("made me suspicious") because I realized that this forum is indeed decidedly only about the devices and not about the underlying psychology.
So the boundary is kind of sharp.
And it is not easy to convince my audiophile friends here in Germany.
But thanks god there is also the 10% subjectivist in me, so there is much understanding!

To be honest, I don't believe people subscribing to audio science collectively write-away cognitive/perceptual/bias effects in audio - quite the contrary.
In my experience it is usually the traditional "everything matters" audiophiles who swear on their hearing acuity and golden ears, as well as immunity to cognitive and perceptual bias.
Of course no human being can be immune to bias, regardless if they're aware of it or not.

Furthermore, if I buy a 2000$ DAC even if I know it sounds the same as a 200$ one, that is perfectly fine if it makes me happy!
It is like how some people will buy a Rolex even if it tells them the time just as well as a cheap fitness band would.

IMO people should be honest and admit that the audio hobby is not *only* about sound quality and enjoyment of music.
There's no problem with that in principle, as long as people are aware that in many case it is not really the "sound quality" that is changing when they invest more.

 

[mocenigo]

Online tests have their limitations, I acknowledged that in my thread as well. This fact however doesn't make them pointless nor useless.

I never meant that they are pointless. If they show a difference, the difference is proven. But when they do not, there is a larger margin of error (mind you, this might be a purely theoretical one!).

While this is by no means proof that "all DACs sound the same", it should at least make people skeptical whether "different DACs have night-and-day sound signatures".

ACK 99.9%.

Roberto