Master Thread: Are measurements Everything or Nothing?

[Aleksandar RS]

You keep repeating this, but refuse to accept what science defines as "proper testing", e.g. properly implemented blind testing that can be independently verified.

Are you not aware of the large body of science that provides massive evidence that sighted listening comparisons are not reliable for detecting sonic differences in audio systems?

Are you really that clueless about the well researched and documented concept of confirmation bias?

I am sure that the test that I did certainly does not meet high standards. Based on everything I've heard, let's end these discussions, and buy a DAC of max 100 Euros, and close all those companies that make more expensive ones. As far as I understand they are all scammers, because all DACs have the same sound. That's what almost everyone claims here. I sincerely hope the same is not true of amplifiers and speakers. I'm afraid to ask.

 

[antcollinet]

Was the signal amplitude level matched, and how was the test blinded?

And how many times out of 10 were you able to reliably detect which DAC was playing.

 

[antcollinet]

I am sure that the test that I did certainly does not meet high standards. Based on everything I've heard, let's end these discussions, and buy a DAC of max 100 Euros, and close all those companies that make more expensive ones. As far as I understand they are all scammers, because all DACs have the same sound. That's what almost everyone claims here. I sincerely hope the same is not true of amplifiers and speakers. I'm afraid to ask.

There is more than sound quality to the choice. There's features, build quality, after sales service, aesthetics, location of supply - etc etc. But yes - once you've got beyond the price level justified by the sum total of all these things, you are certainly flirting with the borders of scam territory.

 

[Blumlein 88]

I am sure that the test that I did certainly does not meet high standards. Based on everything I've heard, let's end these discussions, and buy a DAC of max 100 Euros, and close all those companies that make more expensive ones. As far as I understand they are all scammers, because all DACs have the same sound. That's what almost everyone claims here. I sincerely hope the same is not true of amplifiers and speakers. I'm afraid to ask.

Well the big thing is precise level matching.

 

[Killingbeans]

Based on everything I've heard, let's end these discussions, and buy a DAC of max 100 Euros, and close all those companies that make more expensive ones.

Hell no.

Build quality, reliablity, aesthetics, tactile feel, user interface, customer support, warranty, availability. Those are all things that give a need for variety.

As far as I understand they are all scammers, because all DACs have the same sound. That's what almost everyone claims here.

Nope, that's not the claim. Some DACs have a sound because they are badly designed. Some have a sound because the designer baked in some tonality and tried to sell it as greater transparency.

But searching for audible "personality" in honest well designed DACs is a waste of time, IMO.

I sincerely hope the same is not true of amplifiers and speakers. I'm afraid to ask.

No need for fear. Now we're entering the realm of actual night and day differences, at least when talking about speakers.

 

[HarmonicTHD]

I am sure that the test that I did certainly does not meet high standards. Based on everything I've heard, let's end these discussions, and buy a DAC of max 100 Euros, and close all those companies that make more expensive ones. As far as I understand they are all scammers, because all DACs have the same sound. That's what almost everyone claims here. I sincerely hope the same is not true of amplifiers and speakers. I'm afraid to ask.

Your “tests” are simply invalid as others have explained multiple times. And the control and level matching applies to the test of other audio components too, practically everytime you need to exclude human biases tainting your tests.

And yes beyond a certain measurable performance there is very very little evidence that amps do not sound transparent too unless broken. So I guess we are all in for another stupid round of claims without proof.

Speaker not (yet), too many variables in their room interactions and lack of scientific studies to correlate all to perceptions. But certain plenty of scientific tests that flat anechoic is preferred. But what am I saying, all of that won’t be valid to you which begs the question why are you here? Trolling?

 

[voodooless]

It’s always funny how some random secret unmeasurable thing can cause huge differences in sound, yet something simple as a difference in level is brushed aside as being unimportant

 

[VintageFlanker]

As far as I understand they are all scammers, because all DACs have the same sound.

Well, you first need to give ASR a proper reading...

Science says that "all transparent DACs sound the same", ≠ "all DACs sound the same". Try to understand the difference before claiming such BS about this place.

In addition, "sound" is hopefully not the only factor when buying an audio product. There're quite a few differences between a Topping D10S, a RME ADI-2 PRO and a Matrix Element X2...

I sincerely hope the same is not true of amplifiers and speakers. I'm afraid to ask.

That all speakers sound the same? Why on earth they would? Since I see the "speakers measuring the same but sounding different" thing incoming... By anticipation, you should watch this video of Erin:

Amplifiers are a tricky topic. You may find two transparent amps that sound the same at some very specific level, but it's not that simple: clean power costs money. That's about it.

 

[HarmonicTHD]

Well, you first need to give ASR a proper reading...

Science says that all transparent DACs sound the same, ≠ all DACs sound the same. Try to understand the difference before claiming such BS about this place.

In addition, "sound" is hopefully not the only factor when buying an audio product. There's quite a few differences between a Topping D10S, a RME ADI-2 PRO and a Matrix Element X2...


That all speakers sound the same? Why on earth they would?! Since I see the "speakers measuring the same but sounding different" thing incoming... By anticipation, you should watch this video of Erin:

Amplifiers are a tricky topic. You may find two transparent amps that sound the same at some very specific level, but it's not that simple: clean power costs money. That's about it.

You are correct.

Given through how many circles people here went through with him trying to explain. I very much get the impression he simply doesn’t want to be educated and rather sticks to his faulty beliefs. Nothing in the world one can do about that.

 

[presence]

So you agree? We ought to be critical of manufacturers who don't substantiate their claims yet sell products for good money?

Yes, of course I do agree that data and results should be seen from objective point, but ears are subjective tools, which make you or brake you as a mastering engineer.

The point is something is still missing from audio AI solutions, which can be added by experienced engineer with ears. Otherwise the engineers are not needed. Still most prefer the engineers.

I would be happy to have a company with AI only and I do benefit from it in daily basis. Science is interesting, because it is continious process and fits well for people who are never satisfied : )

 

[presence]

And I it is almost impossible to measure that DACs sound different, but still I do prefer AKM over ESS. Dont ask me why, it is something intuitive, which maybe cant be measured.

 

[antcollinet]

And I it is almost impossible to measure that DACs sound different, but still I do prefer AKM over ESS. Dont ask me why, it is something intuitive, which maybe cant be measured.

The "something intuitive that can't be measured" is almost certainly your own cognitive biases in play. That is not a criticism - everyone is subject to these, no-one is immune. It is simply a part of being human: It is how we are built.

 

[pma]

It’s always funny how some random secret unmeasurable thing can cause huge differences in sound, yet something simple as a difference in level is brushed aside as being unimportant

I do not think it makes any sense to persuade someone that he is wrong in his attitude to sound quality subjective testing. We all need to take the path ourselves.

Measurements are an excellent aid but there are not many who are able to make a useful interpretation of measurements. However, some people are rather willing to learn about measurements than those who believe in their sighted, uncontrolled tests only.

Listening is of course important and is the final goal, however listening tests have to be done in a reasonable way.

 

[Sokel]

As far as I understand they are all scammers, because all DACs have the same sound. That's what almost everyone claims here. I sincerely hope the same is not true of amplifiers and speakers. I'm afraid to ask.

DAC's (new ones) sounds the same,yes,but conditions matters as well.Level,music material (see the horror of intersample overloads in Multitone thread),poor set-up with devices downstream,(even the type of file,as for example some handles PCM different than DSD),etc,can make a big difference in performance.
That's why a good sanity check is measuring the chain as is.

 

[bodhi]

I very much get the impression he simply doesn’t want to be educated and rather sticks to his faulty beliefs.

What I'd like him to understand is that nobody is asking him to give up his beliefs. I'm pretty sure most of ASR folk don't give rat's ass if somebody wants to spend money on cables and such. It's not about DACs, speakers, beliefs or personal preferences.

He hit a pain point when he suggested that it might not be honest to tell people that well measuring DACs sound the same but it would be better to encourage people to try them anyway. The same thing was on in B&W vs KEF thread as in should speakers measuring flat in anechoic chamber always be recommended instead of speakers not measuring as well, all other performance things being about equal.

There are many forums where this kind of scientific argumentation will get you banned and I guess many here have vacated them for that reason.

 

[ahofer]

I think some of the posts become hysterical. To be perfectly clear, I am not selling any audio component and I am not persuading people to buy anything. I suggest to people that it is good to try something, even something different, to hear or not the difference in sound, before buying (comparing that to sick people and homeopathic medicines, for God's sake).

All analogies are flawed; some are useful.

 

[TheBatsEar]

Fundamental to this discussion is the fallacy that brains are reliable instruments.
I can only guess that some people believe that because it fits their intuition that "what one sees and hears must be real".

Here are some pictures and sounds that show that our brains are not reliable and make stuff up all the time, every day, every minute, every second.

An impossible 3D object in our brain:


Movement where there is none:



Grey points appear but there are none:


Not only optical, but all sensors are subject to artifacts that stem from the need for high data compression and fast pattern recognition.
Put your hand in ice for a minute, then in very cool water. The cold water will feel warm.

That includes audible illusions as well.

What you see will change what you hear (this goeas for DAC bling as well, a Wadia must sound better):

An experiment that can be done with speakers:

A few illusions that work with headphones:

Again, the fundamental problem is that some people think the sensory information they get is reliable when it's not. These people lack the ability to acknowledge that their brains are not reliable instruments. The results of a listening test has more to do with blood sugar, amount of sleep we had last night, and what we see, than with anything else.


Also, some people have to be special and just don't get it. And sadly, some simply harvest attention.

 

[Aleksandar RS]

Hell no.

Build quality, reliablity, aesthetics, tactile feel, user interface, customer support, warranty, availability. Those are all things that give a need for variety.



Nope, that's not the claim. Some DACs have a sound because they are badly designed. Some have a sound because the designer baked in some tonality and tried to sell it as greater transparency.

But searching for audible "personality" in honest well designed DACs is a waste of time, IMO.



No need for fear. Now we're entering the realm of actual night and day differences, at least when talking about speakers.

If you think that build quality, reliability, aesthetics, tactile feel, user interface, customer support, warranty, availability (without changing the sound) can make the difference between $100 and over $70,000, then you are very naive. For the speakers, I assumed you meant they didn't all sound the same (thank god that was confirmed). But it seems to me from the answer that this is not the case with amplifiers, I hope I misunderstood.

 

[HarmonicTHD]

If you think that build quality, reliability, aesthetics, tactile feel, user interface, customer support, warranty, availability (without changing the sound) can make the difference between $100 and over $70,000, then you are very naive. For the speakers, I assumed you meant they didn't all sound the same (thank god that was confirmed). But it seems to me from the answer that this is not the case with amplifiers, I hope I misunderstood.

Please explain where you and how you detected differences in amps?
Be specific please eg amp model and facts please.

 

[Aleksandar RS]

All analogies are flawed; some are useful.

And some are not, and some are morbid.