Master Thread: Are measurements Everything or Nothing?

[welwynnick]

I can't hear any differences in noise or distortion between them but I definitely hear differences in sound stage between the different DACS and will pass any blind test in my room with tracks I know very well.

This bit rings true with me. Years ago I used to have a huge turnover in audio equiment, and did lots of AB comparisons: DACS, processors, pre-amps, power amps, and I wrote them all up (sometimes doing myself no favours). There were two occasions when things sounded the same, but in general the differences were usually subtle, but with technique and perseverence* they were readily distinguishable.

Things like noise and distortion were rarely the discriminator, tonally most equipment sounds clean and uncoloured, as it should. What always gave it away was the stereo imaging or soundstage. Some equipment let you think you were looking through a clean window with everything in view and in clear focus, and with full depth of field. Even with 16 bit audio, the soundstage can be deep, wide and tall, with everything laid out with stable and precisely defined locations. Other equipment made it seem like you were viewing a scene through an obscured bathroom window. It really made it quite easy to distinguish equipment.

* technique and perseverence? This means endlessly listening to a very short piece of music over and over again many times. Some people say you have to listen for a long time to judge something. I don't agree with that at all, that just makes everything much more difficult. Movie soundtracks are useless. Short repeated music clips makes it all much easier.

How do you measure stereo imaging, or how do you know what measurements to look for?
I'm not sure, but I presume it boils down to linearity and dynamic range - simply a matter of fidelity, in other words.
Whatever, I'm completely confident that we can measure it. Whatever it is.

 

[antcollinet]

I've heard this quote before somewhere....

Probably here - I use it as often as needed.

 

[rs172]

Objective measurements are not subject to opinion. Whether we like it or not, those two things are in different worlds.

Let's take space exploration, just for one instance. A tremendous number of experiments and analyses over many years have given us:
1) the ability to use a rocket to orbit the earth
2) the ability to control the rocket
3) the ability to carry a payload on that rocket
4) the ability to escape the earth's gravity and travel throughout the solar system, and
5) the ability to communicate with the satellites sent forthwith, and gather information in various forms (including photographs).

Opinion has no place in these accomplishments. Regarding a rocket vector, no one has said, "Well, in my opinion this thruster should fire at .08 seconds, not .04 seconds." Instead, the correct time to fire the thruster, the correct amount of thrust, was scientifically calculated and implemented with authority ... or else the rocket flight failed. Opinion had no place in that implementation.
The same goes for communication with satellites and the rovers on Mars. Opinion has no place in that implementation. Either the process is deeply understood and correctly implemented or the process doesn't work. This includes photography from the outer planets and photography and audio (!) from Mars.
If someone were to say, "Well, in my opinion they should aim that antenna a little bit to the left", what would you think of that?

When it comes to scientific implementation, there is no room for opinion. That means that there is also no room for opinion in the survey of the different foundational sciences that led up to the successful implementation. In other words, if there is no room for opinion in the successful communication with faraway satellites, then there is no room for opinion regarding the science of solid-state circuitry that makes it possible, the science of time analysis that makes it possible, nor the analysis of pressure gradients that make it possible. If any one of those foundational sciences were faulty, the final implementation (the communication with the satellite) would not be successful.

So the fact that we can communicate with (and control) faraway satellites verifies and validates all the scientific principles that have contributed, for years and years, to the final process.

The lowly audio hobby that we enjoy today is governed by the use of the same scientific principles. The implementation of solid-state circuitry that is successful in interplanetary communication is the same implementation that exists in our DACs, our amplifiers and our streamers. It's also what enables us to use computers and cell phones.
The time analysis for interplanetary communications is the same time analysis that allows us to implement digital data recording and playback.
Which brings up another point; today, successful communications with satellites is performed with digital forms of communication. As I said before, the successful implementation of final use validates the foundational principles upon which it rests. So successful digital communications, either with cell phones, streamers or with interplanetary satellites, validates both the principles and uses of digital technology.

I would like to say that it isn't rocket science, but I would be wrong. It IS rocket science ... and more. Much more.

That does not mean that there is no room in our lives for opinion. Of course there is! But ... opinion, correctly used, is simply an expression of the degree to which we like or dislike something. In that, it is valid.
As a comment on the degree to which scientifically proven processes have been verified or are valid, opinion has absolutely no place. It occupies another world.

Hope this helps.

Jim

Thankyou Jim for your detailed explanation!

 

[solderdude]

Of course I'm willing, the only problem is that I think measurements don't cover everything.
I think no one should be at the end of the spectrum between subjective hearing and objective measurements. I think its a bit of both.

You are correct, 'measurements' don't cover everything.
A: Not everything that should be measured, to get a good idea of electrical performance, IS always measured
B: Not all measurements are always done correctly or with enough precision
C: Not everyone can actually understand the complexity of multiple measurements and have enough knowledge of perception
D: Measurements in the acoustical plane are very limited and indicative at best
E: At home the circumstances may differ substantially from lab conditions
.... to name but a few...

That, however, does not excuse anyone to test with rigor and correctly (blind and level matched with statistical relevance) nor to distrust measurements.

 

[rs172]

This bit rings true with me. Years ago I used to have a huge turnover in audio equiment, and did lots of AB comparisons: DACS, processors, pre-amps, power amps, and I wrote them all up (sometimes doing myself no favours). There were two occasions when things sounded the same, but in general the differences were usually subtle, but with technique and perseverence* they were readily distinguishable.

Things like noise and distortion were rarely the discriminator, tonally most equipment sounds clean and uncoloured, as it should. What always gave it away was the stereo imaging or soundstage. Some equipment let you think you were looking through a clean window with everything in view and in clear focus, and with full depth of field. Even with 16 bit audio, the soundstage can be deep, wide and tall, with everything laid out with stable and precisely defined locations. Other equipment made it seem like you were viewing a scene through an obscured bathroom window. It really made it quite easy to distinguish equipment.

* technique and perseverence? This means endlessly listening to a very short piece of music over and over again many times. Some people say you have to listen for a long time to judge something. I don't agree with that at all, that just makes everything much more difficult. Movie soundtracks are useless. Short repeated music clips makes it all much easier.

How do you measure stereo imaging, or how do you know what measurements to look for?
I'm not sure, but I presume it boils down to linearity and dynamic range - simply a matter of fidelity, in other words.
Whatever, I'm completely confident that we can measure it. Whatever it is.

I agree. Especially with part about short pieces of music. Regarding soundstage and imaging measurements, I have no idea

 

[rs172]

You are correct, 'measurements' don't cover everything.
A: Not everything that should be measured to get a good idea of electrical performance IS measured
B: Not all measurements are always done correctly or with enough precision
C: Not everyone can actually understand the complexity of multiple measurements and have enough knowledge of perception
D: Measurements in the acoustical plane are very limited and indicative at best
E: At home the circumstances may differ substantially from lab conditions
.... to name but a few...

That, however, does not excuse anyone to test with rigor and correctly (blind and level matched with statistical relevance) nor to distrust measurements.

Totally agree

 

[JustJones]

Might be more apt to rename this the Broken Record thread.

Every few months new member proclaims measurements don't tell us everything about dacs because they've had X,Y,Z dacs and they sound different.

Did they do proper controlled tests to come to this conclusion?
No

Then there's pages of reasons why proper controlled tests aren't needed because it's obvious these dacs sound different attributing characteristics to dacs that they don't have like soundstage and pinpoint imaging of instruments.

Then hand waving about jitter and distortion even though measurements show the dacs mentioned are way beyond human hearing thresholds.

Why are they here? Oh, to learn while ignoring every piece of well meaning explanations of why what they're claiming isn't possible.

Nobody says you can't have a favorite dac, or you think one sounds better but just leave it at that. Without showing rigorous methods of testing to prove these opinions you're wasting your time.

 

[welwynnick]

That's what this thread is for.

 

[DonR]

I have no idea

yes

 

[rs172]

Might be more apt to rename this the Broken Record thread.

Every few months new member proclaims measurements don't tell us everything about dacs because they've had X,Y,Z dacs and they sound different.

Did they do proper controlled tests to come to this conclusion?
No

Then there's pages of reasons why proper controlled tests aren't needed because it's obvious these dacs sound different attributing characteristics to dacs that they don't have like soundstage and pinpoint imaging of instruments.

Then hand waving about jitter and distortion even though measurements show the dacs mentioned are way beyond human hearing thresholds.

Why are they here? Oh, to learn while ignoring every piece of well meaning explanations of why what they're claiming isn't possible.

Nobody says you can't have a favorite dac, or you think one sounds better but just leave it at that. Without showing rigorous methods of testing to prove these opinions you're wasting your time.

No one mentioned they could hear distortion or noise thus contributing to the way dacs sound different
No one ignored what was written here
But hey you gave your comment and feel strong behind the keyboard.

 

[BDWoody]

Every few months new member proclaims measurements don't tell us everything about dacs because they've had X,Y,Z dacs and they sound different.

Well, since we get 20-30 new members every day, it makes sense that the early learning curve looks very similar.

This is the never-ending thread, so we have a place to move these conversations from the other thousands of threads. Here, we can try to answer those questions in good faith, and engage them in a way that might make them want to learn more, while reducing the 'noise' on the rest of the forum.

We don't move posts here for the posters to be told they don't belong, or to have their motives questioned.

If this thread isn't for you, by all means ignore it.

 

[DonR]

My guess is at best 1 out of 10 want to learn more.

 

[Blumlein 88]

Might be more apt to rename this the Broken Record thread.

Every few months new member proclaims measurements don't tell us everything about dacs because they've had X,Y,Z dacs and they sound different.

Did they do proper controlled tests to come to this conclusion?
No

Then there's pages of reasons why proper controlled tests aren't needed because it's obvious these dacs sound different attributing characteristics to dacs that they don't have like soundstage and pinpoint imaging of instruments.

Then hand waving about jitter and distortion even though measurements show the dacs mentioned are way beyond human hearing thresholds.

Why are they here? Oh, to learn while ignoring every piece of well meaning explanations of why what they're claiming isn't possible.

Nobody says you can't have a favorite dac, or you think one sounds better but just leave it at that. Without showing rigorous methods of testing to prove these opinions you're wasting your time.

+1000. Broken record thread about broken perceptions and my feels.

 

[JustJones]

Well, since we get 20-30 new members every day, it makes sense that the early learning curve looks very similar.

This is the never-ending thread, so we have a place to move these conversations from the other thousands of threads. Here, we can try to answer those questions in good faith, and engage them in a way that might make them want to learn more, while reducing the 'noise' on the rest of the forum.

We don't move posts here for the posters to be told they don't belong, or to have their motives questioned.

If this thread isn't for you, by all means ignore it.

Oh no, it's quite entertaining though I get pages behind.

 

[Newman]

Well, since we get 20-30 new members every day, it makes sense that the early learning curve looks very similar.

All the more reason for there to be an automatic “welcome” message to all new members, outlining a decent introduction to audio science (links as needed), including perception and sighted listening issues. Also listing audio myths that audio science has already effectively debunked, and some sort of note that holding to those myths here with baseless “because I heard it” argumentation and no evidence but plenty of endless repetitions of “I assert, to me, so there”, is not what this site is all about.

The absence of such a welcome message makes the ‘early learning curve’ you mentioned unnecessarily tedious for even the most patient and well-meaning of educators here.

This is the never-ending thread, so we have a place to move these conversations from the other thousands of threads. Here, we can try to answer those questions in good faith, and engage them in a way that might make them want to learn more, while reducing the 'noise' on the rest of the forum.

In good faith yes, for as long as we see good faith in them too. We have all seen, and appreciate, your prompt action once their comments don’t look like good faith to you.

We don't move posts here for the posters to be told they don't belong, or to have their motives questioned.

But of course it is relevant to ask them exactly why they did come here, when they start acting recalcitrant and petulant, if not downright down-their-nose, when their recitations of myths as “I heard it so it’s in the sound waves, fact, end of discussion” are challenged.

 

[Newman]

And please give me some credit that I'm not biased towards:

• It’s a Legendary Ladder DAC (cool:)
• It is specced with Discrete Components (cool:)
• It costs 2X / 4X / 12X more than the other 3 DACs
• The above dot points mean the internet is smothered with subjective rave reviews about it. OMG I'm not kidding.

It’s not about giving you credit. It’s not about “gee you must think I am silly to be influenced by such things”. Our cognitive biases are a combination of conscious and unconscious, the latter especially being unknown to the individual and beyond conscious control. It’s complex and multifaceted. Best to assume that they are there, and are in play. To assume they aren’t happening is completely at odds with a mountain of evidence. That’s why almost 100% of lay audiophiles are completely, cockily arrogant about their ‘sonic perceptions’ being entirely related to attributes of the sound waves themselves, and quick to conclude that ‘the measurements must be missing something that I’m hearing’. The carefully controlled evidence is almost exactly the opposite, and indicate that cognitive biases are typically so strong that they will dominate the ‘sonic perceptions’, not just when the sound waves are almost identical, but even when the sound waves themselves are different at clearly audible levels in controlled tests.

If you think that the subjective rave reviews of the R26 all over the internet are completely unrelated to the dot points I listed, and are reasonable descriptions of the sound waves emanating from the R26 in comparison to other equally competent DACs, then you actually have a bit of learning to do. Exactly what you said you came here for.

It’s a simple fact that all four DACs you mentioned are, if performing to spec and not set to an especially slow filter, audibly transparent. If you want to continue to assert that “well I hear this and that, so it is definitely in the sound waves”, then you have reached the point where the onus is now on you to back that up with rigorous evidence. Because your claims are simply flying in the face of audio science. We don’t do that here lightly. If you reckon you have advanced the science, then great, but don’t just assert, demonstrate.

 

[BDWoody]

All the more reason for there to be an automatic “welcome” message to all new members, outlining a decent introduction to audio science (links as needed), including perception and sighted listening issues. Also listing audio myths that audio science has already effectively debunked, and some sort of note that holding to those myths here with baseless “because I heard it” argumentation and no evidence but plenty of endless repetitions of “I assert, to me, so there”, is not what this site is all about.

That reminds me that we need to revive the FAQ project we got started a while back. @AdamG has already done the heavy lifting on that, so it isn't far from reality.

The absence of such a welcome message makes the ‘early learning curve’ you mentioned unnecessarily tedious for even the most patient and well-meaning of educators here.

I don't disagree, but that is also part of why we try to quickly move these subjects that come up on repeat from the rest of the forum here. I understand that there are some who more enjoy the explaining part, and are able to do it in a way that doesn't make people feel like they've gone from making what is often (not always, I'm not as naïve as I play on TV) a very innocently intentioned comment to being strapped into a chair with the spotlights on, surrounded by questions that we already know they aren't going to have very good answers for.

In good faith yes, for as long as we see good faith in them too. We have all seen, and appreciate, your prompt action once their comments don’t look like good faith to you.

That's always got to be the deal. I don't move posts here if I don't believe that is the starting place. People can express a bit of exasperation without automatically being trolls who are here in bad faith. If they start spouting up all over the forum, that's different and they don't tend to prosper.

But of course it is relevant to ask them exactly why they did come here, when they start acting recalcitrant and petulant, if not downright down-their-nose, when their recitations of myths as “I heard it so it’s in the sound waves, fact, end of discussion” are challenged.

I think it depends. If someone is here, for example mostly to chat about what music they like on the music threads, then made some comment about cables (or whatever) that gets moved here, where they are basically challenged on why they are here, I don't think it matters. I'm happy to have him here talking about music, aren't you? Some tend to overlook their own snark at times when deeming others to be responding inappropriately, and pushing people to answer like they've got a finger in their chest may not be the best way to convince anyone of anything, but that you are someone they don't really want to interact with.

Sometimes, we'd be better off just turning off the spotlights once they've reached saturation point. Sometimes people need time to mentally burn-in to the reality of this place. Giving them things they can think about on their own as they go about their travels may do more than forcing them to either submit or else justify their place.

Anyway, we do need to get on that FAQ.

 

[Mnyb]

I agree. Especially with part about short pieces of music. Regarding soundstage and imaging measurements, I have no idea

This is actually not in the domain of properly designed DAC’s and small signal electronics. So if you heard sound stage differences in DAC’s I think you did it wrong.
That’s not how sh*t works .

And the differences in well designed electronics are so small so all non blind test are by definition invalid and does not provide any information they rest in the “not even wrong category”
I’m sure you heard something but the why can’t be teased out from the bad data and will be forever unknowable .
Was it real was it imagined it can’t be decided, so not even wrong ?

 

[Mnyb]

.. also all off us should remeber that there is nothing special about audio electronics it follows the same rules as every other electronic design Ohms law etc . And there are far more sensitive applications that don’t experience the need for directional cables and quantum purification

 

[welwynnick]

Thank you for the patronising responses.

RS172 works in cyber security, and I work in electronic warfare, so we do actually understand electronic systems and the exacting requirements to test them.