Master Thread: Are Measurements Everything or Nothing?

[04gto]

Which „things“ ?

My hypothesis would be this or a lack therefore.

Audio Blind Testing - You Are Doing It Wrong! (Video)

Just uploaded a video on the basics of performing controlled tests in audio. It was motivated by saving myself text in having to write all of this down when telling someone who to do these tests right. And as a counter to a few online personalities to keep saying they do "blind" tests yet when...

www.audiosciencereview.com

But hey we might just take your word for it.

(Ps. Just to be clear - no I am not a fan of manufactures, which try to get to other people’s money by telling bogus stories about the miracle performance of their product such as Chord. But neither am I a fan of claims based on anecdotes to the opposite either).

Soundstage, note weight, vocal timbre, vocal realism, micro details, dynamic impact- In direct relation to how an amplifier effects slam or even texture differences between driver types (dynamic, planar etc). We don't need to "just take my word for it". Sometimes it can be explained by impedance matching sometimes not. If one cannot hear, or indeed, sometimes feel the differences between driver types, then I would say they are in the minority.

 

[DavidEdwinAston]

Impressive post 0.
Your last sentence, I think on ASR, you may be wrong.

 

[Robbo99999]

View attachment 235010

Ha, to be honest that's what I aspire to.....but with the knowledge that I'm the weakest (most inconsistent) link in that chain! (Ok, and we'll remove the cable, spike, rack comments!)

 

[Killingbeans]

Yes, I have heard the difference that power products can make in blind tests.

Sorry for being one of those pig-headed indiscriminate naysayers, but...

How was the test blinded? And how high was the statistical significance?

I don't know you, so nothing personal. It's just that I've seen people "interpret" the concept of a blind testing in some really creative (read: useless) ways.

For instance people who think that having the audio gear and test subjects in two different rooms is all it takes to make the test "blind". Or those who think that if you just keep testing, you can stop and call "victory" once you get your desired result enough times in a row, because that somehow negates the risk of random guessing

 

[ahofer]

I believe one of the biggest mistakes made in utilizing subjective assessments - whether audio, visual or tactile - is the belief that other people must, most assuredly, sense the same thing in the same way or to the same degree.

Although one sure fire way to get agreement is for one to suggest the difference to the other (‘listen to that snare!, do you sense the slam?’)

 

[Womaz]

I have a scale for how much measurements matter for each category of products:

DACs: 100%
Amplifiers (headphone and speaker): 80 to 90% due to variability of available power. Hard to internalize how much power is available/enough without listening tests.
Speakers: 70 to 80%
Headphones: 50 to 80% (measurements too variable)

This is why you see me do listening tests for the last two categories and half of second (headphone amps).

This is brilliant and will be my starting point in the future if any purchases are to be made

 

[drewdawg999]

I've been pretty subjective in the past but I'm slowly coming around to being more objective. For one thing, I'm a cheapskate and not rich, so I'm priced out of the high end where subjectivity is so dominant. In the past I wanted high end high dollar stuff so bad, but as the years passed by it's become apparent that I'll never be able to afford that stuff. So I've bought 3 things now on the strength of Amir's reviews and I've been very happy with my purchases. 2 of those have been cheap DACs and it's a bit shocking how good the sound quality is that I'm getting. In the shopping process I read a lot of reviews here and it's slowly sinking in, the meaning behind all these measurements and graphs.

I'm wondering though how my perfectly measuring entry level DAC (SMSL DO100) would compare to a much more expensive DAC, namely the Holo Spring or May. Would it basically sound the same? Or would my DAC even sound better, beating those R2R DACs in SINAD and the multitone test. Or would it just be a small difference that's a matter of taste? How would it compare to something ridiculous, like the 6-figure MSB Select DAC? I'm thinking the differences would be rather small, but I'll never know unless I go audition the stuff with no intention of buying. Something like a Topping D90LE is within my grasp, but why would I when it measures so similar to the DO100? Just wondering, as I'm very happy with the sound I'm getting now.

 

[voodooless]

I'm wondering though how my perfectly measuring entry level DAC (SMSL DO100) would compare to a much more expensive DAC, namely the Holo Spring or May. Would it basically sound the same?

Probably. Even though your DAC will have 10 to 15 dB better performance. The “NOS” mode may be audible due to the HF roll off and added intermodulation. But resample to a high rate first, and that difference will be gone as well.

Something like a Topping D90LE is within my grasp, but why would I when it measures so similar to the DO100? Just wondering, as I'm very happy with the sound I'm getting now.

I wouldn’t bother unless you need some specific features. You may want to try this test:

DAC ABX Test Phase 1: Does a SOTA DAC sound the same as a budget DAC if proper controls are put in place? Spoiler: Probably yes. :)

This thread is meant to once and for all settle the age-old question on whether different non-broken DACs have a "sound" or if all DACs sound the same! :cool: ...just kidding - those debates will probably never end. :p Let me start by providing the link to the ABX test. Note: There are 16...

www.audiosciencereview.com

 

[DonR]

I've been pretty subjective in the past but I'm slowly coming around to being more objective. For one thing, I'm a cheapskate and not rich, so I'm priced out of the high end where subjectivity is so dominant. In the past I wanted high end high dollar stuff so bad, but as the years passed by it's become apparent that I'll never be able to afford that stuff. So I've bought 3 things now on the strength of Amir's reviews and I've been very happy with my purchases. 2 of those have been cheap DACs and it's a bit shocking how good the sound quality is that I'm getting. In the shopping process I read a lot of reviews here and it's slowly sinking in, the meaning behind all these measurements and graphs.

I'm wondering though how my perfectly measuring entry level DAC (SMSL DO100) would compare to a much more expensive DAC, namely the Holo Spring or May. Would it basically sound the same? Or would my DAC even sound better, beating those R2R DACs in SINAD and the multitone test. Or would it just be a small difference that's a matter of taste? How would it compare to something ridiculous, like the 6-figure MSB Select DAC? I'm thinking the differences would be rather small, but I'll never know unless I go audition the stuff with no intention of buying. Something like a Topping D90LE is within my grasp, but why would I when it measures so similar to the DO100? Just wondering, as I'm very happy with the sound I'm getting now.

Take some hearing tests. Go to an audiologist and find out where your hearing drops off (mine is 13kHz). Take a Klippel test https://www.klippel.de/listeningtest/ to find out what level of distortion you can discern. You might be surprised at how cheap a DAC you can buy where there is no discernable difference to you.

 

[drewdawg999]

Take some hearing tests. Go to an audiologist and find out where your hearing drops off (mine is 13kHz). Take a Klippel test https://www.klippel.de/listeningtest/ to find out what level of distortion you can discern. You might be surprised at how cheap a DAC you can buy where there is no discernable difference to you.

From a YT test I find my hearing drops off at around 14kHz, no spring chicken. Took the Klippel test and I got -21dB, right smack in the mean. I'm thinking I've reached my endgame DAC on the cheap, nice!

 

[drewdawg999]

I wouldn’t bother unless you need some specific features. You may want to try this test:

DAC ABX Test Phase 1: Does a SOTA DAC sound the same as a budget DAC if proper controls are put in place? Spoiler: Probably yes. :)

This thread is meant to once and for all settle the age-old question on whether different non-broken DACs have a "sound" or if all DACs sound the same! :cool: ...just kidding - those debates will probably never end. :p Let me start by providing the link to the ABX test. Note: There are 16...

www.audiosciencereview.com

Tried the test and I was guessing the whole time. Got 9/16 correct. So 90dB SINAD is already transparent to me, I'm not that sensitive. I should've known being that I was a happy vinyl listener for the longest time, though my turntable's not even hooked up right now. Reason being I like the DAC plugged straight into power amp with balanced XLRs, so single source at the moment. I guess I'm all digital until I get that Pre90 for Christmas.

 

[slsmaster]

Quite amazing how much subjective nonsense you can cram into 3 paragraphs...

Maybe it is the worse channel matching that made a difference here? There are unit variations due to the analog pot. The A90D doesn’t have that issue.
It seems to me that at least from a measurement point of view, that every other metric is so close in performance that it shouldn’t be audible?
But there also could be some other performance criteria that hasn’t yet been specified in form of an measurable metric.

 

[Lupin]

But there also could be some other performance criteria that hasn’t yet been specified in form of an measurable metric.

And what might that be?
Audio is just an electric signal, nothing more nothing less. So what part of this electric signal is not measured and/or looked at yet?

Keeps amazing me that people don't or hardly care about the electrical signal when it comes to their TV, the CPU or GPU in their computer. But once it is an audio signal suddenly all kinds of mythical and magical properties are attached to that signal...
Audio is an electric signal. Audio reproduction has been solved years ago. All the rest is just audiophile snake-oil and plain old marketing.

 

[raif71]

And what might that be?
Audio is just an electric signal, nothing more nothing less. So what part of this electric signal is not measured and/or looked at yet?

Keeps amazing me that people don't or hardly care about the electrical signal when it comes to their TV, the CPU or GPU in their computer. But once it is an audio signal suddenly all kinds of mythical and magical properties are attached to that signal...
Audio is an electric signal. Audio reproduction has been solved years ago. All the rest is just audiophile snake-oil and plain old marketing.

I thought the mantra here is that DACs measured similarly, sound the same. This is not necessarily the same between different amps, ie amps that measure similarly can sound different, right?

 

[Cidious]

I thought the mantra here is that DACs measured similarly, sound the same. This is not necessarily the same between different amps, ie amps that measure similarly can sound different, right?

Of course they can, but you'll always have forum Maffioso like Lupin that have to repeat their same lines in every topic. It's probably ignorance or not having enough funds to buy and test some of the devices him/herself. Who knows. I'm just speculating at that.

I've got plenty of kit here. And if it was all the same I'd have never bought anything else anymore after I would have noticed after the first two or maybe three times that it wouldn't matter going off of just measurements. But it does differ. For good or for worse is subjective. All I can say. Devices don't sound the same because they contain an audio signal as mr wise guy tries to sell above.

How much some of the subtleties are worth to someone in terms of money and how much one can hear of it is up to someone on it's own. Some products are snake-oil for sure. But AMPs easily sound different not just by volume level. More measurements and tests need to be included to get a fuller picture of these differences or maybe a better interpretation of the ones already measured.

Having said all that. I stuck with my A30Pro purely for convenience of use and thinking it wasn't worth the extra coin to me personally. A practical decision. D30Pro + A30Pro work just fine for most situations.

 

[Veri]

How much some of the subtleties are worth to someone in terms of money and how much one can hear of it is up to someone on it's own. Some products are snake-oil for sure. But AMPs easily sound different not just by volume level.

Not *just* by volume level?

How much some of the subtleties are worth to someone in terms of money and how much one can hear of it is up to someone on it's own. Some products are snake-oil for sure. But AMPs easily sound different not just by volume level. More measurements and tests need to be included to get a fuller picture of these differences or maybe a better interpretation of the ones already measured.

This is a bit of a red herring, "more measurements and tests need to be included to get a fuller picture of these differences" basically corresponds to "that which proves my anecdotal findings is yet to be found but it's surely there". You could explain anything with that.

 

[heatwave]

And this got a 5 star review on What HiFi

They quote:

"Streaming SBTRKT’s Trials Of The Past, the AVC-X3800H delivers a surprisingly nuanced musical performance with an assured attack at the start of each synth note and the envelope of each reverb tail given space to breathe. The hefty bass is solidly controlled, while at the top end of the spectrum, the shakers are nicely defined and spread wide"

The difference being - What Hi-Fi? based their findings on what you experience with real world usage - actually allowing the receiver to be used the way everyone who owns will be using it. Measurements are never any guarantee of what to expect from a perofrmance point of view. How are senses enjoy what a piece of home audio offers has to offer actually means something to the consumer. Lines on a graph are for would-be engineers, engineers and people with a science delusion.
It would take a complete fool to believe they senses are always going to agree with 'scope measurements.
Here's another thing to consider with measruements. In the late 1970's, Hitachi manufactured an integrated amplifier which measured 'perfectly'. But it was a flop because it apparently sounded very boring.

 

[heatwave]

Great Review Amir, Thanks
I had high expectations but what a disappointment, oh well, I will keep enjoying my X3700H

Give the AVR-X3800H a listen. The proof is much more in the experience than it is in the measurements.

 

[heatwave]

Pure gobbledygook from a person that apparently does not have a understanding of electronics. The circuitry is reacting to each electron passing through it and the designers are basically counting electrons when designing due to the ability to easily convert current and voltage to electron flow ~quantity. Electronics theory(s) has been around for a long time and it has been proven time and time again to be accurate. So when a layman comes along with big ideas about electronics it's amusing if not ridiculous.

Your rant doesn't explain how measurements are any guarantee of outright performance.
When you spend a good deal of time in front of the equipment, comparing with others in it's range, it's very clear to see how mearurements and performance are not necessarily going to correlate.
If you love how something sounds after having compared with the other models of it's type by other brands, but then read the measurements were not good, would you no longer be interested?
That's like falling in love with someone, but deciding to have nothing to do with them because someone else said they didn't like certain little things about their personality.
Mis-matched systems are built around buying based on measurements.
An open mind will trust thy senses and pick based on the sound which was most enthralling. The rest can enjoy their system by looking at the spec's over and over.

 

[mctron]

Your rant doesn't explain how measurements are any guarantee of outright performance.
When you spend a good deal of time in front of the equipment, comparing with others in it's range, it's very clear to see how mearurements and performance are not necessarily going to correlate.
If you love how something sounds after having compared with the other models of it's type by other brands, but then read the measurements were not good, would you no longer be interested?
That's like falling in love with someone, but deciding to have nothing to do with them because someone else said they didn't like certain little things about their personality.
Mis-matched systems are built around buying based on measurements.
An open mind will trust thy senses and pick based on the sound which was most enthralling. The rest can enjoy their system by looking at the spec's over and over.

So you've come to Audio Science Review to tell everyone that measurements don't matter and trust your ears