• WANTED: Happy members who like to discuss audio and other topics related to our interest. Desire to learn and share knowledge of science required. There are many reviews of audio hardware and expert members to help answer your questions. Click here to have your audio equipment measured for free!

Master Thread: Are measurements Everything or Nothing?

Sal1950

Grand Contributor
The Chicago Crusher
Forum Donor
Joined
Mar 1, 2016
Messages
14,072
Likes
16,604
Location
Central Fl
Yes, we all want gear that measures well. But we want it to sound good too!
Good measuring gear always sounds good.
On that you can be guaranteed
 

TrevC

Active Member
Joined
Apr 11, 2021
Messages
180
Likes
124
Blind listening tests. I do not have a "theory" behind the improvements. But I was able to hear them and distinguish when the conditioners were being used.

Yes, we all want gear that measures well. But we want it to sound good too! So why not do unbiased blind testing to see if a product makes your ears happy? It costs you nothing but a bit of time.

Then again this forum is full of cranky people that want cheap gear with good measurements regardless of how it sounds. I am very lucky to have access to tons of gear to test and some of the stuff on hear that measures well sounds like crap in real life.
No point in blind testing things that make no difference though. It appears to be you that is cranky.
 

Mnyb

Major Contributor
Forum Donor
Joined
Aug 14, 2019
Messages
2,640
Likes
3,608
Location
Sweden, Västerås
I understand and can appreciate your point. In my view it's a serious difficulty that's probably never going to be solvable within the limits of internet-based discussion that does not involve scientific studies.

What I mean is this:

- You state you have done multiple blind tests, and if I read your comment correctly, you are saying that in those blind tests you have heard, to a degree of statistical significance, the difference a power conditioner makes and you have also heard differences that other power products make.

- Your claim is what many here would describe as an extraordinary claim. They would describe it as such not because they think it's really, really hard to hear such sonic differences, but rather because there is no plausible or sensible explanation for how a power condition could even make any difference in the audible sound, given how a power condition works, what it does, and what it does no do. (Unless, as someone suggested above, there's an unusual level of audible interference of a particular kind in the system in the first place.) So the root cause of people's skepticism isn't that they doubt you for any personal reason, but rather that your claim simply doesn't seem to add up.

- Given that, there are only two possible explanations: (1) your blind tests were not actually properly blind tests and you are glossing over key details that would allow listener bias to creep in, or (2) our knowledge of electronics, physics, human hearing, and possibly math must be fundamentally wrong or incomplete in a way that does not manifest itself in other areas (like, say computer data transmission or video performance) but somehow only manifests itself in the narrow realm of hi-fi sound product listening.

- Option (1) is pretty much impossible for anyone to articulate without you feeling like they're calling you a liar, a fool, or both - so naturally you take offense, and I understand how you might feel that people are simply hypocrites because they say they are for the scientific method but when you present them with a scientific result they don't like, they refuse to accept it.

- Conversely, option (2) is pretty much impossible for anyone to accept simply based on your assurance that you've done blind tests and you've reliably detected these differences. If that were the standard - "you don't know me, you don't even necessarily know my real name, and this is my first post here but take my word for it" - then there would be no reliable basis for scientific knowledge.

So I see it as an impasse. And the only way to resolve it would be for a researcher to conduct the kinds of blind tests you have gone through and document the procedure and results in a peer-reviewed study. (Really two peer-reviewed studies, since repeatability of results is also key.)

Unless or until that happens, this same pattern is going to be repeated, and the same arguments and slogans will be employed: "your claim of blind tests is not evidence, it's just what you're claiming"; "you say you want blind testing but if you don't like the results you reject them"; "extraordinary claims require extraordinary evidence - there's no way these devices can make a difference so the onus is on you to suggest a hypothesis of how they could"; "science is about observation and open-mindedness to what we don't know - how arrogant to think that it's impossible for us to hear things we haven't figured out how to measure yet"; and on and on and on.

I don't have a solution; I just think it's worth trying to understand this as a communicative problem, so we don't go around and around in circles forever.
The solution is as always repeated independent testing by several people and labs.

Thats why some scientist are testing already done deals and sometimes turn up surprises and sometimes not.
This is a large part of the everyday grind of science to confirm results . You can bet that if you claim something new 10 other scientist are going to dive into it and try to disprove you , thats part of the method

Some forgets the repeatable part of a test , someone else should be able from your description to put together the same test and get the same result.
If the results cant be repeated it's not given any credit. A one-off does not really help .

Therefore we have meta studies some one statically compare many studies in the same subject and use statistics .

And I like your argument . In some cases it's simple very implausible to get the described results so much so that experimental error is the most likely explanation even if no one knows what the explanation is and the OP gets very upset about everyone question the findings .
And it cant be setled on internet forums .
 

HarmonicTHD

Major Contributor
Forum Donor
Joined
Mar 18, 2022
Messages
3,326
Likes
4,829
I had one of the most disappointing audio experiences ever recently. I attended CanJam SoCal 2022. Towards the end of the show I spotted a TH900 connected to a TT2. Both of which I was always curious to try. TH900 is a known bass cannon. TT2 owners always talking about how 'special' the sound is......I started by playing a couple familiar tracks. take the headphones off...confused. Not good bass and thin sounding. Maybe something is wrong with the headphones. I look them over, stock pads, look new. Hmmmm. I plug in my 7 hz Timeless to make sure something is not 'wrong' with the TT2. Timeless sounds 'good' on the TT2, though not even .01% better than from my Qudelix 5K. I plug in a Focal Celestee, sounds good, but again, bass is weird and sound is thin. Celestee is kind of a thick and not at all thin sounding headphone. If this is the Chord 'special sound'- no thank you. I actually preferred the Mojo 2. It sounded good and normal. I will have to try the TH900 another time on another better piece of equipment. Which to my ears, means almost anything. The Chord products are beautiful, Inlove the look and feel of them. But I am a music lover and for the kind of money they are charging, these products should not lack bass energy or note weight and sound so thin.

So I get up to leave the table and the guy says "what do you think?" I reply "that Chord gear really is something". He says "yep that M scaler really adds some magic!"

I had not even realized that it was connected to an M Scaler! I was listening to a $12k+ set up and it was mediocre. From now on whenever someone says they own Chord Gear, I will have to giggle a little and mutter 'sucker' under my breath. (unless it is the Mojo or Mojo 2. Those are good portable devices).
Despite the incredible bad value to performance of the Chord and topped with plenty of snake oil stories from the designer.

Your subjective listening impression can not be explained by what Amir measured and heard (aside from the only ca 75dB headphone noise performance).

How would you explain your impressions based on the facts shown?
 

04gto

Member
Joined
May 20, 2020
Messages
44
Likes
67
Lots if things cannot be explained by measurements. So the'facts' as you put it will have no correlation. Example: When I had my Hifiman Ananda and Hifiman Arya at the the same time, I had them both EQ'd to the same frequency target. Ananda still sounded thinner (less weghty notes) even though it measured the same (within .5db throughout). This Chord experience reminded me of the day I used a 3.5 to rca jack from my PC into my THX789, just out of curiosity. Did it sound bad? Nope. But it did sound similarly thin. All amps (or dac/amps) do not sound the same, some are perhaps overly thick (most tube amps to me). Some sound thinner TT2 (to an extreme), Topping A90. Some are right down the middle, like my RME ADI-2 PRO. In most cases it is subtle like switching filters. In some cases it is distracting.
 
  • Like
Reactions: Els

Ken Tajalli

Major Contributor
Forum Donor
Joined
Sep 8, 2021
Messages
1,995
Likes
1,772
Location
London UK
Lots if things cannot be explained by measurements. So the'facts' as you put it will have no correlation. Example: When I had my Hifiman Ananda and Hifiman Arya at the the same time, I had them both EQ'd to the same frequency target. Ananda still sounded thinner (less weghty notes) even though it measured the same (within .5db throughout). This Chord experience reminded me of the day I used a 3.5 to rca jack from my PC into my THX789, just out of curiosity. Did it sound bad? Nope. But it did sound similarly thin. All amps (or dac/amps) do not sound the same, some are perhaps overly thick (most tube amps to me). Some sound thinner TT2 (to an extreme), Topping A90. Some are right down the middle, like my RME ADI-2 PRO. In most cases it is subtle like switching filters. In some cases it is distracting.
I am sure you had a bad experience, however, You seem to be one of the very very few people that have complained about Chord TT2's sound quality, that I have seen on the web or otherwise.
Sure enough, as @HarmonicTHD has mentioned, there are unhappy folks out there with TT2 and its designer, but NONE with its subjective or objective sound quality performance.
Perhaps something else was at play on that day, what? - I don't know! since Chord DACs reliability is also very high, so your unit being faulty is also hard to accept.
Have it this way, if its designer would take a break and they drop Chord DAC prices to Topping level, Amir himself would buy one for his own use, if not already, so would most ASR members.
Me? I already have a couple!.
 

04gto

Member
Joined
May 20, 2020
Messages
44
Likes
67
I am sure you had a bad experience, however, You seem to be one of the very very few people that have complained about Chord TT2's sound quality, that I have seen on the web or otherwise.
Sure enough, as @HarmonicTHD has mentioned, there are unhappy folks out there with TT2 and its designer, but NONE with its subjective or objective sound quality performance.
Perhaps something else was at play on that day, what? - I don't know! since Chord DACs reliability is also very high, so your unit being faulty is also hard to accept.
Have it this way, if its designer would take a break and they drop Chord DAC prices to Topping level, Amir himself would buy one for his own use, if not already, so would most ASR members.
Me? I already have a couple!.
I think it is fine if other people like it. I really love the build and feel of all the Chord products I have touched. I really just expected the TT2 to sound 'normal', like most amps do. But in the same way I also loved the DCA Stealth headphones, I was just disappointed by the sound. Hell I wanted to buy those even after I decided they were not for me sound wise. Because they are works of art construction wise. If I bought audio gear with my eyes, I would be so broke right now. :) I do think that some people (not all) need to justify spending so much on gear that sounds no better (or in some cases worse) than gear costing a small fraction. Some may even feel that way about my RME ADI-2 PRO? Maybe I just like the light show on the RME better? :) No. To my ears it just sounds more natural. Like I am listening to the music and not the gear so much.
 

GXAlan

Major Contributor
Forum Donor
Joined
Jan 15, 2020
Messages
3,867
Likes
5,953
This did not make the front page but here is an example of two amplifiers that sound different yet both measure well.

 

HarmonicTHD

Major Contributor
Forum Donor
Joined
Mar 18, 2022
Messages
3,326
Likes
4,829
Lots if things cannot be explained by measurements. So the'facts' as you put it will have no correlation. Example: When I had my Hifiman Ananda and Hifiman Arya at the the same time, I had them both EQ'd to the same frequency target. Ananda still sounded thinner (less weghty notes) even though it measured the same (within .5db throughout). This Chord experience reminded me of the day I used a 3.5 to rca jack from my PC into my THX789, just out of curiosity. Did it sound bad? Nope. But it did sound similarly thin. All amps (or dac/amps) do not sound the same, some are perhaps overly thick (most tube amps to me). Some sound thinner TT2 (to an extreme), Topping A90. Some are right down the middle, like my RME ADI-2 PRO. In most cases it is subtle like switching filters. In some cases it is distracting.
Which „things“ ?

My hypothesis would be this or a lack therefore.


But hey we might just take your word for it.

(Ps. Just to be clear - no I am not a fan of manufactures, which try to get to other people’s money by telling bogus stories about the miracle performance of their product such as Chord. But neither am I a fan of claims based on anecdotes to the opposite either).
 

04gto

Member
Joined
May 20, 2020
Messages
44
Likes
67
Which „things“ ?

My hypothesis would be this or a lack therefore.


But hey we might just take your word for it.

(Ps. Just to be clear - no I am not a fan of manufactures, which try to get to other people’s money by telling bogus stories about the miracle performance of their product such as Chord. But neither am I a fan of claims based on anecdotes to the opposite either).
Soundstage, note weight, vocal timbre, vocal realism, micro details, dynamic impact- In direct relation to how an amplifier effects slam or even texture differences between driver types (dynamic, planar etc). We don't need to "just take my word for it". Sometimes it can be explained by impedance matching sometimes not. If one cannot hear, or indeed, sometimes feel the differences between driver types, then I would say they are in the minority.
 

Robbo99999

Master Contributor
Forum Donor
Joined
Jan 23, 2020
Messages
6,874
Likes
6,672
Location
UK

Killingbeans

Major Contributor
Joined
Oct 23, 2018
Messages
4,089
Likes
7,546
Location
Bjerringbro, Denmark.
Yes, I have heard the difference that power products can make in blind tests.

Sorry for being one of those pig-headed indiscriminate naysayers, but...

How was the test blinded? And how high was the statistical significance?

I don't know you, so nothing personal. It's just that I've seen people "interpret" the concept of a blind testing in some really creative (read: useless) ways.

For instance people who think that having the audio gear and test subjects in two different rooms is all it takes to make the test "blind". Or those who think that if you just keep testing, you can stop and call "victory" once you get your desired result enough times in a row, because that somehow negates the risk of random guessing :facepalm:
 

ahofer

Major Contributor
Forum Donor
Joined
Jun 3, 2019
Messages
4,951
Likes
8,696
Location
New York City
I believe one of the biggest mistakes made in utilizing subjective assessments - whether audio, visual or tactile - is the belief that other people must, most assuredly, sense the same thing in the same way or to the same degree.
Although one sure fire way to get agreement is for one to suggest the difference to the other (‘listen to that snare!, do you sense the slam?’)
 

Womaz

Active Member
Joined
Sep 28, 2022
Messages
216
Likes
67
Location
Newcastle - England
I have a scale for how much measurements matter for each category of products:

DACs: 100%
Amplifiers (headphone and speaker): 80 to 90% due to variability of available power. Hard to internalize how much power is available/enough without listening tests.
Speakers: 70 to 80%
Headphones: 50 to 80% (measurements too variable)

This is why you see me do listening tests for the last two categories and half of second (headphone amps).
This is brilliant and will be my starting point in the future if any purchases are to be made
 

drewdawg999

Active Member
Joined
Dec 25, 2021
Messages
186
Likes
168
Location
Suburban Los Angeles
I've been pretty subjective in the past but I'm slowly coming around to being more objective. For one thing, I'm a cheapskate and not rich, so I'm priced out of the high end where subjectivity is so dominant. In the past I wanted high end high dollar stuff so bad, but as the years passed by it's become apparent that I'll never be able to afford that stuff. So I've bought 3 things now on the strength of Amir's reviews and I've been very happy with my purchases. 2 of those have been cheap DACs and it's a bit shocking how good the sound quality is that I'm getting. In the shopping process I read a lot of reviews here and it's slowly sinking in, the meaning behind all these measurements and graphs.

I'm wondering though how my perfectly measuring entry level DAC (SMSL DO100) would compare to a much more expensive DAC, namely the Holo Spring or May. Would it basically sound the same? Or would my DAC even sound better, beating those R2R DACs in SINAD and the multitone test. Or would it just be a small difference that's a matter of taste? How would it compare to something ridiculous, like the 6-figure MSB Select DAC? I'm thinking the differences would be rather small, but I'll never know unless I go audition the stuff with no intention of buying. Something like a Topping D90LE is within my grasp, but why would I when it measures so similar to the DO100? Just wondering, as I'm very happy with the sound I'm getting now.
 

voodooless

Grand Contributor
Forum Donor
Joined
Jun 16, 2020
Messages
10,223
Likes
17,800
Location
Netherlands
I'm wondering though how my perfectly measuring entry level DAC (SMSL DO100) would compare to a much more expensive DAC, namely the Holo Spring or May. Would it basically sound the same?
Probably. Even though your DAC will have 10 to 15 dB better performance. The “NOS” mode may be audible due to the HF roll off and added intermodulation. But resample to a high rate first, and that difference will be gone as well.
Something like a Topping D90LE is within my grasp, but why would I when it measures so similar to the DO100? Just wondering, as I'm very happy with the sound I'm getting now.
I wouldn’t bother unless you need some specific features. You may want to try this test:
 

DonR

Major Contributor
Joined
Jan 25, 2022
Messages
2,970
Likes
5,614
Location
Vancouver(ish)
I've been pretty subjective in the past but I'm slowly coming around to being more objective. For one thing, I'm a cheapskate and not rich, so I'm priced out of the high end where subjectivity is so dominant. In the past I wanted high end high dollar stuff so bad, but as the years passed by it's become apparent that I'll never be able to afford that stuff. So I've bought 3 things now on the strength of Amir's reviews and I've been very happy with my purchases. 2 of those have been cheap DACs and it's a bit shocking how good the sound quality is that I'm getting. In the shopping process I read a lot of reviews here and it's slowly sinking in, the meaning behind all these measurements and graphs.

I'm wondering though how my perfectly measuring entry level DAC (SMSL DO100) would compare to a much more expensive DAC, namely the Holo Spring or May. Would it basically sound the same? Or would my DAC even sound better, beating those R2R DACs in SINAD and the multitone test. Or would it just be a small difference that's a matter of taste? How would it compare to something ridiculous, like the 6-figure MSB Select DAC? I'm thinking the differences would be rather small, but I'll never know unless I go audition the stuff with no intention of buying. Something like a Topping D90LE is within my grasp, but why would I when it measures so similar to the DO100? Just wondering, as I'm very happy with the sound I'm getting now.
Take some hearing tests. Go to an audiologist and find out where your hearing drops off (mine is 13kHz). Take a Klippel test https://www.klippel.de/listeningtest/ to find out what level of distortion you can discern. You might be surprised at how cheap a DAC you can buy where there is no discernable difference to you.
 

drewdawg999

Active Member
Joined
Dec 25, 2021
Messages
186
Likes
168
Location
Suburban Los Angeles
Take some hearing tests. Go to an audiologist and find out where your hearing drops off (mine is 13kHz). Take a Klippel test https://www.klippel.de/listeningtest/ to find out what level of distortion you can discern. You might be surprised at how cheap a DAC you can buy where there is no discernable difference to you.
From a YT test I find my hearing drops off at around 14kHz, no spring chicken. Took the Klippel test and I got -21dB, right smack in the mean. I'm thinking I've reached my endgame DAC on the cheap, nice!
 

drewdawg999

Active Member
Joined
Dec 25, 2021
Messages
186
Likes
168
Location
Suburban Los Angeles
I wouldn’t bother unless you need some specific features. You may want to try this test:
Tried the test and I was guessing the whole time. Got 9/16 correct. So 90dB SINAD is already transparent to me, I'm not that sensitive. I should've known being that I was a happy vinyl listener for the longest time, though my turntable's not even hooked up right now. Reason being I like the DAC plugged straight into power amp with balanced XLRs, so single source at the moment. I guess I'm all digital until I get that Pre90 for Christmas.
 
Top Bottom