• Welcome to ASR. There are many reviews of audio hardware and expert members to help answer your questions. Click here to have your audio equipment measured for free!

Master Thread: Are Measurements Everything or Nothing?

That's a major reason why @amirm (and many other reviewers and testers) runs 32-tone signals, distortion tests at multiple frequencies from 20Hz-20kHz, and intermodulation distortion tests. Those tests of complex signals can be enlightening in various ways - but these complex-signal tests rarely (if ever) reveal gear that performs one way with a simpler signal and another way with a more complex one.

It's a similar principle to digital sampling: the sampling frequency needs to be 2x the highest frequency you want to reproduce (with a little buffer/wiggle room for filtering and reconstruction of course). It doesn't matter how simple or complex the signal is: that rule holds true regardless, and it's not a perceptual rule or a "most of the time" rule - it's a mathematical rule.
addictedtoaudio.com.au/blogs/how-to/how-to-pick-the-best-filter-setting-for-your-dac
sure filters sound different- that's the point of different filters even if sometimes the differences are small enough not to be heard. this isn't the same as saying 2 dacs that measure the same sound different.
I wrote “all DACs that measure well don’t sound the same”. In fact, I expect they would measure different if they sound different. And I would expect much of the difference for DACs that measure well (but not identically the same) would be from different filters, that will measure differently.
 
Last edited:
Did you come here to learn the science of audio?
You never will if you continue to refuse to properly test your beliefs.
Or just waste our time repeating that garbage about being able to hear things we can't measure?
Beware the self-appointed expert.
 
Beware the self-appointed expert.
..but respect those - like @Sal1950 - who prefer to study and learn from the real experts, and who are aware of audiophile myths and the all-to-common refusal to accept the verified findings of audio engineering and the science of psychoacoustics with respect to audibility of differences in audio systems.

Examples of the Dunning-Kruger effect are rampant in the subjectivist audiophile community.

Dunning-Kruger effect, in psychology, is a cognitive bias whereby people with limited knowledge or competence in a given domain greatly overestimate their own knowledge or competence in that domain relative to objective criteria...
 
Examples of the Dunning-Kruger effect are rampant in the subjectivist audiophile community.
@Xulonn
Hello my friend, thanks for the kind words.
It's good to hear from you, I hope you've been feeling well.
 
So, full disclosure. I am 58 and a musician by profession - classically trained, went to Trinity College of Music in London and have worked mostly in orchestras and shows, playing wind instruments. I have also had, unusually for a musician, a lifelong interest in Hi-Fi.
Floyd Toole has made some interesting observations on musicians and their relative capability to assess audio playback gear, based on solid experimental evidence:

"Ando et al. (2000) found that musicians judge reflections to be about seven times greater than ordinary listeners, meaning that they derive a satisfying amount of spaciousness from reflections at a much lower sound level than ordinary folk: “Musicians prefer weaker amplitudes than listeners do.” "

"Musicians have long been assumed to have superior abilities to judge sound quality. Certainly, they know music, and they tend to be able to articulate opinions about sound. But what about the opinions themselves? Does living “in the band” develop an ability to judge sound from the audience’s perspective? Does understanding the structure of music and how it should be played enable a superior analysis of sound quality? When put to the test, Gabrielsson et al. (1979) found that the listeners who were the most reliable and also the most observant of differences between test sounds were persons he identified as hi-fi enthusiasts, a population that also included some musicians. The worst were those who had no hi-fi interests. In the middle, were musicians who were not hi-fi oriented."

Pasted Graphic.png



"Figure 19.4b puts hearing loss into an easily understandable, and disturbing, context. Plotted on top of the ISO 226 equal-loudness contours are the hearing threshold measurements of listeners who exhibited high variability in their ratings of loudspeaker sound quality. All of these listeners were audio professionals, and many were part-time musicians."


--all quotes from the first edition of Sound Reproduction.
 
Thanks for the welcome and responses!

I’ve been wondering whether to stick around and engage with some of the replies or to just back out quietly while I can! ;) Obviously the majority opinion here sees things rather differently to the way I do, although some obviously agree.

However, I have no problem with scientific results or the scientific method and I’m interested to learn all I can in the hope of enlightenment, so I’ll keep reading, even if I post infrequently.

Instead of engaging with specific responses to my original post, which will likely end up in pointless confrontation, I’d like to ask a question; is it possible that some audible differences, perceived by some but not others, and are simply unmeasurable at the moment? In other words the current science says there is no difference between X and Y, but does that by definition mean that there IS no difference? Scientific measurement is changing and progressing all the time, isn’t it? As Richard Dawkins says (paraphrasing) ‘We might not know the answer yet, but we’re getting there.....’
 
Floyd Toole has made some interesting observations on musicians and their relative capability to assess audio playback gear, based on solid experimental evidence:

"Ando et al. (2000) found that musicians judge reflections to be about seven times greater than ordinary listeners, meaning that they derive a satisfying amount of spaciousness from reflections at a much lower sound level than ordinary folk: “Musicians prefer weaker amplitudes than listeners do.” "

"Musicians have long been assumed to have superior abilities to judge sound quality. Certainly, they know music, and they tend to be able to articulate opinions about sound. But what about the opinions themselves? Does living “in the band” develop an ability to judge sound from the audience’s perspective? Does understanding the structure of music and how it should be played enable a superior analysis of sound quality? When put to the test, Gabrielsson et al. (1979) found that the listeners who were the most reliable and also the most observant of differences between test sounds were persons he identified as hi-fi enthusiasts, a population that also included some musicians. The worst were those who had no hi-fi interests. In the middle, were musicians who were not hi-fi oriented."

"Figure 19.4b puts hearing loss into an easily understandable, and disturbing, context. Plotted on top of the ISO 226 equal-loudness contours are the hearing threshold measurements of listeners who exhibited high variability in their ratings of loudspeaker sound quality. All of these listeners were audio professionals, and many were part-time musicians."


--all quotes from the first edition of Sound Reproduction.
Thank you for this, informative, very adult.
If nothing else, at least scholars above were putting "listening to music through one's ears" as the main subject.
With all respect to everyone here whom I have argued with, ask yourselves, what is the point of Hifi or as a concept?
- I put it to you, that it is a Trick show! Smoke and mirrors! All geared up to create a realistic Illusion that one is hearing an actual piece of music, played by actual players, while in fact it is nothing but artificially created sound waves using a transducer.
- And to what end? Again, I put it to you, to satisfy one's ear/brain.
Realistically, for him to take his wallet out!
If you accept the above and keep that in mind at all times, it becomes very clear that Science is used as a means to achieve that end.
It is ludicrous to suggest to a Hifi buyer/end user that what he hears is just an illusion, and he should not really trust his ears, but he should look at lab results and charts instead.
Lab tests, measurements, are just a means to check for flaws and mistakes - nothing more. Obviously, a device with a flawed lab test result is not going to sound good, but the reverse may not be true. (here it comes ...... all the name-calling!). In my culture (Amir's too) we have a saying, A walnut is round, but not anything round is a walnut.
We choose what to measure based on scientific understanding of the design, to believe we have reached the zenith of knowledge is foolish and dangerous, as it would stop further research.
No I am not suggesting there is magic in there we yet do not know! Not at all. We should use all our known scientific knowledge to consistently better ourselves, but loosing sight of the end goal is just short-sightedness.
Hifi was conceived to trick us into believing that we are listening to real music while we are on a plane, train or in our homes.
Our ears, our brains, our desires together with our fallibilities are the major part and consideration of a Hifi system.
Please see the bigger picture.
 
I’d like to ask a question; is it possible that some audible differences, perceived by some but not others, and are simply unmeasurable at the moment? In other words the current science says there is no difference between X and Y, but does that by definition mean that there IS no difference?

You would think someone would have demonstrated that to be a false premise by now.

There has not been a single instance, ever, anywhere, by anyone that has identified an audible difference that wasn't measurable.

Don't you think that if you were selling based on SQ improvements, that you'd want to demonstrate that if you possibly could? I would.

There are thousands of people on this forum who would pay absurd amounts for a component that could be demonstrated to improve the sound. What most of us won't do is buy based on a story with no evidence.
 
‘There has not been a single instance, ever, anywhere, by anyone that has identified an audible difference that wasn't measurable.’

But if your only method for discerning a difference is measurement then this is just self fulfilling, isn’t it? i.e. There is no difference because I can’t be measured........
 
‘There has not been a single instance, ever, anywhere, by anyone that has identified an audible difference that wasn't measurable.’

But if your only method for discerning a difference is measurement then this is just self fulfilling, isn’t it? i.e. There is no difference because I can’t be measured........
I have to agree with @BDWoody here.
Think about it, if we can not quantify something, we can not recreate it, or do anything about it.
 
addictedtoaudio.com.au/blogs/how-to/how-to-pick-the-best-filter-setting-for-your-dac

I wrote “all DACs that measure well don’t sound the same”. In fact, I expect they would measure different if they sound different. And I would expect much of the difference for DACs that measure well (but not identically the same) would be from different filters, that will measure differently.
You never provided, after having been asked several times by multiple posters to present evidence for your claim „that DACs sound different“ although they are SOTA and measure the same. Therefore your argument is fundamentally flawed.

You most likely perceive a difference due to your perceptional biases, like every human being. You can therefore only proof the correctness of your claim by excluding those biases with an ABX listing test. See the link I provided to you previously.

Everything else is just hearsay, anecdotes …
 
Last edited:
I have to agree with @BDWoody here.
Think about it, if we can not quantify something, we can not recreate it, or do anything about it.
Right. But that’s why the statement is self fulfilling, isn’t it? I’m really not sure about that I have to confess - I’m no philosopher - I’m just a humble musician who likes his hi-fi!

But here is the problem. What do I do with information that I’m 100% sure about that can’t be proven scientifically? For instance (and this is where I don my flame suit) I’m absolutely sure that I can hear the differences, for instance, between different digital coax cables. I hasten to add that I can’t hear any difference between optical cables, but between electrical digital connect cables I’m 100% sure there are audible differences. It’s possible that those differences could be measurable through a pair of speakers, but I’m not sure that the wires could be measured to be different? They are just carrying digital information after all....? ‍
 
But if your only method for discerning a difference is measurement then this is just self fulfilling, isn’t it? i.e. There is no difference because I can’t be measured........

Why is that the only method?

We want Danny, or Darko, or Hans or you or anyone who makes claims to have heard differences where none are expected to exist, to demonstrate that with level controlled double blind AB/X testing.

No one has yet.
 
For instance (and this is where I don my flame suit) I’m absolutely sure that I can hear the differences, for instance, between different digital coax cables. I hasten to add that I can’t hear any difference between optical cables, but between electrical digital connect cables I’m 100% sure there are audible differences

 
Whenever I test something new I do the same REW measurements of the chain+room.If that reveal any improvement in any of the separate measurements AND does not annoys my brain,stays.
Is that simple for me.
I'm sure there is a myriad other measurements but these will not include everything in my chain+room.So...
 
Right. But that’s why the statement is self fulfilling, isn’t it? I’m really not sure about that I have to confess - I’m no philosopher - I’m just a humble musician who likes his hi-fi!

But here is the problem. What do I do with information that I’m 100% sure about that can’t be proven scientifically? For instance (and this is where I don my flame suit) I’m absolutely sure that I can hear the differences, for instance, between different digital coax cables. I hasten to add that I can’t hear any difference between optical cables, but between electrical digital connect cables I’m 100% sure there are audible differences. It’s possible that those differences could be measurable through a pair of speakers, but I’m not sure that the wires could be measured to be different? They are just carrying digital information after all....? ‍
Of course you have to respect the notion that others can discard experiences an individual may had, that can not be re-created or measured.
But statistically, if enough people in a controlled test can repeatedly hear something we can not measure yet, then it becomes a challenge for engineers to come up with a method of quantifying it, but first, the controlled test! To rule out the possibility of external factors, mere chance . . etc. That is the scientific approach.
Regarding you hearing differences in coax cables, there is a small possibility that you may be on to something!
- There is the issue of Noise. More accurately, Random RF Noise leaking into a DAC.
Digital circuits are pretty much immune to this noise, hell that was one of the first reasons they invented digital audio. Analogue circuits, not so much.
If a certain DAC/amp is sensitive to this noise, such as Chord portable DACs as they have no isolation, this noise can interfere with their amp (analogue) section, spicing up the output.
For that to be the reason here, you need a noisy environment and/or noisy DAP and such a DAC that is sensitive, together with cables that have some issue with noise. A lot of If's.
Toslink or optical is electrically an insulated interface by default.
 
I’m absolutely sure that I can hear the differences, for instance, between different digital coax cables
You need to understand how we hear, as humans we can convince ourself of virtually anything. So when you say that you heard a difference I believe you even though I know, because it has been scientifically proven, there is no difference.
The only way to be sure is by demonstrating to yourself, trough abx blind test that you can really hear a difference, as minute and insignificant it might be.
Does it matter, that is up to you, I believe my system sounds fantastic for a lot of reasons, not all of them totally rational and/or scientifically proven, but it really does sound fantastic, to me, and that is all that matter to me, no little graph will deny me any of it.
 
Obviously, a device with a flawed lab test result is not going to sound good, but the reverse may not be true.
No it's the other way around. If it does not measure so well it can still 'sound good' since there is a threshold of hearing beyond which it doesn't matter.

It will have to measure very poorly to sound bad. As long as it measures past the hearing threshold it will 'sound' like whatever recording you play through it aka 'transparent'.
 
No it's the other way around. If it does not measure so well it can still 'sound good' since there is a threshold of hearing beyond which it doesn't matter.

It will have to measure very poorly to sound bad. As long as it measures past the hearing threshold it will 'sound' like whatever recording you play through it aka 'transparent'.

This is why I am not stressed about what I may be missing out on when using my quality vintage gear. My old Luxman preamp gives me a noticeable hum when I crank the gain waaay up with no signal where my Schiit Freya S doesn't, but in terms of real world use, there are no veils in front of one or the other that I would ever notice.

People take different things from this site. The common misconception 'out there' is that we are all just chasing meaningless numbers, when they are only meaningless when you don't understand them or apply them inappropriately.

I did one controlled test shortly after joining the site between DACs I had on hand, and instantly had a different view of reviews based on sighted listening comparisons.
 
This is why I am not stressed about what I may be missing out on when using my quality vintage gear. My old Luxman preamp gives me a noticeable hum when I crank the gain waaay up with no signal where my Schiit Freya S doesn't, but in terms of real world use, there are no veils in front of one or the other that I would ever notice.

People take different things from this site. The common misconception 'out there' is that we are all just chasing meaningless numbers, when they are only meaningless when you don't understand them or apply them inappropriately.

I did one controlled test shortly after joining the site between DACs I had on hand, and instantly had a different view of reviews based on sighted listening comparisons.
If I had a quid for everyone who had said to me 'ASR reviewed my DAC and they panned it' I could retire to the Bahamas.

I explain:

They didn't review it, not in the sense you mean, they measured it, that's not the same thing. And just because it fell short of the state of the art in measurements does not mean that it will sound in the slightest bit inferior.

But it becomes hard work, it really does, as 'magazine thinking' (as I call it) is so deeply entrenched. Everything must have its own 'sound signature' like wines from a different vineyard. There's decades of brainwashing to overcome and often it's impossible.
 
Back
Top Bottom