• WANTED: Happy members who like to discuss audio and other topics related to our interest. Desire to learn and share knowledge of science required. There are many reviews of audio hardware and expert members to help answer your questions. Click here to have your audio equipment measured for free!

Master Thread: Are measurements Everything or Nothing?

tmtomh

Major Contributor
Forum Donor
Joined
Aug 14, 2018
Messages
2,835
Likes
8,393
Measurements are taken with relatively simple signals to allow analysis.

IMO, ASR should continue to present the data and let the consumer decide.

- Rich

That's a major reason why @amirm (and many other reviewers and testers) runs 32-tone signals, distortion tests at multiple frequencies from 20Hz-20kHz, and intermodulation distortion tests. Those tests of complex signals can be enlightening in various ways - but these complex-signal tests rarely (if ever) reveal gear that performs one way with a simpler signal and another way with a more complex one.

It's a similar principle to digital sampling: the sampling frequency needs to be 2x the highest frequency you want to reproduce (with a little buffer/wiggle room for filtering and reconstruction of course). It doesn't matter how simple or complex the signal is: that rule holds true regardless, and it's not a perceptual rule or a "most of the time" rule - it's a mathematical rule.
 

Ken Tajalli

Major Contributor
Forum Donor
Joined
Sep 8, 2021
Messages
2,200
Likes
1,980
Location
London UK
Let's say he's wrong and most people here did not used to believe as you currently believe. What difference would that make in your or Sal's arguments?
Nothing!
But the assumption that he is right, knows more than the next guy, and learning is always for others.
 

Sal1950

Grand Contributor
The Chicago Crusher
Forum Donor
Joined
Mar 1, 2016
Messages
14,395
Likes
17,289
Location
Central Fl
Nothing!
But the assumption that he is right, knows more than the next guy, and learning is always for others.
Did you come here to learn the science of audio?
You never will if you continue to refuse to properly test your beliefs.
Or just waste our time repeating that garbage about being able to hear things we can't measure?
 

SuicideSquid

Addicted to Fun and Learning
Joined
Jan 20, 2022
Messages
722
Likes
1,701
Don't know what to say to that!!
Are you suggesting that I listen to music, while checking charts from a lab test?
Or should I watch a movie with my taste buds?
All the lab tests , charts and SINAD results on ASR don't amount to a hill of beans, if my ears disagree.
You know what they say about "proof of the pudding . . ."
I was around in the 70's, and I remember the same empty claims then "with modern transistor audio equipments all problems are solved, there is nothing we don't know"
And this is the topic of this thread, after all.
Obviously you belong to the camp ".. it is everything" , and I from the camp ".... it is a necessary first step" .
We agree to disagree.

If measurements say there's no difference between two devices, and your ears disagree, it's time for double-blind testing. If you're not bothering to compare these devices blind, then it's more likely than not that the perceived differences are in your head. If you run a properly-blinded comparison and you can hear a difference, then there's something wrong with the measurements.

More often than not, though, hearing a difference when none is measurable demonstrates the fallibility of your brain, not the fidelity of your ears.
 

Sal1950

Grand Contributor
The Chicago Crusher
Forum Donor
Joined
Mar 1, 2016
Messages
14,395
Likes
17,289
Location
Central Fl
More often than not, though, hearing a difference when none is measurable demonstrates the fallibility of your brain, not the fidelity of your ears.
Amen
 

elvisizer

Active Member
Joined
Sep 16, 2018
Messages
267
Likes
214
www.addictedtoaudio.com.au/blogs/how-to/how-to-pick-the-best-filter-setting-for-your-dac

Above is a link that explains how different filters measure and sound, on an ASR fav DAC. While some may claim these differences are inaudible, in my experience with DACs that I’ve owned that allow one to select various filters, one could easily hear differences in real time a-b comparisons. Many highly respected engineers such as Dr Johnson, E Meitner will tell you they measure and listen. These are folks that have also made huge contributions not just advancing consumer products, but also in the pro recording arena designing and manufacturing some of the best sounding PCM and DSD A to D and D to A equipment, used by many of the best studios to produce some of the best recordings ever.
sure filters sound different- that's the point of different filters even if sometimes the differences are small enough not to be heard. this isn't the same as saying 2 dacs that measure the same sound different.
 

Andrew s

Member
Joined
May 9, 2021
Messages
69
Likes
129
It's a similar principle to digital sampling: the sampling frequency needs to be 2x the highest frequency you want to reproduce (with a little buffer/wiggle room for filtering and reconstruction of course). It doesn't matter how simple or complex the signal is: that rule holds true regardless, and it's not a perceptual rule or a "most of the time" rule - it's a mathematical rule.
True provided all the assumptions of the mathematical proof are met e.g. the input is suitably bandwidth limited. These constraints on input and reconstruction are often not met.
Regards Andrew
 

PGAMiami

Active Member
Joined
Apr 25, 2021
Messages
241
Likes
252
That's a major reason why @amirm (and many other reviewers and testers) runs 32-tone signals, distortion tests at multiple frequencies from 20Hz-20kHz, and intermodulation distortion tests. Those tests of complex signals can be enlightening in various ways - but these complex-signal tests rarely (if ever) reveal gear that performs one way with a simpler signal and another way with a more complex one.

It's a similar principle to digital sampling: the sampling frequency needs to be 2x the highest frequency you want to reproduce (with a little buffer/wiggle room for filtering and reconstruction of course). It doesn't matter how simple or complex the signal is: that rule holds true regardless, and it's not a perceptual rule or a "most of the time" rule - it's a mathematical rule.
addictedtoaudio.com.au/blogs/how-to/how-to-pick-the-best-filter-setting-for-your-dac
sure filters sound different- that's the point of different filters even if sometimes the differences are small enough not to be heard. this isn't the same as saying 2 dacs that measure the same sound different.
I wrote “all DACs that measure well don’t sound the same”. In fact, I expect they would measure different if they sound different. And I would expect much of the difference for DACs that measure well (but not identically the same) would be from different filters, that will measure differently.
 
Last edited:

JamesRF

Member
Joined
Jun 24, 2021
Messages
24
Likes
10
Did you come here to learn the science of audio?
You never will if you continue to refuse to properly test your beliefs.
Or just waste our time repeating that garbage about being able to hear things we can't measure?
Beware the self-appointed expert.
 

Xulonn

Major Contributor
Forum Donor
Joined
Jun 27, 2018
Messages
1,828
Likes
6,316
Location
Boquete, Chiriqui, Panama
Beware the self-appointed expert.
..but respect those - like @Sal1950 - who prefer to study and learn from the real experts, and who are aware of audiophile myths and the all-to-common refusal to accept the verified findings of audio engineering and the science of psychoacoustics with respect to audibility of differences in audio systems.

Examples of the Dunning-Kruger effect are rampant in the subjectivist audiophile community.

Dunning-Kruger effect, in psychology, is a cognitive bias whereby people with limited knowledge or competence in a given domain greatly overestimate their own knowledge or competence in that domain relative to objective criteria...
 

Sal1950

Grand Contributor
The Chicago Crusher
Forum Donor
Joined
Mar 1, 2016
Messages
14,395
Likes
17,289
Location
Central Fl
Examples of the Dunning-Kruger effect are rampant in the subjectivist audiophile community.
@Xulonn
Hello my friend, thanks for the kind words.
It's good to hear from you, I hope you've been feeling well.
 

Newman

Major Contributor
Joined
Jan 6, 2017
Messages
3,608
Likes
4,483
So, full disclosure. I am 58 and a musician by profession - classically trained, went to Trinity College of Music in London and have worked mostly in orchestras and shows, playing wind instruments. I have also had, unusually for a musician, a lifelong interest in Hi-Fi.
Floyd Toole has made some interesting observations on musicians and their relative capability to assess audio playback gear, based on solid experimental evidence:

"Ando et al. (2000) found that musicians judge reflections to be about seven times greater than ordinary listeners, meaning that they derive a satisfying amount of spaciousness from reflections at a much lower sound level than ordinary folk: “Musicians prefer weaker amplitudes than listeners do.” "

"Musicians have long been assumed to have superior abilities to judge sound quality. Certainly, they know music, and they tend to be able to articulate opinions about sound. But what about the opinions themselves? Does living “in the band” develop an ability to judge sound from the audience’s perspective? Does understanding the structure of music and how it should be played enable a superior analysis of sound quality? When put to the test, Gabrielsson et al. (1979) found that the listeners who were the most reliable and also the most observant of differences between test sounds were persons he identified as hi-fi enthusiasts, a population that also included some musicians. The worst were those who had no hi-fi interests. In the middle, were musicians who were not hi-fi oriented."

Pasted Graphic.png



"Figure 19.4b puts hearing loss into an easily understandable, and disturbing, context. Plotted on top of the ISO 226 equal-loudness contours are the hearing threshold measurements of listeners who exhibited high variability in their ratings of loudspeaker sound quality. All of these listeners were audio professionals, and many were part-time musicians."


--all quotes from the first edition of Sound Reproduction.
 

Mike F

Member
Joined
Jun 8, 2022
Messages
6
Likes
3
Thanks for the welcome and responses!

I’ve been wondering whether to stick around and engage with some of the replies or to just back out quietly while I can! ;) Obviously the majority opinion here sees things rather differently to the way I do, although some obviously agree.

However, I have no problem with scientific results or the scientific method and I’m interested to learn all I can in the hope of enlightenment, so I’ll keep reading, even if I post infrequently.

Instead of engaging with specific responses to my original post, which will likely end up in pointless confrontation, I’d like to ask a question; is it possible that some audible differences, perceived by some but not others, and are simply unmeasurable at the moment? In other words the current science says there is no difference between X and Y, but does that by definition mean that there IS no difference? Scientific measurement is changing and progressing all the time, isn’t it? As Richard Dawkins says (paraphrasing) ‘We might not know the answer yet, but we’re getting there.....’
 

Ken Tajalli

Major Contributor
Forum Donor
Joined
Sep 8, 2021
Messages
2,200
Likes
1,980
Location
London UK
Floyd Toole has made some interesting observations on musicians and their relative capability to assess audio playback gear, based on solid experimental evidence:

"Ando et al. (2000) found that musicians judge reflections to be about seven times greater than ordinary listeners, meaning that they derive a satisfying amount of spaciousness from reflections at a much lower sound level than ordinary folk: “Musicians prefer weaker amplitudes than listeners do.” "

"Musicians have long been assumed to have superior abilities to judge sound quality. Certainly, they know music, and they tend to be able to articulate opinions about sound. But what about the opinions themselves? Does living “in the band” develop an ability to judge sound from the audience’s perspective? Does understanding the structure of music and how it should be played enable a superior analysis of sound quality? When put to the test, Gabrielsson et al. (1979) found that the listeners who were the most reliable and also the most observant of differences between test sounds were persons he identified as hi-fi enthusiasts, a population that also included some musicians. The worst were those who had no hi-fi interests. In the middle, were musicians who were not hi-fi oriented."

"Figure 19.4b puts hearing loss into an easily understandable, and disturbing, context. Plotted on top of the ISO 226 equal-loudness contours are the hearing threshold measurements of listeners who exhibited high variability in their ratings of loudspeaker sound quality. All of these listeners were audio professionals, and many were part-time musicians."


--all quotes from the first edition of Sound Reproduction.
Thank you for this, informative, very adult.
If nothing else, at least scholars above were putting "listening to music through one's ears" as the main subject.
With all respect to everyone here whom I have argued with, ask yourselves, what is the point of Hifi or as a concept?
- I put it to you, that it is a Trick show! Smoke and mirrors! All geared up to create a realistic Illusion that one is hearing an actual piece of music, played by actual players, while in fact it is nothing but artificially created sound waves using a transducer.
- And to what end? Again, I put it to you, to satisfy one's ear/brain.
Realistically, for him to take his wallet out!
If you accept the above and keep that in mind at all times, it becomes very clear that Science is used as a means to achieve that end.
It is ludicrous to suggest to a Hifi buyer/end user that what he hears is just an illusion, and he should not really trust his ears, but he should look at lab results and charts instead.
Lab tests, measurements, are just a means to check for flaws and mistakes - nothing more. Obviously, a device with a flawed lab test result is not going to sound good, but the reverse may not be true. (here it comes ...... all the name-calling!). In my culture (Amir's too) we have a saying, A walnut is round, but not anything round is a walnut.
We choose what to measure based on scientific understanding of the design, to believe we have reached the zenith of knowledge is foolish and dangerous, as it would stop further research.
No I am not suggesting there is magic in there we yet do not know! Not at all. We should use all our known scientific knowledge to consistently better ourselves, but loosing sight of the end goal is just short-sightedness.
Hifi was conceived to trick us into believing that we are listening to real music while we are on a plane, train or in our homes.
Our ears, our brains, our desires together with our fallibilities are the major part and consideration of a Hifi system.
Please see the bigger picture.
 

BDWoody

Chief Cat Herder
Moderator
Forum Donor
Joined
Jan 9, 2019
Messages
7,116
Likes
23,753
Location
Mid-Atlantic, USA. (Maryland)
I’d like to ask a question; is it possible that some audible differences, perceived by some but not others, and are simply unmeasurable at the moment? In other words the current science says there is no difference between X and Y, but does that by definition mean that there IS no difference?

You would think someone would have demonstrated that to be a false premise by now.

There has not been a single instance, ever, anywhere, by anyone that has identified an audible difference that wasn't measurable.

Don't you think that if you were selling based on SQ improvements, that you'd want to demonstrate that if you possibly could? I would.

There are thousands of people on this forum who would pay absurd amounts for a component that could be demonstrated to improve the sound. What most of us won't do is buy based on a story with no evidence.
 

Mike F

Member
Joined
Jun 8, 2022
Messages
6
Likes
3
‘There has not been a single instance, ever, anywhere, by anyone that has identified an audible difference that wasn't measurable.’

But if your only method for discerning a difference is measurement then this is just self fulfilling, isn’t it? i.e. There is no difference because I can’t be measured........
 

Ken Tajalli

Major Contributor
Forum Donor
Joined
Sep 8, 2021
Messages
2,200
Likes
1,980
Location
London UK
‘There has not been a single instance, ever, anywhere, by anyone that has identified an audible difference that wasn't measurable.’

But if your only method for discerning a difference is measurement then this is just self fulfilling, isn’t it? i.e. There is no difference because I can’t be measured........
I have to agree with @BDWoody here.
Think about it, if we can not quantify something, we can not recreate it, or do anything about it.
 

HarmonicTHD

Major Contributor
Joined
Mar 18, 2022
Messages
3,326
Likes
4,846
addictedtoaudio.com.au/blogs/how-to/how-to-pick-the-best-filter-setting-for-your-dac

I wrote “all DACs that measure well don’t sound the same”. In fact, I expect they would measure different if they sound different. And I would expect much of the difference for DACs that measure well (but not identically the same) would be from different filters, that will measure differently.
You never provided, after having been asked several times by multiple posters to present evidence for your claim „that DACs sound different“ although they are SOTA and measure the same. Therefore your argument is fundamentally flawed.

You most likely perceive a difference due to your perceptional biases, like every human being. You can therefore only proof the correctness of your claim by excluding those biases with an ABX listing test. See the link I provided to you previously.

Everything else is just hearsay, anecdotes …
 
Last edited:
Top Bottom