• WANTED: Happy members who like to discuss audio and other topics related to our interest. Desire to learn and share knowledge of science required. There are many reviews of audio hardware and expert members to help answer your questions. Click here to have your audio equipment measured for free!

Master Thread: Are measurements Everything or Nothing?

krabapple

Major Contributor
Forum Donor
Joined
Apr 15, 2016
Messages
3,168
Likes
3,715
addit : this was moved from the thread about denafrip ares ii, but that was actually the point of the post, not to fan a flame war about measurable vs. subjective splendour

still, now it's over here, and i guess i can either read 200 pages of conflicting opinions, or look somewhere else

perhaps this is not the forum i am seeking? anyway, back to the original post :

been reading a lot about this technology, and watching youtube reviews by people who generally seem impressed by it

yeah, i know that readings are some indication of technology's "measure", but ears are an individual's unique portal to the brain

so when the purists are outraged that people couldn't possibly "hear" the differences, i would like them to explain how they can be so assured that their experience is the real one?

i am a would-be audiophile, and i can quite clearly hear the differences between the components that led me to start my upgrade journey


Based on the torrent of same old same old nonsense that followed, I'd bet good money you can't.

Perhaps this is not the forum you are seeking.
 
Last edited:

krabapple

Major Contributor
Forum Donor
Joined
Apr 15, 2016
Messages
3,168
Likes
3,715
I want to listen to the source, not to the gear.
That's not really possible, unless you are at the original acoustic event. Or if by source, you mean what the production engineer(s) heard when they played back the final recording for approval, that too can only be approximated. Fortunately, well enough.

This is the main point that we do not share. At all.
I do not see us (the human kind) being anywhere close to 100%. About anything. Not even sure if we should.

And audio in particular is (still) a very tough nut:
  • even Law of Gravity level of certainty is almost impossible in audio, if only because brains/preferences are in the middle of the "problem".
  • many around here see those DACs/AMPs at -120 dB THD and seem to think: houray, distortion will be extinct by next week! Really!? Will amazon deliver my 120 dB speakers next week? Or this century?
  • Probably even more important, will I like those 120 dB speakers? Will anyone? (particularly with the current music recorded with whatever mics at whatever SINAD ). Oh wait, we do not even have a most basic metric to connect those "measurements are always right" with the things people actually want to know: audibility and preference.
  • we are 60+ dB away from 120 dB (a sort of "magic number of SINAD" according to some). And we have no idea about the distance between that number and the actual 100% 'perfection'. That doesn't even fit my definition of 'close'. Look ma, we are this far

All of your complaints point to the transducers in an audio system. No is claiming those are audibly transparent. But the rest can be. So yes, '100% black and white' of predicting performance from measurements is certainly approachable for other parts of the system. And when gear is sonically identically *under controlled conditions* then 'preference' must be entirely determined by factors other than sound.
 

Vacceo

Major Contributor
Joined
Mar 9, 2022
Messages
2,635
Likes
2,753
That's not really possible, unless you are at the original acoustic event. Or if by source, you mean what the production engineer(s) heard when they played back the final recording for approval, that too can only be approximated. Fortunately, well enough.
It is perfectly possible as my objective is listening to what is recorded. Sure, it will not be 100% true to the recorded product, but I can get quite close.

What I am clearly not interested is in listening to what the gear manufacturer thinks is an improvement.
 

Killingbeans

Major Contributor
Joined
Oct 23, 2018
Messages
4,089
Likes
7,547
Location
Bjerringbro, Denmark.
Pollworkers get the results they are hired to get. Whether the results are accurate is usually (but not always) beside the point. :( Jim

True... I really hope that's what we're seeing in effect here. Still, polls easily being rigged is just another sad fact of life.
 

lashto

Major Contributor
Forum Donor
Joined
Mar 8, 2019
Messages
1,045
Likes
535
A couple of fallacies here. Of course we can study brains/preferences, and how we hear. It's being approached from different directions - measurements, blind testing, study of auditory centres, Science can and is coming along in telling us a fuller story here. And if you think we don't know a lot about audibility and preference, it's time for you to read Toole's book and the science in that field.

Secondly, you confuse SINAD numbers and potentialities with the actual "problem". Defining the problem is one of the things audiophiles have difficulty with, and I'll include myself in that. But just consider that most rooms won't let us near 120dB dynamic range above the noise floor. And consider that no instrument has 120 dB dynamic range from the usual position of a listener to a performance - nor does the human voice. You're asking the wrong question.
.
Thirdly, you are making the common mistake of wanting the best, or expecting perfect reproduction. It's best to think of audio as a representation, just as a TV picture or a home theatre system does no more than a representation. Looked at in that way, we've already achieved a huge amount.

Thanks for the effort but we seem to have some serious communication issues: my 'questions' are rhetorical. There might be some bad rhetoric but I am not "asking the wrong question". I am not even asking. And there is no need to answer.

Communication issue no2: I like sarcasm (bad habit I know) and your detector seems to be off. Your answers contain many good points but you are not talking to me. The 'person' you are answering to might be a sort of opposite of me. I blame sarcasm. Baad bad sarcasm :)

With that in mind, you may re-read my post and figure out that I am simply annoyed by those 'some' who present 120dB devices as a sort of "final perfect solution for everyone" (not even sure to what problem). And whoever does "the common mistake of expecting perfect reproduction" might not be me. I even use a SET amp sometimes. With tubes. The horror! :)

P.S.
Looks like at least @Xulonn got my point about 'the some', so my message was not impossibly thick. That was also the only point (somewhat) worth of an answer.
Not sure what everyone else is answering to but yeah, that is the usual result of posting "rhetorical sarcasm". Just move along...
 
Last edited:

Xulonn

Major Contributor
Forum Donor
Joined
Jun 27, 2018
Messages
1,828
Likes
6,311
Location
Boquete, Chiriqui, Panama
I even use a SET amp sometimes. With tubes. The horror! :)

P.S.
Looks like at least @Xulonn got my point about 'the some', so my message was not impossibly thick. That was also the only point (somewhat) worth of an answer.
Not sure what everyone else is answering to but yeah, that is the usual result of posting "rhetorical sarcasm". Just move along...
My main system uses either a 78db (according to Amir's testing, dominated by 3rd harmonic distortion) SINAD ICEpower 200ASC amplifier, or a 30lb YarLand Class A 6CA7 PP vacuum tube amplifier. The YarLand is probably in the 60dB SINAD range. I can hear no buzz or hum with my ears very close to the speakers, and with my IOTAVX AVP volume control at a fairly loud normal listening level, but no source playing.

Twenty years ago, in the living room, I had a pair of Klipsch Forte II's with a Tektron 2A3 SET driving them. Sources were a YBA CD player and a McIntosh MR74 tuner. In the dining room was a second audio system with a Bryston B60R integrated driving a pair of Apogee Centaurus monitors.

All of those systems were dead quiet and played music that I enjoyed thoroughly. In my world, 120dB SINAD is not nirvana - at least for enjoyig music.
 

lashto

Major Contributor
Forum Donor
Joined
Mar 8, 2019
Messages
1,045
Likes
535
All of your complaints point to the transducers in an audio system. No is claiming those are audibly transparent. But the rest can be. So yes, '100% black and white' of predicting performance from measurements is certainly approachable for other parts of the system. And when gear is sonically identically *under controlled conditions* then 'preference' must be entirely determined by factors other than sound.
In my 'complaints' I only considered the reproduction part, between the 'CD' and the ears, so yes transducer-only-complaints. Some talk about a much 'longer' audio system, i.e. everything between the microphone and the ears. A lot more problem-parts in there and one can easily extend the complaints ...

In terms of "predicting performance from measurements" I'm only asking for a bit less: predicting audibility/preference from measurements. A 'simple' metric to correlate HD and preferences/audibility. People used to try and a lot of metrics were proposed between Shorter/195x and GedLee/200x. Apparently none of them won the day. And nowadays nobody seems to be even trying. So, I'm complaining :)
 
Last edited:

sonitus mirus

Active Member
Joined
Apr 19, 2021
Messages
256
Likes
336
The part between the CD and an ear requires something that plays/runs a CD to interpret a digital file and produce an analog output signal, and that output signal requires a transducer to make audible sounds that humans can hear.
 

Galliardist

Major Contributor
Joined
Jun 26, 2021
Messages
2,558
Likes
3,273
Location
Sydney. NSW, Australia
Thanks for the effort but we seem to have some serious communication issues: my 'questions' are rhetorical. They might be 'bad' but I am not "asking the wrong question". I am not even asking. So, no need to answer.

Communication issue no2: I like sarcasm (bad habit I know) and your detector seems to be off. Your answers contain many good points but you are not talking to me. The 'person' you are answering to might be a sort of opposite of me. I blame sarcasm. Baad bad sarcasm :)

With that in mind, you may re-read my post and figure out that I am simply annoyed by those 'some' who present 120dB devices as a sort of "final perfect solution for everyone" (not even sure to what). And whoever does "the common mistake of expecting perfect reproduction" might not be me. I even use a SET amp sometimes. With tubes. The horror! :)

P.S.
Looks like at least @Xulonn got my point about 'the some', so my message was not impossibly thick. That was also the only point (somewhat) worth of an answer.
Not sure what everyone else is answering to but yeah, that is the usual result of posting "rhetorical sarcasm". Just move along...
I assumed that the sarcasm was addressed at me: that in turn you are mistaking my position for the one which you attacked.
But you are still making the same mistakes as in your previous post: you are claiming that there are things which are not understood, when they are: you are mistaking SINAD for science and a scientific approach. Just because this is framed in a sarcastic attack on a perceived position doesn't make those mistakes go away.

In my 'complains' I only considered the reproduction part, between the 'CD' and the ears, so yes transducers-only. According to some, the "audio system" is much 'longer', i.e. everything between the microphone and the ears. A lot more problem-parts in there and one can easily extend those complains ...

In terms of "predicting performance from measurements" I'm only asking for a bit less: predicting audibility/preference from measurements. A 'simple' metric to correlate HD and preferences/audibility. People used to try and a lot of metrics were proposed between Shorter/195x and GedLee/200x. Apparently none of them won the day. And nowadays nobody seems to be even trying. So, I'm complaining :)
Much better, and maybe closer to what you mean. But have you not notice the scoring done in response to speaker measurements here? This goes some way to what you are looking for, doesn't it? Though I suspect the speaker/room/listener part is somewhat simplified for the purpose.

The metrics you refer to seem to fail in repeatibility across a larger population. We can see that in the statistical analysis of preference testing in more recent studies, very little if anything scores 100% across a valid sample. So the best we can do for an individual is determine where in the population they lie. On top of that, the point about understanding people that you also made comes into it. In practice, we don't listen blind to our systems, and the hole is there. Our ears are imperfect transducers and the brain has to sort out what is being heard.

My complaint would be that we don't have a decent definition of how we interpret sound. Sure, around here you'll see lots of references to cognitive bias and such, but I don't think that cuts it.
And I think there is another dichotomy at play here, which is that the system/equipment purchasing process and the listening process may work against each other. The theory of planned behaviour would sit well with me for our relationships with our systems when it comes to buying, upgrading and installation. And it may fit what music we choose to listen to. But it doesn't fit the activity of listening to music, and I'm not sure where to look for to describe that process.
 

scruffy1

Member
Joined
Sep 19, 2019
Messages
80
Likes
53
here we go

I know what I like
And I like what I know
Getting better in your wardrobe
Stepping one beyond your show
When the sun beats down and I lie on the bench
I can always hear them talk

(somewhat superfluously:
Me, I'm just a lawnmower
You can tell me by the way I walk)

anyway, measuring distortion (or lack of it) will apparently tell me the engineers have made a good piece of gear; what it won't tell me is how it sounds to me

rather apparent is that better numbers mean it is better, with regards to numbers; whether one can extrapolate the perceived beauty of the output from that parameter seemingly will remain a vexed question, because the engineers like a good graph, and their antagonists like what they like and frustrate them by being dogmatic that numbers aren't all, whereas engineers seem dogmatic that they are

I think it's time we stop
Children, what's that sound?
Everybody look, what's going down?

There's battle lines being drawn
Nobody's right if everybody's wrong
Young people speaking their minds
Getting so much resistance from behind

ohm my goodness, by some measure

(runs back into the wardrobe)
 

scruffy1

Member
Joined
Sep 19, 2019
Messages
80
Likes
53
thank you, thank you :p

i'm here all week (if you're unlucky) :facepalm:
 

voodooless

Grand Contributor
Forum Donor
Joined
Jun 16, 2020
Messages
10,226
Likes
17,803
Location
Netherlands
because the engineers like a good graph, and their antagonists like what they like and frustrate them by being dogmatic that numbers aren't all, whereas engineers seem dogmatic that they are
This isn’t about dogma. People don’t just “like what they like”. There are reasons for this. The scientific method lets you figure out what those reasons are. Turns out: in many cases, it has very little to do with the actual objective performance of the device. Many people get veils lifted by putting crystals on chips and enclosures, putting stones on cupboards, or putting strange wooden contraptions (which are not diffusers) in their rooms. These are obviously the extremes. But when is it no longer extreme? Where is the dividing line? When does common sense say: well this can't be right... To then ignore it because some reviewers said the product is a game changer :facepalm:. This is the problem right there: we need science to distinguish between what is actually real, and what is really not, and if not, why that is.
 

scruffy1

Member
Joined
Sep 19, 2019
Messages
80
Likes
53
alles gut! i just watched (and enjoyed) amir's video @

i now appreciate where the scientists are coming from, and their requirement for reproducible appreciation of "better", but it raises a greater angst for me

short of auditioning any potential new component, with rigorous set up of the necessary testing equipment and blinded switching (none of which i can easily manage, or even uneasily), where to from here to consider "upgrades" ? or why to bother

at risk of being stoned as a heretic for the suggestion, some components are a very (to my unblinded, non-scientific appraisal) definite step up : the switch from onboard sound to a decent external dac (yay! the khadas tone board before rebranding to tone 1), the resolution of headphones (audiotechnica ad700 < akg q701 < focal elear) - maybe i am fooling myself, but doing any blinded testing of headphones would seem impossible

i have often suggested the key to audio happiness is being pleased with what you have, and then make it your business to avoid exposure to anything that sounds better

so how to know (or actually achieve gnosis) of whether a new item is a better choice than what i currently possess ?

do the non-professional audioscientists have a method, or just randomly select anything and do the leg work with amir's techniques ? if they do, i salute their ocd as time and finances and patience to set up the bits is unlikely to eventuate in my universe

price vs. performance is one confounder (the khadas broke that idea); the sinad measure of dacs sorta simplifies comparison, but for every other component, maybe not so easy

thanks for some enlightenment, but on digesting amir's points, i am dissuaded from acquiring other equipment because there seems to be no reliable information in "opinion", and no particularly decipherable expectation of "my unique personal experience" in scientific monitoring - one subject's ability to achieve p< 0.05 has no bearing on what my results might be

and i am, after all is measured, lazy wrt the rigours involved in proving i like something i don't have if i like something i have quite a lot already


perhaps my energies are better spent designing and building little aeroplanes, and enjoying music with my existing hardware
 

BDWoody

Chief Cat Herder
Moderator
Forum Donor
Joined
Jan 9, 2019
Messages
6,948
Likes
22,625
Location
Mid-Atlantic, USA. (Maryland)
so how to know (or actually achieve gnosis) of whether a new item is a better choice than what i currently possess ?

It's called learning. Educate yourself.
 

scruffy1

Member
Joined
Sep 19, 2019
Messages
80
Likes
53
Although you may use the information here to fuel a more canny path to upgrades, I have never viewed the purpose here as fostering "upgrade-itis".

i'm not big on upgrading for upgrade's sake, as much as for improvement's sake

my personal path has indeed been influenced by "opinion" (esp. wrt the khadas, but also the elears), and i found it interesting that the sinad assessment of the khadas supported my own experience that it kicked serious arse (yes, even sans double blinded trials), which has remained kicked... and naively hoped that the hype around the denafrips might insinuate a similar discovery to be made

sometimes there is reason for hype, and "proof" of why is a later discovery

It's called learning. Educate yourself.

thank you for your input

i retract my earlier comment - i get the message, but the presentation is a bit terse to appreciate the good intention i now think you meant; curse the interweb for making nuance a guess more often than might be ideal
 
Last edited:

voodooless

Grand Contributor
Forum Donor
Joined
Jun 16, 2020
Messages
10,226
Likes
17,803
Location
Netherlands
i now appreciate where the scientists are coming from, and their requirement for reproducible appreciation of "better", but it raises a greater angst for me
That's exactly what the marketing departments of many audio companies capitalize on. The same goes for review outlets, you must remember that to them, you are not the customer, but the product, sold to audio companies as potential buyers.
short of auditioning any potential new component, with rigorous set up of the necessary testing equipment and blinded switching (none of which i can easily manage, or even uneasily), where to from here to consider "upgrades" ? or why to bother
Sometimes you should not bother. It's always good to know when. This needs knowledge and understanding. Blind faith in your senses won't give you that.
at risk of being stoned as a heretic for the suggestion, some components are a very (to my unblinded, non-scientific appraisal) definite step up : the switch from onboard sound to a decent external dac (yay! the khadas tone board before rebranding to tone 1), the resolution of headphones (audiotechnica ad700 < akg q701 < focal elear) - maybe i am fooling myself, but doing any blinded testing of headphones would seem impossible
But how do you know when something is an obvious definite step up, vs an imagined one? Trusting in science gives much more consistent answers than in senses.
i have often suggested the key to audio happiness is being pleased with what you have, and then make it your business to avoid exposure to anything that sounds better
That sounds like excellent advice.
so how to know (or actually achieve gnosis) of whether a new item is a better choice than what i currently possess ?
Knowledge is achieved by learning.
do the non-professional audioscientists have a method, or just randomly select anything and do the leg work with amir's techniques ? if they do, i salute their ocd as time and finances and patience to set up the bits is unlikely to eventuate in my universe
Measurements are only part of the story. There are also features, looks, prices, etc. And in the case of speakers, there is really no other option than to listen, because you have so much interaction with your room, and preference is such a big part of the speaker experience as well. Still, measurements will tell you already a lot before you get a speaker in your home. And science also tells us something about how on average people will like a certain speaker. That doesn't mean you'll like it of course.
 

Killingbeans

Major Contributor
Joined
Oct 23, 2018
Messages
4,089
Likes
7,547
Location
Bjerringbro, Denmark.
thank you for your input; your snide rating is high

He's blunt, but he's not wrong. If you want actual upgrades, educating yourself will let you focus on the 20% that makes 80% of the difference.

Going from an on-board DAC to an externeal DAC doesn't guarantee anything. They can both be garbage. They can both be audibly sufficient. They can both cause ground loops.

You need measurements on both the on-board solution and the external "upgrade", if you want to verify any increase in performance reliably*.

If you can't or won't do the measurements, educating yourself is a great alternative. It gives you the option of making an educated guess on the odds of the "upgrade" doing anything that isn't placebo, and whether that thing has a magnitude that's worth your time and effort. It's not easy to suppress FOMO, but accumulating well-founded knowledge about what should and shouldn't worry you helps a lot.

Besides, placebo can be accounted for, but it can't be eliminated. Nothing wrong with using it to your advantage. Just don't mix up the causes and effects.

*Or do a proper blind comparison.
 
Last edited:

Killingbeans

Major Contributor
Joined
Oct 23, 2018
Messages
4,089
Likes
7,547
Location
Bjerringbro, Denmark.
i have often suggested the key to audio happiness is being pleased with what you have, and then make it your business to avoid exposure to anything that sounds better

Probably not a bad idea.

Especially when you consider that the quality of the exposure can be heavily dependent on both the situation and the circumstances of it.

Also, mental burn-in can be both your best friend and your worst enemy.

But then again, it wouldn't be a hobby if it was totally rational? :D
 
Last edited:

scruffy1

Member
Joined
Sep 19, 2019
Messages
80
Likes
53
knowledge is achieved by learning.
so i am told, repeatedly here

i am a lifelong learner, but some subjects are beyond my limited resources, both financially and available time and energy

triage applies

Measurements are only part of the story. There are also features, looks, prices, etc. And in the case of speakers, there is really no other option than to listen, because you have so much interaction with your room, and preference is such a big part of the speaker experience as well. Still, measurements will tell you already a lot before you get a speaker in your home. And science also tells us something about how on average people will like a certain speaker. That doesn't mean you'll like it of course.
headphones likewise; i do know i much prefer open backed options
 
Top Bottom