• WANTED: Happy members who like to discuss audio and other topics related to our interest. Desire to learn and share knowledge of science required. There are many reviews of audio hardware and expert members to help answer your questions. Click here to have your audio equipment measured for free!

Master Thread: Are measurements Everything or Nothing?

D

Deleted member 23982

Guest
Can I recommend the following video, and particularly the 39:00-41:00 minute mark?
There is a lot of good info in this video regarding placebo and testing, but that particular section makes an important point: being susceptible to placebo does not make you psychotic - it makes you human.
thanks for the video, i really enjoyed watching this guy :) he tells alot of truth and also watched some more videos of him because he has a really nice way to talk/explain things

also good to hear that professionals actually hear a difference between interconnect cables... (mainly cause of better shielding, inductance, capatitance etc but its certainly not just "if sound gets through it sounds the same", sure there is a point easly reached where it gets hard to distingiush different cables (diminishing returns..) but this point isnt reached with 10$ cables from what i can tell)

and sure placebo is a thing but its not as "universal" as people put it here, i mean it "can" be, it doesnt mean "it has to" ...

but yea i dont wanna start discuss again about it, see my previous post... just wanted to say thanks for the video/recommending this guy
 

storing

Active Member
Forum Donor
Joined
Aug 27, 2021
Messages
226
Likes
220
Here we go again :)

I won't anymore, but one final tip: refrain from using diminuitive phrasing like "sciences" stuff, quoted. It makes you look like a troll who isn't willing to learn anything at all and will lead to people not taking you seriously no matter what you say..
 

Killingbeans

Major Contributor
Joined
Oct 23, 2018
Messages
4,098
Likes
7,578
Location
Bjerringbro, Denmark.
because apparently having a own opinion counts as "trolling" lol

Opinions are fine. They are just hardly ever being presented as such.

It's never: "This is just my opinion. I know there's a risk that it's all in my head, but I'm having fun, so I don't really care. I see the points you're making, and I get it, but it's just not my bag."

99% of the time it's: "Don't tell me what I'm hearing! You guys are just condescending and narrow minded!"

The opinions that seem dreary to ASR members are those that have been mislabeled as facts in the audiophile hobby/industry, despite mountains of reasearch pointing to them as make believe. They are not new or revelating in any way, just excruciatingly persistent.

for me it doesnt really matter because i know what differences are there/real

Again, that old saying "you can lead a horse to water but you can't make it drink" ;)
 

BDWoody

Chief Cat Herder
Moderator
Forum Donor
Joined
Jan 9, 2019
Messages
7,082
Likes
23,537
Location
Mid-Atlantic, USA. (Maryland)
also good to hear that professionals actually hear a difference between interconnect cables...

Can you show that result from a test with proper controls? Without them, folks are likely to hear all kinds of things that may not be real.

This isn't evidence...its just presented as if it were.
 

Plcamp

Addicted to Fun and Learning
Joined
Jul 6, 2020
Messages
860
Likes
1,318
Location
Ottawa
Seems like so many review threads get challenged with:

1. Measurements are not everything.

2. You all never listen.

3. I trust my ears, not graphs.

4. I don't listen to graphs. I listen to music.

5. You all must not listen to music at all.

6. Why don't you all buy the best SINAD gear?

7. I have heard your best SINAD gear and they sound terrible. I don't like any of this Chinese stuff.

8. You don't trust your ears. I/we do.

9. All these reviewers/youtubers/audophiles say these amps, DACs, etc. sound different and you say they don't. They can't all be wrong.

10. Surely designers have created certain house sound for each equipment which your measurements don't show.

11. Your measurements are only at one frequency. You need to also measure X, Y and Z like impulse response, slew rate, etc., etc.

12. You guys run a cult here where you only go by measurements and no one is allowed to disagree.

On and on...

I have had to answer these so many times that I thought it is time to stop having them go into every review as they are not product specific. From here on, any such questions should be posted here. Answers will be given in this thread and simply referenced in future challenges in other threads.

@AdamG247 and @BDWoody, please direct any future posts in review threads to here and not allow discussions there.

Thanks. You all are free to discuss this topic, provide answers, argue, whatever, in this thread. :)
I think measurements combined with statistical tests that demonstrate human thresholds of detection are “everything useful we’ve got” even if not fully comprehensive. That’s the message to doubters.

There are areas less than comprehensive (I would suggest speaker intermodulation distortion might be one) either because we lack best understanding of what humans do detect, or we don’t have a scalable metric to judge how close we are to it. In such cases I see folks working on that.

I come here because my suspicions of what matters can be validated or dismissed by folks with cogent arguments, and I don’t find it difficult to exclude opinion from others. Is pretty obvious who knows what IMO.
 

Digby

Major Contributor
Joined
Mar 12, 2021
Messages
1,632
Likes
1,560
I don't want to read the entire thread, it's so long, but I want to make a point that I hope has some value and hasn't been done to death already (apologies if so).

I don't understand the, to my mind false, binary split. Surely there is plenty of subjective information that has value, especially when gathered from many listeners in a controlled setting; even if the setting isn't a controlled one, it doesn't mean what a listener hears isn't related to actual playback, as a matter of course. A subjective opinion can be purely a figment on one's imagination and some people are more imaginative than others, but there are many occasions when humans have to trust their instincts before an abundance of data comes back to affirm something (or else there would be far fewer of us on this planet). We may not be as accurate as machines, but that doesn't mean we are wholly inaccurate.

If a subjective reviewer is tested and it is found their opinions seem to chime with the objective data, then surely their opinions have value, to some extent. The correlation would prove it isn't purely a feat of imagination on their part.

This is an argument, to paraphrase, one hears a lot :

"objective measurements are everything and if you, as an individual, are hearing something that doesn't chime wholly with the measurements, then you must be the one at fault"

Well, maybe that is the case. Almost certainly (as Amir said, further up this thread) if you are hearing differences in cables and such, when they are proven to be working within spec, it is a figment of your imagination.

However, if you are hearing differences between loudspeakers, and that difference is not obvious from the measurements typically taken, is the science of loudspeaker measurement so settled that it can be said, without question, that what you are hearing is just a phantom, a figment of your imagination? I don't believe so.

The science isn't settled. The way the measurements are taken is not settled. Which measurements are taken and how is not settled. It may be 70% or 80% of the way there, higher still perhaps, but it is not 100%; therefore it seems unlikely that the measurements alone (as it pertains to loudspeakers) would convey 100% of the listener experience and/or preference for any particular loudspeaker.

This would suggest, regarding loudspeakers at least, you cannot trust the data on its own, and that you do you own ears a disservice when you say they have no role to play in the process, let alone suggest that they exist only to trick you.

Look at the Harman curves. This is subjective data collected, en masse, for the purposes of understanding listener preferences. The preference curves seem to suggest that people prefer, by and large, one or more downward slopes at high frequencies. Well, this isn't accurate reproduction, is it? An accurate reproduction of the signal would be a completely flat FR in room, no? Yet, if the overwhelming majority of people will find such a scenario "too bright" for their tastes, then it is likely neither here nor there whether that would be the accurate thing to do regarding reproduction.

That there were strong correlations in how people preferred sound reproduction in the Harman tests, suggest that subjective preferential information is not anywhere near as scattershot and random as some suggest. You can't account for one individual very well, but when you have many people, then preferences often chime with each other to a greater extent than they don't.

The subjective way in which people appreciate sound has value. The value depends on the individual, of course, but manufacturers like Harman seem to be under no illusion that a deviation from accurate, isn't necessarily a deviation for the worse; if they thought that way, they wouldn't have bothered doing the studies in the first place and, if it was found that many/most peoples tastes deviate from accurate reproduction, they would have stuck the information in the trash and not published it.

Will people look back at our posts in 100 years (as we look back at the writings of those 100 years in the past) and giggle at how silly we were to believe x thing, when that thing was proven to not be the case, as far as they are concerned, long ago.

We are only following the science as we know it, but it is not settled, therefore the idea that the measurements taken, as they stand, represent wholly the experience a person might have listening to any given speaker, seems unlikely. To say that subjective impressions have no value whatsoever, or exist only to trick us, is also unlikely.

Any extreme position in either direction seems not to lead to a full, scientific understanding of the question.
 
Last edited:

AudioSceptic

Major Contributor
Joined
Jul 31, 2019
Messages
2,736
Likes
2,628
Location
Northampton, UK
He may not be the one who misunderstands...

That is all included in the analog output, which is what is actually measured. Our host doesn't open up the box and break out the chip...he is measuring the output where you plug in your cables.
Thanks for explaining what should be bleedin' obvious.
 

Purité Audio

Master Contributor
Industry Insider
Barrowmaster
Forum Donor
Joined
Feb 29, 2016
Messages
9,183
Likes
12,476
Location
London
Ultimately the decision over one speaker or another is always subjective, but it really is ( for me at least) useful to be able to initially narrow the choice through their measured performance .
Keith
 

AudioSceptic

Major Contributor
Joined
Jul 31, 2019
Messages
2,736
Likes
2,628
Location
Northampton, UK
You obviously do not understand the essence of the problem. DAC is not the only factor involved in the final sound. The complete design of the device is important, from the power supply, clock, analog part, etc. And ultimately a certain listening experience.
By "DAC" we mean the whole device, from digital input to analogue output, so that includes power supply, etc. We would say "DAC chip" or similar if we mean just that component.
 

SimpleTheater

Addicted to Fun and Learning
Forum Donor
Joined
Jun 6, 2019
Messages
929
Likes
1,814
Location
Woodstock, NY
“A product that does not measure well will never sound great. A product that does measure well may or may not sound great.”
—me
 

Frgirard

Major Contributor
Joined
Apr 2, 2021
Messages
1,737
Likes
1,043
I don't want to read the entire thread, it's so long, but I want to make a point that I hope has some value and hasn't been done to death already (apologies if so).

I don't understand the, to my mind false, binary split. Surely there is plenty of subjective information that has value, especially when gathered from many listeners in a controlled setting; even if the setting isn't a controlled one, it doesn't mean what a listener hears isn't related to actual playback, as a matter of course. A subjective opinion can be purely a figment on one's imagination and some people are more imaginative than others, but there are many occasions when humans have to trust their instincts before an abundance of data comes back to affirm something (or else there would be far fewer of us on this planet). We may not be as accurate as machines, but that doesn't mean we are wholly inaccurate.

If a subjective reviewer is tested and it is found their opinions seem to chime with the objective data, then surely their opinions have value, to some extent. The correlation would prove it isn't purely a feat of imagination on their part.

This is an argument, to paraphrase, one hears a lot :

"objective measurements are everything and if you, as an individual, are hearing something that doesn't chime wholly with the measurements, then you must be the one at fault"

Well, maybe that is the case. Almost certainly (as Amir said, further up this thread) if you are hearing differences in cables and such, when they are proven to be working within spec, it is a figment of your imagination.

However, if you are hearing differences between loudspeakers, and that difference is not obvious from the measurements typically taken, is the science of loudspeaker measurement so settled that it can be said, without question, that what you are hearing is just a phantom, a figment of your imagination? I don't believe so.

The science isn't settled. The way the measurements are taken is not settled. Which measurements are taken and how is not settled. It may be 70% or 80% of the way there, higher still perhaps, but it is not 100%; therefore it seems unlikely that the measurements alone (as it pertains to loudspeakers) would convey 100% of the listener experience and/or preference for any particular loudspeaker.

This would suggest, regarding loudspeakers at least, you cannot trust the data on its own, and that you do you own ears a disservice when you say they have no role to play in the process, let alone suggest that they exist only to trick you.

Look at the Harman curves. This is subjective data collected, en masse, for the purposes of understanding listener preferences. The preference curves seem to suggest that people prefer, by and large, one or more downward slopes at high frequencies. Well, this isn't accurate reproduction, is it? An accurate reproduction of the signal would be a completely flat FR in room, no? Yet, if the overwhelming majority of people will find such a scenario "too bright" for their tastes, then it is likely neither here nor there whether that would be the accurate thing to do regarding reproduction.

That there were strong correlations in how people preferred sound reproduction in the Harman tests, suggest that subjective preferential information is not anywhere near as scattershot and random as some suggest. You can't account for one individual very well, but when you have many people, then preferences often chime with each other to a greater extent than they don't.

The subjective way in which people appreciate sound has value. The value depends on the individual, of course, but manufacturers like Harman seem to be under no illusion that a deviation from accurate, isn't necessarily a deviation for the worse; if they thought that way, they wouldn't have bothered doing the studies in the first place and, if it was found that many/most peoples tastes deviate from accurate reproduction, they would have stuck the information in the trash and not published it.

Will people look back at our posts in 100 years (as we look back at the writings of those 100 years in the past) and giggle at how silly we were to believe x thing, when that thing was proven to not be the case, as far as they are concerned, long ago.

We are only following the science as we know it, but it is not settled, therefore the idea that the measurements taken, as they stand, represent wholly the experience a person might have listening to any given speaker, seems unlikely. To say that subjective impressions have no value whatsoever, or exist only to trick us, is also unlikely.

Any extreme position in either direction seems not to lead to a full, scientific understanding of the question.
Electronics: the war is finished since three or four decade.
Speaker: Measuring and listening are two separate fields that psychoacoustics tries to unite.
Start by acquiring there.
For science, the cheese maker does not need it to make cheese. Its presence on this forum and the others is only speculation.
 

Blumlein 88

Grand Contributor
Forum Donor
Joined
Feb 23, 2016
Messages
20,771
Likes
37,635
I don't want to read the entire thread, it's so long, but I want to make a point that I hope has some value and hasn't been done to death already (apologies if so).

I don't understand the, to my mind false, binary split. Surely there is plenty of subjective information that has value, especially when gathered from many listeners in a controlled setting; even if the setting isn't a controlled one, it doesn't mean what a listener hears isn't related to actual playback, as a matter of course. A subjective opinion can be purely a figment on one's imagination and some people are more imaginative than others, but there are many occasions when humans have to trust their instincts before an abundance of data comes back to affirm something (or else there would be far fewer of us on this planet). We may not be as accurate as machines, but that doesn't mean we are wholly inaccurate.

If a subjective reviewer is tested and it is found their opinions seem to chime with the objective data, then surely their opinions have value, to some extent. The correlation would prove it isn't purely a feat of imagination on their part.

This is an argument, to paraphrase, one hears a lot :

"objective measurements are everything and if you, as an individual, are hearing something that doesn't chime wholly with the measurements, then you must be the one at fault"

Well, maybe that is the case. Almost certainly (as Amir said, further up this thread) if you are hearing differences in cables and such, when they are proven to be working within spec, it is a figment of your imagination.

However, if you are hearing differences between loudspeakers, and that difference is not obvious from the measurements typically taken, is the science of loudspeaker measurement so settled that it can be said, without question, that what you are hearing is just a phantom, a figment of your imagination? I don't believe so.

The science isn't settled. The way the measurements are taken is not settled. Which measurements are taken and how is not settled. It may be 70% or 80% of the way there, higher still perhaps, but it is not 100%; therefore it seems unlikely that the measurements alone (as it pertains to loudspeakers) would convey 100% of the listener experience and/or preference for any particular loudspeaker.

This would suggest, regarding loudspeakers at least, you cannot trust the data on its own, and that you do you own ears a disservice when you say they have no role to play in the process, let alone suggest that they exist only to trick you.

Look at the Harman curves. This is subjective data collected, en masse, for the purposes of understanding listener preferences. The preference curves seem to suggest that people prefer, by and large, one or more downward slopes at high frequencies. Well, this isn't accurate reproduction, is it? An accurate reproduction of the signal would be a completely flat FR in room, no? Yet, if the overwhelming majority of people will find such a scenario "too bright" for their tastes, then it is likely neither here nor there whether that would be the accurate thing to do regarding reproduction.

That there were strong correlations in how people preferred sound reproduction in the Harman tests, suggest that subjective preferential information is not anywhere near as scattershot and random as some suggest. You can't account for one individual very well, but when you have many people, then preferences often chime with each other to a greater extent than they don't.

The subjective way in which people appreciate sound has value. The value depends on the individual, of course, but manufacturers like Harman seem to be under no illusion that a deviation from accurate, isn't necessarily a deviation for the worse; if they thought that way, they wouldn't have bothered doing the studies in the first place and, if it was found that many/most peoples tastes deviate from accurate reproduction, they would have stuck the information in the trash and not published it.

Will people look back at our posts in 100 years (as we look back at the writings of those 100 years in the past) and giggle at how silly we were to believe x thing, when that thing was proven to not be the case, as far as they are concerned, long ago.

We are only following the science as we know it, but it is not settled, therefore the idea that the measurements taken, as they stand, represent wholly the experience a person might have listening to any given speaker, seems unlikely. To say that subjective impressions have no value whatsoever, or exist only to trick us, is also unlikely.

Any extreme position in either direction seems not to lead to a full, scientific understanding of the question.
One subtle distinction. The target as discovered by Harman in preferences is flat on axis response anechoicly. The downward sloping you refer to is a measurement artifact of measuring in room. So the discovered preferences of most of their listening panels isn't away from accuracy.
 

Blumlein 88

Grand Contributor
Forum Donor
Joined
Feb 23, 2016
Messages
20,771
Likes
37,635
“A product that does not measure well will never sound great. A product that does measure well may or may not sound great.”
—me
I'm afraid I disagree. A product that does not measure well may never sound accurate. But not all recordings are accurate and some happy coincidences make some products capable of sounding pretty great.

A product measuring well is accurate. Accuracy isn't always great sounding.
 

Digby

Major Contributor
Joined
Mar 12, 2021
Messages
1,632
Likes
1,560
Ultimately the decision over one speaker or another is always subjective, but it really is ( for me at least) useful to be able to initially narrow the choice through their measured performance .
Keith
True, and that narrowing can be done, likely for anyone (if you understand their preferences), but I don't think it is, as yet, a 100% foolproof way of choosing a better or worse speaker, likely because the measurements themselves aren't wholly settled and there is disagreement of the precedence of one factor over another.

For example, flat FR and low distortion seems to largely take precedence over SPL ability for many on this forum. Many speakers reviewed are limited in their SPL, yet they are considered accurate by way of a reasonably flat FR and low distortion. They are accurate in some ways, but not in all ways.

Accuracy obviously has many dimensions and to boil it down to one or two measurements is not correct. Inaccuracy in one area can be balanced by accuracy in another. That a speaker has relatively low SPL abilities, may not matter to a particular user, but it does not make the speaker wholly accurate, even if it has a flat FR and low distortion.

One thing has been exchanged for another. Accuracy has been reduced in one area, for gains in another. The decision as to what matters is, to an extent, a subjective one (based on personal preference, room size, SPL requirements and so on).
 
Last edited:
D

Deleted member 23982

Guest
“A product that does not measure well will never sound great. A product that does measure well may or may not sound great.”
—me
agreed!

A product measuring well is accurate. Accuracy isn't always great sounding.
atleast it will be somewhat "closer to source", which it comes down to for me, preferences is a whole other story tho
accuracy is exactly what studios aim for and so should the listener imo
probably one reason why i really dislike headphones, 98% of headphones are "tinkered" to some preferences, specially all the consumer stuff

one thing that opened my eyes is how easy its to understand the "true meaning" of a song (most of the time kinda "funny" meanings/jokes :D people who got it know what im talking about) with studio monitors, its just not possible with non-flat speakers to hear this stuff or much, much harder to hear
 

voodooless

Grand Contributor
Forum Donor
Joined
Jun 16, 2020
Messages
10,405
Likes
18,366
Location
Netherlands
However, if you are hearing differences between loudspeakers, and that difference is not obvious from the measurements typically taken, is the science of loudspeaker measurement so settled that it can be said, without question, that what you are hearing is just a phantom, a figment of your imagination? I don't believe so.
Nobody denies audible differences in loudspeakers. This is just a straw man.
Look at the Harman curves. This is subjective data collected, en masse, for the purposes of understanding listener preferences. The preference curves seem to suggest that people prefer, by and large, one or more downward slopes at high frequencies. Well, this isn't accurate reproduction, is it? An accurate reproduction of the signal would be a completely flat FR in room, no?
Not nessesarily. It totally depends on how the recording and mastering was done. Next there is room interaction, which was also in effect at the moment of recording. You’ll need to compensate for this as well. Rooms are different, so the Harman curve is an average curve of an average room. This is quite a complex thing that can never be settled because the amount of variables are just to large, and can even differ from song to song.
Yet, if the overwhelming majority of people will find such a scenario "too bright" for their tastes, then it is likely neither here nor there whether that would be the accurate thing to do regarding reproduction.
People not liking something does not mean it’s not accurate.
The subjective way in which people appreciate sound has value. The value depends on the individual, of course, but manufacturers like Harman seem to be under no illusion that a deviation from accurate, isn't necessarily a deviation for the worse;
I doubt Harman claims “accurate”, they claim “preference”. Not the same thing.
 

Digby

Major Contributor
Joined
Mar 12, 2021
Messages
1,632
Likes
1,560
Nobody denies audible differences in loudspeakers. This is just a straw man.
That isn't what I meant. I meant does the measured data fully explain the difference in sound between one speaker and the next. Fully explain, in that everything we could want to know about how a speaker sounds can be seen from the data (as it is currently taken). Amir says it represents about 70%, so does that mean the data gathered is flawed or not extensive enough, or is the remaining data just subjective? (what is that 30% made up of)

People not liking something does not mean it’s not accurate.
I suppose the question is what is more important? Imagine 9 out of 10 people prefer a given loudspeaker that is less accurate (in some dimension) than a more accurate one. Accuracy is measured over many dimensions, what is accurate in one place may not be in another. How the inaccuracies in one place interact with the greater accuracy elsewhere might be interesting to study?

Essentially, where is greatest accuracy needed, and where is least needed. That kind of thing.

These are questions that don't need asking with cables, you can have it all at once and for pennies; with speakers you have to pick and choose your accuracies.
 

voodooless

Grand Contributor
Forum Donor
Joined
Jun 16, 2020
Messages
10,405
Likes
18,366
Location
Netherlands
That isn't what I meant. I meant does the measured data fully explain the difference in sound between one speaker and the next. Fully explain, in that everything we could want to know about how a speaker sounds can be seen from the data (as it is currently taken). Amir says it represents about 70%, so does that mean the data gathered is flawed or not extensive enough, or is the remaining data just subjective? (what is that 30% made up of)
I’d say on average, yes the data does exactly that. But note that only the spinorama is not enough. You’d need to take the waterfall plot into account, as well as maxSPL capabilities. An KH80 might be very accurate, it doesn’t go very loud. If you like loud, you’ll ever find the thing satisfying. And also note that with the law of averages it also means plenty of people will not fall into the middle part.
I suppose the question is what is more important?
Preference definitely is. I can have a super accurate speaker, if I don’t like it, I still won’t listen to it.
 
Last edited:
D

Deleted member 23982

Guest
An KH80 might be very accurate, it doesn’t go very loud. If you like loud, you’ll ever find the thing satisfying.
1. 108db (111db with two) isnt loud? what are you talking about lol 2. they are made for nearfield monitoring, you are not supposed to listen to them with listening position 6meters or more away
 

Digby

Major Contributor
Joined
Mar 12, 2021
Messages
1,632
Likes
1,560
108db at what frequency, not 40hz or 60hz and with what degree of distortion?

Nearfield speakers are limited in SPL, this is a lack of accuracy in one aspect. Whether this matters or not is subjective, but it is a type of inaccuracy (even though it isn't often regarded as such).
 
Top Bottom