• WANTED: Happy members who like to discuss audio and other topics related to our interest. Desire to learn and share knowledge of science required. There are many reviews of audio hardware and expert members to help answer your questions. Click here to have your audio equipment measured for free!

Master Thread: Are measurements Everything or Nothing?

Pdxwayne

Major Contributor
Joined
Sep 15, 2020
Messages
3,219
Likes
1,172
The level of diff indicated as you were using them they weren't 0.000x
SINAD differences were not the reason why I heard differences.

I have compared:
KTB
Topping L30
Topping D30pro
Gustard X16

All are 0.000x.

There are other reasons why people heard a difference.
 

audio2design

Major Contributor
Joined
Nov 29, 2020
Messages
1,769
Likes
1,830
SINAD differences were not the reason why I heard differences.

I have compared:
KTB
Topping L30
Topping D30pro
Gustard X16

All are 0.000x.

There are other reasons why people heard a difference.

If I am not mistaken when you did a test with the difference tool the difference was greater than two units with a 0.000x SINAD could produce. Hence how can you make said conclusion??
 

Andrej

Member
Joined
Mar 21, 2021
Messages
94
Likes
129
I am always baffled and amused by the deliberate overlapping of disjoint measures we use in audio. On the one side we have "objectivists", who are determined to achieve absolute fidelity, be it in a microphone transducing acoustic energy into voltage/current, an amplifier, which gives you more of the same thing, or a loudspeaker, transducing voltage/current into sound pressure (etc.). On the other side we have subjectivists who primarily care about how the final sound "sounds" to them. The problem is that the measures of "goodness" in the two cases are drastically different, both in kind and in the way one does the measuring. They do have a tiny overlap, but they are mostly orthogonal.

On the "objectivist" side the primary goal is fidelity which can be measured in numerous ways, some of which are easy and common, others not as easy and more rare. The benefit is that anybody doing them can replicate what the others have done with a very high probability of obtaining the same results. This is all part of the scientific process and the backbone of technological progress. Part of these evaluations are also subjective evaluations. However, in order to satisfy the repeatability condition, given that they are based on statistical analyses, it necessitates double blind testing on a large population with a focused set of objectives. We all know about people and their biases. Removal of those is part of those requirements, too.

On the "subjectivist" side the primary goal is to find out how much "satisfaction" one gets from listening to reproduced sound under some predefined conditions. I call it "pleasure", other terms can be applied with equal success. Since most of the time those are centered around a specific reproduction chain in a specific acoustic environment, at a specific time, anything can throw it off, making it impossible to be made repeatable to the level which satisfies scientific rigor. I am not implying that it is the wrong thing to do, just that it is different and not nearly as useful to others, if at all.

Here are a couple of specific experiences touching on both of these sides of the coin. I recently built a room for enjoying sound reproduction, equipped it with the best equipment I was willing to pay for, and was extremely happy with the sound. I did a lot of measurements and adjustments over time. However, one thing I noticed was that the same track of music could elicit a very different emotional response from me at different times. Sometimes it would amaze me how faithful and realistic the reproduction was, and at others I would be disappointed, expecting the same elation as some of the previous times. All I can come up is that I changed, as nothing else had! It made me doubt (even more) my ability to subjectively evaluate sound reproduction "quality" (in quotes, as it means different things to different people - it is imperative that we define the metric when stating that something is "better").

At another time, several years earlier, I was using a Behringer digital crossover which I found to be a terrifically useful piece of equipment. I had doubts about it's ability in the sense of fidelity (ie. SINAD etc.) but no real evidence. I was not equipped to measure it, but I could conduct a subjective test. As close to double blind as we could mke it, with changes between stimuli talking only several seconds. I collected half a dozen audiofile friends, came up with a varied list of good recordings, generated a random list to use in the experiments, and decided to just look for the ability to determine if there was any perceived difference in sound, depending on whether the device was in the signal path or not. (four types of stimuli: yes-yes, yes-no, no-yes, and no-no. Question was: are the segments the same or not. It took us some time to find the optimal length for the stimuli. Very short, so that you can remember it - not enough information. Very long, and you could not remember the first version. About 20 seconds worked for us. Ultimately, the listeners' choice (same or different) was completely random. And post-experiment comments mostly centered on: They sounded the same, but we KNEW that some were going to be different so we took that option every once in a while.)

The lesson I learned is that most electronics is so good these days that the burden for achieving good reproduction (both faithful to the original performance at the place of the microphone (objectivist) and satisfying my "pleasure" parts of the brain (subjectivist) - admittedly those get triggered in my brain if the objectivist targets are met:) fall on the energy trunsduction components (speakers) and the acoustics of the listening room.
I always measure, and always listen. It is rare that the two disagree, and if they do, after some investigation, I usually learn something new, useful and non-contradictory. Perhaps if we all tried to put ourselves into the other camp's shoes, we could stop bickering. But more than anything else, please define the metric before defining the performance!-) It would go a long way towards promoting harmony
 

Pdxwayne

Major Contributor
Joined
Sep 15, 2020
Messages
3,219
Likes
1,172
If I am not mistaken when you did a test with the difference tool the difference was greater than two units with a 0.000x SINAD could produce. Hence how can you make said conclusion??
What difference are you talking about?

All DAC I have, as measured by Amir, have sinad of 0.000x. My tools are not not good enough to show that 0.000x, but it doesn't mean those DAC I have do not have 0.000x.

With 0.000x DAC, some people still hear differences between them can easily be explained with:
1. Did not voltage matched
2. Did not filter matched
3. Can't channel balance matched
 
Last edited:

Geert

Major Contributor
Joined
Mar 20, 2020
Messages
1,938
Likes
3,526
I always measure, and always listen. It is rare that the two disagree, and if they do, after some investigation, I usually learn something new, useful and non-contradictory. Perhaps if we all tried to put ourselves into the other camp's shoes, we could stop bickering.
An important step is also understanding the 2 camps. The audio or sound engineering camp where people don't listen only exists in the confused mind of some subjectivist who believe to know better than the pro's.
 

rdenney

Major Contributor
Forum Donor
Joined
Dec 30, 2020
Messages
2,235
Likes
3,857
The lack of empathy shows in your post. I don’t want to be rude, sorry if it came out like that. The point is that you are leading the horse to the water ASSUMING he is thirsty and when he shows you he is not, you call him stubborn and/or dumb.

I think that is why when in your position, we are stuck.
Or, maybe you haven’t read enough yet to realize the depth and never-ending flow of the stream of apparently committed anti-data antagonists, and the effort many long-standing contributors here go to to help them shift their paradigm.

How much empathy is required? Maybe some of those antagonists ought to reciprocate by deciding not get their feelings hurt when experts are terse with them.

One can have a legitimate conversation about subjective evaluation, but not if one stubbornly insists that what they can perceive is beyond measurement. That leaves no ground for discussion. One can’t place all the blame for that on the people already here, who have defined the forum’s point of view.

Rick “time for the long pants” Denney
 
D

Deleted member 21219

Guest
The lack of empathy shows in your post.

With all due respect, this is aimed at the person, not the measurements, not the logic and not the science. It is a comment about emotion. Science, measurements and logic are not emotional. People are emotional. If I'm not mistaken, this thread is about measurements. It's not about people, their emotions, the causes they espouse ..... or their empathy or lack thereof.

5 VDC is 5 VDC, no matter how you feel about it. 10% distortion at 100 Hz is 10% distortion at 100 Hz regardless of opinions. When facts are described as "cold, hard facts", that doesn't mean that they are cruel. It means that they are without emotion. And that's the way it should be. A person with a fever doesn't run through the supply at a pharmacy and discard the red pills he needs because he thinks they look "hot", and instead looks only for blue pills because they look "cool". He'd be a dead person. It would be regrettable (emotion) .... but he would still be dead (measurement). Jim
 
Last edited by a moderator:

Pugsly

Member
Joined
Jun 25, 2021
Messages
98
Likes
122
the problem is: for those guys everything which is not measureable is placebo, thats the real problem here, they are in a massive deny about everything they cant "explain", i bet this has a way greater effect then placebo :)
also because, atleast me (and i bet many "true" audiophiles too) go to a subjective listening test way more objective than the "science guys" here, because they have a mindset of just denying things that cant be measured right from the start, and/or dont have the right gear to evaluate those things, if i cant hear them i would also believe they dont matter
Why do you so stubbornly refuse to understand how blind ABX testing works (and why it is important)?

Claim - there is an audible difference not captured by current measurements, so that two pieces of electronics that should sound identical according to all known relevant measured standards sound night and day different
Ok, let's prove it - ABX - if the difference in sound is really night and day, then you should be able to reliably identify which is which by sound alone.
Listening is kind of the crucial ingredient - indeed, the point of blind testing is to rely on hearing and not sight and other biases.
Results thus far -zilch. Lots of 'true' audiophiles have tried - and failed over, and over, and over.

Could you be the one to prove such an unknown factor exists because you (unlike the many who have come before you) can reliably differentiate two such pieces of equipment? In principle, sure, but you should not expect people to take your personal confidence and credulity seriously as arguments. Prove you can do this and you would be famous and would provide a great benefit to modern science. Asserting that you just know it is true while offering no evidence, however, is acting like the nerdy kid who expects everyone to believe them when they insist that their girlfriend who lives in a different city and who no one has ever met is super hot, but then insisting that the reason he has no pictures of her is because she is just so camera shy. Don't expect more than a polite 'Uh huh, sure.'
 

BoredErica

Addicted to Fun and Learning
Joined
Jan 15, 2019
Messages
629
Likes
899
Location
USA
They're obviously upset at being banned and very much not open to changing their mind anymore.

--
The problem is there are people in this forum who pretend they are on the fence or 'just asking questions' when deep down they formed an opinion long ago. I've been in conversation with people who say measurements are limited because of X, Y, Z, when it's clear they lack a very basic understanding of what they criticize. When I explain to them actually yes they do, and here's why, they often just flat out ignore my post or latch on to some other thing I write that's unrelated to try to further their argument because they were never interested in learning, it's a simple objectivist vs subjectivist debate. I dislike it when people hide their power level like that.

I can imagine someone in my shoes who keeps having dialogue with 'the other side' or people who 'just have questions' would be annoyed and cynical even when a person who genuinely is just asking questions for their own education. That's understandable but not ideal.

(In the case of ghostknight I didn't bother to read the entire exchange, and I'm not specifically talking about them.)
 
Last edited:
D

Deleted member 23982

Guest
Why do you so stubbornly refuse to understand how blind ABX testing works (and why it is important)?
i never said blind testing does "nothing", i just say it can be flawed just as much as a subjective opinion so its no "fool-proof" evidence either (specially with nocebo like i said in my last post)

atleast speaking for me i "refuse" to do blind test because i dont have the right gear nor the time, people helping me, actual incentive to do this for hours
tho i had the idea to record speaker output to make a easy a/b test in fubar but yea, any information gathered with this wouldnt be enough for you guys from what i readed

i also dont get the talking about needing 500 peoples, if 50 audiophiles say they hear a difference and actually can proof with ABX test they can hear a difference, what does it matter if 450 other people dont hear a difference? clearly there is a difference then.
 

BoredErica

Addicted to Fun and Learning
Joined
Jan 15, 2019
Messages
629
Likes
899
Location
USA
i never said blind testing does "nothing", i just say it can be flawed just as much as a subjective opinion so its no "fool-proof" evidence either (specially with nocebo like i said in my last post)

atleast speaking for me i "refuse" to do blind test because i dont have the right gear nor the time, people helping me, actual incentive to do this for hours
tho i had the idea to record speaker output to make a easy a/b test in fubar but yea, any information gathered with this wouldnt be enough for you guys from what i readed

i also dont get the talking about needing 500 peoples, if 50 audiophiles say they hear a difference and actually can proof with ABX test they can hear a difference, what does it matter if 450 other people dont hear a difference? clearly there is a difference then.
I thought you said multiple times you didn't want to have this conversation anymore? If so, why not just ignore replies here?

If you change your mind, I have a quick question. If there was a way to show you without a doubt that a large amount of people can have a major experience of something and for it all to be based on placebo/hallucination/etc, would that do anything to increase your uncertainty when it comes to reading people's uncontrolled subjective experiences? (This is a hypothetical, so just entertain the idea that such a thing could be shown.)

When I have an opinion/belief, sometimes I ask myself: What would it take to change my mind?
 
Last edited:

Pugsly

Member
Joined
Jun 25, 2021
Messages
98
Likes
122
If one person can prove that they can reliably, repeatedly, and to a statistically significant extent differentiate between two pieces of electronic equipment or cables that measures identically (or near enough to be considered until that time to be inaudible) then that would be enough... assuming that there are not methodological problems with the test (not really double-blind) or the results are not due to failing to adhere to already known required parameters (e.g., level matched, not clipping, etc. etc.) or equipment failure.
Isn't it neat how falsification works in science? An inductive argument can be defeated by a single counter-example. By contrast, there is nothing that can falsify your '100% certainty.'
 
D

Deleted member 23982

Guest
If you change your mind, I have a quick question. If there was a way to show you without a doubt that a large amount of people can have a major experience of something and for it all to be based on placebo/hallucination/etc, would that do anything to increase your uncertainty when it comes to reading people's uncontrolled subjective experiences?
i pretty much always take subjective expierences/descriptions with a grain of salt, specially because 2 individuals could describe the same thing with way different words/metaphors etc, tho for me it "becomes a fact" if i actually hear things myself (probably like 99% of people...)
thats the reason why i said, atleast for me speaking, i do listening test pretty much as objective as possible and not with the mindset of "whoa this will take my setup to a whole new level" if i read something new... (because there is also alot of stuff that indeed doesnt matter or does make things even worse, even if people still praise it...)

i just dont know the term for it but im aware of "studys" showing like a a lot of people getting sick because they are "aware" of others getting sick without getting "really" sick, pretty sure you are refering to this or something similar

When I have an opinion/belief, sometimes I ask myself: What would it take to change my mind?
well someone/something convincing you and the own opinion is pretty much the "greatest force" in this so it takes alot... thats why blindtests are not "off the table" for me and pretty surely the only thing that could convince me otherwise (and still there could be room for error specially with mic`ed speakers...) instead of trusting other peoples opinions, i guess thats the only common ground we have here guys x)

thats also the worst thing about it, i got this opinion of just trusting myself in the end because when i started, pretty surely everyone does google how much cable matter if they get their first pair of interconnects or whatever and if you read they dont matter and buy a 10$ pair on amazon and 1 year later actually changing them and you actually hear a difference.... you are just mindboggled once again after this expierence, same expierence with a palmer DI-BOX, once removed it just sounded so much different beside people saying they dont alter the sound, if for the better or worse is debatable i guess ( tho it could be the effect of the interconnects )
(while thinking about it, i started with me being biased on your side guys, till i heared those differences myself.... thats the thing... how could have this been placebo???)
 

voodooless

Grand Contributor
Forum Donor
Joined
Jun 16, 2020
Messages
10,227
Likes
17,807
Location
Netherlands
surely everyone does google how much cable matter if they get their first pair of interconnects or whatever and if you read they dont matter and buy a 10$ pair on amazon and 1 year later actually changing them and you actually hear a difference....
And surely you didn’t read the multitude of content claiming magical differences?
 
D

Deleted member 23982

Guest
And surely you didn’t read the multitude of content claiming magical differences?
not really at this point, it mainly became obvious because i started "easy" with 3,5mm to 3,5mm -> passive volume control -> rca to 6,3mm -> DI BOX -> XLR to XLR
and the first thing i noticed how it sounded way different as i removed the passive volume control because one channel was dead beside many claiming hearing no difference, pretty sure that was the first thing that got me going down the rabbit hole... surely now i will hear stuff like "yea passive controls can matter the rest is still placebo tho" but yea, just being honest
(it was this one btw https://www.thomann.de/be/fostex_pc...Ba4YKZtHDVHDFAHM3QDDbi1-HByn-eIUaAlXeEALw_wcB )

EDIT: and surely subjectivity is debatable but audiophiles claiming "night and day differences" just put them that way because they listen way more critical/closely than the usual listener, so while for one being "no difference at all or very very little difference" it could be for another one "night an day" depending on how you put things
and atleast for the volume control example, it became obvious that it does "something" before i went down the rabbit hole because if you get "free" sq (well not really free but without major speaker update) i said to myself why not... specially if new speakers would probably benefit from it just the same
 
Last edited by a moderator:

tvrgeek

Major Contributor
Joined
Aug 8, 2020
Messages
1,017
Likes
566
Location
North Carolinia
Omission of volume matching? :p
As far as the 5 DACs I had, No.. VERY clear differences in "glare" . I mean big walk by noticeable, wife in the other room notices. Able to reduce the 3100 band cut on the eq and bring up the top end a little on "bad" recordings.
I might give you as little of that for the low level detail resolution with headphones, but not much.
 

Geert

Major Contributor
Joined
Mar 20, 2020
Messages
1,938
Likes
3,526
i just say it can be flawed just as much as a subjective opinion so its no "fool-proof" evidence either
Can you explain how a methodology that's takes the necessary measures to avoid erroneous perception can be as wrong as not following a methodology to avoid bias or the placebo effect at all?
 

tvrgeek

Major Contributor
Joined
Aug 8, 2020
Messages
1,017
Likes
566
Location
North Carolinia
And surely you didn’t read the multitude of content claiming magical differences?
I actually had some "prestige" interconnects that did sound different. WORSE. Lack of top end. " How smooth" they were rated as. Back to my $6 Amazon, DIY 75 Ohm coax, or if you like well made, WBC from Amazon as it is the famous brand name wire. One of these days I may try 12 gauge zip cord and see it I notice any difference on my sub. Sticking with 16 on the mains.

Many years ago I had a problem with an illegal CB radio operator. He had a 1000W amp in his truck. My long cables picked it up. I put in Kimber which solved it until I had opportunity to "pin his coax" which solved the "footwarmer" problem for everyone.

No magic needed. Explainable and measurable physics.
 

Geert

Major Contributor
Joined
Mar 20, 2020
Messages
1,938
Likes
3,526
And surely subjectivity is debatable but audiophiles claiming "night and day differences" just put them that way because they listen way more critical/closely than the usual listener,
Which is exactly one if the reasons why they start hearing things that aren't there, especially if you're not a trained listener
while for one being "no difference at all or very very little difference" it could be for another one "night an day" depending on how you put things
If the difference is night and day than it shouldn't be a problem to identify it in a blind test. That looks reasonable to me.
 
Top Bottom