You any many others want, so very hard, to believe that. And that is why I don't think you will find objectivist forums very enjoyable.
-
WANTED: Happy members who like to discuss audio and other topics related to our interest. Desire to learn and share knowledge of science required. There are many reviews of audio hardware and expert members to help answer your questions. Click here to have your audio equipment measured for free!
Master Thread: Are measurements Everything or Nothing?
- Thread starter aeroborne
- Start date
You any many others want, so very hard, to believe that. And that is why I don't think you will find objectivist forums very enjoyable.
No, doesn't seem to work.
What we do is move the posts that bring up the same challenges to one or two places so they don't pollute the rest of the forum. This is one of those threads.
As soon as people stop using uncontrolled subjective claims as if they represented evidence.
solderdude
Grand Contributor
In every review there are basically a few 'parameters'.
1: technical performance (measurable)
2: longevity and build quality
3: impressions from owners including operational, and devices breaking, service, seller and manufacturer 'issues'
4: Operational issues (difficult menus, buttons not easy to operate/reach)
About ...
1: To make a full recomendation/evaluation many, many measurements should be done. Yet only some basic measurements are posted/discussed.
Sometimes interaction with other gear is not taken into account, common mode/groundloop/RF susceptibility/emission is not measured and could be of importance.
So here (at ASR) a choice is made by the reviewer (in most cases Amir) which measurements are made and which one is shown.
In some cases Amir is asked to make specific measurements, in some cases this is not possible, sometimes Amir does them.
2: This is not easy to predict unless a torture test is done and the device is fully taken apart. A teardown often is not possible as the gear is from owners who send one in.
3: Yes, here you need the feedback from owners, for parameter 1 you do not need that at all.
4: Sometimes this is duscussed by the reviewer or owners. An owners opinion helps but some things are so obvious one can see it looking at pictures or manuals.
Owning a device may help bring issues forward but is not a necessity in a discussion.
Not a work of art, rather a labour of love and care taken by the owner that values it. A personal thing and VERY biased because of it.
Also most folks do not go for 'optimal signal fidelity' (despite them thinking they do) but go with what 'sounds best' to them.
This, however, is merely a personal opinion based on their personal taste. It has no relevance to reviews on ASR. These are about signal fidelity and the assurance for most people that desire optimum signal fidelity.
To most of these people (ASR followers) optimal signal fidelity is an assurance that possible 'not so good' sound is either the recording or room or personal preference being in the way and NOT signal fidelity of the chosen components.
That may NOT be what everyone is looking for. For them there are many, many other review sites. Some also publish a few measurements.
Last edited:
Killingbeans
Major Contributor
If only it was that easy. We've already done a thousand iterations of that, and none of them seem to work.
The newcommers are usually very insistent on ignoring them, and just start presenting their impressions as a solid foundation for facts.
Call it "parroting" if you will, but the most sensible thing is to keep making the implausibility of things as clear as possible.
Zensō
Major Contributor
Wha? Sorry, but I have to strongly disagree.
Pretty much. It’s not rocket science… or art.
Cote Dazur
Addicted to Fun and Learning
If you appreciate then you understand why it is needed to be repeated every time someone comes with a subjective opinion on anything that has zero value for anyone, not even for the one making the subjective comment.
The tone and content of the balance of your post clearly shows you are not really as appreciative as you write in your first paragraph (of your first post on ASR).
Hopefully, with time, you will truly understand the immense value that ASR brings to the people interested in pursuing high fidelity sound and be well prepared to detect snake oil peddlers.
Measurements are essential for
checking whether or not an appliance is working electrically as intended.
With regard to the purpose for which we use hi-fi, I must say that data is almost as valuable as the indication of the alcohol content of a bottle of wine.
Why am I here, because I am an engineer and interested in how data for hi-fi can achieve the same value and meaning as data for a measuring instrument, which paradoxically is the only way we have to measure hi-fi besides the ear and the brain.
checking whether or not an appliance is working electrically as intended.
With regard to the purpose for which we use hi-fi, I must say that data is almost as valuable as the indication of the alcohol content of a bottle of wine.
Why am I here, because I am an engineer and interested in how data for hi-fi can achieve the same value and meaning as data for a measuring instrument, which paradoxically is the only way we have to measure hi-fi besides the ear and the brain.
solderdude
Grand Contributor
When you are an engineer I would say that data like only a single THD number in a spec sheet is as valuable as the indication of the alcohol content of a bottle of wine.
When you are an engineer that knows about audio gear and look at a more elaborate set of measurements then you should already have far more valuable information at hand.
When you are an engineer that knows about audio gear and perception and a full measurement suite (so more than the average review on ASR) that would be all you need to know.
The correlation between measurements and perception is something that will not be there for most people.
The reasons for that could be:
Lack of proper measurements
Lack of understanding measurements
Lack of faith in measurements
When you are interested in measurement data and listening you could best look into perception and audible limits before venturing in understanding what measurements can show what and when higher fidelity does not lead to improved sound quality any more.
When you are an engineer that knows about audio gear and look at a more elaborate set of measurements then you should already have far more valuable information at hand.
When you are an engineer that knows about audio gear and perception and a full measurement suite (so more than the average review on ASR) that would be all you need to know.
The correlation between measurements and perception is something that will not be there for most people.
The reasons for that could be:
Lack of proper measurements
Lack of understanding measurements
Lack of faith in measurements
When you are interested in measurement data and listening you could best look into perception and audible limits before venturing in understanding what measurements can show what and when higher fidelity does not lead to improved sound quality any more.
Michael Larkin
New Member
- Joined
- May 9, 2021
- Messages
- 3
- Likes
- 1
o
mindmatters.ai
studiousguy.com
also this:
In respect of audio gear, I think the point some make is that the "placebo effect" is responsible for subjectivists thinking that some gear is better than others. But my point is that this could apply both for subjectivists and objectivists.
How so for objectivists? Well, if one believes that measurements are best in indicating effectiveness of DACs, amps and speakers, one is more likely to consult a site like ASR, and to choose from the best-measuring gear that is within one's budget. And when one purchases it, one is more likely than otherwise to think it sounds good because one believes in measurements.
But does a piece of gear that measures well necessarily sound good? That's a $64,000 question. As is the obverse: if it sounds good based on a personal, subjective evaluation, does it necessarily measure well?
I genuinely believe that Amir is somewhat unusual in that he has trained himself to pick up on subjective differences in sounds that correlate with certain objective measurements. But has he also inadvertently trained himself (because of his objectivist leanings), to think this means his measurements are reliable indicators of sound quality -- judgements that more subjectively-oriented people would be silly to ignore?
I idly wonder whether subjectivists and objectivists might be two sides of the same coin. The main thing that differentiates them is that one side believes passionately in measurements and the linkage of those with sound perception. As for the other side, the belief is that good sound depends more on subjective impressions.
The common element is that there's belief on both sides. But how is to one to know that a degree of correlation of sound quality with objective measurements is the whole story? I mean, why does Amir's table of SINADs for DACs go from "best" (highest) to "worst" (lowest)? If anyone can objectively show gradual degradations in the perception of sound as one goes from highest to lowest, then please supply the relevant data, because I'm not confident in my own mind that that would prove to be the case. The 18th DAC down in the list might or might not sound better than the 65th to at least some listeners who were unware of Amir's findings.
Why do we prefer some kinds of sound perception to others? Why do I rate the four DACs I own, from best to worst, in the order [prices at the time of purchase]: Aune X8 viii + sparkos op amp + Aune XP2 power supply (total £470) -- Earmen Tradutto (£699) -- Ifi Zen DAC V2 (£139) -- Topping E30 (£110)? Buggered if I know, but it sure as heck has little to do with measurements or online subjective reviews (even though the latter did to some extent influence my purchases). Others might rate them differently, and if so, that's fine, but where would they stand according to SINAD measurements? I think Amir places the E30 above the Tradutto, but to me, it's the other way round. I'm not sure he's reviewed the other two.
At the moment, I try to avoid both excessive objectivism and subjectivism. But I can't avoid the latter completely because in the final analysis, given acceptable equivalence of features and convenience, the deciding factor is how a piece of kit sounds subjectively to me. Amir's measurements may say something ranks objectively very highly, and yet to me it might sound worse than something that ranks lower. And vice-versa for that matter. I guess I'm suggesting that proponents on both sides might chill out a bit and accept that there could be a degree of merit on both sides of the objectivist/subjectivist divide.
OTOH, there's quite a lot of evidence that placebos in their proper medical context actually do work, and sometimes permanently (as with fake knee operations):
Yes, the Placebo Effect Is Real, Not a Trick
The fact that you may start to get better if you believe you are receiving treatment is one of the best-attested facts in medicine. Despite that, far from being accepted, this “placebo effect” is seen…
mindmatters.ai
8 Placebo Effect Examples in Real Life – StudiousGuy
studiousguy.com
also this:
In respect of audio gear, I think the point some make is that the "placebo effect" is responsible for subjectivists thinking that some gear is better than others. But my point is that this could apply both for subjectivists and objectivists.
How so for objectivists? Well, if one believes that measurements are best in indicating effectiveness of DACs, amps and speakers, one is more likely to consult a site like ASR, and to choose from the best-measuring gear that is within one's budget. And when one purchases it, one is more likely than otherwise to think it sounds good because one believes in measurements.
But does a piece of gear that measures well necessarily sound good? That's a $64,000 question. As is the obverse: if it sounds good based on a personal, subjective evaluation, does it necessarily measure well?
I genuinely believe that Amir is somewhat unusual in that he has trained himself to pick up on subjective differences in sounds that correlate with certain objective measurements. But has he also inadvertently trained himself (because of his objectivist leanings), to think this means his measurements are reliable indicators of sound quality -- judgements that more subjectively-oriented people would be silly to ignore?
I idly wonder whether subjectivists and objectivists might be two sides of the same coin. The main thing that differentiates them is that one side believes passionately in measurements and the linkage of those with sound perception. As for the other side, the belief is that good sound depends more on subjective impressions.
The common element is that there's belief on both sides. But how is to one to know that a degree of correlation of sound quality with objective measurements is the whole story? I mean, why does Amir's table of SINADs for DACs go from "best" (highest) to "worst" (lowest)? If anyone can objectively show gradual degradations in the perception of sound as one goes from highest to lowest, then please supply the relevant data, because I'm not confident in my own mind that that would prove to be the case. The 18th DAC down in the list might or might not sound better than the 65th to at least some listeners who were unware of Amir's findings.
Why do we prefer some kinds of sound perception to others? Why do I rate the four DACs I own, from best to worst, in the order [prices at the time of purchase]: Aune X8 viii + sparkos op amp + Aune XP2 power supply (total £470) -- Earmen Tradutto (£699) -- Ifi Zen DAC V2 (£139) -- Topping E30 (£110)? Buggered if I know, but it sure as heck has little to do with measurements or online subjective reviews (even though the latter did to some extent influence my purchases). Others might rate them differently, and if so, that's fine, but where would they stand according to SINAD measurements? I think Amir places the E30 above the Tradutto, but to me, it's the other way round. I'm not sure he's reviewed the other two.
At the moment, I try to avoid both excessive objectivism and subjectivism. But I can't avoid the latter completely because in the final analysis, given acceptable equivalence of features and convenience, the deciding factor is how a piece of kit sounds subjectively to me. Amir's measurements may say something ranks objectively very highly, and yet to me it might sound worse than something that ranks lower. And vice-versa for that matter. I guess I'm suggesting that proponents on both sides might chill out a bit and accept that there could be a degree of merit on both sides of the objectivist/subjectivist divide.
Last edited:
So that's why every proper medical study has a group of test subjects who receive placebo? To figure out how to effectively fool patients into being healthy? Or is it to eliminate the possibility of imaginary effects from skewing the (objective) results of the study? Hmm....
fpitas
Master Contributor
No offense intended, but you have a lot to learn about audio.
VQR
Active Member
- Joined
- Oct 15, 2021
- Messages
- 145
- Likes
- 342
Two opposing sides means nothing toward validity. One asks for proof, the other refuses and makes excuses. When pointed toward evidence, what invariably happens is:
Objectivists: I think you fell for placebo.
Subjectivists: No u
It's a childish example, but when you scrape past word fluff you often find that to be the case. Sometimes they invoke lies that Amir and others say a DAC with worse SINAD, say, 100 dB, is going to sound worse than one with top tier SINAD which is currently 123 dB SINAD. There's a good thread that gives a starting point on when our hearing won't notice what the analyzer can.
Amir's gone on record several times saying higher SINAD is not a reason to dump your transparent and good DAC for the latest and greatest. Often, SINAD is just a way to tease out if there's any immediate flaws with a DAC, such as PSU noise and harmonics.
There are 32 tone tests than can tease out IMD or distortion across the audio band. We get linearity tests to see how consistent the signal is across all frequencies. It's not uncommon for some boutique HPA or speaker amps to fail at lower frequencies.
For amplifiers especially, it's tested how they react at different resistance loads, what power you get, and at what points the power sags. We see what the noise differences are between gain settings. This is all good information to know!
The lie often found on Head-fi and SBAF that ASR only cares about SINAD and doesn't listen to music. Any headphone, speaker, and HPA review proves that false upon just reading.
Last edited:
Killingbeans
Major Contributor
There's no doubt that both "sides of the fence" are equally susceptible to bias.
It's been said before (a lot) but the whole "subjectivist" vs. "objectivist" deal is completely pointless. We are all subjectivists.
I think the thing that makes an "objectivist" like me, is a focus on probability rather than possibility.
I prefer "That's not very probable, so lets spend our time and energy on something else" over "Think of the possibilities! Lets just roll with it".
I would instead make the radical guess that nearly all DACs on the list would sound practically indistinguishable when tested blind and level matched.
You'd have a hard time finding users in here who believe that 'better measurements == better sound'. Most of us will make a judgement on the "sound" of a piece of gear based on our best current understanding of the thresholds of human hearing and the different masking effects being at play.
Last edited:
- Joined
- Jan 15, 2022
- Messages
- 1,212
- Likes
- 1,575
You should read Cork Dork. The author becomes a sommelier and explains the process of how you do it. You can get a pretty good idea of what a bottle of wine should taste like by knowing four things: the amount of alcohol, how sweet/dry it is, how full-bodied it is, and its (legitimate*) tasting notes. The first three can be measured by instruments. Perhaps the 4th as well. So give the number of elements measure for electronics on this site, it would be more like printing the alcohol, sugar, and specific gravity on a bottle of wine, along with the vintage (Though the vintage alone will tell you those things to a large amount if you are really in the know). Sommeliers use the first three and a ton of experience in tasting to identify down to the vintage an unknown wine.
*interestingly, like hearing, we often overestimate our ability to discriminate by taste. Also like hearing, taste is very open to suggestion and bias. In Cork Dork, the author points out that we can only process two or three-ish tastes at any given time, so those long lists of tasting notes are pretty useless as the more minor ones (if they exist) will be overwhelmed by the more prominent ones. Masking effects apply to taste as well. Wine could easily have its own science site (maybe it does, I’m not into wine).
Cote Dazur
Addicted to Fun and Learning
Of course it does apply to both, we are subject to bias. Still there is a lot of differences between both approach. As already mentioned, we are all subjectivist, no one is immune. Where we differ is how we proceed. Objectivist know they are subject to bias, in order to help themselves get a better grasp, they will try to gather knowledge to help understand what is objectively happening to make better decision. Subjectivist usually know little about anything and believe their ears can be trusted above all else.
We are all fumbling in the dark, subjectivist are totally blind, objectivist have a little bit of light, not enough to understand everything about their surrounding but enough to avoid falling off the cliff.
Last edited:
The wine analogy is always applied (to audio) incorrectly.
The wine is the music. Not the gear. The analogy to the gear is the bottle and glass. If they are dirty they contaminate the wine.
You don't need to measure wine to find out how it is going to taste - you just enjoy it, like you just enjoy music. You do need QA to check the cleanliness of the bottles, and you need to make sure you are using a clean glass so as not to spoil the wine - just as you need to make sure your audio reproduction gear doesn't apply distortion and noise, and spoil the music.
Last edited:
- Joined
- Jan 15, 2022
- Messages
- 1,212
- Likes
- 1,575
True, it is a bad analogy. Also wine, like music, (and paintings, dance, etc) can be measured and described. It’s just that the value can not be assigned, as that is largely at the discretion of the experiencer.
Art isn’t indescribable.
I enjoyed many inexpensive fine wines in the day based on blind tasting sessions performed by groups of friends. The hosts were wealthy and could easily afford any wine they wanted, but they preferred value to marketing hype and "vinophile" reputation.
As with audio, "accuracy" in sighted wine tasting is limited by confirmation bias. Great cheap wines were extremely rare, but there were many gems in the mid-price range. And of course, there are superb wines in the expensive classes, but also some brands and releases that were vastly over-rated.
Ab out 25 years ago, I brought a bottle of Napa Valley wine and 3 years later, paid the $10 corkage fee to drink it with friends at a nice little obscure Sausalito restaurant. It was for a birthday party for the wife of the host of those blind tasting gatherings. The wine was a Grgich Hills Cabernet Sauvignon from a young Napa vineyard that I had purchased for $25 from a local wine shop in El Cerrito, California, across the bay from San Francisco. We agreed thst it was one of the best Cabs we ever had, and when the birthday lady went to buy some, the price had gone up to $60 - in 3 years. So she didn't buy it, and continued her quest for low-priced bargains.
Typically, unlike wine, audio component prices don't go up sharply after a good review and increased popularity - but if anyone has stories about such price increases, I'd enjoy reading them.
Michael Larkin
New Member
- Joined
- May 9, 2021
- Messages
- 3
- Likes
- 1
Reminds me of the Mulla Nasrudin story about the man who looked for his lost keys under a lamppost rather than in the dark where they were actually lost. To what extent might shining a light on things like SINAD parallel this? Could one be looking in areas that don't have conclusive significance when it comes to sound perception?
BTW, I just realised that I inadvertently posted this in a thread I didn't mean to -- hence my inclusion of the video from gavagai, posted elsewhere. But no matter, luckily I don't seem to have posted it in a totally irrelevant thread.
Cote Dazur
Addicted to Fun and Learning
Of course, once you have objective information, you still have to decide how relevant it is to your situation.
On the other hand no knowing any information does not allow for any decision, just fumbling in the dark. Leaving our self open to any snake oil argument from any peddlers so happy to deal with a group of ignorant, superstitious people.
Objective information piles up, their is more that SINAD to objective information. Once you are educated you can help yourself and help other. You are very new here, take you time, learn as much as you can, it gets easier the more you know.
Similar threads
