Master Thread: Are measurements Everything or Nothing?

[raz1969]

I can't help but noticed there are subjectivists on ASR (or anywhere else). These subjectivists believes different electronics sound different even if measurements are the same or are below hearing threshold. They would often describe the sound they hear as "smooth," "warm," "harsh," etc. to differentiate different electronics.

I have asked the question: what makes these electronics sound different then? And the answer I usually get from these subjectivists (and sometimes pseudo-engineers) goes something like this: "Each part and component used (i.e. resistors, capacitors, etc.) along with the circuit design will affect the sound."

As a trained EE, I can tell you with 100% certainly, that is a load of bullshite. It is a myth fabricated by the industry (manufacturers, reviewers, dealers, distributors) and pushed on to the consumers to make HiFi appear as an infinite choice requiring the art of component matching for a true connoisseur to enjoy. Manufacturers, dealers and distributors want this myth to exist because it helps them sell more HiFi, reviewers wants this myth to exist because it also helps their business but it also helps stroke their ego to make them come across more sophisticated and knowledgeable than they really are.

If you are not convinced that this is bullshite, let me attempt to use layman's logic to explain why.

If you want to build/engineer anything, one of the first steps in doing so, you must define the desired end state specifications. By definition, all specifications can be measured. For instance, building a home, you need to defined how many floors, sq. footage, how many bathrooms, height of ceiling. No builders will eyeball the dimensions and say, "Yeah, that looks about 10 ft wide."

If you cannot measure or quantified in standardized units, how do you know how much more "smooth" or "warmth" or "air" an electronic needs (or how much you are putting in there.)? How do you know resistor A vs. resistor B is the right one to use to reduce the "harshness" by X amount? If you were the engineer and you got feedback from a customer saying they need more "lush" in the midrange, what changes do you make in your design?

All that can be measured in electronics are measurable today (actually decades ago). So the next time you think you noticed a difference between two electronic components that are the same in spec or specs below hearing threshold, I encourage you to level match using a multimeter and have someone administer a blind A/B test to you. It will be eye opening.

EDIT/SOME SUMMARY POINTS FROM THIS THREAD:

1) Golden Ear Myth. So long as one is able to pass an hearing test with a good margin conducted by an audiologist and done some listening training (Such as Harman's How to Listen app, only available on Windows.), no one can claim they have a better hearing than you. And anyone who claims they have golden ear are straight out a charlatan.

2) The what if? What if we really aren't measuring everything out there. First, there must be scientific evidence that there are sonic difference that we have yet to be able to measure or even know about. Then the scientific community can investigate on how to measure it. Such as a blind AB conducted in a scientific manner demonstrating such a sonic difference exists. To date, no such scientific evidence of "what if" exists.

Couldn’t disagree more. dont buy based on price, buy what sounds best. I don’t think I have “golden ears” but gear does sound different. Equipment is “voiced” by EVERY manufacturer, choose what you like and move on. assuming everything sounds the same is folly. It’s not about measurements but what YOU like. We don’t all have the same hearing. Some of us are old. Some of us don’t tolerate sibilant sounds. This hobby should be fun, explore gear and find what you like.

 

[JSmith]

Couldn’t disagree more. dont buy based on price, buy what sounds best.

I buy on what measures best, has quality components, good build quality and looks nice... when it comes to amps and DAC's, there is absolutely no need to "listen" to them.

Speakers, same criteria... but would want to listen to them first at the very least, if not trial in room.

Equipment is “voiced” by EVERY manufacturer

If a manufacturer claims this and doesn't provide reliable measurement data, then I would move on quickly... as the snake oil will be concentrated.

It’s not about measurements but what YOU like.

When it comes to the performance, it is absolutely about measurements... it has to be due to the nature of the devices and the fallibility of our perception.

This hobby should be fun, explore gear and find what you like.

It is fun, especially listening to music and movies on a nice system. Going through the device roundabout looking for a "sound" simply detracts from time that could be used enjoying the music/movies.


JSmith

 

[Ambient384]

Fair enough. This may be a dumb question but you are encoding to mp3 from a lossless source, right? You aren’t reencoding mp3 to mp3 I hope!

It Lossless archive, I switched to V1 as It seems to behave more like 256kbps in that It will reach 130kbps. V0 seems to behave like CVBR 256kbps and has few filters turned off like low pass.

Another dirty trick I've learned Is using LAME dev encoders for --allshort + 48KHz resample shows MP3 can be transparent on pre-echo. My likely theory is that MDCT codecs can't see very narrow sharp attacks(very common in Noise music) an codecs like AAC get around this by having It short blocks active at all times.

Even FHG & Other research labs have said that If MP3 was a pure MDCT but still 576(long) x 192(short) sample wise It would've fared much better. MP3 basically HE-MP2 in that It a subband codec with MDCT taped on.

 

[Waxx]

There are two measurable factors that give an amp a sound that matters to me. And both are technically inperfections of an amplifier circuit (or an other device but you only want that on a (pre)amp, not on your dac or so. And yes, you can measure them, Amir does measure them in his standard tests. (or at least the results).

Low order harmonic distortion (H2 and H3) give a subjective warmer sound, but needs to be dosed right for the subjective taste of the listener. The dificulty there is to keep H2 and H3 at a high enough level to work that way and keep the rest of the distortion low enough.

Low damping factor can give a boost in the lower frequencies, what may be wanted with certain speakers. Especially vintage speakers from the tube era rely on that low damping factor to give bass and will sound bad with a modern class D amp with a very high damping factor.


But both are not what is wanted by most here, and by ASR standards a fault in the design. Some like me like that fault altough and consider it a feature. And no, that feature does not have to cost much.

But all the rest is just audiophool nonsens

 

[Killingbeans]

Equipment is “voiced” by EVERY manufacturer, choose what you like and move on.

Sorry to burst that particular bubble, but it's just plain wrong.

The audiophile press and its myriad of reviewers would like you to have that idea, but in reality the "voice" disappears once its magnitude gets below the thresholds of human hearing. Not saying that gear can't possibly have a "voice", but the vast majority of products don't present large enough deviations from neutrality to actually do so.

The only thing that EVERY manufacturer does, is claiming that their products are better at achieving an approximation of absolute neutrality than any other competing product.

Measurements here on ASR have shown that those claims are quite often complete BS, and some of the most BS riddled products might be far enough off the mark to have an actual "voice", but those are relatively far between. More often the measurements show how "pleb" products have no problem keeping nonlinearities far below audibility.

But yes, in cases where you want or can't avoid a "voiced" product, choose what you like and move on.

 

[Frank2]

A little anecdote I like to trot out in threads like this...

At my previous job we occasionally put out firmware updates for Bluetooth speakers and headphones.

Once, we put out a firmware update that simply changed how the lights blinked and some timing on buttons, which solved some small issues people had raised. That's all we did. We touched nothing else.

I put out a survey to make sure people were happy with the update. Feeling cheeky, I also added a question about sound quality. Had it improved, stayed the same, or gotten worse?

Most answered that it got better, some answered that it got MUCH better.

I repeat, it did absolutely nothing to the sound.

These were all honest people with no skin in the game. People who liked their headphones and wanted to keep them up to date - no more than that. It was a survey simply for the sake of checking in with our customers. And yet, there was a massive placebo effect going on, with people who would probably solemnly swear they heard something, even though they definitely didn't.

Imagine how bad it can get when your and your friends' reputations are riding on finding a positive result?

I totally recognize that. Some weeks ago I stumbled on a nice, small, sturdy remote keyboard that was perfect for remote controlling my audio PC. Much better than the bulky one I used up till then. When I installed it and was using it the first time, I was so happy with it that the music actually seemed to sound better! So maybe there's a market for audiophile keyboards as wel...

Years ago I had the weird experience of tweaking the settings of my subwoofers a little and actually hearing a difference while I was slightly changing them. The surprise came later, when I discovered I forgot to switch on my subwoofer amps...

Cognitive bias can be very powerful. That's why a double blind test is the golden standard. This eliminates the cognitive bias of the one controlling the test from 'spilling over' into the test subject. I think that may have gone wrong with the cable test posted on page 1 of this thread. The person controlling the test said he heard a difference after switching cables (his cognitive bias). After that he stepped to the side but his facial expression and other non-verbal communication may have influenced the person doing the listening.

Thanks for your replies to my questions I posted earlier, by the way!

 

[sweetsounds]

"Liberal conservative" sounds oxymoronic to me.

Not at all.

Liberalism = political concept of rights of the individual, secularism, personal liberty, human rights, democracy
Conservatism = political concept of maintaining the status quo with mild adoptions

As we are living in a liberal democracy for a long time now, I prefer to maintain it against the forces to reinstall autocratic regimes.
It is true the conservatists are usually referring to authorities, individual autocratic leaders, institutions, hegemonies, personal limitations, religions as "the good old times". But from today's standpoint this should be called "restoration", not conservatism.

I am also a subjective objectivist in the sense, that I want to find out with objective measurements and data, what I prefer and why. We need to be ready to challenge ourselves what are the limits of perception and reproducibility. I believe that preference can be qualified and quantified, but that it has probabilistic, biased elements to it.

On a higher philosophical level it means, that the concept of free will is objectively doubtful, but it is subjectively clear that there is consciousness in humans, which yet is not understood.

 

[Sokel]

Sorry to burst that particular bubble, but it's just plain wrong.

The audiophile press and its myriad of reviewers would like you to have that idea, but in reality the "voice" disappears once its magnitude gets below the thresholds of human hearing. Not saying that gear can't possibly have a "voice", but the vast majority of products don't present large enough deviations from neutrality to actually do so.

The only thing that EVERY manufacturer does, is claiming that their products are better at achieving an approximation of absolute neutrality than any other competing product.

Measurements here on ASR have shown that those claims are quite often complete BS, and some of the most BS riddled products might be far enough off the mark to have an actual "voice", but those are relatively far between. More often the measurements show how "pleb" products have no problem keeping nonlinearities far below audibility.

But yes, in cases where you want or can't avoid a "voiced" product, choose what you like and move on.

Then again...

D70 Pro SABRE Fully Balanced DAC-TOPPING

 

[Killingbeans]

Of course. The absolute most effective way of "voicing" the experience is to use actual signal processing, rather than trying to use each link of the reproduction chain as a hardwired "EQ preset" and match by ear.

I always strongly advocate getting as close to a "flat" reproduction chain as you can at first, and then use "surgical" introduction of deviations to get your prefered sound.

It's by far the fastest way of reaching a point where you can just focus 100% on enjoying the music.

Problem is that the "dicking around" is 99% of the hobby for most people. Take that away, and there's no "fun" left. It's a game of masochism

 

[MaxwellsEq]

Easy. Absolute polarity can be an audible giveaway with certain non-symmetrical musical content and test signals, as can inverted polarity in one speaker of a pair.

I agree. In theory anything in the time domain can be a transform of the frequency domain (making practical decisions about infinite time duration) but I think there are no standard tests for strongly non symmetrical content.

For me, recordings of applause in a large auditorium can help identify absolute phase, which would not be audible with sine waves. I've not tried this double-blind ABX tested, however!

 

[Sokel]

Of course. The absolute most effective way of "voicing" the experience is to use actual signal processing, rather than trying to use each link of the reproduction chain as a hardwired "EQ preset" and match by ear.

I always strongly advocate getting as close to a "flat" reproduction chain as you can at first, and then use "surgical" introduction of deviations to get your prefered sound.

It's by far the fastest way of reaching a point where you can just focus 100% on enjoying the music.

Problem is that the "dicking around" is 99% of the hobby for most people. Take that away, and there's no "fun" left. It's a game of masochism

The interesting thing about the add is that what we call "fault" and usually cover it with tar and feathers they call it "timbre" and "fine tuning".
To a new user that's plain bad education.

 

[DSJR]

No offence to anyone here, but I think where 'enthusiast hobby' audio is concerned, you lot are WAY over-estimating the basic quality of some of the products out there

Yes, I'm going back to the dark times of the 1980's UK audio scene, where two makers (one source/speakers and the other amplification) had taken all but a stranglehold on the upper mid to lower high end market. One could EASILY hear differences between many lower cost amps and imagination had next to nothing to do with it...

...How can I say that with such certainty - just look at the bloody measurements of such amps - borderline IMD, basic harmonic distortion in the 60's and 70's quite often and almost ALL of them had often wild RIAA playback eq curves to balance wayward response pickups back in the days before digital took over. Couple that with limited power bandwidth and iffy clipping performance of what weren't hugely powerful amps and the 'difference' between them (on the contemporary music we played back then) started this entire subjectivist thing!

Interestingly for me anyway, was how even the better far eastern amps often sounded 'lean, dry and rather clinical' in comparison. Sure, some had tiny power supplies and a lack of 4 ohm drive, so basically collapsed when pushed beyond a basic voltage output, but the top models with ample output and distortions down in the -90's all 'sounded' similar, just not to the reviewer's taste. Said reviewer, who became something of a guru for a while in these parts and maybe the US too for a while, raved over two US makes which were very popular in the UK (one of them still is and the other designer now makes steam punk excessive confections with copper trimming and a huge round 'meter' on the front - you know the make I'm sure) and neither measured well at all apart from one of the maker's range of power amps desiogned to double power into 4 and then 2 ohm loads (to go with the hideously tricky ribbon and panel speakers becoming popular) and even these weren't anywhere near sota apart from the power available into 2 ohm loads.

Bringing it all up to date, maybe things have changed a bit (I know they have in digital and good amplifier circles away from the bling models), but the now older left over bods from thoise 80's times (I'm one at 66 yrs old) will still rely on that old conditioning and experience (and, well, brainwashing) of their audio youth and will be easily taken in by foo add-ons and products at over-inflationary high prices together with speakers which now fizzle and scream at them, trying to make up for their now age related depleted hearing.

 

[BDWoody]

I just brought in another thread to keep all this together.

 

[Bridges]

Seems like so many review threads get challenged with:

1. Measurements are not everything.

2. You all never listen.

3. I trust my ears, not graphs.

4. I don't listen to graphs. I listen to music.

5. You all must not listen to music at all.

6. Why don't you all buy the best SINAD gear?

7. I have heard your best SINAD gear and they sound terrible. I don't like any of this Chinese stuff.

8. You don't trust your ears. I/we do.

9. All these reviewers/youtubers/audophiles say these amps, DACs, etc. sound different and you say they don't. They can't all be wrong.

10. Surely designers have created certain house sound for each equipment which your measurements don't show.

11. Your measurements are only at one frequency. You need to also measure X, Y and Z like impulse response, slew rate, etc., etc.

12. You guys run a cult here where you only go by measurements and no one is allowed to disagree.

On and on...

I have had to answer these so many times that I thought it is time to stop having them go into every review as they are not product specific. From here on, any such questions should be posted here. Answers will be given in this thread and simply referenced in future challenges in other threads.

@AdamG247 and @BDWoody, please direct any future posts in review threads to here and not allow discussions there.

Thanks. You all are free to discuss this topic, provide answers, argue, whatever, in this thread.

Seems like so many review threads get challenged with:

1. Measurements are not everything.

2. You all never listen.

3. I trust my ears, not graphs.

4. I don't listen to graphs. I listen to music.

5. You all must not listen to music at all.

6. Why don't you all buy the best SINAD gear?

7. I have heard your best SINAD gear and they sound terrible. I don't like any of this Chinese stuff.

8. You don't trust your ears. I/we do.

9. All these reviewers/youtubers/audophiles say these amps, DACs, etc. sound different and you say they don't. They can't all be wrong.

10. Surely designers have created certain house sound for each equipment which your measurements don't show.

11. Your measurements are only at one frequency. You need to also measure X, Y and Z like impulse response, slew rate, etc., etc.

12. You guys run a cult here where you only go by measurements and no one is allowed to disagree.

On and on...

I have had to answer these so many times that I thought it is time to stop having them go into every review as they are not product specific. From here on, any such questions should be posted here. Answers will be given in this thread and simply referenced in future challenges in other threads.

@AdamG247 and @BDWoody, please direct any future posts in review threads to here and not allow discussions there.

Thanks. You all are free to discuss this topic, provide answers, argue, whatever, in this thread.

For electronics / I will take your word for it. For speakers/ No way

 

[Killingbeans]

To a new user that's plain bad education.

I wouldn't call it bad education. Technically it is a "fault", regardsless of whether you enjoy it or not.

 

[restorer-john]

For me, recordings of applause in a large auditorium can help identify absolute phase, which would not be audible with sine waves. I've not tried this double-blind ABX tested, however!

That’s really interesting. I had always put applause into the almost pink noise type category.

 

[lisgotan]

It cannot be different because it's not measurable to be different. It's the same.

Such different interpretation is called cognitive bias. There are decades of research done on this.

Cognitive bias is all in the head. . .once you take that cognitive bias away, these people cannot tell a difference, therefore none of them are able to pass a blind ABX test.

Of course it's all in the head. Our ears are only to convert sound into electrical signals that is sent to our brain. Regardless of what sight, sound, taste, feel etc etc, they are still all just electrical signals fed int our brain.

If you take cognitive bias away, here will be definitely no difference. Because you will be dealing with robots, not humans.

That's why I have been saying the world is not in black and white. There are over 8 billion souls on this planet and they are all unique individuals.

It is impossible to study every individual so the practical way is to take a small sample size, study them and extrapolate the results to the population. But, it doesn't mean everyone must behave in the same way the research says.

 

[SIY]

Of course it's all in the head. Our ears are only to convert sound into electrical signals that is sent to our brain. Regardless of what sight, sound, taste, feel etc etc, they are still all just electrical signals fed int our brain.

If you take cognitive bias away, here will be definitely no difference. Because you will be dealing with robots, not humans.

That's why I have been saying the world is not in black and white. There are over 8 billion souls on this planet and they are all unique individuals.

It is impossible to study every individual so the practical way is to take a small sample size, study them and extrapolate the results to the population. But, it doesn't mean everyone must behave in the same way the research says.

So?

 

[lisgotan]

So?

So, stop trying to say that everyone must be the same.

 

[Mnyb]

A little anecdote I like to trot out in threads like this...

At my previous job we occasionally put out firmware updates for Bluetooth speakers and headphones.

Once, we put out a firmware update that simply changed how the lights blinked and some timing on buttons, which solved some small issues people had raised. That's all we did. We touched nothing else.

I put out a survey to make sure people were happy with the update. Feeling cheeky, I also added a question about sound quality. Had it improved, stayed the same, or gotten worse?

Most answered that it got better, some answered that it got MUCH better.

I repeat, it did absolutely nothing to the sound.

These were all honest people with no skin in the game. People who liked their headphones and wanted to keep them up to date - no more than that. It was a survey simply for the sake of checking in with our customers. And yet, there was a massive placebo effect going on, with people who would probably solemnly swear they heard something, even though they definitely didn't.

Imagine how bad it can get when your and your friends' reputations are riding on finding a positive result?

Absolutely

I remember the days when i was frequenting the squeezebox forum on a daily basis ( i have a lot of those streamers ) , there where always lunatics claiming that firmware version sounded differently , all the time .

And to add even more insult to common sense they of course sounded different on the spdiff output to ...