Master Thread: Are measurements Everything or Nothing?

[Cidious]

I thought the mantra here is that DACs measured similarly, sound the same. This is not necessarily the same between different amps, ie amps that measure similarly can sound different, right?

Of course they can, but you'll always have forum Maffioso like Lupin that have to repeat their same lines in every topic. It's probably ignorance or not having enough funds to buy and test some of the devices him/herself. Who knows. I'm just speculating at that.

I've got plenty of kit here. And if it was all the same I'd have never bought anything else anymore after I would have noticed after the first two or maybe three times that it wouldn't matter going off of just measurements. But it does differ. For good or for worse is subjective. All I can say. Devices don't sound the same because they contain an audio signal as mr wise guy tries to sell above.

How much some of the subtleties are worth to someone in terms of money and how much one can hear of it is up to someone on it's own. Some products are snake-oil for sure. But AMPs easily sound different not just by volume level. More measurements and tests need to be included to get a fuller picture of these differences or maybe a better interpretation of the ones already measured.

Having said all that. I stuck with my A30Pro purely for convenience of use and thinking it wasn't worth the extra coin to me personally. A practical decision. D30Pro + A30Pro work just fine for most situations.

 

[Veri]

How much some of the subtleties are worth to someone in terms of money and how much one can hear of it is up to someone on it's own. Some products are snake-oil for sure. But AMPs easily sound different not just by volume level.

Not *just* by volume level?

How much some of the subtleties are worth to someone in terms of money and how much one can hear of it is up to someone on it's own. Some products are snake-oil for sure. But AMPs easily sound different not just by volume level. More measurements and tests need to be included to get a fuller picture of these differences or maybe a better interpretation of the ones already measured.

This is a bit of a red herring, "more measurements and tests need to be included to get a fuller picture of these differences" basically corresponds to "that which proves my anecdotal findings is yet to be found but it's surely there". You could explain anything with that.

 

[heatwave]

And this got a 5 star review on What HiFi

They quote:

"Streaming SBTRKT’s Trials Of The Past, the AVC-X3800H delivers a surprisingly nuanced musical performance with an assured attack at the start of each synth note and the envelope of each reverb tail given space to breathe. The hefty bass is solidly controlled, while at the top end of the spectrum, the shakers are nicely defined and spread wide"

The difference being - What Hi-Fi? based their findings on what you experience with real world usage - actually allowing the receiver to be used the way everyone who owns will be using it. Measurements are never any guarantee of what to expect from a perofrmance point of view. How are senses enjoy what a piece of home audio offers has to offer actually means something to the consumer. Lines on a graph are for would-be engineers, engineers and people with a science delusion.
It would take a complete fool to believe they senses are always going to agree with 'scope measurements.
Here's another thing to consider with measruements. In the late 1970's, Hitachi manufactured an integrated amplifier which measured 'perfectly'. But it was a flop because it apparently sounded very boring.

 

[heatwave]

Great Review Amir, Thanks
I had high expectations but what a disappointment, oh well, I will keep enjoying my X3700H

Give the AVR-X3800H a listen. The proof is much more in the experience than it is in the measurements.

 

[heatwave]

Pure gobbledygook from a person that apparently does not have a understanding of electronics. The circuitry is reacting to each electron passing through it and the designers are basically counting electrons when designing due to the ability to easily convert current and voltage to electron flow ~quantity. Electronics theory(s) has been around for a long time and it has been proven time and time again to be accurate. So when a layman comes along with big ideas about electronics it's amusing if not ridiculous.

Your rant doesn't explain how measurements are any guarantee of outright performance.
When you spend a good deal of time in front of the equipment, comparing with others in it's range, it's very clear to see how mearurements and performance are not necessarily going to correlate.
If you love how something sounds after having compared with the other models of it's type by other brands, but then read the measurements were not good, would you no longer be interested?
That's like falling in love with someone, but deciding to have nothing to do with them because someone else said they didn't like certain little things about their personality.
Mis-matched systems are built around buying based on measurements.
An open mind will trust thy senses and pick based on the sound which was most enthralling. The rest can enjoy their system by looking at the spec's over and over.

 

[mctron]

Your rant doesn't explain how measurements are any guarantee of outright performance.
When you spend a good deal of time in front of the equipment, comparing with others in it's range, it's very clear to see how mearurements and performance are not necessarily going to correlate.
If you love how something sounds after having compared with the other models of it's type by other brands, but then read the measurements were not good, would you no longer be interested?
That's like falling in love with someone, but deciding to have nothing to do with them because someone else said they didn't like certain little things about their personality.
Mis-matched systems are built around buying based on measurements.
An open mind will trust thy senses and pick based on the sound which was most enthralling. The rest can enjoy their system by looking at the spec's over and over.

So you've come to Audio Science Review to tell everyone that measurements don't matter and trust your ears

 

[Doodski]

Your rant doesn't explain how measurements are any guarantee of outright performance.
When you spend a good deal of time in front of the equipment, comparing with others in it's range, it's very clear to see how mearurements and performance are not necessarily going to correlate.
If you love how something sounds after having compared with the other models of it's type by other brands, but then read the measurements were not good, would you no longer be interested?
That's like falling in love with someone, but deciding to have nothing to do with them because someone else said they didn't like certain little things about their personality.
Mis-matched systems are built around buying based on measurements.
An open mind will trust thy senses and pick based on the sound which was most enthralling. The rest can enjoy their system by looking at the spec's over and over.

You came to ASR a objective website to preach your subjective nonsense and want me to provide the proof to answer your subjective questions. It does not work that way. The electronics theories have been proven over hundreds of years of research. Better specs means better design and transparency. You are apparently not educated in electronics and I say apparently because I don't actually know and we have had peeps come here with a electronics education and subjectively go at us for fun and games. How can I answer your subjective questions when you do not have a fundamental understanding of electronics? It takes 8 months to 10 months of full time study to get the electronics core education under ones belt and that is serious attention to study and lots of hours of studying for tests and stuff. It takes 4 years to be a engineer. So why should any peep here take any layman seriously when they come here preaching listening tests, trusting their senses and whatever else it is you subjectively think. You are deluded and have fantasies about what is what with electronics and want me to explain them for you.

 

[solderdude]

Your rant doesn't explain how measurements are any guarantee of outright performance.

Performance.... for certain there is a high correlation.
With performance is meant 'signal fidelity'. With signal fidelity is meant, the described waveform (analog or digital) is reproduced as closely as possible to that waveform.
With performance is not meant how one evaluates a complete sound system in a room including personal preference.

When you spend a good deal of time in front of the equipment, comparing with others in it's range, it's very clear to see how measurements and performance are not necessarily going to correlate.

Correlation with perceived sound and technical performance can be very poor for those that have a certain preferences indeed. Blame the preference of the listener, not the measurements.
Aside from that... There is a difference in quality between 'measurements' and one needs several different measurements (what is called a measurement suite) and knowledge HOW to interpret them AND how to translate that to conditions for that individual.
Do not blame that on measurements but lack of understanding of a bunch or measurements.

If you love how something sounds after having compared with the other models of it's type by other brands, but then read the measurements were not good, would you no longer be interested?

Some would, some won't. See my remark above.

That's like falling in love with someone, but deciding to have nothing to do with them because someone else said they didn't like certain little things about their personality.

Could be they were right about that person but infatuation makes you not see it or do not care about such aspects personally.

Mis-matched systems are built around buying based on measurements.

A bold statement. Any examples of this ?

An open mind will trust thy senses and pick based on the sound which was most enthralling. The rest can enjoy their system by looking at the spec's over and over.

An open mind will realize preference is involved and not discard measurements because of lack of understanding them.
Disregarding measurements and only trusting 'senses' is not open minded behavior.
Open minded means approaching something without prejudice and able to accept any outcome.

 

[beren777]

What I'd like to see in ASR reviews are two separate conclusions: technical excellence, and audible transparency/utility. Many people obsess over and admire which cars can get from 0 to 60 the fastest, or have the highest fuel economy over various scenarios. Many of us just want to get to the grocery store affordably, reliably, and in comfort. I believe ASR can serve both performance and utility-driven objectivists. ASR is never going to satisfy all or even a minority of subjectivists, because by its very nature, subjectivism is only going to be satisfied by putting reality aside and embracing a marketing-driven approach. Other sites and some manufacturers already serve the "tell me how special my ears are and what sounds good to me on Tuesday" crowd.

What I want to know, for a given piece of audio gear, is am I likely under reasonable use and blind testing to notice any audio reproduction quality differences, for better or worse, between it and other available options? If AVR A and AVR B have a SINAD difference of 10 and the IMD is different, but the differences are generally imperceivable either before or after room correction up to normal (or even reference level) listening volume, for example, then I'd like to know so I can make an informed choice. If one AVR costs $5,000 more than another but all features and performance are audibly identical, I'd like to know.

I also like to see what liquid nitrogen-cooled 2-channel DAC has just reached 160 SINAD so I can marvel at it from a "holy crap that's amazing for $200" perspective.

I also like to know if key difference-making features (say, DIRAC support) require additional licensing, whether that licensing is locked per-device, whether it's stable, and whether it degrades audio quality. Does turning it on for a given unit reduce SINAD from 85 to 75, and can I hear the difference or is it off-set by other factors?

Are there areas of research that aren't being addressed these days

Personally, I believe that SINAD is a more useful benchmark of general quality the simpler the audio chain is and ASR weighs this in its bucket approach and in grouping AVRs separate from the 2-channel DACs. There's less going on with a 2-channel DAC intended for music reproduction than an 11.4 channel AVR that has to decode multiple audio and video formats and incorporate all the features expected of a modern AVR, while also running stable and having an easy to use interface for configuration and management. A 120 SINAD AVR is useless if it crashes and experiences weird, movie-interrupting glitches every few minutes, or if the manufacturer provides crappy support and deletes users from its forum if they fail to start their posts with "HAIL MANUFACTURER!"

 

[Killingbeans]

If you love how something sounds after having compared with the other models of it's type by other brands, but then read the measurements were not good, would you no longer be interested?

Depends on how bad those measurements are.

They could be bad only from an engineering point of view, but still be good enough to exceed the expected abilities of human hearing. In that case, I could simply tell myself that the product was good enough for my hearing, and I could appreciate it for its other abilities like looks, functionality and what have you. If wanted another product with more functionality or a different visual design, I could look for one, well knowing what thresholds of "bad" measurements I should aim for to avoid worrying about any audible downsides.

Or they could be so bad that they'd give indisputable audible effects. In that case, I'd have an indication of what "bad" things pleases me, and I'd have the option to seek out those things in other products, well knowing what to look for. I could hone in on a product that satisfies my lust for those effects, but also has the looks, functionalities and other features that gives me an enjoyable experience. The measurements would help me reach that goal without the need of simply throwing money at products and keeping my fingers crossed.

 

[irontortoise]

Your rant doesn't explain how measurements are any guarantee of outright performance.
When you spend a good deal of time in front of the equipment, comparing with others in it's range, it's very clear to see how mearurements and performance are not necessarily going to correlate.
If you love how something sounds after having compared with the other models of it's type by other brands, but then read the measurements were not good, would you no longer be interested?
That's like falling in love with someone, but deciding to have nothing to do with them because someone else said they didn't like certain little things about their personality.
Mis-matched systems are built around buying based on measurements.
An open mind will trust thy senses and pick based on the sound which was most enthralling. The rest can enjoy their system by looking at the spec's over and over.

I would recommend doing a little research and discovering a little about the science of sound. Or, I guess you could try and tell engineers, scientists, and educated hobbyists, how they’re wrong based upon your own emotions.

 

[TimF]

Are measurements everything? Sometimes in music there is more than fidelity if you know what I mean.

 

[pau]

I have huge respect for the engineering and research done and very much enjoy educating my self via information shared.

One aspect often forgot while just looking at the measurements is the environment noise floor and what we can utilize from the devices on our typical listening levels and rooms.

I was shocked to find i cannot hear more than <48db dynamic range with my typical listening volumes and old ears. Doesn't stop me from enjoying any less, but just points out that there might come other aspects to consider than just equipment measures.

Hunting better measuring devices where the improvements are not audible in many cases will end up money wasted that could be used to better aestetics & acoustic treatments.

"Whatever's good for your soul.. Do That."

Online Audible Dynamic Range Sound Test

Measures the dynamic range of your audio system and listening room.

www.audiocheck.net

 

[Robbo99999]

What I'd like to see in ASR reviews are two separate conclusions: technical excellence, and audible transparency/utility. Many people obsess over and admire which cars can get from 0 to 60 the fastest, or have the highest fuel economy over various scenarios. Many of us just want to get to the grocery store affordably, reliably, and in comfort. I believe ASR can serve both performance and utility-driven objectivists. ASR is never going to satisfy all or even a minority of subjectivists, because by its very nature, subjectivism is only going to be satisfied by putting reality aside and embracing a marketing-driven approach. Other sites and some manufacturers already serve the "tell me how special my ears are and what sounds good to me on Tuesday" crowd.

What I want to know, for a given piece of audio gear, is am I likely under reasonable use and blind testing to notice any audio reproduction quality differences, for better or worse, between it and other available options? If AVR A and AVR B have a SINAD difference of 10 and the IMD is different, but the differences are generally imperceivable either before or after room correction up to normal (or even reference level) listening volume, for example, then I'd like to know so I can make an informed choice. If one AVR costs $5,000 more than another but all features and performance are audibly identical, I'd like to know.

I also like to see what liquid nitrogen-cooled 2-channel DAC has just reached 160 SINAD so I can marvel at it from a "holy crap that's amazing for $200" perspective.

I also like to know if key difference-making features (say, DIRAC support) require additional licensing, whether that licensing is locked per-device, whether it's stable, and whether it degrades audio quality. Does turning it on for a given unit reduce SINAD from 85 to 75, and can I hear the difference or is it off-set by other factors?

Are there areas of research that aren't being addressed these days

Personally, I believe that SINAD is a more useful benchmark of general quality the simpler the audio chain is and ASR weighs this in its bucket approach and in grouping AVRs separate from the 2-channel DACs. There's less going on with a 2-channel DAC intended for music reproduction than an 11.4 channel AVR that has to decode multiple audio and video formats and incorporate all the features expected of a modern AVR, while also running stable and having an easy to use interface for configuration and management. A 120 SINAD AVR is useless if it crashes and experiences weird, movie-interrupting glitches every few minutes, or if the manufacturer provides crappy support and deletes users from its forum if they fail to start their posts with "HAIL MANUFACTURER!"

I'll just give my thoughts on this, they're not answers to the problem, but they're my thoughts on it. About your second paragraph, I don't think enough is known about how to relate measurements to audibility and enjoyment for a review site to compare those aspects and draw the conclusions that you're looking for. The way I see it with SINAD and IMD, etc is that if the price is right then I'll just choose the piece of equipment that measures the best for something simple like a DAC or amp. I agree though that it would be great if there was some scientifically proven information out there that could thresholds for these various measurements and therefore we'd be in a position to evaluate the real world importance of the measurements. In my opinion you want your DAC to be over 100dB SINAD for psychological/theoretical comfort, that said I've got a miniDSP 2x4 that's hooked up to my speakers and the output side of that DSP solution says it's measuring minus 80-85dB of noise floor when not even playing anything and I'd say it's not negatively impacting my listening experience - but I'd always opt for over 100dB SINAD in my buying choices as long as it was priced OK.

Re, your 5th paragraph, single sentence on research - that could be an interesting inclusion into the site, it could be interesting if there was maybe a seperate section on HiFi News that could contain anything interesting re new products or new research......it might dilute the purity of the site though in being a review site......but the research side of "what's happening" now and in future would be interesting.

Your last paragraph re AVR reviews - I agree there could be a greater emphasis on real world experience or real world features rather than relying just on the current suite of measurements. AVRs are complicated beasts, and I'm sure there's more to their experience than just the SINAD numbers. The ability of the entire solution to create a perhaps measurably & subjectively positive experience at the listening positions would be an interesting addition. There'd be a lot of time investment I imagine to thoroughly review an AVR. I don't actually own an AVR, but for me they're an aspirational somewhat impractical aspiration - I haven't married the concept of 2-channel high quality music playback with a practical/affordable AVR solution, I'm currently stranded (happily) on a 2.1 channel quality (ASR measured for the speakers) system for TV/Music/Movies....but I know from experience what even an old relatively inexpensive properly setup 5.1 AVR can do for movies!

 

[heatwave]

I would recommend doing a little research and discovering a little about the science of sound. Or, I guess you could try and tell engineers, scientists, and educated hobbyists, how they’re wrong based upon your own emotions.

It makes perfect sense for the engineers to concern themselves with measyrements, but not the average consumer. It might if measurements could guarantee performance, but they do not. This is where spec heads fall of the rails with their argument every time - they cannot answer the question of what does it matter what the specs are if it sounds great? Or why are spec's so important to a consumer if they do not guarantee performance?
I get the impression such people are just out to try to shame the manufacturer if their product doesn't measure well (regardless of who good the sonic results are). Tube amplifiers don't measure well, but the good ones sound sublime...or at least allow the speakers to sound sublime. In a way solid state cannot hold a candle to.

 

[heatwave]

So you've come to Audio Science Review to tell everyone that measurements don't matter and trust your ears

Measurements matter to engineers, and for good reason. Consumers are not so good at explaining their fixation with measurements, when what matters most is sonic performance - if indeed how something sounds means anything to the consumer.
When I started to reaize a product which hasn't measured well could perform impressively, I became skeptical about what the measurements were really telling us about performance.
If the engineer gets the product to perform in a way which really pleases him or her, they have the choice of reducing the perofrmance for the sake of pleasing the spec heads, or leaving the mearurements where they are to please those who want the product to perform like a champ.
If my system - which measures really well - doesn't impress me, I would be a fool to think it's a good system because I am safe in the knowledge it measures well.

 

[Blumlein 88]

Measurements matter to engineers, and for good reason. Consumers are not so good at explaining their fixation with measurements, when what matters most is sonic performance - if indeed how something sounds means anything to the consumer.
When I started to reaize a product which hasn't measured well could perform impressively, I became skeptical about what the measurements were really telling us about performance.
If the engineer gets the product to perform in a way which really pleases him or her, they have the choice of reducing the perofrmance for the sake of pleasing the spec heads, or leaving the mearurements where they are to please those who want the product to perform like a champ.
If my system - which measures really well - doesn't impress me, I would be a fool to think it's a good system because I am safe in the knowledge it measures well.

I see the circle of confusion is strong for you. Measurements can break that circle, and if your preference is for something else it can be changed to suit your preference. Otherwise the cliche of throwing the baby out with the bathwater applies.

 

[mctron]

If the engineer gets the product to perform in a way which really pleases him or her, they have the choice of reducing the perofrmance for the sake of pleasing the spec heads, or leaving the mearurements where they are to please those who want the product to perform like a champ.

You must be trolling

 

[Jim Taylor]

It makes perfect sense for the engineers to concern themselves with measyrements, but not the average consumer. It might if measurements could guarantee performance, but they do not

Tube amplifiers don't measure well, but the good ones sound sublime...or at least allow the speakers to sound sublime. In a way solid state cannot hold a candle to.

what matters most is sonic performance

Basically, your comments adhere to the subjectivist manifesto. You use words ("performance" , in re: the system, and "sublime" ) which cannot transfer meaning to another person. Those words have no reference. The only meaning that WE can derive is that there is something out there that you like, that it doesn't seem to be accuracy, and that we cannot otherwise tell a single thing about what it is.
This is the typical problem of subjective terminology; the description (and the phraseology used to provide the description) mean something to the speaker, but not anything to other people. People can't meld minds.

If my system - which measures really well - doesn't impress me, I would be a fool to think it's a good system because I am safe in the knowledge it measures well.

Actually, this statement is 100% backwards. It should read, "If my system measures really well, I would be a fool to think it's not a good system." It may or may not be the best, but if it measures well, then it is accurate and neutral.

One other thing: no system that is accurate should ever "impress" anyone .... ever. Audio systems are not made to impress people. They are made to play back a recording of a musical performance that was entered into them. If that musical performance was lousy (and many are) then the system will present you with lousy music. If the musical performance was good, then the system will present you with good music.
And if the musical performance was truly magnificent .... well, then yes, you will be impressed ..... by truly magnificent music. Not a truly magnificent system, but truly magnificent music.

That doesn't happen all the time. Despite that, some people want it to happen all the time. Some people want to be impressed all the time. They want a magic box that turns garbage into gold, that makes what is naturally mundane into an exciting and visceral experience.
I understand that. I think everyone here understands that. And I think quite a few of the people here are attracted to that sort of system.
But they understand it for what it is: a mirage, an attractive lie. And they don't evangelize their personal choice to other people. They don't claim it's "better". They simply enjoy it for what it is.

With the apps available today, you can modify any system that is neutral and accurate to sound like any "impressive" system .... and then cancel or modify as you please at any later date. With an "impressive" system, that's either not possible, not easy or not cheap. You're stuck with it. The sound you hear is its native response, and you may not be able top get rid of all of it. And the odds are that you WILL want to get rid of all, or at least a part, of that system sound. Over the years, I've seen it time and time again. I've seen "upgrade-itis" fueled by gradual diffidence and dissatisfaction. Initial sound was "impressive", but gradually the newness wears off. And make no mistake about it; newness is what really causes the impressiveness. The brain locks onto something new and different, and we're impressed ..... for a while. Then we get used to it. Oops!

Now you may say, "You're wrong! I'm different! I've had a system that impresses me greatly, and I've had it for a long time!" Good. Congratulations! However, don't think that the vast majority of listeners out there are like you. They're not. Instead, they go from one "impressive" system to another, trying to chase that ephemeral Holy Grail. What causes them to continue? Testimonies such as yours. They read what you've written, and they trusted you. They believed you.

Talk about frustration and dissatisfaction! If they had just gotten a reasonably neutral system, they could fiddle around with all sorts of apps and find what they want .... inexpensively.

Or they could get a neutral system and face the facts: some recordings are garbage. Live with it.

Jim

 

[GXAlan]

Experiment
Go ahead and try a UMIK-1 or other calibrated microphone. Record your own voice and play it back on your best setup.

Unless you're a voice actor, the way your recorded voice sounds to you probably doesn't sound the way it's being played back. This is the mirage that @Jim Taylor talks about -- we want to sound better/cooler than we really do. You can add a little bit of bass to your bass or even a bit of DRC and then it sounds better.

That doesn't happen all the time. Despite that, some people want it to happen all the time. Some people want to be impressed all the time. They want a magic box that turns garbage into gold, that makes what is naturally mundane into an exciting and visceral experience... they understand it for what it is: a mirage, an attractive lie. And they don't evangelize their personal choice to other people. They don't claim it's "better". They simply enjoy it for what it is.
...
Or they could get a neutral system and face the facts: some recordings are garbage.

Why not have both? No one ever made the rule that you only get one speaker/sound system.