Master Thread: Are measurements Everything or Nothing?

[lashto]

My main system uses either a 78db (according to Amir's testing, dominated by 3rd harmonic distortion) SINAD ICEpower 200ASC amplifier, or a 30lb YarLand Class A 6CA7 PP vacuum tube amplifier. The YarLand is probably in the 60dB SINAD range. I can hear no buzz or hum with my ears very close to the speakers, and with my IOTAVX AVP volume control at a fairly loud normal listening level, but no source playing.

Twenty years ago, in the living room, I had a pair of Klipsch Forte II's with a Tektron 2A3 SET driving them. Sources were a YBA CD player and a McIntosh MR74 tuner. In the dining room was a second audio system with a Bryston B60R integrated driving a pair of Apogee Centaurus monitors.

All of those systems were dead quiet and played music that I enjoyed thoroughly. In my world, 120dB SINAD is not nirvana - at least for enjoyig music.

Your system plus the warm weather sounds like a good recipe for a long retirement. I am still working on that, the "warm place" is missing from the list and still (too) many years left of work, work, work...
That is a cool looking tube amp/pre, hope you do not have to move it often . I would have 'fought' for the Tektron from your old system. No idea if it did sound any 'better' but I like its mean looks even more. Was it exactly this one?

My system is also a sort of yin/yang using two 'parallel' lines: one with tube 'horrors' and one with (reasonably) linear SS amps. Don't really need anything but thinking about single drivers with 95+ dB efficiency (my speakers are only ~87dB and the SET is struggling a bit). Looking for a single driver/speaker that is not priced for bankers like the ones in the video linked above.

P.S.
and so we went from 120 SINAD to SETs/tubes. This thread (title) is such an open-ended click bait

 

[Galliardist]

do the non-professional audioscientists have a method, or just randomly select anything and do the leg work with amir's techniques ? if they do, i salute their ocd as time and finances and patience to set up the bits is unlikely to eventuate in my universe

EQ of headphones takes very little time, and assuming a computer source, potentially no money. I did mine with Equalizer APO and a downloaded profile from someone who had measured and generated the appropriate EQ. It took me about an hour.
Of course, you have to have phones with low enough distortion figures that you can apply the equalisation and still have a clean sound. You need someone who has measured to check that as well. Hardly "audio science" on my part.

What it is possible to blind test with headphones, by the way, are different frequency responses applied to the same headphones. This is what the professionals have done. We can take advantage of that: we can still find ourselves choosing a different sound, but it's probably best to start with what the majority prefer, simply because you are more statistically likely to be in that majority than not.

 

[scruffy1]

True, true. It calls people to a thread wherein they converse and, hopefully, justify their cherished beliefs ..... or not. Jim

i wasn't called

they threw my uneducated arse over here from a thread about the denafrips ares, so i might see the light


guess it worked, a bit

 

[BDWoody]

i now think you meant; curse the interweb for making nuance a guess more often than might be ideal

I'd go further and say that much of the hi-fi related interweb in particular makes the impossible seem plausible, with a good enough story. The learning part isn't that hard, it is the unlearning that is nearly impossible for many who have relied on bad information fed to them by people who have taken advantage of their ignorance for years.

Taking the time to test yourself may be more revealing than you want to believe. It was the before/after moment in my hi-fi journey and it doesn't have to be as hard as many want to make it. For me, I took my new fancy DAC and compared it with my non-fancy DAC, followed the basic directions to take a multimeter to match the output levels on the two DAC's, mixed up the cables so I didn't know which was connected to which, connected the 'anonymous' cable ends to my preamp, then switched between them.

It was obvious beforehand which was better. Once the levels were matched and I didn't know which was which, the differences magically disappeared. I played around with a few more boxes I had around with the same results. Humbling? Definitely. Eye opening? Definitely.

 

[scruffy1]

it's a compelling invitation, but my inertia is less about concern of discovering my ignorance, than having other distractions that currently offer better reward for time invested, given i enjoy the arrangements i have here for listening (+/- delusional expectations of quality)

presently, rearranging cables is not as enticing as building a more sophisticated discus launch glider, where the performance improvement of that sort of toy is much more easy to recognise, and there is a seasonal imperative to make that priority as summer approaches

but again, i thank you for supplying food for thought

i already feel cleverer - even though that may be yet another delusion

 

[lashto]

I assumed that the sarcasm was addressed at me: that in turn you are mistaking my position for the one which you attacked.

could very well be..

But you are still making the same mistakes as in your previous post: you are claiming that there are things which are not understood, when they are: you are mistaking SINAD for science and a scientific approach. Just because this is framed in a sarcastic attack on a perceived position doesn't make those mistakes go away.

Only 'mistaking' it for the single measurement/metric we have for HD. Excellent for engineers, not so good for 'the people'. A better HD metric may save tons of internet-ink per day. All those tiring discussions about what is audible/not...

Much better, and maybe closer to what you mean. But have you not notice the scoring done in response to speaker measurements here? This goes some way to what you are looking for, doesn't it? Though I suspect the speaker/room/listener part is somewhat simplified for the purpose.
...

I'd argue that the point you call "much better" is exactly the same as before. But no need for more internet-ink.

Speaker measurements are very useful but only the links from FR & room to audibility are (somewhat) clear. Last I checked, Toole did a a cop-out on HD with stuff like "on nonlinear distortion .. there is no unanimity of opinion .. there is a need for more work .."

You bring out other points about populations, repeatability, purchasing habits... Those might part of the HD-to-preference metric but no need to go that far. Just give me HD-to-audibility and I'll stop complaining .. for a while

 

[Blumlein 88]

it's a compelling invitation, but my inertia is less about concern of discovering my ignorance, than having other distractions that currently offer better reward for time invested, given i enjoy the arrangements i have here for listening (+/- delusional expectations of quality)

presently, rearranging cables is not as enticing as building a more sophisticated discus launch glider, where the performance improvement of that sort of toy is much more easy to recognise, and there is a seasonal imperative to make that priority as summer approaches

but again, i thank you for supplying food for thought

i already feel cleverer - even though that may be yet another delusion

Trolling, Trolling, Trolling keep them dogies Trolling, troll on........

 

[lashto]

True, true. It calls people to a thread wherein they converse and, hopefully, justify their cherished beliefs ..... or not. Jim

“I don't have pet peeves - I have major psychotic hatreds.”
― George Carlin

 

[krabapple]

In my 'complaints' I only considered the reproduction part, between the 'CD' and the ears, so yes transducer-only-complaints. Some talk about a much 'longer' audio system, i.e. everything between the microphone and the ears. A lot more problem-parts in there and one can easily extend the complaints ...

In terms of "predicting performance from measurements" I'm only asking for a bit less: predicting audibility/preference from measurements.

Audibility can be predicted very well from measurements.

Preference for sound, where that is possible (meaning, there is a real audible difference), less so. Surely you're aware of Harman's work in that aread, on speakers. No one claims it's "100% black and white" though. Including Harman/Olive/Toole.

 

[krabapple]

thank you, thank you

i'm here all week (if you're unlucky)

I'm applauding. Not for your tedious, blathering act, but for the ignore option. Toodle-oo.

 

[antcollinet]

presently, rearranging cables is not as enticing as building a more sophisticated discus launch glider.....

I think you have your priorities 100% (measured to the 10th decimal place) correct.

 

[scruffy1]

Trolling, Trolling, Trolling keep them dogies Trolling, troll on........

um, no..... see below

your incorrect judgement of my motivation is unwarranted - trust me, i'll get around to testing my equipment, but sorry for your frustration that it won't be this afternoon

i am intrigued by what i have had explained to me here in the last few days, but it is now on my to do list rather than my "must get right on to that now" list; as stated, the rule of triage applies because unlike many here seem to manage, i have far more tasks to undertake within my current interests (okay, obsessions), and only a certain aliquot of waking hours left after also earning a living and managing a family

I think you have your priorities 100% (measured to the 10th decimal place) correct.

written like a person who understands an appreciation of music trumped by a love of cumuli

 

[RayDunzl]

presently, rearranging cables is not as enticing as building a more sophisticated discus launch glider

I flew RC sailplanes - ridge, thermal, hand and bungee launch.

While considering building another (after 25 years or so), I viewed the remains of the last one in a paper grocery bag in the garage.

My present hobby solution has progressed to this point:

1980 Schweizer 1-35C to call my own.

Haven't crashed it even once!

 

[scruffy1]

i see your aeroplane, and present my current toy - i started making balsa and newspaper kites around 1968, and worked up through free flight gliders, rubber (peanut scale) and control-line (ukie, as americans say), then after a hiatus during university, restarted in gliders and then r/c gliders (and rubber and co2 and sometimes now, electric powered stuff, which had come of age by the late naughties)

i love it when people ask me if i have ever considered flying a real plane, and i get to reply "i do, they're just small"

discus launch glider is really the apex of my sensibilities - no extra gear to tote, a bit of exercise, silent and graceful... and beloved of the pursuit dog

there are two much higher end craft that i need to fit into my schedule, hence my reluctance to go off on an audiophile tangent - it's spring where i am

cables (tow line, bungee... rca, coaxial, optical) have lost their attraction in comparison

cheers

anyway /thread crap

 

[ahofer]

Making the rounds on Audiogon:

Why comparing specifications is completely pointless

Bitrates, sampling rates, bit sizes, wattages, amplifier classes…. as an audio enthusiast, there are countless specifications to compare. But it is – virtually – all meaningless. Why? Because the specifications that matter are not reported ánd because every manufacturer measures differently...

alpha-audio.net

Aargh. For instance, when we deliberately leave out specs that are provided, the other specs are “useless” (oh, and “microdetails”)

To get straight to the point: power (wattages) means nothing. In many cases it says for example: 100 watts into 8 Ohms. And if you are lucky, there is a specification for 4 Ohms. That gives a little insight into the reserve; with a very stable amplifier the power doubles. What is often not mentioned, however, is how it is measured, and how much distortion occurs at maximum power. Is that 1%? 10%? Is it at 1 kHz? Full spectrum? In short: it is useless to compare wattages if you do not know this.

It is much more interesting to know what the behavior of the amplifier is at very low power levels. Below 1 watt for example. Below are three power measurements. Look carefully at the vertical axis … that is the distortion. The horizontal axis is the power. The Fosi amp has nearly 90% distortion at 39 uWatts. And well over 15% at 1 mW. 1 watt comes to 0.5% which is horribly high by today’s standards. In fact, most speakers are well below that. The Pass Labs X150.8 shows 0.35% at 37 uWatt, and then drops down. At 1mW it shows 0.07%. The Advance shows neat values in its price range: 1 mW comes in at 0.4%.




Why do we insist on this? Simple: the first watt is crucial. All microdetail is in the low regions of power. These values are nowhere to be found. Only maximum power, but never the distortion for 1 watt or a complete graph, when that’s what it’s all about.

Then they say the quiet part out loud:

However, the purpose of this background story is to prevent you – the reader – from focusing on specifications. Above all, go listen and trust your ears: not just the specifications of a product.

 

[scruffy1]

in their defence, they don't say it's meaningless - they say it's "virtually meaningless"

that's doublespeak for "i don't know what it means"

 

[Blumlein 88]

Making the rounds on Audiogon:

Why comparing specifications is completely pointless

Bitrates, sampling rates, bit sizes, wattages, amplifier classes…. as an audio enthusiast, there are countless specifications to compare. But it is – virtually – all meaningless. Why? Because the specifications that matter are not reported ánd because every manufacturer measures differently...

alpha-audio.net

Aargh. For instance, when we deliberately leave out specs that are provided, the other specs are “useless” (oh, and “microdetails”)



Then they say the quiet part out loud:

Actually they are lying. The graph is thd+N. Not distortion.

At those lower levels it likely is low distortion and mostly noise. So this article is intended to misinform people before lying to them. Highly reprehensible. I might think the author is incompetent, but my first guess is intent to mislead.

 

[BDWoody]

in their defence, they don't say it's meaningless - they say it's "virtually meaningless"

that's doublespeak for "i don't know what it means"

I think that gives them too much credit. The last thing they want is an informed customer.

 

[MacCali]

Not sure what you are really going on about here. First blush is this being a covert attempt to re-insert subjective arbitration of fidelity. I'll take you at your word that you aren't attacking the approach informed by measurements.

So perhaps a good idea to start a new thread with you laying out in a little more detail what you really are thinking about. Also the "two forms" you mention above one of which you've given us a taste of in this post I'm replying to here.

I am starting to preach the reality of audio to our ASR members in two forms.

Well I will be glad to bring up the second one, from personal experience. We as objectivists always look at measurements as a defining factor and as stated by @Purité Audio that something that measures well does sound good. I pointed out already two things I personally heard that have phenomanal measurements and absolutely did not sound good to me, I own the 887 and I have heard the ABH2 three times and did not enjoy it even in bridged configuration which would clearly create more noise/distortion/etc.

As stated originally, IT IS NOT BAD SOUNDING, so with Purite's point I see eye to eye. It just does not sound typical to everything else and may come off as unnatural, and again I pointed out the HPA4 which is using the 888 technology I believe and is very clean measurement wise does sound good. I also have a FiiO Q5s with the THX AMD amp module which is amazing too and clean as well.

But back to the topic at hand, the second thing I have realized in my audio journey which has only been just over 2 years now. Is that even products with HORRIBLE sinad, for example less than -40 db DO NOT sound as bad as you would imagine. However the threshold we seek is at least-110db and Amir at -115 for clearly inaudible issues.

I have two stereo setups at my home, one is my main with everything "flagship" measurements and it is extremely enjoyable. My other system is just for background music and sometimes "critical" listening or full attention listening. My second system has a component which does measure at less than -40 db. The dac and speakers are garbage too. So overall the entire system is crap.. and it cost me about 220 just for context. So even with these extremely poor measurements I still enjoy critical listening on this system. It absolutely doesn't bother me..

The thing that got me is this, I never knew what measurements of my chain were on this secondary system and I would listen and have a pleasant experience. Then all of sudden I found out this amplifier was actually measured and the results were grotesque. It's probably one of the worst measuring amps in every aspect and undoubtly the bottleneck of my system.

So this now raised the question, how bad is bad really? Before I saw these measurements I was having a great time listening and even after these findings it still did not bother me. Have you yourself owned any components which are this bad and also had a "flagship" system to compare. Clearly this -40 db sinad amp should be the definition of bad but when it comes to perception it actually doesn't sound "bad". I am not sure how else to get my point across for you to understand what I mean. I am in no way saying the poor measuring system is better than my main system. Yes the tweeter does hiss and a lot of nasty things are audible but when the music is playing to me it's not as bad as I would of thought it would be based on these measurements and personally I do not hate it and it does not in any way make me want to not want to listen anymore.

I cannot complete a blind test, but what I can do is literally play music in my main listening room and just walk into my living room and switch that very same track and listen to it on this trash system. I cannot tell you a quantified amount of how much improvement I hear even though it does exist without a doubt, but all I am saying is it is not as bad you would imagine.

Furthermore, this tells me that when these people come in here saying this sounded amazing exactly like what brought this exact threat back into discussion, and that these measurements are BS this is the exact reason why they are saying that. I can see where those people are coming from and also that if you had for example an amp or equipment which is -80 db sinad you would surely be satisfied with that if -40 isn't as bothersome as you would assume or hypothesize.

In addition, this would make sense as well when you look at so many subjective reviews when people say my god this is an amazing dac or amp. Yet as soon as we see the measured performance of the unit we are like omg that's total crap and overpriced junk. That's exactly how all these companies have for years gotten away with selling you mid grade performance for a high price tag.

Lastly, with all of this in mind, you as a consumer should have an even harder time being able to differentiate the difference between products -80 to -120 cause I am telling you from experience the difference between -40 and -120[my main system] is not as crazy as it may seem or be perceived. You would think that something that is that poorly implemented would have a dramatic difference and that it would make the experience unpleasant.

The only reason I preach this is because I feel most people have never had this experience and you should experience it, hell you should have a blind test to really understand it better. 10 out of 10 times you will know which system is better measuring even if you aren't a pro listener but AGAIN the point is it's not that bad. Clearly this is my opinion, and if you really want to sit there and dissect everything bad rather than listen and enjoy the music you're going to have a miserable time with this poor measuring system

This entire post is not for the purpose for people to stop looking at measurements or ignore objectivity, it is merely a learning experience and for me it created a foundation of what the difference between a "floor" and ceiling in audio measurements truly are.

 

[Blumlein 88]

Well I will be glad to bring up the second one, from personal experience. We as objectivists always look at measurements as a defining factor and as stated by @Purité Audio that something that measures well does sound good. I pointed out already two things I personally heard that have phenomanal measurements and absolutely did not sound good to me, I own the 887 and I have heard the ABH2 three times and did not enjoy it even in bridged configuration which would clearly create more noise/distortion/etc.

As stated originally, IT IS NOT BAD SOUNDING, so with Purite's point I see eye to eye. It just does not sound typical to everything else and may come off as unnatural, and again I pointed out the HPA4 which is using the 888 technology I believe and is very clean measurement wise does sound good. I also have a FiiO Q5s with the THX AMD amp module which is amazing too and clean as well.

But back to the topic at hand, the second thing I have realized in my audio journey which has only been just over 2 years now. Is that even products with HORRIBLE sinad, for example less than -40 db DO NOT sound as bad as you would imagine. However the threshold we seek is at least-110db and Amir at -115 for clearly inaudible issues.

I have two stereo setups at my home, one is my main with everything "flagship" measurements and it is extremely enjoyable. My other system is just for background music and sometimes "critical" listening or full attention listening. My second system has a component which does measure at less than -40 db. The dac and speakers are garbage too. So overall the entire system is crap.. and it cost me about 220 just for context. So even with these extremely poor measurements I still enjoy critical listening on this system. It absolutely doesn't bother me..

The thing that got me is this, I never knew what measurements of my chain were on this secondary system and I would listen and have a pleasant experience. Then all of sudden I found out this amplifier was actually measured and the results were grotesque. It's probably one of the worst measuring amps in every aspect and undoubtly the bottleneck of my system.

So this now raised the question, how bad is bad really? Before I saw these measurements I was having a great time listening and even after these findings it still did not bother me. Have you yourself owned any components which are this bad and also had a "flagship" system to compare. Clearly this -40 db sinad amp should be the definition of bad but when it comes to perception it actually doesn't sound "bad". I am not sure how else to get my point across for you to understand what I mean. I am in no way saying the poor measuring system is better than my main system. Yes the tweeter does hiss and a lot of nasty things are audible but when the music is playing to me it's not as bad as I would of thought it would be based on these measurements and personally I do not hate it and it does not in any way make me want to not want to listen anymore.

I cannot complete a blind test, but what I can do is literally play music in my main listening room and just walk into my living room and switch that very same track and listen to it on this trash system. I cannot tell you a quantified amount of how much improvement I hear even though it does exist without a doubt, but all I am saying is it is not as bad you would imagine.

Furthermore, this tells me that when these people come in here saying this sounded amazing exactly like what brought this exact threat back into discussion, and that these measurements are BS this is the exact reason why they are saying that. I can see where those people are coming from and also that if you had for example an amp or equipment which is -80 db sinad you would surely be satisfied with that if -40 isn't as bothersome as you would assume or hypothesize.

In addition, this would make sense as well when you look at so many subjective reviews when people say my god this is an amazing dac or amp. Yet as soon as we see the measured performance of the unit we are like omg that's total crap and overpriced junk. That's exactly how all these companies have for years gotten away with selling you mid grade performance for a high price tag.

Lastly, with all of this in mind, you as a consumer should have an even harder time being able to differentiate the difference between products -80 to -120 cause I am telling you from experience the difference between -40 and -120[my main system] is not as crazy as it may seem or be perceived. You would think that something that is that poorly implemented would have a dramatic difference and that it would make the experience unpleasant.

The only reason I preach this is because I feel most people have never had this experience and you should experience it, hell you should have a blind test to really understand it better. 10 out of 10 times you will know which system is better measuring even if you aren't a pro listener but AGAIN the point is it's not that bad. Clearly this is my opinion, and if you really want to sit there and dissect everything bad rather than listen and enjoy the music you're going to have a miserable time with this poor measuring system

This entire post is not for the purpose for people to stop looking at measurements or ignore objectivity, it is merely a learning experience and for me it created a foundation of what the difference between a "floor" and ceiling in audio measurements truly are.

You might be over-reaching with your conclusions from your experience.

First I can agree with you in some of this. Tape machines, the source of recorded music for 50 years, only have a Sinad of something in the mid 40 db range. Even pro units. With noise reduction in use you will have better SNR, but the THD will still rarely be less than -50 db if ever. Those as a source even with the finest cleanest gear otherwise can be highly highly enjoyable. It is my opinion, with some considerable justification, that a Sinad of -80 db is indistinguishable from anything better when listening to music at least 98% of the time. I've written many times the old adage that frequency response is 85% of high fidelity again with some justification. In a sense we are far more sensitive to minor FR differences than we are to relatively high distortion. For over a decade I only used vacuum tube power amps.

Transducers (tape heads, phono cartridges, microphones, headphones and speakers) are the fidelity bottleneck and the main source of any sound character that isn't transparent.

So where I think you overreach is thinking because you found sound not enjoyable when a Benchmark amp or other clean amp was in use that an overly clean presentation is inherently less enjoyable (assuming I am not misinterpreting your thinking). Could be many things like which speakers were in use. Or your level of enjoyment was less than anticipated and gave you a poor opinion of the Benchmark even if it seemed fine. Or possibly your preferences are not for fidelity (which is okay, we do this because music is fun). Or a less clear system covers some issues in a recording that a clean clear system exposes and that ruins your experience. And yes, if that ruins your experience your experience is not wrong. However the issue is the recording and not how clean the amp is. You can have highly enjoyable experiences with really clean gear.

So on a measurement oriented forum, if a given piece of gear costs $5000 and has -80 db SINAD it is judged not a good option. Maybe it will in no way hurt the sound. When you can get SOTA performance for $500 it is a poor deal. Often the $5k unit also comes with some dubious reasons that it sounds better despite measuring mediocre. Usually those dubious reasons are actually just flat out wrong. A really fancy $5k unit with some good story attached will find plenty of accolades among reviewers and consumers. One is likely to be affected by that. Anyone, all of us. Measurements however are not affected.

A device with -120 db SINAD, flat frequency response to beyond 20 khz and no peaks in ultrasonic response is beyond reproach. It will not add to or subtract from the sound of the source signal in a way that is audible. Less good gear may be as good audibly, but you have no doubt with the best of gear. That does not mean if you don't have SOTA performance a device is junk or that one unit being -90 db SINAD will sound better if you swap to one that is -120 db SINAD. The middling numbers need more nuance in understanding to know if they sound transparent or not.

I think a sensible approach in current times is to have very good clean setup every step of the way up to the speakers. And speakers will make the most difference in what you hear (or headphones). If you want a particular type of non-super fidelity sound, DSP can provide most anything you want. Yes, none of this means "junky" systems cannot be enjoyable. If you rely upon a "dirty" system to cover up mistakes in speakers or recordings you will find really good recordings don't sound as good. In the past you may have needed to pick your poisons and work around it. Now one can for a not tremendous amount of money get a clean system, and adjust it for less than great recordings or to a limited extent less than great speakers.

Looking back, until the introduction of CD, all of our sources were no better than -50 db SINAD. Reel tape, cassette tape, 8 track tape, LP vinyl, and FM radio. Those sources even now can be enjoyable to listen to all day long.

Here is another post from a few hours ago with some links that might interest you. One is Distort software from pkane here on ASR. It lets you adjust levels of distortion and each harmonic any way you please. You can use it to play your music files and see what kinds of distortion you can hear. The other links have online tests to determine what levels of distortion are audible to you. HINT HINT: you aren't likely to find you can hear -80 db THD vs lower levels.

Ideas about blind test with ADDA

here's the thing, blind test can be carrying out: we need a bunch of different DACs all set under the same output level like 2vrms or 4vrms DAC - AD - digital files there're some decent adda out there, but anything within 10 or even 15 years can do that (audio interfaces, the one I daily use...

www.audiosciencereview.com