Master Thread: Are measurements Everything or Nothing?

[BDWoody]

Sorry Jim, I know this may not the place, but all amplifiers sound different.Try them and you'll know.

That's ok, I moved it to a 'better' place.

OMG ... here we go again!

Jim

Got this one moved in early!

 

[SIY]

I can't fool myself with what I hear and I play the violin and go to classical concerts.

Well, that's that, then.

 

[voodooless]

Well, that's that, then.

Sorry SIY, that’s a rock solid argument. I guess we’re done here.. finally, after only 472 pages

 

[Benesyed]

Your wife sounds like smart cookie

His wife is the little child pointing out the the emperor is naked

 

[Deleted member 21219]

Sorry Jim, I know this may not the place, but all amplifiers sound different.Try them and you'll know. Measurements are useful for electronic performance but listening to music and synergy is another.

Even perfect measured DACs and amps may sound lifeless, dry, monochromatic or clinical. Music reproduction of the real performance is not mirrored in measurements of a full scale 1KHz, frequency sweeps or a 32-tone test.

I bought a Hypex NC122MP based amp, brother of the NC250MP that got a high review here, and violins and guitars sound metallic with the Elacs. A cheaper Onkyo A-9010 integrated sounds better, more realistic tonality. I can't fool myself with what I hear and I play the violin and go to classical concerts.

First point: Have you perused this site? Have you scanned through any of the threads? Have you gleaned ANY of the information at this site? Did you look at this page?

Audio Reference Library

Formal educational posts and articles on science and engineering of digital audio. Only site admin can create new threads. But anyone can comment.

www.audiosciencereview.com

Second point: Amir has put time and energy into several videos explaining various scientific points (and debunking various subjectivist points) in a manner that is cogent and easily understood. Some are here:

Master List of Instructional Teaching Video presentations (How to read and interpret/understand all the Graphs and Charts!

Collection of Amir’s instructional Videos for reference, refresher and new members: (“Behind the Curtain @ ASR University”) Welcome New and returning Members of ASR. Our Host Amir has put together a collection of comprehensive instructional videos that might help new and returning Members...

www.audiosciencereview.com

You might want to watch the one on blind listening tests - especially as it applies to electronic circuits.

Thirdly: The differences to which you refer are either A) audible or they are B) not audible. If they are audible, that means that they can be heard. SO ... you will be able to identify them in a rigorously controlled double-blind test, is that not correct?
Conversely, if you CANNOT identify them in a rigorously controlled double-blind test, then, they are, ipso facto, not audible.

BTW: audibility is a different issue than circuit characteristics. There are many different circuits, with many different measurable characteristics. That does not mean that the differences are audible.

Fourthly: People are subject to various biases. A bias is the brain's shortcut in the decision-making process. However, like many shortcuts, accuracy is sacrificed in order to gain speed and utility. After all, if you hear a roar in the jungle, you're not going to make a wide-spectrum frequency analysis to figure out whether it's a lion or not.
Here is a list of confirmation biases known to cause problems:

List of cognitive biases - Wikipedia

en.wikipedia.org

Note that there have been visitors to this site who claimed to be able to control their biases. That has never been proven to be true.

And lastly ... neither being a musician nor going to concerts mean anything to music reproduction in the listener's room. A musician, either in an orchestra pit or onstage, doesn't hear what a listener hears in the audience. Not only that, but a concert is recorded by sound engineers through microphones, and the signal presented to your system depends on the way they handle the recording process and the goals that they have. Comparisons based on either of these attributes you mentioned are invalid.

You might also wish to read this, just to make sure that you've not made this mistake:

False Premise: When Arguments Are Built on Bad Foundations – Effectiviology

effectiviology.com

Jim

 

[solderdude]

Even perfect measured DACs and amps may sound lifeless, dry, monochromatic or clinical.

Why ?
Have you ever seen some irrefutable evidence that this is indeed the case and was this test done in any scientific sound setting ?

Music reproduction of the real performance is not mirrored in measurements of a full scale 1KHz, frequency sweeps or a 32-tone test.

Agreed SINAD says nothing about the total performance.
Some sweeps or multitones already say more.
A multitone test also lifts a part of the total performance.

Signal fidelity of electronics under actual load can be captured well and much more reliable and in depth than with listening tests.

Things go sour when measuring speakers in a room and when listening tests are done without proper controls.

 

[Killingbeans]

Music reproduction of the real performance is not mirrored in measurements of a full scale 1KHz, frequency sweeps or a 32-tone test.

I agree, but for a different reason.

Contrary to layman's logic, the simple test signals are way less forgiving in terms of audibility.

Once you switch to music, you're entering a world of hurt in the form of masking effects.

 

[Benesyed]

First point: Have you perused this site? Have you scanned through any of the threads? Have you gleaned ANY of the information at this site? Did you look at this page?

Audio Reference Library

Formal educational posts and articles on science and engineering of digital audio. Only site admin can create new threads. But anyone can comment.

www.audiosciencereview.com

Second point: Amir has put time and energy into several videos explaining various scientific points (and debunking various subjectivist points) in a manner that is cogent and easily understood. Some are here:

Master List of Instructional Teaching Video presentations (How to read and interpret/understand all the Graphs and Charts!

Collection of Amir’s instructional Videos for reference, refresher and new members: (“Behind the Curtain @ ASR University”) Welcome New and returning Members of ASR. Our Host Amir has put together a collection of comprehensive instructional videos that might help new and returning Members...

www.audiosciencereview.com

You might want to watch the one on blind listening tests - especially as it applies to electronic circuits.

Thirdly: The differences to which you refer are either A) audible or they are B) not audible. If they are audible, that means that they can be heard. SO ... you will be able to identify them in a rigorously controlled double-blind test, is that not correct?
Conversely, if you CANNOT identify them in a rigorously controlled double-blind test, then, they are, ipso facto, not audible.

BTW: audibility is a different issue than circuit characteristics. There are many different circuits, with many different measurable characteristics. That does not mean that the differences are audible.

Fourthly: People are subject to various biases. A bias is the brain's shortcut in the decision-making process. However, like many shortcuts, accuracy is sacrificed in order to gain speed and utility. After all, if you hear a roar in the jungle, you're not going to take a wide-spectrum frequency plot to figure out whether it's a lion or not.
Here is a list of confirmation biases known to cause problems:

List of cognitive biases - Wikipedia

en.wikipedia.org

Note that there have been visitors to this site who claimed to be able to control their biases. That has never been proven to be true.

And lastly ... neither being a musician nor going to concerts mean anything to music reproduction in the listener's room. A musician, either in an orchestra pit or onstage, doesn't hear what a listener hears in the audience. Not only that, but a concert is recorded by sound engineers through microphones, and the signal presented to your system depends on the way they handle the recording process and the goals that they have. Comparisons based on either of these bases are invalid.

You might also wish to read this, just to make sure that you've not made this mistake:

False Premise: When Arguments Are Built on Bad Foundations – Effectiviology

effectiviology.com

Jim

I need to go through those videos so I can appreciate the full meaning of the measurements and where the thresholds are for audible differences.

My own personal understanding from dwelling here is that most low and mid price components (dac, amp, or any other non transducer) are already above the audible fidelity and difference threshold. In addition, many are very reliably fabricated or are reliable enough that you could purchase them again and still be under the cost of high end components for as long as you live.

It seems the issue is really with high end components. Some try to sell themselves as better than what anyone could hear and some are actually worse! On top of that some don't even have nice build quality.

That's where ASR is helpful. Amirs criticism doesn't mean you can't enjoy the thing you want to enjoy. It's just pointing out that it might not live up to the hype you are ascribing it.

I am often reminded of this Keanu Reeves interview:

ASR is fighting the good fight for 1+1=2

 

[Mart68]

I agree, but for a different reason.

Contrary to layman's logic, the simple test signals are way less forgiving in terms of audibility.

Once you switch to music, you're entering a world of hurt in the form of masking effects.

Or you can use that recording they always play at shows which is just a woman singing while someone hits a block of wood with a stick.

'OMG listen to that block of wood it's so realistic!'

'Can we hear some Thin Lizzy now please mister?'

'No.'

 

[Benesyed]

Why ?
Have you ever seen some irrefutable evidence that this is indeed the case and was this test done in any scientific sound setting ?


Agreed SINAD says nothing about the total performance.
Some sweeps or multitones already say more.
A multitone test also lifts a part of the total performance.

Signal fidelity of electronics under actual load can be captured well and much more reliable and in depth than with listening tests.

Things go sour when measuring speakers in a room and when listening tests are done without proper controls.

The lifeless argument makes little sense to me for dacs.

For tube amps I can understand it a little bit. I think of the distortion tubes add as pepper being added to food. Some people like a lot of pepper added to their food, no matter what it is. That's certainly a preference but It's not exactly hi fidelity to the food you are eating and you definitely will be altering the intended taste.

But dacs typically don't really do that, from best to worst dac it's unlikely they are introducing anything enough to change the audible sound

 

[Andrej]

I thas been a long time since I decided (mostly, but not entirely, based on sighted comparisons, but also having done many automated blind comparisons as a subject at work) that I am wasting time comparing competent DACs and most pre- and power-amps. The big obstacles to fidelity have been transductions and room acoustics. Conversion of acoustical/mechanical/electrical energies is where the real problems are, where SINAD is vastly lower than in electronics. I read somewhere in these pages that if a unit (call it a DAC) has SINAD >10dB higher than the next item (pre-amp) then pre-amp defines the final fidelity. Applying that premise to the whole chain, it is a no-brainer that speakers are the only place with significant shortcomings. I pick non-trunsducers by measurements/price/features (in different order depending what it is and how expensive it is). Listening does not provide any insights. Even if I am able to detect minute (percieved) differences, I have never, EVER, been able to assign preference.

I know nothing about microphones, and have no control over those, so I ignore them. Room acoustics is a major obstacle for me, as it is inconvenient to change, and I still need to learn a lot more about what constitutes a genuine improvement (lots of conflicts here - improving one aspect can hurt another aspect of percieved change). So I devote my time to speakers, and I use DIY, as I see a big benefit in cost (for what I am trying to achieve). I still mostly depend on measurements - and here is where ASR has helped a lot. The trick is to measure significant parameters, and have a rational way to convert the measurement results into audability. I do not agree with every common practice, as I think my priorities are somewhat different than the common ones, and I do not necessarily think this is just a matter of taste, but we should always accomodate decision based on taste, but not as something that is "better" if using a common metric. "I like it" != "better". Defining the metrics carefully and precisely is the only way to make progress (in addition to coming up with better metrics, of course!).

 

[ads_cft222]

How about from someone who actually designs amps?

Here are a couple of snippets from an interview done with Quad's Peter Walker many years ago:

"Question:
Have you any opinions you'd share on the relative merits of distortion tests,
such as harmonic, two-tone IM, transient IM, or slew rate limiting, as clues to
amplifier quality?

"An amplifier should, within its limits of voltage and rate of change of voltage,
(which is slew rate limiting) if you keep within those two it should be very much
better than any program material. These are the things that are measured at .01
per cent or .05 per cent. But what is listened to is usually a program with 2 or 3
per cent distortion in the first place. That's the least you can get on records,
tapes, and such things. Listening tests are usually not done in this region of .01
percent distortion. I'm quite convinced within that range the amplifier is just as
perfect as you like to make it. It's quite possible to put 50 amplifiers in cascade,
each one into a load, potted down into the next one, and to listen to the 50th one
or to listen to the first one, and the sound will be virtually the same.

...

The peripheral effects are what get people into trouble. You can
see why you find these differences in amplifiers. You can always find them. If
people test two amplifiers and say, "These sound different," there's no magic in
it. Spend two days, maybe a whole week in the lab, and you find out exactly why
they're different and you can write the whole thing down in purely practical,
physical terms. This is why these two sound different, and the cause is usually
peripheral effects. It is not really a case of good or bad amplifiers, it's that the
termination impedances are wrong, or something of that sort. "

"TAA: How do you rate the merits of listening tests to instrument tests?

PW: We designed our valve (tube) amplifier, manufactured it, and put it on the market. and never actually listened to it. In fact, the same applies to the 303 and tile 405. People say, "Well that's disgusting, you ought to leave listened to it." However, we do a certain amount of listening tests, but they arc for specific things. We listen to the differential distortion - does a certain thing matte' You've got to have a listening test to sort out whether it matters. You've got to do tests to sort out whether rumble is likely to overload pickup inputs, or whether the high frequency stuff coming out of the pickup due to record scratch is going to disturb the control unit But we aren't sitting down listening to Beethoven's Fifth and saying "l hat amplifier sounds better. let's change a resistor or two. Oh yes, that's now better still." We never sit down and listen to a music record through an amplifier in the design stage. We listen to funny noises, funny distortions, and see whether these things are going to matter. to get a subjective assessment. But we don't actually listen to program material at all."

https://geocities.restorativland.org/ResearchTriangle/Lab/6722/pwint1.txt

Arguably the most important was an “artifact” that was a hiss at high volume with klipsch 8000f for one amplifier compared to my amplifier that produced no audible hiss

 

[Newman]

@Morse27 Welcome to ASR! Are you interested in learning? I mean it sincerely. Because a lot of what you wrote above is incorrect, and I expect friendly educators here are about to give you an opportunity. Cheers

 

[fpitas]

@Morse27 Welcome to ASR! Are you interested in learning? I mean it sincerely. Because a lot of what you wrote above is incorrect, and I expect friendly educators here are about to give you an opportunity. Cheers

Yeah. Don't make me post the Whac-A-Mole picture again.

 

[BDWoody]

@Morse27 Welcome to ASR! Are you interested in learning? I mean it sincerely. Because a lot of what you wrote above is incorrect, and I expect friendly educators here are about to give you an opportunity. Cheers

Well, since that was his fourth puppet, I don't think he is.

 

[fpitas]

It must be great fun trolling us.

 

[BDWoody]

It must be great fun trolling us.

It's very important work.

I think they have to put in two years of going forum to forum as missionaries as part of the deal.

 

[Killingbeans]

Arguably the most important was an “artifact” that was a hiss at high volume with klipsch 8000f for one amplifier compared to my amplifier that produced no audible hiss

Residual noise. Audibility is dependent on the gain structure of your setup and the sensitivity of your speakers.

Hiss is an actual problem in many cases. If you claimed to be plagued by it, I don't think anybody would call it placebo.

When people "tell you what you hear" it usually as a response to much more outrageous claims.

 

[ahofer]

Sorry Jim, I know this may not the place, but all amplifiers sound different.Try them and you'll know. Measurements are useful for electronic performance but listening to music and synergy is another.

Even perfect measured DACs and amps may sound lifeless, dry, monochromatic or clinical. Music reproduction of the real performance is not mirrored in measurements of a full scale 1KHz, frequency sweeps or a 32-tone test.

I bought a Hypex NC122MP based amp, brother of the NC250MP that got a high review here, and violins and guitars sound metallic with the Elacs. A cheaper Onkyo A-9010 integrated sounds better, more realistic tonality. I can't fool myself with what I hear and I play the violin and go to classical concerts.

Wow, cliche trifecta. Still waiting for the wife to run in from the kitchen.

 

[fpitas]

OMG ... here we go again!

Jim

Here, you'll need this.