Master Thread: Are measurements Everything or Nothing?

[Killingbeans]

However, as I said before, I mutated my position from being a 100% measurements and specs guy to a always hear what you buy before one. Because the "transparent" measurements from different products I have bought always, 100% of the time, I have been able to spot sound signatures, defects and virtudes. I have learned that all audio products have audible virtudes and defects no matter how perfectly their measurements or specs are.

Sounds like you were 90% of the way. If you take the last step and do a proper level matched blind test, I'll guarantee you'll be surprised by the outcome.

This can be true, I had topping stack and I didn’t feel it any different from the chord dac I was having. On paper topping should have been better but it was not the case for me.

No surprise there. Both make products that perform orders of magnitude better than what human hearing can reasonably be expected to appreciate.

 

[dogmamann]

You aren't understanding what the measurements mean, if that was what you took from them.

Better numbers, beyond some fuzzy place that is going to be somewhat different from person to person aren't going to make it sound better.

If one DAC has an error the equivalent of a 1mm blip on a signal magnified to be 1 Kilometer high, and another has a 1.5 mm blip, how audible do you expect that to be? That's the scale we are talking about.

I said they didn’t sound different.

 

[ahofer]

This isn't Audiogon.

Thank goodness.

 

[dogmamann]

There is nothing to argue in Audiogon. Everyone agrees to subjective opinions there.

Thank goodness.

 

[BDWoody]

On paper topping should have been better...

I said they didn’t sound different.

Sounded like you expected it to, or maybe I was mistaken. On paper, the Topping and Chord should sound the same. Anything else would be unexpected.

 

[ahofer]

There is nothing to argue in Audiogon. Everyone agrees to subjective opinions there.

And yet the arguments are endless, stupid, uncivil, and pointless, and the moderators encourage it, like parents spoiling their children.

 

[Mart68]

I said they didn’t sound different.

you did but you also said ''On paper topping should have been better but it was not the case for me.''

Which is incorrect - 'on paper' they should be identical.

 

[ahofer]

you did but you also said ''On paper topping should have been better but it was not the case for me.''

Which is incorrect - 'on paper' they should be identical.

Yes.

How many times will I link this before one of these measurement objectors reads it? It's a darn good guide to what to expect in controlled comparison. The difference between "strict" and "lenient" thresholds is the "maybe you can tell a difference zone", although it is likely to be highly dependent on volume and program material (and listener age). Outside "lenient" you would expect to be able to tell a difference with careful listening. However, a lot of failed blind tests have involved differences well outside of lenient.

Audibility thresholds of amp and DAC measurements

Introduction A recurring theme on ASR is whether or not the various measured qualities of the devices are audible. In this post, I'll present some clear and visual thresholds for when those imperfections can be considered a potentially audible concern. I will not explain the basics of amp/DAC...

www.audiosciencereview.com

Add this thread on understanding loudspeaker measurements, and you have a baseline of what we would contend measurements can predict (with a caveat that I am coming to think that more varied compression testing should also be part of the speaker measurement suite in order to understand the implications for listening distance). But most here, I think, acknowledge that speakers in rooms or ear canals are more subjective and measurements are only a triage step.

Genelec 8050B Speaker Review (and how to read speaker measurements)

Here is a rather long (45 minutes or so) video on the Genelec 8050 that I reviewed recently. In the process, I do a 90% deep dive into what the measurements mean and how I read and use them. So it will be a decent tutorial as well on how to read proper speaker measurements and listening tests...

www.audiosciencereview.com

The old conventional wisdom used to be "spend 50% on speakers". My systems are more like 85-90% speakers now, much to my benefit.

 

[IAtaman]

you did but you also said ''On paper topping should have been better but it was not the case for me.''

Which is incorrect - 'on paper' they should be identical.

Or indistinguishable. When configured correctly that is - I guess you can always choose a weird setting or a filter on the Chord devices and make things sound different.

 

[Mart68]

Or indistinguishable. When configured correctly that is - I guess you can always choose a weird setting or a filter on the Chord devices and make things sound different.

True - I couldn't tell between filters but I forget about these youngsters with their as yet undamaged ears.

 

[Mart68]

The old conventional wisdom used to be "spend 50% on speakers". My systems are more like 85-90% speakers now, much to my benefit.

Back in the days of the old UK 'Flat Earth' cult we were told to keep the same £100 pair of dinky 2 way bookshelf speakers until we'd maxed out the turntable and amplification. And people actually did that.

 

[ahofer]

Back in the days of the old UK 'Flat Earth' cult we were told to keep the same £100 pair of dinky 2 way bookshelf speakers until we'd maxed out the turntable and amplification. And people actually did that.

At least there were significant differences between cartridges. Probably still are. and RIAA implementation. But we no longer have those problems of destroying and reconstructing signals at the front end.

 

[Blumlein 88]

I agree with all those arguments. I learned from experience a long time ago that is very easy to bias people if you tell them your impressions before they make their hearing. I have told people that have came to my home to hear stuff that I will not provide them my opinion because I want to hear from them and theirs and be able to compare with mines. I 100% support what you are saying.I haven't learned that from reading papers but from experience. I also think your opinion makes complete sense. You expressed it perfectly. Kudos for that.

First about biasing someone. You can say nothing, ask them to listen and tell you what you think. You know what is likely to happen? They'll hear very similar to you. How? Look up Clever Hans. We are influenced easily without anyone meaning to do so or even knowing they have. I realized over time when some of my friends had some new gear, I could tell what they were thinking about it by which recordings they decided to play. I didn't even intend to notice, but when I did I would know right away "what they were hearing", and realized I probably had done so for a long time.

However, as I said before, I mutated my position from being a 100% measurements and specs guy to a always hear what you buy before one. Because the "transparent" measurements from different products I have bought always, 100% of the time, I have been able to spot sound signatures, defects and virtudes. I have learned that all audio products have audible virtudes and defects no matter how perfectly their measurements or specs are.

I have also disagree with many products reviews as my sound impressions have been very different and my measurements where also different from the ones published. My best guess from this phenomena Is that many manufacturers send golden samples for review or some magazines gave positive reviews in exchange for paid ads (I know It is not the case of ASR)

Respectfully, you almost surely didn't hear actual sound differences from differences in sound. You haven't level matched, you knew what you were listening to etc. It is quite common for people to have the idea cheaper gear is less smooth or has bit less soundstage etc. etc. You get the same from reading reviews as well as the Clever Hans influence. The idea about manufacturers sending golden samples has little merit. I don't know what measurements you were taking that differed. A number of people here if you look around may not have an AP device like Amir, but other devices are not far from it and they get measurements that match quite closely with different samples of the same gear. You'll also find the basic measures by Stereophile are generally in close agreement with Amir's.

This sort of thing is well established. Many of us have been there. It seems too real and unbelievable that what you are hearing is not different. Yet a 1/4 of a db will sound perfectly level matched and will reliably cause you to hear better quality from the louder device. It does not sound louder, it sounds deeper, warmer, more detailed. You'll think it is the better sounding device.

Have you listened to the tracks I made available that were 8th generation copies? If every device has a sound magnifying it 8x should make it obvious. Try that and see what you think.

 

[bodhi]

Where is the difference here? Only the source of the cues is different, not the outcome. Imagination is largely deliberate, illusions are not.

If you don't know what you should hear, only that is should be "better" because of more expensive or impressive looking piece of equipment, you will be hard pressed to describe how the thing is different. The descriptions tend to differ, but include things that are hard to disagree such as "more air", "realistic sound" or "more accurate soundstage".

But when you have already been given detailed descriptions of how the device should sound and already believe you should hear exactly that, then I would call what's happening "imagining". The descriptions are more specific, such as amp being "brighter, more detailed" or power cable tightening the bass.

 

[Blumlein 88]

When I had the Topping DX7s I also had a Chord QTest.

They had small but audible differences. What I could spot was a colder sound signature on the Topping and less sound stage. On the Chord I liked their rhythm precision and their detailed and richer textures. I also had (still have) the Arcam CDS50 (SACD/DAC) the Arcam was also definitely better than the Topping. It had a more balanced sound signature but wasn't at rich or rhythmic as the Chord. It was a very small difference in favor for the Chord vs the Arcam but as the Arcam was more feature rich I keep it and sold the Topping and the QTest.

I have to say that I love that Arcam player. With SACDs sounds wonderful. I have recommended it to many people here and all that bought them loved it's sound too.

Let us talk about soundstage. Do you know how much channel separation is needed for us to hear maximum soundstage over speakers? 25 db is enough. Yes, 25 db is all it takes to hear all the stereo imaging and soundstaging that is possible. I don't know that I've ever seen a DAC that wasn't at least triple that much. You still seem to have 100% faith in sighted uncontrolled listening comparisons. I get it. I also now know it is a much misplaced faith.

 

[Blumlein 88]

If you don't know what you should hear, only that is should be "better" because of more expensive or impressive looking piece of equipment, you will be hard pressed to describe how the thing is different. The descriptions tend to differ, but include things that are hard to disagree such as "more air", "realistic sound" or "more accurate soundstage".

But when you have already been given detailed descriptions of how the device should sound and already believe you should hear exactly that, then I would call what's happening "imagining". The descriptions are more specific, such as amp being "brighter, more detailed" or power cable tightening the bass.

If you are so certain you are hearing reality, get someone let you do so blinded. The same differences will be there and you'll hear them. No peeking. Try the 8th generation test I've mentioned, you don't know which is which, but you should be able to hear the differences. Air and soundstage, very common results of a level mismatch.

 

[Blumlein 88]

I said they didn’t sound different.

Yes, doesn't that surprise you that such an inexpensive device sounds like a more expensive one? Your take seemed to be you expected the Topping to sound better from measurements. The measurements of both of those devices is good enough one would expect they sound the same. Not that the Topping would sound better.

 

[voodooless]

If you don't know what you should hear, only that is should be "better" because of more expensive or impressive looking piece of equipment, you will be hard pressed to describe how the thing is different. The descriptions tend to differ, but include things that are hard to disagree such as "more air", "realistic sound" or "more accurate soundstage".

But when you have already been given detailed descriptions of how the device should sound and already believe you should hear exactly that, then I would call what's happening "imagining". The descriptions are more specific, such as amp being "brighter, more detailed" or power cable tightening the bass.

Fair point! Well said.

 

[bodhi]

Fair point! Well said.

BTW I don't think there is a study to prove that theory, it's just something I have noticed during the years. For example I remember local hifi magazine doing tests where they had several reviewers review the same device. I had to give it to the, they didn't coordinate their reviews, as they didn't hear exactly the same things. But rarely did they say anything that could be considered contradicting another reviewer. But most of the time you couldn't really tell they had been listening to the same device, maybe they just appreciated different aspects of sound. I'm still almost 100% that the dudes were honest and didn't even have much monetary motivation to make things up ( this was LONG time ago).

 

[voodooless]

I had to give it to the, they didn't coordinate their reviews, as they didn't hear exactly the same things. But rarely did they say anything that could be considered contradicting another reviewer.

But they saw the thing, read the marketing materials, heard the rep talk about the product, maybe read other reviews. There is always bias at play.