Master Thread: Are measurements Everything or Nothing?

[Sal1950]

Has stereophile been here on this thread to defend itself? It would be nice to read their side of the "STORY"

Not on this particular thread.
But yes, John Atkinson is a member and posts on rare occasions.
Kal Rubinson also stops in much more often.
I'm sure they have others that follow our discussions and generally know very well what is said here.
They chose not to become involved in the debates for the same reason they chose to ignore the value of DBT and the rest.
It would endanger their reader and advertiser base.

 

[MattHooper]

Imagine if Stereophile wrote every week that one dac sounded exactly the same as every other.

Surprisingly, it occasionally happens.
Two DACs compared:

Benchmark DAC3 HGC D/A preamplifier-headphone amplifier Page 2

www.stereophile.com

“I spent an afternoon of focused listening, then part of the evening with my 19-year-old son, a serious musician, listening to music he's exploring: Led Zeppelin, Eminem, Schoenberg, J.S. Bach. We sat back to listen (he was in the good chair), switching between the two DACs, sometimes quickly, sometimes slowly.

Neither of us heard a difference—nothing. Soundstage depth? Tonality? Reverb tails? No difference. Did the DAC3 sound very slightly cleaner? Perhaps. Did it have a little more energy in the very lowest bass? Maybe. Was the DSD-based DirectStream a touch smoother? I thought it would be, but didn't hear it.

The next day, a friend and fellow audio reviewer stopped by for a listen. I checked to make sure the volume levels were matched, then handed him the remote. He did the same demo I'd done, listening mainly to percussion (he's a drummer), switching at will. His conclusion was the same as mine: No difference.”

 

[NTK]

Surprisingly, it occasionally happens.
Two benchmark DACs compared:

Benchmark DAC3 HGC D/A preamplifier-headphone amplifier Page 2

www.stereophile.com

“I spent an afternoon of focused listening, then part of the evening with my 19-year-old son, a serious musician, listening to music he's exploring: Led Zeppelin, Eminem, Schoenberg, J.S. Bach. We sat back to listen (he was in the good chair), switching between the two DACs, sometimes quickly, sometimes slowly.

Neither of us heard a difference—nothing. Soundstage depth? Tonality? Reverb tails? No difference. Did the DAC3 sound very slightly cleaner? Perhaps. Did it have a little more energy in the very lowest bass? Maybe. Was the DSD-based DirectStream a touch smoother? I thought it would be, but didn't hear it.

The next day, a friend and fellow audio reviewer stopped by for a listen. I checked to make sure the volume levels were matched, then handed him the remote. He did the same demo I'd done, listening mainly to percussion (he's a drummer), switching at will. His conclusion was the same as mine: No difference.”

No. For the Benchmark DAC1 and DAC3, the difference "wasn't subtle".

He only said he (and his son, and a friend) could not hear any difference between the DAC3 and the PS Audio DirectStream.

 

[MattHooper]

No. For the Benchmark DAC1 and DAC3, the difference "wasn't subtle".

View attachment 404965

He only said he (and his son, and a friend) could not hear any difference between the DAC3 and the PS Audio DirectStream.

Oh right, in skimming, I’d read it as being the reviewer “heard” differences between the DAC1 and DAC2, but when he finally bothered to level match, then the differences disappeared . I missed that they had switched to the PS audio DAC and level matched for those DACs.

In any case, same point. A rare case of the reviewer level matching for comparison and not “hearing” a difference between two DACs.

 

[Sal1950]

He only said he (and his son, and a friend) could not hear any difference between the DAC3 and the PS Audio DirectStream.

I'm not sure if that version of the DirectStream and the one we tested are the same but still my thoughts remain.

Review and Measurements of PS Audio PerfectWave DirectStream DAC

This is a review and detailed measurements of the PS Audio PerfectWave DirectStream DAC. It is on kind loan from a member. The unit that I have has the network "bridge" streaming option and costs US $6,899. Without that option it costs US $5,999. The DS DAC is quite heft with a textured matt...

www.audiosciencereview.com

Our test of the DirectStream revealed an extremely poorly measuring DAC and one which Amir was able to hear issues with.
He pointed to the transformer as the main reason for the audible failures.
That said, I wonder why the Golden Ears at Stereophile didn't hear anything against a SOTA Benchmark. HUMMM

 

[sigbergaudio]

I'm not sure if it's the NFS but I remembered reading about using a Klippel.

Post in thread 'Sigberg Audio Manta (12" wideband cardioid active speakers) development thread' https://audiosciencereview.com/foru...peakers-development-thread.28255/post-1067079

We use the anechoic chamber at Seas, which has Klippel software. So they do not have an NFS rig, but an actual and pretty large anechoic chamber that is accurate down to around 50hz. They have a motorized pidestal/thing so you can change the angle and do multiple measurements from the measurement console. The chamber was compared to NFS measurements by Ascend and got similar results as their NFS. So in essence the same accuracy, just not automated. And a bit more problematic to do vertical measurements as then the speaker has to be on its side (difficult with tower speakers for instance).

We also have a semi-anechoic chamber in our workshop that works well enough for the gruntwork (relatively accurate above 100hz, and I can calibrate against Seas since I've measured the same speaker both places). Then I travel to Seas (20 minutes away) to verify and fine tune now and then.

 

[Newman]

[to Matt] Your always happy to try and defend nearly any BS subjective crap that comes up.
Don't you ever tire of riding that fence?
I'd think by now it would wear sores in your crack.

A good debating team captain never tires of his or her mission, and indeed draws greatest pleasure from being given the incorrect side of an assertion to argue, and using every trick of the trade to 'win at all costs', whether it be via smoking keyboards, verbosity, eloquence, indefatigability, insinuation, personableness and amicability, vast repetition...the true debating champion is admirable in a way, but a genuine menace when unleashed into discussions of fact, especially science.

I think it's not right for some here on ASR to call it constructive dialog. Constructive dialog on matters of scientific inquiry requires participants in said dialog to be actual content experts, or if not, to refrain from personal opinion while presenting materials from actual content experts. The first sign of debating tactics, laced with personal anecdotes, ....well, it's a red flag to the pursuit of truth.

cheers

 

[Newman]

Not for me they haven't. A review is nothing more than someone else's opinion. You get to decide how much influence it has. I generally go for what I like and don't worry about others opinions. Just do what you like and enjoy it. It's your system after all so it should make you happy first and foremost.

Rob

Actually, the bit I made bold is much more complex than you think. You don't really get to decide, you just think you do. Illusions of conscious control.

 

[dshreter]

Very little is needed. Audio is a solved technology.
The numbers will tell most all.

The numbers DONT tell it all. They might contain the information necessary for interpretation but that interpretation is still required. And the best way to communicate interpretation is with words.

Even for people that well understand how to read the measurements (which I would presume is also rare), fewer still have experienced actually hearing a wide range of speakers with available measurements.

So take a top of the line Revel and top of the line Genelec and ask “what’s the difference in how these sound?” To which you could answer “it’s all right there in the measurements!”

Well, they both measure flat. They both measure smoothly on and off axis. They both have full range frequency response. But they don’t sound the same.

I’m not saying the difference lies beyond the measurements. It’s all right there. But what that actually sounds like is best communicated in the language of the senses because listening is a sensory experience.

 

[antcollinet]

The numbers DONT tell it all. They might contain the information necessary for interpretation but that interpretation is still required. And the best way to communicate interpretation is with words.

Even for people that well understand how to read the measurements (which I would presume is also rare), fewer still have experienced actually hearing a wide range of speakers with available measurements.

So take a top of the line Revel and top of the line Genelec and ask “what’s the difference in how these sound?” To which you could answer “it’s all right there in the measurements!”

Well, they both measure flat. They both measure smoothly on and off axis. They both have full range frequency response. But they don’t sound the same.

I’m not saying the difference lies beyond the measurements. It’s all right there. But what that actually sounds like is best communicated in the language of the senses because listening is a sensory experience.

Did you see the words "very little" and "most" in the post you quoted from?

 

[dshreter]

Did you see the words "very little" and "most" in the post you quoted from?

Apologies. The measurements don’t tell MOST ALL. I stand by everything else I wrote.

 

[sigbergaudio]

We're not quite there yet with the measurements, and I (as a manufacturer) think this is kind of a good thing. In some ways it would be too easy if that were the case.

I can listen to speakers that measure extremely good, but they still don't necessarily sound perfectly right to me. And during development I can measure speakers, but I can't know if they sound right until I listen. Often they sound better and more real if one does not try to beat the response into perfect submission.

If I look at the measurements, I can have some assumptions (often right) about how the speakers sound.
If I listen to speakers, I can have some assumptions (often right) about how the speakers measure.

The most elusive factors (at least for me) are things like soundstage and imaging. It appears to be a combination of many things that makes them work. And relatively small adjustments can make them come undone.

 

[Axo1989]

Apologies. The measurements don’t tell MOST ALL. I stand by everything else I wrote.

You can tell all of the numbers most of the time. And most of the numbers all of the time. But you can't tell most all of the numbers almost of the time?

 

[MattHooper]

A good debating team captain never tires of his or her mission, and indeed draws greatest pleasure from being given the incorrect side of an assertion to argue, and using every trick of the trade to 'win at all costs', whether it be via smoking keyboards, verbosity, eloquence, indefatigability, insinuation, personableness and amicability, vast repetition...the true debating champion is admirable in a way, but a genuine menace when unleashed into discussions of fact, especially science.

Deep breaths Newman. You’re going to have to put up with a Genuine Menace for quite a while

Constructive dialog on matters of scientific inquiry requires participants in said dialog to be actual content experts, or if not, to refrain from personal opinion while presenting materials from actual content experts.

How would this work precisely? You have posted all sorts of references to scientific literature including, of course, lots of Toole’s work, and you have included your own interpretations and opinions.

Can you tell us specifically what direct scientific expertise you have in psychoacoustics, etc., in order to comment?
Can you put this in your signature so we know you are a proper expert? If you are an expert in the relevant areas, is there a reason you don’t have an “ expert” badge here?


Your proposal that folks like me can present expert comment but refrain from comment or interpretation seems a tad convenient, coming shortly after I have Shown several times your own interpretations of Toole to be incorrect.

Here is how discussion or debate actually works: if someone writes something unreasonable or factually incorrect, the intellectually honest response is to rebut what they have said with a better argument or correcting the facts. That’s how everybody learns, including onlookers.

The wrong way to engage (bad faith) is to reply with what amounts to “ shut up, he’s an expert and you are not” (“and by the way, my own interpretation of that experts work is correct, and you have no standing to disagree with my own interpretation”).

That is a type of fallacious argument from authority that you seem unfortunately prone to.

By all means, if I write something unreasonable or factually incorrect, then address those points, rather than do a fallacious appeal to authority.

If you’d like, Here’s another chance because…horrors!…Here goes the great menace again, daring to Discuss a quote from an expert..

 

[goat76]

The most elusive factors (at least for me) are things like soundstage and imaging. It appears to be a combination of many things that makes them work. And relatively small adjustments can make them come undone.

Maybe you just need to make some adjustments to the positioning of the speakers, even if the changes you made seemed relatively small. Just a thought.

 

[Axo1989]

Deep breaths Newman. You’re going to have to put up with a Genuine Menace for quite a while

How would this work precisely? You have posted all sorts of references to scientific literature including, of course, lots of Toole’s work, and you have included your own interpretations and opinions.

Can you tell us specifically what direct scientific expertise you have in psychoacoustics, etc., in order to comment?
Can you put this in your signature so we know you are a proper expert? If you are an expert in the relevant areas, is there a reason you don’t have an “ expert” badge here?

Your proposal that folks like me can present expert comment but refrain from comment or interpretation seems a tad convenient, coming shortly after I have Shown several times your own interpretations of Toole to be incorrect.

Here is how discussion or debate actually works: if someone writes something unreasonable or factually incorrect, the intellectually honest response is to rebut what they have said with a better argument or correcting the facts. That’s how everybody learns, including onlookers.

The wrong way to engage (bad faith) is to reply with what amounts to “ shut up, he’s an expert and you are not” (“and by the way, my own interpretation of that experts work is correct, and you have no standing to disagree with my own interpretation”).

That is a type of fallacious argument from authority that you seem unfortunately prone to.

By all means, if I write something unreasonable or factually incorrect, then address those points, rather than do a fallacious appeal to authority.

If you’d like, Here’s another chance because…horrors!…Here goes the great menace again, daring to Discuss a quote from an expert..

Genuine Menace is the name of my new hardcore band.

Also note: credentials fallacy it's a house favourite.

 

[sigbergaudio]

Maybe you just need to make some adjustments to the positioning of the speakers, even if the changes you made seemed relatively small. Just a thought.

Positioning is certainly one of the many factors that affect imaging and soundstage, yes. And yes, in combination with other things.

 

[MattHooper]

Genuine Menace is the name of my new hardcore band.

Cool. I think you’ll find my licensing fees to be reasonable.

Also note: credentials fallacy it's a house favourite.

Indeed. Tip of the iceberg, really…

 

[MattHooper]

Often they sound better and more real if one does not try to beat the response into perfect submission.

Interesting. I’ve heard some imperfectly measuring loudspeakers sound a bit more real to me in some aspects with some content.

We have quite a bit of data as to overall preferences in loudspeaker design over a range of different music.

It would’ve been neat to have plenty of good data where loudspeakers were rated in terms of realism (even more cool if somebody had managed to produce real versus live comparisons that weren’t problematic).

 

[sigbergaudio]

Interesting. I’ve heard some imperfectly measuring loudspeakers sound a bit more real to me in some aspects with some content.

We have quite a bit of data as to overall preferences in loudspeaker design over a range of different music.

It would’ve been neat to have plenty of good data where loudspeakers were rated in terms of realism (even more cool if somebody had managed to produce real versus live comparisons that weren’t problematic).

At the end of the day, the purpose of speakers is to be listened to by human beings. So a perfectly objective evaluation of the performance is practically impossible.

Each human being puts different weight on different qualities, has different ideas about which imperfections they will accept, etc. Most people also have a preference with regards to genre(s), despite claiming "I listen to everything". This too affects which qualities are more important, and less important.

That being said, we see a general consensus on many things. We also have some data on what measurements indicate a good speaker. At the same time, a number of the speakers that measure the best (and as such are presented as the best on this forum) does not necessarily evoke that "this sounds real" and "this feels right" experience. At least not to me.

Not a perfect world, and no access to perfect data. So we are not in a position to make a perfectly objective, perfectly logical choice. But that's not what we humans do anyway. And still, a lot of great speakers out there, which is good.