Master Thread: Are Measurements Everything or Nothing?

[TTT15]

Frequency response. Young people might with a lot of practice be able to hear that diffrence. Statistically your probably not young, so there is no way you'll every hear the difrence.

Yes discussing the estimated frequency response calculated between the samples. I’ve not heard a NOS DAC but presumably you are saying then there is no difference between a NOS DAC which does no interpolation, and R2R DAC an FPGA DAC, the different DAC chips and the different filters is the argument. Indeed questionable to extent audible other than in ideal listening conditions but the maths which is one of the few knowns we have says technically there is a difference.

 

[voodooless]

I’ve not heard a NOS DAC but presumably you are saying then there is no difference between a NOS DAC which does no interpolation, and R2R DAC an FPGA DAC, the different DAC chips and the different filters is the argument.

Most of these will probably sound the same if properly engineered. A NOS DAC at a base sample rate of 44.1 or 48 kHz can very well be audible. The frequency response droop is significant, and there are other side effects that might be audible as well. In 96 kHz and above, it’s probably as inaudible as all the others.

 

[TTT15]

That’s precisely why measurements are so useful, they allow you to discard poor designs, assuming you seek fidelity.
Amir measures the efficacy of the filters too, just purchase well designed equipment, again if your goal is fidelity.
Keith

Thanks yes that is the way I perceive the debate. I’m certainly taking more interest in products that measure well and less in those that don’t as a filter. How are the filter measurements called if I may ask.

 

[Adis]

Panthers.

 

[antcollinet]

How are the filter measurements called if I may ask.

They will typically look like this. Sometimes if the filter is selectable, all the filter responses will be plotted - in this case only the default filter is shown.

 

[solderdude]

I listen to music for enjoyment so if there is no link to measurements I’m not sure what the point of measurements are.

To check for technical performance (signal fidelity).
That is the only thing measurements can show.
Not if person A, B or Z likes what he hears.

still not sure how we measure and rank that approach compared to other filters.

By using white noise, or multitones or a sweep and measuring (and ultimately displaying) what the response is.
Then there is interpretation which requires knowledge.

Yes discussing the estimated frequency response calculated between the samples. I’ve not heard a NOS DAC but presumably you are saying then there is no difference between a NOS DAC which does no interpolation, and R2R DAC an FPGA DAC, the different DAC chips and the different filters is the argument. Indeed questionable to extent audible other than in ideal listening conditions but the maths which is one of the few knowns we have says technically there is a difference.

Technically there are differences between conversion methods, oversampling algorithms and filters.
They are very measurable and in some cases might even reach audible thresholds.
There are different solutions to every problem and then there are patents that need to be respected so other manufacturers need to make things in another way.
Ask 10 (audio)designers to make a circuit that has a certain function and you get 10 different designs using different components and all of those designs could meet the requirements.
That's why there are so many different devices. And then the manufacturers need to sell those devices and use everything in the book they are allowed to use (and sometimes go beyond that).
That's how the (audio) business works.

A well built R2R (with proper filtering) or used at fs > 88.4kHz is audibly indistinguishable from a good Delta Sigma implementation or FPGA or other conversion method.

The goal is to convert digital described values (numbers) to the corresponding voltages and to 'connect' the analog sample points seamlessly so a smooth waveform is the result.
Math and oversampling and/or an analog filter ensures this as long as it conforms (or is close) to the sampling theorem.

 

[MattHooper]

I certainly get your point, but I would add some caveats…

Hifi equipment has a job: to reproduce the recordings with as high fidelity (faithfulness, accuracy) as possible.

That is one view of the job of high end/hi fi equipment. There are other points of view, for instance high Fidelity to the experience of hearing live instruments in real acoustic space. (Hi Fidelity started off as a goal to re-create the sound of real voices and instruments).

So it’s possible for a system to be doing its job in this sense well, not necessarily in the sense you are advocating.

So in the case of hifi equipment, yes, better is objective and measurable.

Some people may not prefer better hifi equipment, of course, and there is nothing wrong with that at all, but the equipment they prefer is objectively worse at doing its job (reproducing recordings).

So you’ve suggested there’s a criteria by which you can decide which piece of equipment is objectively doing a worse or better job than another. Fair enough. But let’s look at a challenge to this.

Let’s take an example.

Let’s see you have a recording you want to reproduce: Leonard Bernstein conducting Gustav Mahler’s Symphony No. 5 with the Vienna Philharmonic Orchestra, recorded in 1987 for Deutsche Grammophon.

Now we want to reproduce that recording accurately.

So what is on that recording? It’s a recording of an entire Symphony Orchestra playing an extremely dynamic orchestral piece of music.
That’s what the recording represents.

Now let’s say you have a very nice room for playback acoustically, and you set up two different loudspeakers on which to play this recording:

The Genelec G Three active speaker:

The specs for this speaker are : 54Hz–20kHz, ±2.5dB. And they measure almost impeccably neutral.

Now here is the second loudspeaker:

Wilson Wamm Master Chronosonic loudspeaker:

Frequency Response: 20Hz-33kHz ±2dB

No measurements available for this loudspeaker, but Wilson speakers have often been slammed here for their design and lack neutral measurements.
Measurements for one of their large loudspeakers here
So let’s assume that these large speakers have some significant deviations from neutrality.

So, for the goal of reproducing our selected recording, here’s the question:

How would you determine which one of these loudspeakers is objectively worse at doing its job?

Even if that job is to re-create the information in the recorded signal?

If you went with strict neutrality as the criteria, it suggest you would choose that tiny little Genelec speaker.

But that Genelec speaker is missing plenty of the lower frequencies - frequencies in the recording - that the Wilson speaker is able to reproduce.

More pointedly, if the job of the loudspeaker is to faithfully reproduce the information represented in the recorded signal, again the information represents a bombastic performance by an entire Symphony Orchestra. So which speaker is going to be better (even if not perfect) at doing that job?

It seems the huge Wilson speaker, even if not fully neutral, would be much more adept at producing the sense of scale and power and dynamics of that orchestral recording than the tiny Genelec. (again we don’t have measurements though listeners have generally reported being in awe in terms of these attributes of those speakers - the ability in terms of sound staging size and imaging and power in creating the sense of being in front of large scale sound sources like symphonies).

(I suspect also that one can find very neutral stand mounted loudspeakers that may also go down to 20Hz, but which would still lose the fight with the Wilson speakers in terms of the impression of convincing scale and power hand dynamics).

So I’m curious how you (or anyone else who wants to chime in), decide which loudspeaker is doing the objectively better job of reproducing the recording.

It seems to me in such scenarios, and given the variety of speaker designs and recordings that could be a great many such comparisons, some level of arbitrariness is going to be involved even when invoking this “objective criteria.”

 

[RexrothPigeon]

There are other points of view, for instance high Fidelity to the experience of hearing live instruments in real acoustic space. (Hi Fidelity started off as a goal to re-create the sound of real voices and instruments).

Yeah, I know some old hifi guys talk this way, but it's nonsense: In some cases, that's the job of recording equipment; it's never the job of hifi equipment. The job of hi-fi equipment is to reproduce what has been recorded, full stop.

 

[RexrothPigeon]

No measurements available for this loudspeaker
[...]
How would you determine which one of these loudspeakers is objectively worse at doing its job?

I have no way to evaluate that, sorry.

If you went with strict neutrality as the criteria, it suggest you would choose that tiny little Genelec speaker.

Certainly not. I don't know what the Wilson does (since there are no measurements available), so it would be unreasonable to make that assessment.

 

[MattHooper]

Yeah, I know some old hifi guys talk this way, but it's nonsense:

What do you think the origin of sound reproduction - especially high-quality high fidelity sound reproduction- was about?

It was about, generally speaking, taking a sound source - a voice or an instrument or many instruments - and reproducing that sound.

Grok: The term “high fidelity” (or hi-fi) which gained traction in the 1930s and 1940s, explicitly referred to the pursuit of reproducing the original acoustic event—whether a voice, instrument, or ensemble—with minimal distortion or alteration.

Many of the advances in sound quality were spurred by this very approach - you wanted to be able to accurately reproduce voices and instrumental timber, the more challenging the acoustic object, e.g. ensembles or even orchestras, required advances in wider frequency response, loudness, dynamic capability etc. It was the characteristics of life sound itself, which people were trying to reproduce, that pushed all these envelopes and got us closer to be able to reproduce the information captured about those live events.

Are you sure you really want to wave away all of this as nonsense? (while having benefitted from that history?)

It’s clearly not nonsense that the reproduction of a Sonic event, a voice or an instrument, can be closer or further from the real thing.
Why wouldn’t that be an interesting and relevant concept even today? (It is!)

In some cases, that's the job of recording equipment; it's never the job of hifi equipment.

if certain equipment re-creates the sensation of the live event, certainly it can be the job of the hi-fi equipment as well to do so.

I have no way to evaluate that, sorry.

Which was my point

In the case I gave - and there could be countless other examples - you wouldn’t be able to decide non-arbitrarily which speaker is “ doing the job of most accurately reproducing the recording.”

And the point also was that it it’s worth accounting for what the recording itself represents, which is an account of a live orchestra playing, and so it may be that the loudspeaker that gets closer to a convincing presentation of the orchestra playing that might be doing his job better, which isn’t decided by strict frequency response neutrality.

 

[RexrothPigeon]

What do you think the origin of sound reproduction - especially high-quality high fidelity sound reproduction- was about?

It was about, generally speaking, taking a sound source - a voice or an instrument or many instruments - and reproducing that sound.

I don't think that bears any relevance. The original meaning of watches was so monks could know when to pray, after all.

 

[RexrothPigeon]

In the case I gave - and there could be countless other examples - you wouldn’t be able to decide non-arbitrarily which speaker is “ doing the job of most accurately reproducing the recording.”

You would be able to if you had the measurements, yes. Without the measurements, it's not possible.

 

[MattHooper]

I don't think that bears any relevance. The original meaning of watches was so monks could know when to pray, after all.

That doesn’t seem to address what I said.
And if watches were invented to tell time, which is what the monks wanted to know, then of course that is still relevant for watches today.

Likewise, with audio equipment. But I just be repeating what I already wrote on this.

You would be able to if you had the measurements, yes. Without the measurements, it's not possible.

I gave you the measurements for the Genelec. For the purposes of the question, you can assume the Wilson measures like the measurements I provided for another large Wilson speaker.

So assuming those measurements for each speaker, which one is doing its job objectively better reproducing the recording, and why?

 

[RexrothPigeon]

Likewise, with audio equipment. But I just be repeating what I already wrote on this.

And it will remain irrelevant. Hifi equipment reproduces recorded sound and is agnostic to what is on those recordings (orchestras? sure. Rock bands? sure. Electronic synthesis? sure. Harsh noise walls? sure. Sine waves? why the hell not). That's its job, unequivocally.

While personal preference is of course much more important to you or me or any other individual listener, it is subjective sensation and thus variable, capricious, and non-transferrable to others except in metaphors, since writing (or talking) about music is like dancing about architecture. And that's how hifi discourse is undertaken virtually everywhere but ASR.

 

[MattHooper]

And it will remain irrelevant. Hifi equipment reproduces recorded sound and is agnostic to what is on those recordings (orchestras? sure. Rock bands? sure. Electronic synthesis? sure. Harsh noise walls? sure. Sine waves? why the hell not). That's its job, unequivocally.

Well, that’s simply a reassertion of the very claim under debate, without actually addressing the counter argument.

While personal preference is of course much more important to you or me or any other individual listener, it is subjective sensation and thus variable, capricious, and non-transferrable to others except in metaphors,

That statement seems to overreach what the science says so far. See the work of Floyd Toole on how studying group preferences allowed for a high level of predictability for individual and group preferences.

since writing (or talking) about music is like dancing about architecture.

I disagree, but that will be more of a distraction to get into.

I guess you’re not gonna answer the question I posed that was derived from your claim that we can tell objectively which gear is doing its job better with a recording. So I’m throwing in the towel on this conversation.

I’d be interested in what anybody else has to say to the question I posed in evaluating the Genelec versus Wilson.

 

[TTT15]

To check for technical performance (signal fidelity).
That is the only thing measurements can show.
Not if person A, B or Z likes what he hears.


By using white noise, or multitones or a sweep and measuring (and ultimately displaying) what the response is.
Then there is interpretation which requires knowledge.


Technically there are differences between conversion methods, oversampling algorithms and filters.
They are very measurable and in some cases might even reach audible thresholds.
There are different solutions to every problem and then there are patents that need to be respected so other manufacturers need to make things in another way.
Ask 10 (audio)designers to make a circuit that has a certain function and you get 10 different designs using different components and all of those designs could meet the requirements.
That's why there are so many different devices. And then the manufacturers need to sell those devices and use everything in the book they are allowed to use (and sometimes go beyond that).
That's how the (audio) business works.

A well built R2R (with proper filtering) or used at fs > 88.4kHz is audibly indistinguishable from a good Delta Sigma implementation or FPGA or other conversion method.

The goal is to convert digital described values (numbers) to the corresponding voltages and to 'connect' the analog sample points seamlessly so a smooth waveform is the result.
Math and oversampling and/or an analog filter ensures this as long as it conforms (or is close) to the sampling theorem.

Most recordings including new releases are still at 44/48 kHz though therefore the DAC technology in theory could make a difference. I agree in principle that if the recording was at a higher sample rate with a well made DAC designed to operate at that sample rate then any potential marginal differences should disappear. Any differences are only in ideal listening conditions according to studies and they may well disappear as the trend to hi res increases and chip processing capabilities increases driven by AI.

 

[voodooless]

How would you determine which one of these loudspeakers is objectively worse at doing its job?

Then you first have to objectively define “better”.

Maybe the Wilson has so much overpowering bass, that it still sounds like shit. Yes it can do more frequencies, but is it any good at it?

I think the preference score does take bass extension into account, so if it’s accurate enough, it should also work in this case. I highly doubt one parameter is the defining factor here.

Never mind other factors. What If I sit 1m in front of the speakers? Is the Wilson still a good idea? What if my wife divorces me when I try to put them in my living room?

 

[TTT15]

I certainly get your point, but I would add some caveats…


That is one view of the job of high end/hi fi equipment. There are other points of view, for instance high Fidelity to the experience of hearing live instruments in real acoustic space. (Hi Fidelity started off as a goal to re-create the sound of real voices and instruments).

So it’s possible for a system to be doing its job in this sense well, not necessarily in the sense you are advocating.



So you’ve suggested there’s a criteria by which you can decide which piece of equipment is objectively doing a worse or better job than another. Fair enough. But let’s look at a challenge to this.

Let’s take an example.

Let’s see you have a recording you want to reproduce: Leonard Bernstein conducting Gustav Mahler’s Symphony No. 5 with the Vienna Philharmonic Orchestra, recorded in 1987 for Deutsche Grammophon.

Now we want to reproduce that recording accurately.

So what is on that recording? It’s a recording of an entire Symphony Orchestra playing an extremely dynamic orchestral piece of music.
That’s what the recording represents.

Now let’s say you have a very nice room for playback acoustically, and you set up two different loudspeakers on which to play this recording:

The Genelec G Three active speaker:

View attachment 484550

The specs for this speaker are : 54Hz–20kHz, ±2.5dB. And they measure almost impeccably neutral.

Now here is the second loudspeaker:

Wilson Wamm Master Chronosonic loudspeaker:

View attachment 484554

View attachment 484555

View attachment 484556

Frequency Response: 20Hz-33kHz ±2dB

No measurements available for this loudspeaker, but Wilson speakers have often been slammed here for their design and lack neutral measurements.
Measurements for one of their large loudspeakers here
So let’s assume that these large speakers have some significant deviations from neutrality.

So, for the goal of reproducing our selected recording, here’s the question:

How would you determine which one of these loudspeakers is objectively worse at doing its job?

Even if that job is to re-create the information in the recorded signal?

If you went with strict neutrality as the criteria, it suggest you would choose that tiny little Genelec speaker.

But that Genelec speaker is missing plenty of the lower frequencies - frequencies in the recording - that the Wilson speaker is able to reproduce.

More pointedly, if the job of the loudspeaker is to faithfully reproduce the information represented in the recorded signal, again the information represents a bombastic performance by an entire Symphony Orchestra. So which speaker is going to be better (even if not perfect) at doing that job?

It seems the huge Wilson speaker, even if not fully neutral, would be much more adept at producing the sense of scale and power and dynamics of that orchestral recording than the tiny Genelec. (again we don’t have measurements though listeners have generally reported being in awe in terms of these attributes of those speakers - the ability in terms of sound staging size and imaging and power in creating the sense of being in front of large scale sound sources like symphonies).

(I suspect also that one can find very neutral stand mounted loudspeakers that may also go down to 20Hz, but which would still lose the fight with the Wilson speakers in terms of the impression of convincing scale and power hand dynamics).

So I’m curious how you (or anyone else who wants to chime in), decide which loudspeaker is doing the objectively better job of reproducing the recording.

It seems to me in such scenarios, and given the variety of speaker designs and recordings that could be a great many such comparisons, some level of arbitrariness is going to be involved even when invoking this “objective criteria.”

Quite how do you weight different measurements and what do you do if measurements as is the case for many products are not available. Having heard the Wilson speakers I suspect money no object that most people would go for the Wilsons.

 

[solderdude]

Most recordings including new releases are still at 44/48 kHz though therefore the DAC technology in theory could make a difference.

Only when 'poorer' filters are used. You know the 'slow' kind and the 'NOS' type.
Funny thing is that DS DACs with the NOS filter still oversample many times but emulate stairsteps.
These filter options are only available because people 'ask' for it.
The chip manufacturers also put in other filters because people ask for them.
Usually there is also at least one 'proper' filter in there.
One of the fast linear phase ones.

When selecting that one you will know the treble response is good and there will be no mirroring in the audible band.
Besides.... when you don't know or suspect the manufacturer selected the 'wrong' filter on purpose (out of 'theories' they have or to offer 'something different') you can always do software upsampling to say 88.2 or 96kHz and the filters don't matter as the upsampler has a good filter (mostly).

Any differences are only in ideal listening conditions according to studies and they may well disappear as the trend to hi res increases and chip processing capabilities increases driven by AI.

Recording studios all record at 96kHz or higher and all 44.1/48kHz releases are down sampled.
That is not the issue however.

With recording quality I mean producers f'ing up by participating in the loudness wars (still happens) or the mixing guy or recording engineer making (deliberate or not) 'mistakes'. Or the studio is f'ing up the tonal balance somehow... (circle of confusion).

 

[Mart68]

And the point also was that it it’s worth accounting for what the recording itself represents, which is an account of a live orchestra playing, and so it may be that the loudspeaker that gets closer to a convincing presentation of the orchestra playing that might be doing his job better, which isn’t decided by strict frequency response neutrality.

How is it decided though? What criteria can be used for an objective assessment of which speaker is closer to 'real voices and instruments'?

Fidelity to signal is objective, fidelity to someone's idea of 'real' can only ever be subjective.

I take your point about 'scale' or lack of it with the symphony orchestra but I'm not aware that scale is ever a criteria for the 'real voices and instruments' camp. Many of the advocates of using that as a criteria for 'goodness' use small 2 ways like Audionotes or the LS3/5a. Plus it's often used to justify the use of speakers that measure poorly.