Are measurements really telling the whole story?

[JoeKickass]

I've been into audio for a little while now, and it's been a fascinating journey. While I love music, I discovered early on that my critical listening ability is mediocre at best. I might be able to pick one device over another, but it's pretty much at a subconscious level. To me, reading eloquent reviews of audio products was similar to reading reviews about wine, lots of poetry and romance but not much substance I could actually relate to in my own experience.

I am an electrical engineer so I love quantifiable measurements and comparison, they just make life easier! For a long time I just bought what had the best specs I could afford, until I tried a multibit dac by schiit and discovered that somehow something that measured worse and had worse specs, actually sounded better to me... even with my below-average ears!

In science and physics getting unexpected results is actually the most exciting thing because it means there is something new that you still need to understand. I wonder if that's what is happening right now in the audiophile world. We have measurements like THD and freq. response, etc. but they are actually incredibly basic static tests compared to the complexity and dynamics of a music signal...

I think the ultimate quantifiable test would be to feed in a piece of reference music, and then compare and analyze the resulting output waveform with the reference. With the great advancements in CPU power I am hopeful to see that within my lifetime. I suspect until we get to that point where we can actually measure music, there will always be disagreement between the measurements and our own ears.

Here is a good interview that explains it better than me:

I love that there is an audio forum dedicated to the measurements instead of subjective reviews, but I'm curious how many people here really feel that the current measurement test suites are giving a complete picture?

 

[q3cpma]

Try to take imagination out of the equation before trying to solve it.

 

[solderdude]

I suspect until we get to that point where we can actually measure music, there will always be disagreement between the measurements and our own ears.

That point has been reached decades ago. It's called nulling and basically is present in all circuits using feedback.
Also there are nulling test circuits and you can even do that in the digital domain.
You use music as a test signal with actual loads.
Nulling too has its issues.
Then there are well performed blind tests. These are difficult because human brains and acoustics are involved and must be done on a proper technical basis.
Further more you need to differentiate between electrical measurements and acoustical measurements. Acoustical (and above all interpretation related to hearing) certainly do not tell the whole story and you cannot null this either.

Have a nice holiday period.... after that look into nulling for electronics.

 

[SIY]

I think the ultimate quantifiable test would be to feed in a piece of reference music, and then compare and analyze the resulting output waveform with the reference. With the great advancements in CPU power I am hopeful to see that within my lifetime.

If you're alive today, your prediction is correct. See @pkane 's superb DeltaWave software.

 

[pozz]

I've been into audio for a little while now, and it's been a fascinating journey. While I love music, I discovered early on that my critical listening ability is mediocre at best. I might be able to pick one device over another, but it's pretty much at a subconscious level. To me, reading eloquent reviews of audio products was similar to reading reviews about wine, lots of poetry and romance but not much substance I could actually relate to in my own experience.

I am an electrical engineer so I love quantifiable measurements and comparison, they just make life easier! For a long time I just bought what had the best specs I could afford, until I tried a multibit dac by schiit and discovered that somehow something that measured worse and had worse specs, actually sounded better to me... even with my below-average ears!

In science and physics getting unexpected results is actually the most exciting thing because it means there is something new that you still need to understand. I wonder if that's what is happening right now in the audiophile world. We have measurements like THD and freq. response, etc. but they are actually incredibly basic static tests compared to the complexity and dynamics of a music signal...

I think the ultimate quantifiable test would be to feed in a piece of reference music, and then compare and analyze the resulting output waveform with the reference. With the great advancements in CPU power I am hopeful to see that within my lifetime. I suspect until we get to that point where we can actually measure music, there will always be disagreement between the measurements and our own ears.

Here is a good interview that explains it better than me:

I love that there is an audio forum dedicated to the measurements instead of subjective reviews, but I'm curious how many people here really feel that the current measurement test suites are giving a complete picture?

The measurements give a full picture of the engineering.

The perceptual aspect requires a lot of discussion but the combination of psychoacoustics and device performance data gives firm answers.

 

[Zensō]

For a long time I just bought what had the best specs I could afford, until I tried a multibit dac by schiit and discovered that somehow something that measured worse and had worse specs, actually sounded better to me... even with my below-average ears!

As humans, it’s so easy to underestimate the power of our minds to color our perceptions based upon preconceived notions. We’re all incredibly susceptible to it. In the case of the Schiit DAC, were you aware of the “multibit story” before listening? Schiit’s clever, and sometimes disarming and subtle marketing is quite effective (Jason Stoddard was the owner of a very successful marketing firm before starting Schiit). This could have easily colored your expectations and subsequently your experience with the DAC.

The only way to know if you truly heard a difference would be to conduct a blind, volume matched, A/B comparison with other decent measuring DACs. Unless the Schiit measures poorly enough to have audible distortion (this is a real possibility with some multibit DACs), it’s unlikely you’d be able to pick the Schiit from among the others.

 

[MakeMineVinyl]

No

 

[Duke]

In science and physics getting unexpected results is actually the most exciting thing because it means there is something new that you still need to understand.

Totally agree. Unexpected results = invitation to discover something and possibe window of opportunity for improvement.

 

[Robin L]

More to the point, are subjective reviews actually useful or are they a form of gaslighting?

 

[raistlin65]

More to the point, are subjective reviews actually useful or are they a form of gaslighting?

I was thinking more like propaganda

 

[Maxicut]

I've been into audio for a little while now, and it's been a fascinating journey. While I love music, I discovered early on that my critical listening ability is mediocre at best. I might be able to pick one device over another, but it's pretty much at a subconscious level. To me, reading eloquent reviews of audio products was similar to reading reviews about wine, lots of poetry and romance but not much substance I could actually relate to in my own experience.

I am an electrical engineer so I love quantifiable measurements and comparison, they just make life easier! For a long time I just bought what had the best specs I could afford, until I tried a multibit dac by schiit and discovered that somehow something that measured worse and had worse specs, actually sounded better to me... even with my below-average ears!

In science and physics getting unexpected results is actually the most exciting thing because it means there is something new that you still need to understand. I wonder if that's what is happening right now in the audiophile world. We have measurements like THD and freq. response, etc. but they are actually incredibly basic static tests compared to the complexity and dynamics of a music signal...

I think the ultimate quantifiable test would be to feed in a piece of reference music, and then compare and analyze the resulting output waveform with the reference. With the great advancements in CPU power I am hopeful to see that within my lifetime. I suspect until we get to that point where we can actually measure music, there will always be disagreement between the measurements and our own ears.

Here is a good interview that explains it better than me:

I love that there is an audio forum dedicated to the measurements instead of subjective reviews, but I'm curious how many people here really feel that the current measurement test suites are giving a complete picture?

You are correct in your thinking. Measurements are a very good guide, but fall way short of the whole picture... which is why amirm gives his "personnel" subjective opinion with his speaker, monitor & headphone reviews.... even though they all "mostly" test well &/or above our hearing threshold, which is not very science-y. I would suggest that you keep to the "science", or your stay here won't be very enjoyable, as mine hasn't.

 

[ZolaIII]

We are all unique individuals that's what makes us subjects and that's perfectly fine as long as we have the sense of reality. When we lose it we use measurement as a compass to guide us back, of course neither doesn't paint the whole picture that would be pointless.
The real question is can someone be totally objective? My answer is it can't, it can believe that he is and tend to be which is still the best path to take.

Happy holidays everyone.

 

[raif71]

You are correct in your thinking. Measurements are a very good guide, but fall way short of the whole picture... which is why amirm gives his "personnel" subjective opinion with his speaker, monitor & headphone reviews.... even though they all "mostly" test well &/or above our hearing threshold, which is not very science-y. I would suggest that you keep to the "science", or your stay here won't be very enjoyable, as mine hasn't.

I mostly agree with you on measurement is a guide (however not the whole picture) but not being science-y can be enjoyable even on this site. Yes, once in awhile you get plonked on but this is to be expected and I mostly go with the flow.

 

[Blumlein 88]

https://deltaw.org

You can get Delta wave here.

 

[Maxicut]

I mostly agree with you on measurement is a guide (however not the whole picture) but not being science-y can be enjoyable even on this site. Yes, once in awhile you get plonked on but this is to be expected and I mostly go with the flow.

It's the non-science factor that fill's the missing pieces of the audio puzzle.

 

[Maxicut]

https://deltaw.org

You can get Delta wave here.

Pro-Tools does a better job

 

[raif71]

It's the non-science factor that fill's the missing pieces of the audio puzzle.

Yeah, one has to balance between being fully objective and being fully subjective. I don't believe into going far into either sides coz that's when distortion happens and you see people being cranky and defensive with either side of the argument.

 

[RayDunzl]

Re: Video in post $1

What happens in Part 2?

Conversely, did anything interesting happen in Part 1?

 

[whazzup]

Re: Video in post $1

What happens in Part 2? ...

Well, maybe the OP earned his $1 for posting the video and went off for his holidays...

 

[Chromatischism]

You are correct in your thinking. Measurements are a very good guide, but fall way short of the whole picture... which is why amirm gives his "personnel" subjective opinion with his speaker, monitor & headphone reviews....

Yes, but the personal recommendation (or not) from the site owner carries a lot of power that needs to be considered. Perhaps it would be best to skip the recommendation. We can take away what we need from measurements and his evaluation which he does a good job of.