Master Thread: Are measurements Everything or Nothing?

[Ken Tajalli]

It's useful for me and he already mentioned "subjectively" so it is not entirely futile at least not to me.

Thank you for your defence (support?).
It is a bit bothersome situation here.
I realize that a member can come in from out of the blue and talk exclusively about subjective issues, as though he missed the Science bit in Audio Science Review!
I truly get that, but isn't the opposite side equally to blame?!
I mean those with an attitude towards the first group of members, telling them they are full of it, imagining it and if it is not done under strict lab environment, they should take a hike and buzz off, as though they missed the Audio bit in Audio Science Review!
After all, why bother to make so many DACs? According to the (Attitude) group, all DACs these days, perform in a manner that almost all deficiencies are Inaudible ! what gives?
Why is @amirm saying, he wants to replace his beloved DX3 pro with a DX7 pro? after all the differences in lab performance are miniscule and inaudible!
Why is he not dishing out the same sentiments for DX7 pro (as he does on other equipments) that "differences are minor, doesn't justify the extra cost".
If we ignore the final Audio quality (a subjective quantity), approval of the listener, measuring any equipment for noise and distortion, Frequency response .... would be a futile exercise, they all pass!
Amir might as well call it a day.
I want to find out if the superior lab measurement results of the DX7 pro, has translated to good sound quality, not necessarily how it competes with anything else, although I will use Hugo2 as baseline.
No A/B testing, no blind tests, no fixed level matching, after all my ears can not compete with APx-555, comparative lab results are already out there - just living with the device for a couple of weeks, before I open my mouth.

 

[rlanger]

Thank you for your defence (support?).
It is a bit bothersome situation here.
I realize that a member can come in from out of the blue and talk exclusively about subjective issues, as though he missed the Science bit in Audio Science Review!
I truly get that, but isn't the opposite side equally to blame?!
I mean those with an attitude towards the first group of members, telling them they are full of it, imagining it and if it is not done under strict lab environment, they should take a hike and buzz off, as though they missed the Audio bit in Audio Science Review!
After all, why bother to make so many DACs? According to the (Attitude) group, all DACs these days, perform in a manner that almost all deficiencies are Inaudible ! what gives?
Why is @amirm saying, he wants to replace his beloved DX3 pro with a DX7 pro? after all the differences in lab performance are miniscule and inaudible!
Why is he not dishing out the same sentiments for DX7 pro (as he does on other equipments) that "differences are minor, doesn't justify the extra cost".
If we ignore the final Audio quality (a subjective quantity), approval of the listener, measuring any equipment for noise and distortion, Frequency response .... would be a futile exercise, they all pass!
Amir might as well call it a day.
I want to find out if the superior lab measurement results of the DX7 pro, has translated to good sound quality, not necessarily how it competes with anything else, although I will use Hugo2 as baseline.
No A/B testing, no blind tests, no fixed level matching, after all my ears can not compete with APx-555, comparative lab results are already out there - just living with the device for a couple of weeks, before I open my mouth.

I've been lurking on this site for a bit and registered to respond in agreement.

The site may be called ASR, but I certainly don't see a lot of science in the reviews. I see measurements, but measurements are not science.

Now correct me if I'm wrong, but the scientific method is: Observation > Question > Hypothesis > Testing / Experimenting / Measuring > Conclusion.

So, if the review were truly scientific, the question would come from listening / observing e.g. This amp / dac / headphone has flabby bass and rolled off highs > Question > Why? etc...

Or, these two dacs sound exactly the same > Do they measure the same? Yes / No? Why / Why not? etc...

Maybe I'm missing something? But science isn't "Here's the experiment so I conclude X."

Shouldn't we be able to ask, "Why do you think I'm hearing X?"

The challenge with audio is that there are just too many variables that affect the way we all perceive sound, that measurements can't capture.

If Amir never gives subjective insight in his reviews, I can't know if his perception of sound is similar to mine. Spoiler Alert > it likely isn't. But at least I could get a sense of the strength of the correlation over time if he did.

Like any good scientist though, I'm open to having my mind changed.

Edited to add: I have the Mojo 2 and love it's quirkiness, sound and killer DSP. Now, I have some measurements to know why it sounds good to me.
Thanks Amir.

 

[Veri]

The challenge with audio is that there are just too many variables that affect the way we all perceive sound, that measurements can't capture.

Where's your proof? Otherwise that's just a blanket statement to explain just about anything you pull out of your ass, "this DAC flows really well and has great PRAT, it's so amazing but I guess measurements can't capture any of that" see the problem? It's an audiophile snake oil vendor's favorite bullshit...

 

[rlanger]

Where's your proof? Otherwise that's just a blanket statement to explain just about anything you pull out of your ass, "this DAC flows really well and has great PRAT, it's so amazing but I guess measurements can't capture any of that" see the problem? It's an audiophile snake oil vendor's favorite bullshit...

Observation > I can't hear high-frequency sounds above a certain level.
Question > Why?
Hypothesis > I'm 55 and perhaps older people lose the ability to hear some high-frequency sounds.
Measurement > Does not disprove hypothesis. Therefore the statement I pulled out of my ass, stands.
Conclusion > You're younger so you perceive sound differently from me.

Science.

 

[Ken Tajalli]

Where's your proof? Otherwise that's just a blanket statement to explain just about anything you pull out of your ass, "this DAC flows really well and has great PRAT, it's so amazing but I guess measurements can't capture any of that" see the problem? It's an audiophile snake oil vendor's favorite bullshit...

You know, usually those who raise their voice and resort to vulgarity, are because they have no argument, hoping to scare the other side to silence by their loudness.
You are better than that, you don't agree, fine . . . .

 

[Purité Audio]

I've been lurking on this site for a bit and registered to respond in agreement.

The site may be called ASR, but I certainly don't see a lot of science in the reviews. I see measurements, but measurements are not science.

Now correct me if I'm wrong, but the scientific method is: Observation > Question > Hypothesis > Testing / Experimenting / Measuring > Conclusion.

So, if the review were truly scientific, the question would come from listening / observing e.g. This amp / dac / headphone has flabby bass and rolled off highs > Question > Why? etc...

Or, these two dacs sound exactly the same > Do they measure the same? Yes / No? Why / Why not? etc...

Maybe I'm missing something? But science isn't "Here's the experiment so I conclude X."

Shouldn't we be able to ask, "Why do you think I'm hearing X?"

The challenge with audio is that there are just too many variables that affect the way we all perceive sound, that measurements can't capture.

If Amir never gives subjective insight in his reviews, I can't know if his perception of sound is similar to mine. Spoiler Alert > it likely isn't. But at least I could get a sense of the strength of the correlation over time if he did.

Like any good scientist though, I'm open to having my mind changed.

Edited to add: I have the Mojo 2 and love it's quirkiness, sound and killer DSP. Now, I have some measurements to know why it sounds good to me.
Thanks Amir.

‘Now correct me if I'm wrong, but the scientific method is: Observation > Question > Hypothesis > Testing / Experimenting / Measuring > Conclusion.’
The problem is subjectivists rarely get past stage one, they imagine they observe but never verfiy with even the simplest controls in place.
Keith

 

[rlanger]

‘Now correct me if I'm wrong, but the scientific method is: Observation > Question > Hypothesis > Testing / Experimenting / Measuring > Conclusion.’
The problem is subjectivists rarely get past stage one, they imagine they observe but never verfiy with even the simplest controls in place.
Keith

I can't argue with that. But don't you think skipping the observations > question > hypothesis is equally problematic?

Then at least we might start to better understand why something has PRAT or flow or whatever else we want to call it? i.e. In my observations, all of these amps have similar PRAT. Maybe, it's because of X and Y domain measurements. Yes, they all measure similarly in X and Y domains. Therefore, I conclude that any amp that measures similarly in X and Y domains will have similar PRAT.

So now when some ass says "where's your proof?", you can say, "Here you go."

I don't know. I just like music that sounds good.

 

[Ken Tajalli]

‘Now correct me if I'm wrong, but the scientific method is: Observation > Question > Hypothesis > Testing / Experimenting / Measuring > Conclusion.’
The problem is subjectivists rarely get past stage one, they imagine they observe but never verfiy with even the simplest controls in place.
Keith

There are extremists on both campaigns.
There are people who name themselves objective scientists, get a measuring gear together and from the graphs and numbers dictate the observation! if it doesn't match, resort to ridicule and vulgarity.

 

[Another Audiophile]

Thank you for your defence (support?).
It is a bit bothersome situation here.
I realize that a member can come in from out of the blue and talk exclusively about subjective issues, as though he missed the Science bit in Audio Science Review!
I truly get that, but isn't the opposite side equally to blame?!
I mean those with an attitude towards the first group of members, telling them they are full of it, imagining it and if it is not done under strict lab environment, they should take a hike and buzz off, as though they missed the Audio bit in Audio Science Review!
After all, why bother to make so many DACs? According to the (Attitude) group, all DACs these days, perform in a manner that almost all deficiencies are Inaudible ! what gives?
Why is @amirm saying, he wants to replace his beloved DX3 pro with a DX7 pro? after all the differences in lab performance are miniscule and inaudible!
Why is he not dishing out the same sentiments for DX7 pro (as he does on other equipments) that "differences are minor, doesn't justify the extra cost".
If we ignore the final Audio quality (a subjective quantity), approval of the listener, measuring any equipment for noise and distortion, Frequency response .... would be a futile exercise, they all pass!
Amir might as well call it a day.
I want to find out if the superior lab measurement results of the DX7 pro, has translated to good sound quality, not necessarily how it competes with anything else, although I will use Hugo2 as baseline.
No A/B testing, no blind tests, no fixed level matching, after all my ears can not compete with APx-555, comparative lab results are already out there - just living with the device for a couple of weeks, before I open my mouth.

I always feel that people should be polite. After all is a hobby, not a crusade. Life is to short for getting upset over these kind of issues. I am basically neutral and believe both measurements and the way I experience things matter. Nevertheless, both sides exaggerate. For example in the largest audio forum, head-fi, all that matters is experiences. From the other hand here we believe measuring equipment is Science. Be polite, enjoy music and the gear that makes it sound great!

Edit: to be fair with you I've tried 5-6 of these super performing DACs against my chord quest and mojo1/2. I sold all of these DACs and kept the chords Why? Because no other DAC I've tried display overtones, decay and soundstage depth the way chord DACs do.Why? I have no idea. Nevertheless, I don't really try to convince anyone because it doesn't really matters to anyone but me.

 

[BDWoody]

Just moved a few more in.

 

[Ken Tajalli]

I always feel that people should be polite. After all is a hobby, not a crusade. Life is to short for getting upset over these kind of issues. I am basically neutral and believe both measurements and the way I experience things matter. Nevertheless, both sides exaggerate. For example in the largest audio forum, head-fi, all that matters is experiences. From the other hand here we believe measuring equipment is Science. Be polite, enjoy music and the gear that makes it sound great!

From the other hand here we (?) believe measuring equipment is (part of the ) Science.
It takes knowledge and experience to use & measure a device using the complicated Audio Precision machine, knowing its limitations, quirks, what to measure and finally comprehend the raw data output.
That last bit, is where most wanna be scientists fail.
But over all , I agree.

 

[Another Audiophile]

I've been lurking on this site for a bit and registered to respond in agreement.

The site may be called ASR, but I certainly don't see a lot of science in the reviews. I see measurements, but measurements are not science.

Now correct me if I'm wrong, but the scientific method is: Observation > Question > Hypothesis > Testing / Experimenting / Measuring > Conclusion.

So, if the review were truly scientific, the question would come from listening / observing e.g. This amp / dac / headphone has flabby bass and rolled off highs > Question > Why? etc...

Or, these two dacs sound exactly the same > Do they measure the same? Yes / No? Why / Why not? etc...

Maybe I'm missing something? But science isn't "Here's the experiment so I conclude X."

Shouldn't we be able to ask, "Why do you think I'm hearing X?"

The challenge with audio is that there are just too many variables that affect the way we all perceive sound, that measurements can't capture.

If Amir never gives subjective insight in his reviews, I can't know if his perception of sound is similar to mine. Spoiler Alert > it likely isn't. But at least I could get a sense of the strength of the correlation over time if he did.

Like any good scientist though, I'm open to having my mind changed.

Edited to add: I have the Mojo 2 and love it's quirkiness, sound and killer DSP. Now, I have some measurements to know why it sounds good to me.
Thanks Amir.

You are right. Measurements is only one step and tool for science to verify or disqualify the prime motivation in science which is observation.

 

[BDWoody]

Yes, they all measure similarly in X and Y domains. Therefore, I conclude that any amp that measures similarly in X and Y domains will have similar PRAT.

So now when some ass says "where's your proof?", you can say, "Here you go."

Not if you didn't use basic level and sight controls.

This may help.

 

[AdamG]

Just moved a few more in.

Oh Joy! Groundhogs day….

 

[Ken Tajalli]

Just moved a few more in.

Thank you, get my last post over here too, If anything, I continue here.

 

[Another Audiophile]

From the other hand here we (?) believe measuring equipment is (part of the ) Science.
It takes knowledge and experience to use & measure a device using the complicated Audio Precision machine, knowing its limitations, quirks, what to measure and finally comprehend the raw data output.
That last bit, is where most wanna be scientists fail.
But over all , I agree.

Yes, ok. We I meant the forum's mantra and it is indeed part of science but not science itself. Audio part of science review? Maybe longer title but more accurate, if accuracy matters.

 

[BDWoody]

Observation > I can't hear high-frequency sounds above a certain level.
Question > Why?
Hypothesis > I'm 55 and perhaps older people lose the ability to hear some high-frequency sounds.
Measurement > Does not disprove hypothesis. Therefore the statement I pulled out of my ass, stands.
Conclusion > You're younger so you perceive sound differently from me.

Observation:
I can hear bat sonar.
Question:
Why?

Wouldn't you first want me to show I *actually* can before it gets into much more?

 

[Another Audiophile]

Observation:
I can hear bat sonar.
Question:
Why?

Wouldn't you first want me to show I *actually* can before it gets into much more?

Science is a little bit more complicated and has to be challenged and discussed with other scientists. The question is not Why. In Science is prove it. And the long scientific process begins. I am an Engineer and not a Scientist but I feel, especially in the audio circles, many people feel that they are kind of scientists because the "Understand" it. When you think you understood it is the moment you stop being a scientist. From the other hand there are many who discard science. That's fine, whatever float their boats. After all the information is there to make a solid decision. For example I think Amir's measurements on mojo 2 are incomplete but that's ok. Better something than nothing. At least Amir is actually an Engineer who accomplished something (or more). Certainly he knows what he is doing and sometimes Engineers, like myself, are challenged and guess what. We are not always right and that's ok.

To conclude measurements are neither everything or nothing. Is an essential tool which is part of Science as a process.

 

[AdamG]

Bat Sonar

 

[BDWoody]

is a little bit more complicated and has to be challenged and discussed with other scientists. The question is not Why. In Science is prove it.

Exactly.

Here, we often have people claiming things that are in the 'as yet unproven' stage, and are typically offended when asked for more than anecdote. Anecdote vs evidence is tricky for many, but absent evidence, the claims are as easily dismissed as my claim to hear bat sonar.