Equal measurements is not the same as equal sound quality

[MagnusH]

I like this page and the reviews, and I think Amir is doing an excellent work. However, I think some of the conclusions drawn from measurements are wrong. Yes, measurements can be used to (partly) decide how well engineered an HiFi equipment is, but they cannot fully describe sound quality. Take soundstage (depth, width, precision, cohesion and more), no measurement I have seen can measure those.

I can take an example, from 2 DACs I have listened to myself, the Pro-Ject Pre Box S2 Digital and the Schiit Yggdrasil. Looking at measurements the Pro-Ject DAC looks to sound better, but it does not. Not even close! The sound stage, clarity and detail retrieval from the Schiit is several steps ahead, as it should based on the price difference. And if you don't believe me, do yourself a favor and listen yourself.

Then there is the issue of sound quality: what is it? Lots of people out there like tubes and R2R DACs, which measures horrible. In a way, you could say that they add "good" distortion, and while the ears can differentiate between good and bad distortion, measurements cannot.

Btw, I have equipment that measures good myself (RME ADI-2 DAC and NCore 500 power amps), because I like how they sound. But there are better sounding DACs and amps out there, and some of those measure less good.

 

[March Audio]

Stock reply which I a think I am going to keep on my desktop and just copy and paste in future:

Do you perform your subjective comparisons under blind and controlled conditions?

If not Im afraid your opinions on sound quality have no validity.

The conversation sort of stops there because you need to demonstrate your claims above have validity.

 

[MagnusH]

Of course I don't listen to music in "controlled conditions" with blind tests. Blind tests can be good when you want to prove difference (they can never be used to prove equality though), but this thread is about how well measurements reflect sound quality.

Besides, I don't need to prove anything, I am just voicing my opinion and you are free to agree or disagree.

 

[majingotan]

The sound stage, clarity and detail retrieval from the Schiit is several steps ahead, as it should based on the price difference. And if you don't believe me, do yourself a favor and listen yourself.

Don't be delusional. I own the Bifrost 2 which the creator Jason Stoddard himself preferred it subjectively over Yggdrasil (https://www.head-fi.org/threads/schiit-audio-bifrost-2.914084/page-30#post-15405085) which I know for a fact that he is just shilling for more profit obviously. Though I own the Bifrost 2 for its rarity in my country and NOT because it's a superior sounding DAC. Upon sighted A/B volume match with the $9 Apple lightning DAC, there's absolutely ZERO difference between the two. Both sound transparent with just sighted A/B. If you believe that a bigger box means better sound, then you ought to read the ASR Manifesto first before say BS about objective measurements and the validity of DBT volume matched test.

 

[Blumlein 88]

Of course I don't listen to music in "controlled conditions" with blind tests. Blind tests can be good when you want to prove difference (they can never be used to prove equality though), but this thread is about how well measurements reflect sound quality.

Besides, I don't need to prove anything, I am just voicing my opinion and you are free to agree or disagree.

Your methodology is inadequate. Your opinion is one we'll just ignore. Not agree or disagree. It has no foundation for your conclusions.

 

[solderdude]

Funny thread.

It starts with equal measurements is not the same as equal sound quality and then proceeds with stating that 2 very different measuring DACs do not sound the same and has a preference for the worse measuring DAC but obviously chooses the even better measuring DAC because he likes it better (owns the ADI-2)

Then he proceeds to tell everyone he did not compare DACs level matched and blind and can thus tell differences clearly and the reason for not comparing DACs is that he doesn't listen to music this way.
Very funny... no one does but if you want to really know and compare it should be done under equal and blind conditions.

Blind tests can be good when you want to prove difference (they can never be used to prove equality though)

This one is even funnier... blind tests can be good for proving a difference (because you can hear differences or not depending if they are audible or not) but cannot be used for detecting equality ?
So what if I do not hear differences between DACs in blind level matched conditions doe such not prove audible equality or perceived equality ?

Then there is the issue of sound quality: what is it? Lots of people out there like tubes and R2R DACs, which measures horrible. In a way, you could say that they add "good" distortion, and while the ears can differentiate between good and bad distortion, measurements cannot.

You are not talking about sound quality here but about personal preference. people liking altered sound prefer altered sound which they feel sound 'good' despite measurably much poorer signal fidelity. Nothing wrong with that it is just a preference.

Measurements can 'see' far below noise floors and well outside the audible range and can 'detect' differences much much smaller than audible ones.
Understanding a whole suite of measurements (under various loads for amps) can show excellent to problematic behaviour and anything in between. That does not mean someone who does not understand measurements or only looks at some numbers can correlate this to measurements. For that one needs experience.

 

[MagnusH]

Funny thread.

It starts with equal measurements is not the same as equal sound quality and then proceeds with stating that 2 very different measuring DACs do not sound the same and has a preference for the worse measuring DAC but obviously chooses the even best measuring DAC because he likes it better (ADI-2)

The Pro-Ject vs Schiit is just to prove that measurement and sound quality can sometimes be opposite. My own choise of DAC has nothing to do with that.

This one is even funnier... blind tests can be good for proving a difference (because you can hear differences or not depending if they are audible or not) but cannot be used for detecting equality ?
So what if I do not hear differences between DACs in blind level matched conditions doe such not prove audible equality or perceived equality ?

You can use blind tests as a listening test without outer influence. However, if you can't separate the sound from the tests that does not mean it sounds the same, only that you can't tell them apart. Maybe in other conditions you could tell them apart, like with different music, different length of listening sessions, different room etc. Hence, blind tests can never prov equality, only difference, which in my opinion makes them pretty worthless.

You are not talking about sound quality here but about personal preference. people liking altered sound prefer altered sound which they feel sound 'good' despite measurably much poorer signal fidelity. Nothing wrong with that it is just a preference.

Sound quality is personal preference. There is no "global" sound quality, although some type of changes in sound is what almost all people prefer. For example, I am pretty sure that more than 90% of listeners would prefer Schiit Yggdrasil over Pro-Ject Pre Box S2 Digital, but the measurements tell something else.

 

[solderdude]

However, if you can't separate the sound from the tests that does not mean it sounds the same, only that you can't tell them apart. Maybe in other conditions you could tell them apart, like with different music, different length of listening sessions, different room etc. Hence, blind tests can never prov equality, only difference, which in my opinion makes them pretty worthless.

When you can't tell equipment apart in properly controlled blind tests they thus sound the same (= equal) to the listener. Even though they could measure quite differently.

There is nothing stopping anyone to use different music or change the length (could even be spread over weeks) and be done at home.
There is no excuse I am afraid except lazyness, not caring or preferring subjective and sighted tests.

Sighted tests have been proven to be pretty worthless in all disciplines where people are involved.

Sound quality is personal preference.

No sound quality is not personal preference.
Prefering a certain sound is a preference. The word quality is not in there.

 

[Koeitje]

Sound quality is personal preference. There is no "global" sound quality, although some type of changes in sound is what almost all people prefer.

That is Myfi, not Hifi. The only part of a Hifi system where we have to deal with Myfi are the transducers, apart from that you can buy blind based on measurements, usability (such as inputs, format support, etc) and looks (I don't want anything that is not black for my electronics).

 

[VintageFlanker]

Here we go again. Don't know where to start...

The Pro-Ject vs Schiit is just to prove that measurement and sound quality can sometimes be opposite.

How on earth your uncontrolled listening would prove anything here?

Sound quality is personal preference. There is no "global" sound quality, although some type of changes in sound is what almost all people prefer. For example, I am pretty sure that more than 90% of listeners would prefer Schiit Yggdrasil over Pro-Ject Pre Box S2 Digital, but the measurements tell something else.

What @solderdude just said. Anyone may prefer a specific sound signature. Objective sound quality does exist. It is called transparency.

Blind tests can be good when you want to prove difference

is it? Then, why don't you do some to "prove" the difference here?

Maybe in other conditions you could tell them apart, like with different music, different length of listening sessions, different room etc. Hence, blind tests can never prov equality, only difference, which in my opinion makes them pretty worthless.

Worthless? And your subjective and uncontrolled listening should worth something from a scientific point of view? I don't think so...

Not even close! The sound stage, clarity and detail retrieval from the Schiit is several steps ahead, as it should based on the price difference.

One last thing: That is the only proof of something over all you said... The proof of your expectation bias.

Sorry for being rude but... I think you perfectly know what you're doing posting such claims on ASR...

 

[murraycamp]

I like this page and the reviews, and I think Amir is doing an excellent work. However, I think some of the conclusions drawn from measurements are wrong. Yes, measurements can be used to (partly) decide how well engineered an HiFi equipment is, but they cannot fully describe sound quality. Take soundstage (depth, width, precision, cohesion and more), no measurement I have seen can measure those . . .
Lots of people out there like tubes and R2R DACs, which measures horrible. In a way, you could say that they add "good" distortion, and while the ears can differentiate between good and bad distortion, measurements cannot.

 

[pma]

I like this page and the reviews, and I think Amir is doing an excellent work. However, I think some of the conclusions drawn from measurements are wrong. Yes, measurements can be used to (partly) decide how well engineered an HiFi equipment is, but they cannot fully describe sound quality.

I think the reason is that the title statement

Equal measurements is not the same as equal sound quality

cannot be proven in general, because so far we are not providing that many measurements to cover the product completely and if we do something more than is usual, it would be often called as unimportant measurements for audio, though no proof is given that it is unimportant.

I tend to agree partially with you and for me the measurements are just the 1st aid to have some view and opinion what the product would be like. However, I cannot imagine how it would sound from measurements only. Only in case that measurements are bad enough, close to horrible, I can imagine how it would sound. This place is trying to play a scientific-like game, but in fact it often displays oversimplifications and lack of experience.

 

[MagnusH]

Don't know where to start...
How on earth your uncontrolled listening would prove anything here?

What @solderdude just said. Anyone may prefer a specific sound signature. Objective sound quality does exist. It is call transparency.

Ok, "prove" is a strong word, "made me aware" might be a better phrase.

But lets use transparency as a term for sound quality, and lets see where the Pro-Ject vs Yggdrasil comparison takes us. Personally I liked the sound of Yggdrasil much more than the sound from the Pro-Ject DAC, partly because it had an increased clarity and was more detailed both in its music and in its sound stage. And since details in the music hardly pop up from nothing, I would say that Yggdrasil sounded more transparent. And increased clarity should be a sign of increased transparency.

Hence: sound quality is different from good measurements, which is the point of this thread.

 

[March Audio]

Sound quality is personal preference. There is no "global" sound quality, although some type of changes in sound is what almost all people prefer. For example, I am pretty sure that more than 90% of listeners would prefer Schiit Yggdrasil over Pro-Ject Pre Box S2 Digital, but the measurements tell something else.

Except actually there is.

I suggest you take a look at the research of Floyd Toole on the correlation of subjective preference to objective measurements in loudspeakers.

To sum up they found they could predict from the measurements to an accuracy of (IIRC) 86% what (large) speakers would be preferred by listeners. This increased to 96% with small speakers. People consistently liked speakers with a flat on axis anechoic response with smooth of axis response.

Im pretty sure that 90% of people would not be able to tell the difference between those 2 dacs under controlled conditions.

 

[Blumlein 88]

Ok, "prove" is a strong word, "made me aware" might be a better phrase.

But lets use transparency as a term for sound quality, and lets see where the Pro-Ject vs Yggdrasil comparison takes us. Personally I liked the sound of Yggdrasil much more than the sound from the Pro-Ject DAC, partly because it had an increased clarity and was more detailed both in its music and in its sound stage. And since details in the music hardly pop up from nothing, I would say that Yggdrasil sounded more transparent. And increased clarity should be a sign of increased transparency.

Hence: sound quality is different from good measurements, which is the point of this thread.

Can you at least compare them level matched? Otherwise nothing you say is of any use. Level matching is job #1 if your comparison has any validity.

 

[March Audio]

Ok, "prove" is a strong word, "made me aware" might be a better phrase.

But lets use transparency as a term for sound quality, and lets see where the Pro-Ject vs Yggdrasil comparison takes us. Personally I liked the sound of Yggdrasil much more than the sound from the Pro-Ject DAC, partly because it had an increased clarity and was more detailed both in its music and in its sound stage. And since details in the music hardly pop up from nothing, I would say that Yggdrasil sounded more transparent. And increased clarity should be a sign of increased transparency.

Hence: sound quality is different from good measurements, which is the point of this thread.

Personal preference, especially when biased by sighted comparison, is NOT sound quality

 

[Wombat]

More yawn. Sound quality can be measured. Even preference. It may not reflect opinion.

 

[VintageFlanker]

Personally I liked the sound of Yggdrasil much more than the sound from the Pro-Ject DAC, partly because it had an increased clarity and was more detailed both in its music and in its sound stage. And since details in the music hardly pop up from nothing, I would say that Yggdrasil sounded more transparent. And increased clarity should be a sign of increased transparency.

Assuming your analysis would be true (and I actually think there is an audible difference between these two DACs): Increased clarity over an already transparent gear would be called emphasis in my book, certainly not increasing transparency.

 

[oldsysop]

 

[MagnusH]

Personal preference, especially when biased by sighted comparison, is NOT sound quality

If a DAC presents more clarity and details, I think its safe to say that that DAC has more transparency. If you believe my hearing is correct or not is another matter, and if you don't feel free to do the comparison yourself using whatever controlled blind test you like. Then, and only then, are you in a position to actually argue about my conclusions.