Master Thread: Are measurements Everything or Nothing?

[Jim Taylor]

I've never found much correlation between expectation and result.

For a great number of people, expectation is their higher priority. IOW, the result must meet their expectation, not the other way around. Some listeners want a "sweet" sound, some want accentuated bass, and some want prominent vocals.

There's not really anything intrinsically wrong with this (see below), but one must realize that ideally, playback mechanisms are neutral. They give you what is on the recording, be that what it may. Electronic circuits capable of doing this have been available for many years, and gradually, speaker systems have matured to follow suite.

This is my (admittedly personal) take on the situation:

1) I have a slight advantage in expectation, because the majority of my recordings (good or bad) are of acoustic events. Therefore, if the "sound" doesn't meet my expectations, I change my expectations. After all, we can't get Glenn Miller to re-record "In The Mood"; he died long ago.

2) If the recording is direct-to-desk (direct to mixer, like Electronica) then that's a completely different story. In that case, no acoustic event prior to recording exists. A listener is free to adjust the sound to whatever extent they desire, and there is no untoward consequence.

3) Recordings of live events in which all the sound that the audience hears has been electrically amplified is midway between the first and second examples above. The reason is, one must assume that the groups' sound engineer has adjusted everything to give a particular effect that the group wants, and what you hear is deliberate.

Of course, this all makes no sense unless there exists some method for knowing that the playback system is accurate and neutral. In general, the only way we can correlate what is on the recording with what we hear is through tests and measurements of the equipment. If you're the sort of person who wants neutral, the tests and measurements will help you get it. It may not be perfect, but it can be usefully close. (Personally, I'm satisfied with "usefully close". )

OTOH, if you're the sort of person who wants affected sound, tests and measurements can help you in that case, too. You will, however, be in a much more difficult situation. You will need to go through a great deal of trial-and-error, because none of the manufacturers know what your personal standard happens to be, nor would you necessarily know which particular characteristic of measurement satisfies you. There is, after all, no public baseline. (You may try tube circuits to attain your goal, or you may depend on DSP to attain it.)

Whichever method you decide on, I wish you good luck!

Jim

 

[DonR]

I don't think its necessarily true.
I went to listen to some speakers on Friday. I had a very positive attitude to the manufacturer, Amir had just posted a very positive review on ASR, and the dealer was offering an discount I couldn't resist. But I listened to them, under ideal conditions and with a variety of music, and I didn't like them.
I was so convinced that I wanted them that I'd have bought them without listening first.
To my enormous surprise I left empty handed, and it's not the first time, it's happened lots of times.
I agree that sighted listening is much less reliable than blind, but I've never found much correlation between expectation and result.

Could also be the room. It has a tremendous effect on the sound.

 

[Basic Channel]

1) I have a slight advantage in expectation, because the majority of my recordings (good or bad) are of acoustic events. Therefore, if the "sound" doesn't meet my expectations, I change my expectations. After all, we can't get Glenn Miller to re-record "In The Mood"; he died long ago.

2) If the recording is direct-to-desk (direct to mixer, like Electronica) then that's a completely different story. In that case, no acoustic event prior to recording exists. A listener is free to adjust the sound to whatever extent they desire, and there is no untoward consequence.

3) Recordings of live events in which all the sound that the audience hears has been electrically amplified is midway between the first and second examples above. The reason is, one must assume that the groups' sound man has adjusted everything to give a particular effect that the group wants, and what you hear is deliberate.

I am not sure that "acoustic events" are really that different. Especially when they are recorded in multi track and manipulated. And since about the 60s or so quite a lot of those acoustic event recordings have been heavily manipulated.

For a lot of electronic music I am less free to adjust the sound tbh. If I listen to Steve Wonder with the sub up a bit it doesn't sound bad. If I play 2562 I feel like I am getting crushed by bass if the sub is set too high. And similar for treble I'd imagine, the more acoustic the recording is the more 10k+ is "air" rather than than some mad sine wave or something.

 

[welwynnick]

What were the speakers?
Keith

Revel Performa F206 - carefully positioned - in an acoustically treated demo room

 

[Purité Audio]

I haven’t really heard any Revels since the Salon 2‘ left, the measurements/review here are reasonable.
Keith

 

[sonitus mirus]

Revel Performa F206 - carefully positioned - in an acoustically treated demo room

Was the demo music featuring Yoko Ono?

 

[welwynnick]

No! Mostly Elbow and Radiohead, some female vocals and piano.

 

[sonitus mirus]

No! Mostly Elbow and Radiohead, some female vocals and piano.

Ha, I was being flippant with no intent to offend.

You've mentioned that this has happened to you multiple times, but the situation still strikes me as unusual. It feels reminiscent of a 'gotcha' scenario, where the reliability of measurements is questionable, or perhaps an important, yet undisclosed, detail is being overlooked in the performance evaluation. Without a way to verify your claims, and considering the information provided, it seems likely that there is a critical piece missing—something that has led you to conclude that what you were hearing was not ideal. Moreover, do you really expect others to simply concede that your personal account represents a groundbreaking revelation in the fields of measurements and audio reproduction?

 

[welwynnick]

In hindsight I think they may have benefitted from being turned in more, and I think my hearing has become sensitive to a minor error of addition by the F206 . It was positively unpleasant, and I wouldn't have bought them , regardless of whether we compared them to other speakers or not.

I'm not saying this is a revelation in anything. I'm sure the evidence is all in Amir's charts - maybe it's the 5kHz mid-range cone breakup

 

[Bozon]

I don't think its necessarily true.
I went to listen to some speakers on Friday. I had a very positive attitude to the manufacturer, Amir had just posted a very positive review on ASR, and the dealer was offering an discount I couldn't resist. But I listened to them, under ideal conditions and with a variety of music, and I didn't like them.
I was so convinced that I wanted them that I'd have bought them without listening first.
To my enormous surprise I left empty handed, and it's not the first time, it's happened lots of times.
I agree that sighted listening is much less reliable than blind, but I've never found much correlation between expectation and result.

I have been on that situation many times...

 

[sonitus mirus]

In hindsight I think they may have benefitted from being turned in more, and I think my hearing has become sensitive to a minor error of addition by the F206 . It was positively unpleasant, and I wouldn't have bought them , regardless of whether we compared them to other speakers or not.

I'm not saying this is a revelation in anything. I'm sure the evidence is all in Amir's charts - maybe it's the 5kHz mid-range cone breakup

Maybe ear position relative to the speakers is not right for you when you demo the speakers compared to where you commonly listen at home?

 

[antcollinet]

Revel Performa F206 - carefully positioned - in an acoustically treated demo room

Who knows - perhaps it was the room treatment you didn't like.

It is a real problem auditioning speakers anywhere other than your own listening space. The most likely won't sound like the will at home.

This is why measurements are especially important for speakers.

 

[Newman]

...I went to listen to some speakers on Friday. I had a very positive attitude to the manufacturer, Amir had just posted a very positive review on ASR, and the dealer was offering an discount I couldn't resist. But I listened to them, under ideal conditions and with a variety of music, and I didn't like them.
I was so convinced that I wanted them that I'd have bought them without listening first.
To my enormous surprise I left empty handed, and it's not the first time, it's happened lots of times.
I agree that sighted listening is much less reliable than blind, but I've never found much correlation between expectation and result.

For a great number of people, expectation is their higher priority. IOW, the result must meet their expectation, not the other way around. Some listeners want a "sweet" sound, some want accentuated bass, and some want prominent vocals....

Could also be the room. It has a tremendous effect on the sound.

...Without a way to verify your claims, and considering the information provided, it seems likely that there is a critical piece missing—something that has led you to conclude that what you were hearing was not ideal. Moreover, do you really expect others to simply concede that your personal account represents a groundbreaking revelation in the fields of measurements and audio reproduction?

Maybe ear position relative to the speakers is not right for you when you demo the speakers compared to where you commonly listen at home?

Who knows - perhaps it was the room treatment you didn't like....

All the above 'in-the-sound-wave' suggestions, while perfectly plausible in themselves, are glossing over this point:-

This is a variation on the 'I expected to like them and I didn't, so there's no such thing as cognitive bias or cognitive bias could not be a factor in this case.'

But it ignores the fact that our conscious and subconscious expectations are not necessarily the same. And, by definition, we cannot be aware of what our subconscious expectation is.

...to which I say, 'Like'!

We have been over this with our protagonist before:-

[to welwynnick] That is not what it is said.

What is said is that Cognitive bias MIGHT Be involved every time there is an uncontrolled listening test. It means such tests are unreliable every time because we DON"T KNOW if cognitive bias played a part.

A subtle but important difference.


Then when devices that are compared are so good that the measurements tell us they will be audibly indistinguishable - and someone still claims a difference can be heard in uncontrolled listening - we can reasonably assume the reason is bias. Just probabilities.

AND ALSO IMPORTANT.

Bias is not necessarily about what we consciously expect. They are a subconscious process, and will be impacted by all sorts of subconscious "expectations"** based on a lifetime of experience.

** There are also many other biases than expectation bias.

I am re-emphasising this because it is semi-automatic to give credence to sighted listening impressions as being caused by sonic factors, and then proceed to rationalise possible things that could be different in the sound waves themselves.

Similarly, it is semi-automatic to think that conscious expectation bias will dominate other conscious biases and will also dominate unconscious biases, and quickly come to the unwarranted conclusion that you have 'outsmarted' cognitive biases and they are a non-issue. Not so.

And, as soon as one has convinced oneself that sighted listening impressions are all about the sound waves, the yawning chasm of misattribution takes over one's judgement about what gear sounds like, and one is found giving "A thousand likes!" to claims that putting a big capacitor across the SMPS will be a clearly audible improvement to the PA5 II amp, that already measures SINAD at -130 dB.

Then one's posts get moved into this thread.

cheers

 

[Beave]

In hindsight I think they may have benefitted from being turned in more, and I think my hearing has become sensitive to a minor error of addition by the F206 . It was positively unpleasant, and I wouldn't have bought them , regardless of whether we compared them to other speakers or not.

I'm not saying this is a revelation in anything. I'm sure the evidence is all in Amir's charts - maybe it's the 5kHz mid-range cone breakup

What 5kHz midrange cone breakup are you referring to?

 

[welwynnick]

What 5kHz midrange cone breakup are you referring to?

This one:

Revel F206 Tower Speaker Review

This is a review, listening tests, EQ, and detailed measurements of the Revel F206 Floor-standing speaker. It is on kind loan from a member (new) and costs US $1347.50 each. Please excuse the mobile phone picture quality and plastics that are still on the speaker. I try to keep products as...

www.audiosciencereview.com

 

[Beave]

This one:

View attachment 363886

Revel F206 Tower Speaker Review

This is a review, listening tests, EQ, and detailed measurements of the Revel F206 Floor-standing speaker. It is on kind loan from a member (new) and costs US $1347.50 each. Please excuse the mobile phone picture quality and plastics that are still on the speaker. I try to keep products as...

www.audiosciencereview.com

I figured that's what you meant. You are conveniently neglecting all of the discussion in that thread that calls that measurement into question.

Notice, for example, the difference between the two woofer outputs in the region above their passband, from 200Hz to 2kHz. Note how the top woofer has 5 to 10dB more output in this region than does the bottom woofer. But how can that be? They're run together with the same crossover. The explanation is that the top woofer is closer to the midrange driver and so it picks up more leakage output from the midrange in these close-mic measurements than does the bottom woofer, which is farther away from the midrange.

Notice too how the lower woofer has a differing output near the port tune region than the upper woofer (at least it shows that in this plot). That too is not real but is a result of the lower woofer being closer to the port, thus having the port output blend in with the measured woofer output.

In other words, these plots can not be trusted as being real and correct in level, especially in their out of band regions.

 

[welwynnick]

Are you suggesting that the mid range driver output is tweeter leakage? If that was the case, the leakage would be more likely to occur at the tweeter's system resonance, which is inside the mid-range pass band.

I'm not conveniently neglecting anything - I want explanation as much as everyone. I fully expected to like the F206, and I'm struggling to understand why I didn't.

 

[solderdude]

Is that cone break-up or perhaps interference between signal being picked up by the close measurement with both tweeter and midrange being 'active' ?
It would be easier to say something if each speaker would be measured with the others turned off. Alas this is not possible.

I also don't think that specific measurement could give a clue why you did not like the F206 (circumstances ?)

 

[antcollinet]

and I'm struggling to understand why I didn't.

But you've already been given many many reasons why you might not have.

Starting with subconscious perceptive biases, through all the room effects, and (less likely) them not matching your subjective preference. Unfortunately if you want a final answer of which of these are at play, the only way is to conduct a series of blind tests to find out. This is impractical, so you'll just have to accept not knowing which of the many possibilities it is, because without valid testing of your perception, everything is pure speculation.

 

[welwynnick]

Is that cone break-up or perhaps interference between signal being picked up by the close measurement with both tweeter and midrange being 'active' ?
It would be easier to say something if each speaker would be measured with the others turned off. Alas this is not possible.

What do you mean by active? I presume they were disconnected for that test?

One option might be to short the inputs of the drive units that are not being measured, so their motion is damped.