Master Thread: Are measurements Everything or Nothing?

[Romario]

The lack of calibration certification per reviewer's ear ( or my own for that matter) is enough to dissmiss the subjective tests on any forum.

 

[MattHooper]

The lack of calibration certification per reviewer's ear ( or my own for that matter) is enough to dissmiss the subjective tests on any forum.

Why? Could you explain what you mean by a calibration certification per reviewer’s ear?

 

[solderdude]

The lack of calibration certification per reviewer's ear ( or my own for that matter) is enough to dissmiss the subjective tests on any forum.

Fortunately enjoying music does not require any certification nor calibration nor proof of listening skills nor audiograms (taken more than a day ago between 150Hz and 8kHz) showing hearing limits.

 

[Romario]

When you take a microphone to measure a calibration for the various frequencies is taken into account. But when a reviewer takes his ear as the instrument no one knows what freqencies he's able to hear properly.

 

[anphex]

Why? Could you explain what you mean by a calibration certification per reviewer’s ear?

I think he means if the people reviewing actually have working hearing.

 

[Romario]

Fortunately enjoying music does not require any certification nor calibration nor proof of listening skills nor audiograms (taken more than a day ago between 150Hz and 8kHz) showing hearing limits.

That's very true, but enjoying music is not the same as recommend any gear. The only thing one can do is to compare speakers ( one might be better than the other) but not as a single unit.

 

[MattHooper]

When you take a microphone to measure a calibration for the various frequencies is taken into account. But when a reviewer takes his ear as the instrument no one knows what freqencies he's able to hear properly.

I agree that in principle it would be ideal if every reviewer had the unblemished hearing of a 20-year-old who has never attended a loud concert or shot firearms

Practically speaking though, you don’t need perfect hearing to be able to apprehend many salient characteristics about sound reproduction or loudspeakers.

After all, the hearing of ASR participants are likely all over the map, but I presume most are carefully vetting their equipment purchases on the assumption that they will still apprehend important sonic features of their sound system.

I mean, I certainly get your concern, but for me, not knowing somebody’s audiogram is not discrediting, so long as they seem to be accurately describing products I’m familiar with, which gives me some informal level of credence in their reports.

 

[Romario]

Yup, if they describe the products I'm familiar with, I then can relate to the review. If they describe a product I'm not familiar with then it gets tricky. I' ve never seen a reviewer drawn a predicted frequency response by ear and compared it to the measured one.

 

[solderdude]

That's very true, but enjoying music is not the same as recommend any gear. The only thing one can do is to compare speakers ( one might be better than the other) but not as a single unit.

With speakers the room also is important.
When one has a reference speaker around one can certainly say something about the sound of a speaker, just not if someone else will like it and if it will sound good in someone else's room.

 

[Romario]

Indeed, one need a reference to compare. The issue with measurements is that I can't see how the speaker's soundstage and imaging is. I have several pairs and believe it or not, the most expensive ones sound the best.

 

[solderdude]

With measurements (polar patterns) you can only see if they have a 'desirable' dispersion pattern.
How that works out in an actual room is an entirely different matter.

 

[MattHooper]

With measurements (polar patterns) you can only see if they have a 'desirable' dispersion pattern.
How that works out in an actual room is an entirely different matter.

Indeed.

And as I pointed up before, unless one is already familiar with having correlated such measurements to the subjective results, then the average person is just left looking at the numbers and graphs wondering “ OK, so what does it all mean? How does it sound?”

And that’s where the interface between translating measurements into their subjective consequences, which will involve subjective language to give impressions, comes in to play.

Or, in some cases, a perceptive listener can translate what they are hearing into subjective impressions.

Which reminds me: I’ve mentioned this before in another thread, but for a while many years ago I was obsessed with possibly purchasing MBL101 omni speakers. So I was able to hear them in a variety of set ups and rooms.

But in one set up I was turned off by a very odd character to the sound. It wasn’t there in the other set ups, but here this strange sound anomaly was very hard to ignore. (my brother, who was with me during that MBL audition, remarked on it as well.)

Then I read Michael Fremer later on the MBL 101s:

"When everything—or anything—was wrong, there was an odd chesty, compressed, almost grainy midbass coloration, and a sense that female singers centered between the speakers were performing in a closet in the next room. The offending sound was probably due to an unusual combination of placement and reflective interference."

Freemer put into words precisely the annoying artefact I heard. Like he taken the very impression right out of my head and put it into words.

I don’t know about anybody here, but I can’t look at the MBL measurements on Stereophile to predict that particular anomaly and precisely how it would sound when it occurred. So in that case it was nice to see another listener noting the same problem. And for someone reading the review thinking of seeking out those speakers, it’s a good
“ heads up, it’s possible to encounter this strange sonic artefact, but properly set up it’s a non-issue.”

When I had the smaller MBLs in my well treated room, I seem to remember this artefact briefly creeping in with one positioning of the speakers, but otherwise it was very easy to get clean sound with proper adjustment of the speakers in the room.

 

[Mart68]

Indeed.

And as I pointed up before, unless one is already familiar with having correlated such measurements to the subjective results, then the average person is just left looking at the numbers and graphs wondering “ OK, so what does it all mean? How does it sound?”

And that’s where the interface between translating measurements into their subjective consequences, which will involve subjective language to give impressions, comes in to play.

Or, in some cases, a perceptive listener can translate what they are hearing into subjective impressions.

Which reminds me: I’ve mentioned this before in another thread, but for a while many years ago I was obsessed with possibly purchasing MBL101 omni speakers. So I was able to hear them in a variety of set ups and rooms.

But in one set up I was turned off by a very odd character to the sound. It wasn’t there in the other set ups, but here this strange sound anomaly was very hard to ignore. (my brother, who was with me during that MBL audition, remarked on it as well.)

Then I read Michael Fremer later on the MBL 101s:

"When everything—or anything—was wrong, there was an odd chesty, compressed, almost grainy midbass coloration, and a sense that female singers centered between the speakers were performing in a closet in the next room. The offending sound was probably due to an unusual combination of placement and reflective interference."

Freemer put into words precisely the annoying artefact I heard. Like he taken the very impression right out of my head and put it into words.

I don’t know about anybody here, but I can’t look at the MBL measurements on Stereophile to predict that particular anomaly and precisely how it would sound when it occurred. So in that case it was nice to see another listener noting the same problem. And for someone reading the review thinking of seeking out those speakers, it’s a good
“ heads up, it’s possible to encounter this strange sonic artefact, but properly set up it’s a non-issue.”

When I had the smaller MBLs in my well treated room, I seem to remember this artefact briefly creeping in with one positioning of the speakers, but otherwise it was very easy to get clean sound with proper adjustment of the speakers in the room.

Whilst I don't disagree with the general thrust of your argument the MBL isn't a good example to pick with it being an omni design. It's going to be a lot harder to interpret the measurements into what might be heard, as with a dipole, and indeed you do mention this issue was room dependant.

In any case, is everyone not in agreement that measurements of speakers are useful only to the extent of weeding out the chaff when making a shortlist and that listening to them in the room they will be used in is always going to be the final arbiter?

 

[Blumlein 88]

Indeed.

And as I pointed up before, unless one is already familiar with having correlated such measurements to the subjective results, then the average person is just left looking at the numbers and graphs wondering “ OK, so what does it all mean? How does it sound?”

And that’s where the interface between translating measurements into their subjective consequences, which will involve subjective language to give impressions, comes in to play.

Or, in some cases, a perceptive listener can translate what they are hearing into subjective impressions.

Which reminds me: I’ve mentioned this before in another thread, but for a while many years ago I was obsessed with possibly purchasing MBL101 omni speakers. So I was able to hear them in a variety of set ups and rooms.

But in one set up I was turned off by a very odd character to the sound. It wasn’t there in the other set ups, but here this strange sound anomaly was very hard to ignore. (my brother, who was with me during that MBL audition, remarked on it as well.)

Then I read Michael Fremer later on the MBL 101s:

"When everything—or anything—was wrong, there was an odd chesty, compressed, almost grainy midbass coloration, and a sense that female singers centered between the speakers were performing in a closet in the next room. The offending sound was probably due to an unusual combination of placement and reflective interference."

Freemer put into words precisely the annoying artefact I heard. Like he taken the very impression right out of my head and put it into words.

I don’t know about anybody here, but I can’t look at the MBL measurements on Stereophile to predict that particular anomaly and precisely how it would sound when it occurred. So in that case it was nice to see another listener noting the same problem. And for someone reading the review thinking of seeking out those speakers, it’s a good
“ heads up, it’s possible to encounter this strange sonic artefact, but properly set up it’s a non-issue.”

When I had the smaller MBLs in my well treated room, I seem to remember this artefact briefly creeping in with one positioning of the speakers, but otherwise it was very easy to get clean sound with proper adjustment of the speakers in the room.

Seems likely you could do some in room measures over several locations in the room and figure out where this might happen. Had to be a response anomaly. If the MBLs are a fair simulation of a horizontal omni you could put one of them in the LP, and measure for places where the two speakers would work. Sort of like the subwoofer crawl idea only full range. So once again, even when some listening and describing sort of works you could do better understanding what is going on and dialing it in right.

 

[Newman]

At the end of the day, the purpose of speakers is to be listened to by human beings. So a perfectly objective evaluation of the performance is practically impossible.

Each human being puts different weight on different qualities, has different ideas about which imperfections they will accept, etc.

That is a romantic and seemingly logical statement, but if it were true, the work of Toole and Olive would have come to nothing, since we would all be so different.

Instead, they found that we pretty much all give the highest weighting to the same thing: absence of colouration. That includes flatness of frequency response, because anything not flat will not reproduce the rich harmonic structure of vocals and instruments intact, and that will sound coloured. Adding resonances will also sound coloured. Toole has written that the human ear is a remarkably sensitive and reliable instrument in this regard.

The real reason that we all appear to have different preferences on speakers is that we each have a unique set of non-sonic biases that we will mistakenly attribute to the sound waves when we engage in sighted listening, as is usual. Our actual preferences for sound waves are remarkably uniform.

Cheers

 

[sigbergaudio]

That is a romantic and seemingly logical statement, but if it were true, the work of Toole and Olive would have come to nothing, since we would all be so different.

Instead, they found that we pretty much all give the highest weighting to the same thing: absence of colouration. That includes flatness of frequency response, because anything not flat will not reproduce the rich harmonic structure of vocals and instruments intact, and that will sound coloured. Adding resonances will also sound coloured. Toole has written that the human ear is a remarkably sensitive and reliable instrument in this regard.

The real reason that we all appear to have different preferences on speakers is that we each have a unique set of non-sonic biases that we will mistakenly attribute to the sound waves when we engage in sighted listening, as is usual. Our actual preferences for sound waves are remarkably uniform.

Cheers

I assume we are both right in some regard. And it's also perhaps more about accepting imperfections rather than weighting different qualities. I agree that everyone would likely enjoy the theoretical perfect speaker. While that does not exist, we are left with speakers that have different imperfections. What imperfections we can live with differs somewhat based on taste in music, living situation, whether it is important to play loud or not, etc.

And through conversations with listeners I find some interesting differences with regards for instance to the importance but on things like soundstage and imaging. It appears not all experience that illusion equally. Some find it to be very impressive, others don't care much about it at all as long as the tonality is right.

To what extent people react to actual differences in speakers vs various biases is of course also difficult to judge.

I do however still not think it's just a matter of picking the speaker with the highest preference score and then calling it a day (and before being accused of strawman, I am not implying that is what you said).

 

[Axo1989]

I assume we are both right in some regard. And it's also perhaps more about accepting imperfections rather than weighting different qualities. I agree that everyone would likely enjoy the theoretical perfect speaker. While that does not exist, we are left with speakers that have different imperfections. What imperfections we can live with differs somewhat based on taste in music, living situation, whether it is important to play loud or not, etc.

And through conversations with listeners I find some interesting differences with regards for instance to the importance but on things like soundstage and imaging. It appears not all experience that illusion equally. Some find it to be very impressive, others don't care much about it at all as long as the tonality is right.

To what extent people react to actual differences in speakers vs various biases is of course also difficult to judge.

I do however still not think it's just a matter of picking the speaker with the highest preference score and then calling it a day (and before being accused of strawman, I am not implying that is what you said).

Nobody likes a strawmen, but reading those posts (and many more iterations of the same earnest homage) you wonder if that bloke's ever plotted survey data. Or seen an error bar.

 

[Roland68]

That's very true, but enjoying music is not the same as recommend any gear. The only thing one can do is to compare speakers ( one might be better than the other) but not as a single unit.

I agree that in principle it would be ideal if every reviewer had the unblemished hearing of a 20-year-old who has never attended a loud concert or shot firearms

Practically speaking though, you don’t need perfect hearing to be able to apprehend many salient characteristics about sound reproduction or loudspeakers.

After all, the hearing of ASR participants are likely all over the map, but I presume most are carefully vetting their equipment purchases on the assumption that they will still apprehend important sonic features of their sound system.

I mean, I certainly get your concern, but for me, not knowing somebody’s audiogram is not discrediting, so long as they seem to be accurately describing products I’m familiar with, which gives me some informal level of credence in their reports.

The basic problem is that everyone hears differently, because it is a learning process that begins at birth and is wired into the brain and fed with new experiences.
Among other things, I have noticed over the decades that there is a big difference whether someone has experience with how real instruments sound (live music, concerts, musicians, etc.) or whether someone has listened to music (almost) exclusively through loudspeakers/headphones in their life. The preferences/likes of these two groups can be very different.

A sentence that many people have taken offense at: "How can you classify the sound of a system, device, loudspeaker or headphones if you have never heard what it must sound like in reality?"

 

[Galliardist]

Nobody likes a strawmen, but reading those posts (and many more iterations of the same earnest homage) you wonder if that bloke's ever plotted survey data. Or seen an error bar.

Nobody likes an empty ad hominem either. Do you really need to pop up practically every time Newman posts, with comments like this?

 

[Blumlein 88]

The basic problem is that everyone hears differently, because it is a learning process that begins at birth and is wired into the brain and fed with new experiences.
Among other things, I have noticed over the decades that there is a big difference whether someone has experience with how real instruments sound (live music, concerts, musicians, etc.) or whether someone has listened to music (almost) exclusively through loudspeakers/headphones in their life. The preferences/likes of these two groups can be very different.

A sentence that many people have taken offense at: "How can you classify the sound of a system, device, loudspeaker or headphones if you have never heard what it must sound like in reality?"

The testing of loudspeaker preferences say otherwise. Regular people, studio people, musicians, audio reviewers, audio salesmen, and engineers even people from cultures with different native languages have the same preferences in speakers.