How we could finally pin down flowery audiophile subjective descriptions

[Jim Taylor]

TAKE MEASUREMENTS.

 

[solderdude]

TAKE MEASUREMENTS.

chicken - egg problem. To make measurements have any meaning to individuals with their own 'vocabulary' they need to know how to convert squiglies to their vocabulary (understand a full measurement suite) and that takes quite a bit of knowledge, experience. Even if 'agreed upon' descriptors are used (as an explanation along with the measurements) this might not be fully understood by people with a very different vocabulary. They would have to use the elaborate 'descriptors descriptions' list to understand what the 'agreed upon' descriptors means.

For ASR, for instance, such a vocabulary could be agreed upon.... maybe.

 

[raif71]

Many of us throw up our hands when equipment is described as "fast", "slow", "crisp", "warm", etc. It seems impossible to relate these terms to measurable characteristics.

I have a slightly more optimistic view, in that subjective descriptions must be correlated with what people hear, and what people hear tends to be correlated (imperfectly) to measurable output.

To translate subjective descriptions into objective measurements (or the other way around, which might be more interesting), I propose that a machine learning model could be used.

The model would correlate subjective terms used (how often people say a speaker is "crisp", for example) with the measurements of the equipment relative to the median measurements of all equipment in its category.

The output would be a map of how semantically close certain audiophile words are to each other, and how use of those words correlates with measurable characteristics. Imagine a word cloud that groups words like "sharp, bright, tinny" together, and then displays measured characteristics that correlate with those terms - elevated response above 4khz, above-average H3 distortion in that range, etc.

This would be interesting in its own right, but if the ML was sophisticated enough, and fed enough data, it might also reveal trends in preference / subjective experience that go beyond the current preference score models. For example, you might unexpectedly find that some aspect of vertical directivity correlates with "warmth" or "speed", for example. I don't know.

I don't have anywhere near the skills to execute such a project, but it seems like a way to solve the "problem" of people using flowery language that, to many of us, is currently worse than useless. It might also reveal that things audiophiles consider to be "beyond science" are actually very well correlated with simple measurements. Which would be progress in and of itself.

I like that some of us will depend on this AI machine to interpret audio data into flowery words like we're putting all our "hopes" on the APx555.

 

[Axo1989]

Devil's advocate here: I take 'plankton' to mean 'very fine detail in the recording'. Perception of "plankton" would therefore depend on a good transient response and very low noise and distortion, particularly IMD. So I think you can relate even something like "plankton" to measurable quantities. The difficulty comes in knowing whether anyone interprets a colorful term the same way.

Plankton is a great analogy tbh.

Yeah!

View attachment 242715

I really like a lot of music that sounds like that.

This would be interesting in its own right, but if the ML was sophisticated enough, and fed enough data, it might also reveal trends in preference / subjective experience that go beyond the current preference score models. For example, you might unexpectedly find that some aspect of vertical directivity correlates with "warmth" or "speed", for example. I don't know.

That would be the fun bit.

Personally I think the problem is less when people use words, more when they use them as meaningless cliche.

 

[Ricardus]

chicken - egg problem. To make measurements have any meaning to individuals with their own 'vocabulary' they need to know how to convert squiglies to their vocabulary (understand a full measurement suite) and that takes quite a bit of knowledge, experience. Even if 'agreed upon' descriptors are used (as an explanation along with the measurements) this might not be fully understood by people with a very different vocabulary. They would have to use the elaborate 'descriptors descriptions' list to understand what the 'agreed upon' descriptors means.

For ASR, for instance, such a vocabulary could be agreed upon.... maybe.

It's even worse than the chicken egg thing.

The point is, the measurements give us the vocabulary we need. Numbers. DONE.

 

[solderdude]

It's even worse than the chicken egg thing.

The point is, the measurements give us the vocabulary we need. Numbers. DONE.

So.. how would someone, that hasn't got the faintest idea what the plot below tells the initiated (the numbers), know if they might like the sound of this headphone ?
Just telling a casual reader that it should be 'following the dotted line' as close as possible (without showing a tolerance band) is not enough IMO.
There are loads of people liking this headphone and plenty that would never buy it based on the numbers below only.

Would you buy it ?

This is what Amir said about it:

AKG K271 Listening Tests
While not perfect, out of the box tonality was close to what I expected to hear. My reference tracks immediately sounded (almost) right.

Would anyone (even the initiated) come to the same sound description as Amir based on the plots above ?

How would someone 'describe' the sound of this headphone based on the numbers ?

 

[Ricardus]

So how would someone that hasn't got the faintest idea what the plot below tells the initiated (the numbers) have to decide if they would like the sound of this headphone ?

I don't think these discussions even matter. Subjectivity is subjectivity. It cannot be agreed upon. Which is why we have measurements.

People should just go listen and buy. Some will realize their ears are the weakest links in the listening chain and some won't.

There simply is NO WAY to describe audio to another person when we live in a world where people think 3 foot lengths of AC cable makes your system sound different.

If it measures good it is good. Good specs are the definition of high fidelity. FULL STOP.

 

[Jim Taylor]

How would someone 'describe' the sound of this headphone based on the numbers ?

Someone wouldn't. The numbers "speak" for themselves. In other words, the numbers are a language, and that language is separate and different from the language we use to speak to each other.
And like any language, you have to learn it to be fluent in it.

Jim

 

[solderdude]

I agree with everything you write.
But the casual reader does not know 'measures' good means. Nor do many others.
Gear that does not measure good can still sound good, maybe not to very critical people but to mortals.

Let me put it this way. Amir measured the K271 and listened to it and wrote:

AKG K271 Listening Tests
While not perfect, out of the box tonality was close to what I expected to hear. My reference tracks immediately sounded (almost) right.

They might well buy it and think... wow for little money I have a headphone that sounds (almost) right.
Someone that only looks at the numbers (plots) would definitely not buy it based on the plots because... very poor measurements.

What is measuring good ? What tolerances and what aspects need to comply to which 'standards'. How much deviation is allowed and to whom ?
What is perfectly acceptable ? What are the limits for what measurements.

This is the difficult part. What is 'measuring good',what is 'good enough'

 

[phoenixdogfan]

All of these words describe the subjective, inner-world impression and reactions of the listener to an objective phenomenon. As such their capability of mapping to anyone else's inner world impressions and reactions, let alone objectively measured data points, is tenuous at best, and almost certainly not worth the effort unless you're a fan of subjective audio reviewer poesy because that's exactly what it is.

 

[solderdude]

Someone wouldn't. The numbers "speak" for themselves. In other words, the numbers are a language, and that language is separate and different from the language we use to speak to each other.
And like any language, you have to learn it to be fluent in it.

Jim

While that is true the vast majority of people that listen to music is not audiophile minded (music fidelity addicts) and only a small portion of those actually understand the numbers and a much smaller number of them is 'fluent' in interpreting the numbers. The vast majority of 'music consumers' are not going to learn 'numbers language'. They want to read 'flowery and positive' words of 'reviewers' instead. So those words, in practice, are more important than 'numbers' to the vast majority of people. They don't care about 'numbers'.
So a good 'translation' or 'vocabulary/description' is very important.
This 'numbers language' is no option for every day life. Flowery wording and 'descriptions' are a more universal language that is easier to learn.
A 'vocabulary' with descriptions is easier to understand than learning to interpret measurements and taking that skill far beyond 'mount stupid'.

Yes, numbers are a language and when understood say a lot more than the many 'flowery wording vocabularies' as these all differ anyway. But not everyone that goes on vacation has learned to speak fluent Swahili, not even when often visiting the countries it is spoken in.

 

[Jim Taylor]

While that is true the vast majority of people that listen to music is not audiophile minded (music fidelity addicts) and only a small portion of those actually understand the numbers and a much smaller number of them is 'fluent' in interpreting the numbers. The vast majority of 'music consumers' are not going to learn 'numbers language'. They want to read 'flowery and positive' words of 'reviewers' instead. So those words, in practice, are more important than 'numbers' to the vast majority of people. They don't care about 'numbers'.
So a good 'translation' or 'vocabulary/description' is very important.
This 'numbers language' is no option for every day life. Flowery wording and 'descriptions' are a more universal language that is easier to learn.
A 'vocabulary' with descriptions is easier to understand than learning to interpret measurements and taking that skill far beyond 'mount stupid'.

Yes, numbers are a language and when understood say a lot more than the many 'flowery wording vocabularies' as these all differ anyway. But not everyone that goes on vacation has learned to speak fluent Swahili, not even when often visiting the countries it is spoken in.

I agree .... somewhat. There is, however, a catch to your logic. You are correct in saying that " ... the vast majority of people that listen to music is not audiophile minded ...". Therefore, the "flowery and positive words" are of even less utility to them than it is to us. I see the disconnect more within the "audiophile" community rather than with people off the street.

As for "music consumers" not caring about numbers, I believe that you're wrong. I believe that they do care about numbers ... very much. I believe they trust numbers far more than "flowery language." But when they get the feeling that someone is trying to mislead them, they trust neither numbers or language. They want someone they can trust. That was the whole premise behind the formation of ASR.
And as far as the reference to "mount stupid", I also believe that normal, average everyday people are far more intelligent and learn far quicker than you may surmise. Remember; there was a day when none of us could even talk. Our maturation, our innate curiosity and the availability of information changed that. Give people a chance. They might surprise you. Learning doesn't stop at the end of our schooling.

And finally ..... as for something (anything) being "easier to learn" ...... I believe that "easy" does not translate to any certain degree of "useful", "accurate" or "repeatable". As such, it is far from being "universal".

I understand your point. After all, this problem has been around for many years, and if it were easy to solve, there would certainly have been a resolution before now. There hasn't been .... at least not one that is useful and repeatable.

Jim

 

[Mr. Widget]

I have two questions.

1. What the heck is "rhythm and pace" in an audio system? Most of the other flowery language I can pretty wells suss out, but this has me stumped.

2. Why do ASR's reviews often include a purely subject "Rate this device" poll? At best it is a measurement of how much agreement there is in the value of the measured objective performance. It seems antithetical to the main premise of this forum. If it was a poll of the subjective performance of the device under test by actual users of the device that would potentially add real value.

 

[Jim Taylor]

. Why do ASR's reviews often include a purely subject "Rate this device" poll? At best it is a measurement of how much agreement there is in the value of the measured objective performance. It seems antithetical to the main premise of this forum. If it was a poll of the subjective performance of the device under test by actual users of the device that would potentially add real value.

Basically, it's the popcorn that goes with the movie.

Jim

 

[Martin Takamine]

You don't trust that they can hear if they don't hear so well?

Has nothing to do with trust. It's just another set of measurement to compare to my capability. I know the gaps in my hearing because of annual audiograms. Around 4K I need a little boost.

 

[pseudoid]

...To translate subjective descriptions into objective measurements...

I think what you are trying to achieve is similar to the puzzle which has been baffling marketeers/researchers for decades, as to what makes a hit.
This question may be related to your other post "...try[ing] to classify all audiophiles..." thread.
Are you exploring blending such worthy audio discussions/topics with AI to take marketing to a higher plateau?

 

[Blumlein 88]

Has nothing to do with trust. It's just another set of measurement to compare to my capability. I know the gaps in my hearing because of annual audiograms. Around 4K I need a little boost.

I was being facetious.

 

[Axo1989]

TAKE MEASUREMENTS.

It's even worse than the chicken egg thing.

The point is, the measurements give us the vocabulary we need. Numbers. DONE.

I don't think these discussions even matter. Subjectivity is subjectivity. It cannot be agreed upon. Which is why we have measurements.

People should just go listen and buy. Some will realize their ears are the weakest links in the listening chain and some won't.

There simply is NO WAY to describe audio to another person when we live in a world where people think 3 foot lengths of AC cable makes your system sound different.

If it measures good it is good. Good specs are the definition of high fidelity. FULL STOP.

Your keyboard is playing up?

If not, yep, hurling some boldface and all caps at the innumerates will sort 'em out.

You appear to have self-selected out of any discussion [of the topic]. I'm going to read NTK's linked paper, might be interesting.

 

[Robbo99999]

Many of us throw up our hands when equipment is described as "fast", "slow", "crisp", "warm", etc. It seems impossible to relate these terms to measurable characteristics.

I have a slightly more optimistic view, in that subjective descriptions must be correlated with what people hear, and what people hear tends to be correlated (imperfectly) to measurable output.

To translate subjective descriptions into objective measurements (or the other way around, which might be more interesting), I propose that a machine learning model could be used.

The model would correlate subjective terms used (how often people say a speaker is "crisp", for example) with the measurements of the equipment relative to the median measurements of all equipment in its category.

The output would be a map of how semantically close certain audiophile words are to each other, and how use of those words correlates with measurable characteristics. Imagine a word cloud that groups words like "sharp, bright, tinny" together, and then displays measured characteristics that correlate with those terms - elevated response above 4khz, above-average H3 distortion in that range, etc.

This would be interesting in its own right, but if the ML was sophisticated enough, and fed enough data, it might also reveal trends in preference / subjective experience that go beyond the current preference score models. For example, you might unexpectedly find that some aspect of vertical directivity correlates with "warmth" or "speed", for example. I don't know.

I don't have anywhere near the skills to execute such a project, but it seems like a way to solve the "problem" of people using flowery language that, to many of us, is currently worse than useless. It might also reveal that things audiophiles consider to be "beyond science" are actually very well correlated with simple measurements. Which would be progress in and of itself.

If this even would work, how would you get enough data to make relevant conclusions, and how would you set up a machine learning algorithm to do it. Surely there's not enough organised data in order for this to work. Let alone the different rooms that people listen in and other variables - but I get that if you have enough data then some of that could be averaged out. I just don't know how the system would work to gather the data in first place, and secondly would there be enough data, and thirdly how do you design the algorithm. I don't know much about machine learning, but it seems like there are some sticking points.

 

[Axo1989]

If this even would work, how would you get enough data to make relevant conclusions, and how would you set up a machine learning algorithm to do it. Surely there's not enough organised data in order for this to work. Let alone the different rooms that people listen in and other variables - but I get that if you have enough data then some of that could be averaged out. I just don't know how the system would work to gather the data in first place, and secondly would there be enough data, and thirdly how do you design the algorithm. I don't know much about machine learning, but it seems like there are some sticking points.

Just call it Full Self Listening and start taking orders.