• WANTED: Happy members who like to discuss audio and other topics related to our interest. Desire to learn and share knowledge of science required. There are many reviews of audio hardware and expert members to help answer your questions. Click here to have your audio equipment measured for free!

AudioScienceReview discussion on AS.

Katji

Major Contributor
Joined
Sep 26, 2017
Messages
2,990
Likes
2,273
Sounds like Zen. Zen isn't science.

Listen carefully. Think carefully. It is what it is. I said that for good reason.

It is not speculation, implied "subtext" or whatever.
The methods and process are explained.

Not a matter of "sounds like Zen" or "sounds like" whatever. Not mental speculation.
 
Last edited:

rdenney

Major Contributor
Forum Donor
Joined
Dec 30, 2020
Messages
2,235
Likes
3,856
I don't get it. I have come here with hope of getting an objective, scientific audio advice. You guys, as I understand it, are against subjective listening tests cause they are unreliable and you base your knowledge on 'audio science' which suggests that by using different measurements it is possible to objectively, scientifically, one could say, evaluate audio equipment. If so - why can't I simply get an information; Kef LS50 is a better speaker than KRK Rockit 5 but instead I just get the information that there is only a '66% chance on average' that it's better? That's all Amirt's $28 000 measuring equipment can say? How am I supposed to choose good audio equipment then?
The statement is that 66% of the people in the model that is the basis for the score would prefer it. Not that it is "better". This is not a moral question--it's just statistics from a study. Given the scatter in those data, the score itself shows precision in excess of accuracy in predicting what the listeners modeled would actually prefer.

And there is no concern here about subjective data. The concern is about uncontrolled subjective opinions being instructive to others. The score model is based on controlled subjective testing, but it is, of course a model based on somewhat noisy empirical data. All models are false, even if some are useful. That said, the model is good enough that listeners of all types generally prefer speakers that score better, as long as the difference in score is sufficient for statistical significance. In summary, it's a general guide but no model can tell you or me what we do or don't want in a speaker.

The reverse question is: If there is a preference for Speaker X vs. Speaker Y that all audiophiles of sophistication and experience would understand, then shouldn't that be demonstrable using controlled testing? If not, why not?

The Klippel measurements and the spinorama, on the other hand, measure speaker behavior, not listener preferences. How we interpret that is up to us. But Toole and Olive used spin data for testing preferences using blind testing, and found that smoother and flatter frequency response, the smoothness and flatness of which is preserved off-axis, sound better to everyone (from experts to novices). I really don't see how that result should shock anyone. The key result of their research seems to me the importance of directivity in addition to on-axis response, which was not as well understood in prior times.

Rick "this is not a moral question and does not need moral judgments" Denney
 

Midwest Blade

Senior Member
Joined
May 8, 2019
Messages
401
Likes
535
Thanks for stopping in to say hello Jabberwalkie.
Please understand we all enjoy the search for great sound quality and have a very different way to get there preferring an objective and measured approach. This is not any sort of audio conspiracy, just a group verifying through fact what we hear.
 

BDWoody

Chief Cat Herder
Moderator
Forum Donor
Joined
Jan 9, 2019
Messages
6,948
Likes
22,625
Location
Mid-Atlantic, USA. (Maryland)
I don't get it. I have come here with hope of getting an objective, scientific audio advice. You guys, as I understand it, are against subjective listening tests cause they are unreliable and you base your knowledge on 'audio science' which suggests that by using different measurements it is possible to objectively, scientifically, one could say, evaluate audio equipment. If so - why can't I simply get an information; Kef LS50 is a better speaker than KRK Rockit 5 but instead I just get the information that there is only a '66% chance on average' that it's better? That's all Amirt's $28 000 measuring equipment can say? How am I supposed to choose good audio equipment then?

This place will be of biggest benefit to those who understand what is behind the charts and the graphs. There is no preference score that will ever be 100% accurate.

This also isn't always the best place for easy answers, because understanding the answers can involve more work than most are interested in doing, especially when it just doesn't match up with what people have heard before.

Easiest example of this would be sampling theory... It's not intuitive, but when understood it removes the FOMO when it comes to crazy sample rates.

The more you understand the graphs and charts, for both the solid state stuff and the speakers, the easier it will be to translate that into what's helpful for you.

Speakers especially are going to be very room dependent, and because of the nature of the world, the ultimate test for Speakers remains a listening test with your ears in your room.

It isn't that subjective tests are always going to be unreliable...we still need them. However, without listening controls (matching volumes and not peeking to oversimplify) the results are going to be unreliable due to our completely normal human bias.

Well made solid state devices, generally speaking, are somewhat different in that their performance can be very well predicted.

Welcome!
 
Last edited:

dc655321

Major Contributor
Joined
Mar 4, 2018
Messages
1,597
Likes
2,235
So you claim that less than 66% of people on average would prefer LS50 ofer KRK, right?

If the algorithm estimates that probability, I have nothing to refute it except opinion. As do you. I've no idea where that figure comes from, but it doesn't sound outlandish to me.

Are you a ls50 owner by chance? You seem hurt about this idea...
 

Wes

Major Contributor
Forum Donor
Joined
Dec 5, 2019
Messages
3,843
Likes
3,788
I don't get it. I have come here with hope of getting an objective, scientific audio advice. You guys, as I understand it, are against subjective listening tests cause they are unreliable and you base your knowledge on 'audio science' which suggests that by using different measurements it is possible to objectively, scientifically, one could say, evaluate audio equipment. If so - why can't I simply get an information; Kef LS50 is a better speaker than KRK Rockit 5 but instead I just get the information that there is only a '66% chance on average' that it's better? That's all Amirt's $28 000 measuring equipment can say? How am I supposed to choose good audio equipment then?

Use the scores to generate a short list.

2nd, do an "objective" listening test - that means use matched levels and blind testing to ensure that your subjective assessment is base don what you hear with your ears, not what you see

3rd, pick something with good ergonomics and that you like the looks of

- or swap #2 and #3 above
 

Sancus

Major Contributor
Forum Donor
Joined
Nov 30, 2018
Messages
2,923
Likes
7,616
Location
Canada
I don't get it. I have come here with hope of getting an objective, scientific audio advice. You guys, as I understand it, are against subjective listening tests cause they are unreliable and you base your knowledge on 'audio science' which suggests that by using different measurements it is possible to objectively, scientifically, one could say, evaluate audio equipment. If so - why can't I simply get an information; Kef LS50 is a better speaker than KRK Rockit 5 but instead I just get the information that there is only a '66% chance on average' that it's better? That's all Amirt's $28 000 measuring equipment can say? How am I supposed to choose good audio equipment then?

I'm starting to think you're just another troll. The difference is that a subjective assessment from an "audio review magazine" or a trash site like audiophilestyle.com gives you a 0% chance because they've never done any research to show their methodology can tell a good speaker from a barn door.

If you want to learn something, the forum search function is there, Amir's videos, reviews, and posts are all there.

No one around here is proposing that the science is perfect, it is just BETTER THAN NO SCIENCE.

If the algorithm estimates that probability, I have nothing to refute it except opinion. As do you. I've no idea where that figure comes from, but it doesn't sound outlandish to me.

It's actually a bit more than that I'm just too lazy to do the actual math. It's because 1 standard deviation in error from the original study == 0.8. This was discussed somewhere around here in this thread.
 
Last edited:

amirm

Founder/Admin
Staff Member
CFO (Chief Fun Officer)
Joined
Feb 13, 2016
Messages
44,368
Likes
234,386
Location
Seattle Area
I'm far from being an expert but I think it's difficult to disagree in 100% with eg this guy's (who claims he used to be an audio dealer) logic and arguments, I even wonder what would @amirm say:
AS post
You mean this?

"The KRK's get the score of 3.6 according to Amir and I'm ok with that - they were IMO the worst of the three active speakers I compared with each other. Adam Audios which both me and Amir preferred to JBLs get 4.2 points. I might be ok with that if it wasn't for the fact that the JBLs get 4.6 points... This is the moment in which he clearly shows he prefers speakers with the lower score - AAs.. "

I don't publish the preference scores. They are computed by others in the forum. While I think directionally the score is very useful, in the specific they are not to me. I have explained many times why. Here is one reason: I listen for example to power handling and have given otherwise excellent Genelec speakers lower score because they bottom out. Ditto for KH80 Neumann.

From start, I have never liked a preference score with fractions. Whole numbers is about as far as I like to take it.

Also, keep in mind that preference score was computed based on Harman's measurement scheme, not mine. We have never ironed out the difference between scores that Klippel NFS generates and their anechoic measurements. I bought one speaker to test this and that has been it. So you have to assume some degree of error.

This is why I listen to speakers. It is my way of seeing if the measurements are correct in assessing preference or not. Listening time is NOT 10 seconds as they incorrectly mention. Minimum time is half hour, often a full hour. It is a rigorous process, controlling many variables. This listening test disagrees with the score at times. That is the way it should be as neither is foolproof.

My goal is provide a high confidence evaluation of a speaker, not a guarantee. Other sites relying on purely subjective evaluation are producing entertainment, not reliable data. Everything I say in subjective results can be analyzed based on objective data. This is not the case when the reviewer doesn't give you proper measurement data.
 

noobie1

Active Member
Joined
Feb 15, 2017
Messages
230
Likes
155
Location
Bay Area
I don't get it. I have come here with hope of getting an objective, scientific audio advice. You guys, as I understand it, are against subjective listening tests cause they are unreliable and you base your knowledge on 'audio science' which suggests that by using different measurements it is possible to objectively, scientifically, one could say, evaluate audio equipment. If so - why can't I simply get an information; Kef LS50 is a better speaker than KRK Rockit 5 but instead I just get the information that there is only a '66% chance on average' that it's better? That's all Amirt's $28 000 measuring equipment can say? How am I supposed to choose good audio equipment then?

1) You learn to interpret the data and understand which equipments measure well

2) Create a short list of equipment that measure well, include functionalities you need, and priced within your budget

3) Demo as many of the products in your short list as possible

4) Buy the one you like the best using any subjective criteria you prefer
 

MZKM

Major Contributor
Forum Donor
Joined
Dec 1, 2018
Messages
4,240
Likes
11,462
Location
Land O’ Lakes, FL
You mean this?

"The KRK's get the score of 3.6 according to Amir and I'm ok with that - they were IMO the worst of the three active speakers I compared with each other. Adam Audios which both me and Amir preferred to JBLs get 4.2 points. I might be ok with that if it wasn't for the fact that the JBLs get 4.6 points... This is the moment in which he clearly shows he prefers speakers with the lower score - AAs.. "

I don't publish the preference scores. They are computed by others in the forum. While I think directionally the score is very useful, in the specific they are not to me. I have explained many times why. Here is one reason: I listen for example to power handling and have given otherwise excellent Genelec speakers lower score because they bottom out. Ditto for KH80 Neumann.

From start, I have never liked a preference score with fractions. Whole numbers is about as far as I like to take it.

Also, keep in mind that preference score was computed based on Harman's measurement scheme, not mine. We have never ironed out the difference between scores that Klippel NFS generates and their anechoic measurements. I bought one speaker to test this and that has been it. So you have to assume some degree of error.

This is why I listen to speakers. It is my way of seeing if the measurements are correct in assessing preference or not. Listening time is NOT 10 seconds as they incorrectly mention. Minimum time is half hour, often a full hour. It is a rigorous process, controlling many variables. This listening test disagrees with the score at times. That is the way it should be as neither is foolproof.

My goal is provide a high confidence evaluation of a speaker, not a guarantee. Other sites relying on purely subjective evaluation are producing entertainment, not reliable data. Everything I say in subjective results can be analyzed based on objective data. This is not the case when the reviewer doesn't give you proper measurement data.
Also, all those are near-field monitors, of which the formula was not made from.

Also, it is a “tonal balance preference”, it indeed says nothing about power handling. As that’s not what researchers needed to understand, of course more power handling and max SPL is better, that can be achieved almost regardless of tonal balance, the same goes for distortion.

To bad that other than Joel, no other reviewer has a ranking system, as then I could do a large comparison with their rankings and the preference formula.
 

MZKM

Major Contributor
Forum Donor
Joined
Dec 1, 2018
Messages
4,240
Likes
11,462
Location
Land O’ Lakes, FL
I haven't done the most recent speakers, but here is the correlation:

pubchart


Besides the Revel M55XC outdoor speaker, pretty well correlated in that the higher the price, the higher the calculated preference to get recommended (the F328Be had the inaccuracy in the bass, so it likely would score higher, I think Amir said he will remeasure them sometime in the future).

pubchart
 
Last edited:
OP
jabberwalkie

jabberwalkie

Member
Joined
Feb 26, 2021
Messages
10
Likes
5
Thanks for the friendly welcome and all your posts, guys. I think I've got some material to think over but it's got late. I also hope that in case of some further questions I can count on your clarifications. Cheers!
 
OP
jabberwalkie

jabberwalkie

Member
Joined
Feb 26, 2021
Messages
10
Likes
5
This is why I listen to speakers. It is my way of seeing if the measurements are correct in assessing preference or not.
Why don't you listen to amplifiers or DACs then if I may ask? Wouldn't a listening test be a good empirical check if the measurements are correct in assessing preferences in their case as well?

for example the Kef LS50 is not guaranteed to be preferred over the KRK, it's only about a 66% chance on average
If I understand that correctly, since JBL 305 and KEF LS50 have the same score, in their case there would be equal chances of being preferred, right? Aren't KEF speakers much better speakers than JBLs, do you really think guys that majority wouldn't choose Kef when given such an alternative?
 

dc655321

Major Contributor
Joined
Mar 4, 2018
Messages
1,597
Likes
2,235
Why don't you listen to amplifiers or DACs then if I may ask? Wouldn't a listening test be a good empirical check if the measurements are correct in assessing preferences in their case as well?

I believe headphone DAC and amps are listened to, likely because it's fairly easy to test that form factor.
Loudspeaker electronics are not always practical to configure for a listening session.

Do you believe there are aspects of electronic devices' performance that cannot be gleaned by looking at the electronic signals produced?
IOW, what additional information would an electronic-to-acoustic conversion reveal?

EDIT: obviously ignoring look/feel/usability here.

Aren't KEF speakers much better speakers than JBLs, do you really think guys that majority wouldn't choose Kef when given such an alternative?

I think that an interesting experiment would be to compare the two, same listening position, volume-matched and no peeking.
Preferences would certainly be arrived at.
But it may be a coin toss which speaker a majority would prefer when tested this way...
 

Robin L

Master Contributor
Joined
Sep 2, 2019
Messages
5,209
Likes
7,588
Location
1 mile east of Sleater Kinney Rd
Most if not all audio magazines review cables, do you suggest it's a conspiracy?
It's not a conspiracy. If you keep getting ad revenue from a particular company, you print their ads. You want to want to be in alignment with the ads if you want to continue getting ad revenue. You don't print "These wires are expensive bullshit, but buy them anyway, it makes our salaries happy". It's all too out in the open and too obvious to be called a "conspiracy". This is what is known as business as usual.
 

Wes

Major Contributor
Forum Donor
Joined
Dec 5, 2019
Messages
3,843
Likes
3,788
If I understand that correctly, since JBL 305 and KEF LS50 have the same score, in their case there would be equal chances of being preferred, right? Aren't KEF speakers much better speakers than JBLs, do you really think guys that majority wouldn't choose Kef when given such an alternative?

You should certainly do a double or single blinded listening test with matched levels using a multi-meter. Women often love to see guys unable to determine such things, and that can be used to recruit a helper.

You may not find a preference. If you do, then buy that one.
 

rdenney

Major Contributor
Forum Donor
Joined
Dec 30, 2020
Messages
2,235
Likes
3,856
Why don't you listen to amplifiers or DACs then if I may ask? Wouldn't a listening test be a good empirical check if the measurements are correct in assessing preferences in their case as well?

If I understand that correctly, since JBL 305 and KEF LS50 have the same score, in their case there would be equal chances of being preferred, right? Aren't KEF speakers much better speakers than JBLs, do you really think guys that majority wouldn't choose Kef when given such an alternative?
"KEF speakers are much better speakers than JBLs" is a value judgment based on brand awareness, perhaps. I would not necessarily choose KEF over JBL--it would depend on many other things as well.

For example, I love my Advents but am always looking at speakers. One particular pair of relatively inexpensive Revel tower speakers came to my attention. Yes, they measure pretty well. But much more important to me was their bass response and their maximum SPL capabilities. So, I looked at the user manual, and discovered that they can be biwired in ways I had never thought about. One method was "vertical" where two stereo amps are used. Each amp drives the woofers out of one channel and the mid and tweeter out of the other channel. The other method was "horizontal", where one stereo amp is used to drive left and right woofers and the other stereo amp is used to drive left and right mids and tweeters (which is the more common way). In both cases, the amps go through the internal driver filters (aka crossovers), so as long as the amps are identical, no external crossover is required. Thus, I could use my current pair of stereo amps, which I now use to drive two pairs of Advent NLA's. The amps are not bridgeable, nor are they particularly happy with really low impedance despite their high-current designs. Do I want extreme channel separation as with dual monos? Or do I want the bass amp to be uncluttered by the requirement to drive the mids and tweeters (and vice versa)? Those speakers are designed to give me that choice. I found myself in deep lust after those speakers just because they would take advantage of my two unbridgeable amps. But all that makes no difference at all if they measure poorly or wouldn't physically work in my room.

I've owned a lot of DACs, ranging from ChiFi cheapies to one found on the Stereophile Recommended list (though still a budget DAC). They all sound the same to me, comparing a CD rip (through one DAC) to the CD itself (through a different DAC). I trust my ears, too. If I heard a difference where what we know of psychoacoustics suggest I shouldn't, then I would first have to prove that I can hear that difference consistently and repeatably. Blind testing is one way to do that. Then, we can explore what is insufficient about the measurements. But we can't complain that the measurements are inadequate until we can demonstrate that our perceptions are repeatable, and not dominated by what we already know about what we are listening to.

John Atkinson tells a story, which I mentioned in (what I hope is) a different thread, where he participated in a blind test and discovered that he could not tell the difference between good amps. So, in a fit of penury he sold his whatever and bought a different whatever because it was cheaper (the different whatever may have been a Quad). He then found himself not listening to music as much, so he went back to the first whatever amp. Then, he found himself listening happily again. This tells me that the visual and the prior knowledge about an amp is an important factor in our enjoyment. The placebo effect is real, when it comes to perceptions. There's nothing wrong with that. But it is not instructional to others. I did not respect JA any less because he admitted preference for an amp that did not sound better to him in controlled blind testing. In fact I respected him even more.

Rick "who'd be buying those Revel speakers if he could think of a way to arrange tower speakers in his open-plan music room" Denney
 

Xulonn

Major Contributor
Forum Donor
Joined
Jun 27, 2018
Messages
1,828
Likes
6,311
Location
Boquete, Chiriqui, Panama
Correlation is not a binary function - it is a spectrum.
Indeed - correlations range from spurious to accurate. They are useful as indicators of possibilities.


Spurious Correlation.jpg
 
Last edited:

Blumlein 88

Grand Contributor
Forum Donor
Joined
Feb 23, 2016
Messages
20,522
Likes
37,053
Indeed - correlations range from spurious to accurate. They are useful as indicators of possibilities.
View attachment 115516
Looks fine to me. Older Miss America contestants were obviously hotter being more mature. That hotness spreads all over the place and you get a handful of extra murders due to the super hotness of fully mature Miss America winners accidentally going over the steam and hot vapours threshold.
 

StefaanE

Addicted to Fun and Learning
Forum Donor
Joined
May 12, 2020
Messages
528
Likes
929
Location
Harlange, Luxembourg
Top Bottom