Why Audio Science Review's measurements are so different compared to other sources?

[idiomatically]

Keith, as for part 2 of your question around hidden agenda’s. I don’t think it’s a sinister as that, rather a case of vested interest thats probably gotten out of control at Head-Fi. I certainly think its dishonest, but maybe due to cognitive bias more then a deliberate plan to deceive or I could be wrong and they know exactly what they are doing and are just out to line their own pockets with cash for comments.

I’ll be honest I don’t understand half the measurements Amir provides here at ASR, what I do know is my experiences using and listening to gear seem to match the results on ASR. That is I have never heard something sound good that didn’t test well here and vice versa.

Now why I don’t trust Head-Fi is pretty simple I have seen them extol the virtues of products, tell the world how awesome they are only for it to turn out the product was soo poorly engineered it was withdrawn from the market for over 6 months to be redesigned from the ground up. Yet if you believed the initial review on Head-Fi it was the best thing going, and measured fantastically.

I have also never seen Amir sling dirt at other people, but others like Head-Fi are quick to play the man and not the ball. They attack Amir personally rather then approach things pragmatically or with a civil tone. They jump on anyone that questions them. I am quite happy to be apart of the ASR community and not the Head-Fi community which I find zealot like.

When I went some subjective entertainment I watch Darko’s YouTube channel and leave it at that. If I were a betting man I‘d bet on Amirs methods every day of the week just based on how I have seen him conduct himself over the past year and his background.

 

[tomtoo]

Sorry for not being specific, but I found something with a quick Google search (if I am not exact or make a mistake in regards to something, just let me know)

This is what Amir got from the Yggdrasil in regards to FR:
View attachment 71202

This is what Jude got:
View attachment 71203

In term of harmonic distortion and (possible) mains hum:

Amir:
View attachment 71204

prr1n from SBAF:
View attachment 71206

I would like to reiterate that if I make a mistake, I wrongly interpret something, or I miss some information, I would like to be corrected where necessary and thanks for your time.

At the end you have to ask your self who grabs manufactures harder by the balls?

But if you like measurements that just show nice results. than maybe you dont like @amirs measurements.

 

[KeithPhantom]

But if you like measurements that just show nice results

No, I do not like always nice results, I like the truth.

 

[tomtoo]

No, I do not like always nice results, I like the truth.

The truth?! Easy spoken word but hard to get. With your examples i would say @armir is nearer at the truth.

 

[pozz]

The truth?! Easy spoken word but hard to get. With your examples i would say @armir is nearer at the truth.

I think he's being earnest. Better to be direct than beat around the bush.

People are having arguments about ASR on other forums all the time. Just saw one Gearslutz (worth reading the whole page): https://www.gearslutz.com/board/product-alerts-older-than-2-months/1301562-aps-klasik-2020-a-3.html

 

[ZolaIII]

I have no doubts whatsoever in Amir's measuremets. We had this kind of let's say discussion couple of times before & @WolfX-700 jumped in with same analyser and confirmed it with very similar results. Actually his ware usually a tad better but that's contributed to internal chain (MB and power supply) and it best shows how even smallest things can make a difference. Everyone else is pretty much doing this with worser equipment and less methodical aprouch. I even suggested documenting serial numbers of mesured gear but whose explained all do that would solve measurements transparency in some cases would be impossible (Chinise stuff without SN) and in others stigmatic for those manufacturers which equipment did bad. It would also violate privacy of owners who sent gear for measurements and registrated the product. In short that's a no go.

In the end it's up to you who you choose to trust.

 

[SimpleTheater]

LOL! No. THX is a marketing thing with minimal hurdles.

Can you back that up? My impression is very different.

According to THX, there are four classes of performance, the lowest being: THX Certified Compact products for smaller sized rooms, up to 1,000 cubic feet in size, with an 8-foot viewing distance from the screen.

I don't need @amirm to measure these Klipsch ProMedia at 8 ft to know they're garbage. But if you want to buy them, send them in for testing, I'll gladly eat crow if they measure well.

 

[pozz]

I don't need @amirm to measure these Klipsch ProMedia at 8 ft to know they're garbage. But if you want to buy them, send them in for testing, I'll gladly eat crow if they measure well.

I used the older ProMedia 2.1 around 15 years ago. I liked them more than any other "computer" speakers sold then, but ended up replacing them a year or so later with active monitors for my desktop.

I'd like to see them measured just to see what it was that I found so attractive about them back them.

 

[Koeitje]

I think he's being earnest. Better to be direct than beat around the bush.

People are having arguments about ASR on other forums all the time. Just saw one Gearslutz (worth reading the whole page): https://www.gearslutz.com/board/product-alerts-older-than-2-months/1301562-aps-klasik-2020-a-3.html

I think many people there are missing the point that measurements can predict real life performance. They also make it seem Amir only focuses on frequency response, while that is clearly not the case. Regardless I would still say it starts at frequency response and everything else comes after that (distortion, spl etc).

 

[pozz]

I think many people there are missing the point that measurements can predict real life performance. They also make it seem Amir only focuses on frequency response, while that is clearly not the case.

A lot of those guys make purchases based on advice and listening only. I remember trying that approach early on and then realizing the futility when confronted with 30+ possibilities at the store.

Many others will "half read" reviews and comments and so on. Pretty common thing. Anyone who really understands how to read graphs wouldn't say stuff like that.

 

[tmtomh]

I don't want to make this too much about individuals, but I will say that to me it seems like @KeithPhantom began this thread in good faith (even if the title was sort of a misleading question) and has continued to participate in good faith. Not saying he's right or wrong about anything in particular he's written - just saying his interest in this topic seems genuine to me.

As for folks at Head-Fi and elsewhere who trash Amir/ASR, I would have to agree that most of the time it's cognitive bias, or more plainly just a desire to be left alone with one's biases. Even if one's preferred gear measures excellently, I think a lot of folks don't like the idea that a centralized site exists where their equipment preferences, their personal setups (and the money they've invested in them), and their listening acumen can potentially be called into question anytime if Amir happens to test a piece of kit and it comes out poorly. They won't admit it, but shooting the messenger is absolutely the point of the criticisms. Much easier to seize on an individual measurement technique that Amir might have done 3 years ago and long ago tweaked or adjusted, and turn that into a generalized claim that he "doesn't know what he's doing" or "is out to get manufacturers" than to ask oneself why, if listening is your gold standard, you even care what his measurements turn out to be of the gear that you own.

It's the same reason you find so many vinyl and tube aficionados who cannot be content with the idea that they simply like how such gear sounds - so many of them also feel compelled to insist that they are higher fidelity than other equipment, despite how poorly they measure.

 

[SimpleTheater]

I think many people there are missing the point that measurements can predict real life performance. They also make it seem Amir only focuses on frequency response, while that is clearly not the case. Regardless I would still say it starts at frequency response and everything else comes after that (distortion, spl etc).

I apologize for not knowing who said this, but my favorite phrase (paraphrase) is "If it measures badly it will sound badly. If it measures well, it might just sound good."

 

[JohnBooty]

I used the older ProMedia 2.1 around 15 years ago. I liked them more than any other "computer" speakers sold then, but ended up replacing them a year or so later with active monitors for my desktop.I'd like to see them measured just to see what it was that I found so attractive about them back them.

I think you hit the nail on the head already: they sounded better than any other "computer" speakers sold back then! I had a similar experience with them. For a lot of us of a certain age they were our first experience with anything remotely resembling a full-range audio experience. They had dedicated tweeters and a subwoofer.... haha!

They were actually pretty bad, but they were leaps ahead of their contemporaries, and opened our ears and let us know that there were probably even better speakers out there...

I apologize for not knowing who said this, but my favorite phrase (paraphrase) is "If it measures badly it will sound badly. If it measures well, it might just sound good."

The most common criticism of ASR, and one I agree with, is that Amir handed out a lot of "not recommended" reviews to amps/DACs that "measured badly" ...in ways that were just literally inaudible, even by Amir's own admission.

This doesn't affect my enjoyment of ASR, because I generally know enough to look at the numbers and know what differences might be audible and which aren't.

But it's understandably upsetting for manufacturers (particularly those who cater to the same niche market that reads ASR) to see their products slammed for inaudible flaws. I have also seen a lot of newbies confused by ASR's numbers, thinking that they really need a DAC with 120dB+ of SINAD or whatever if they actually want something that sounds good.

 

[SimpleTheater]

I think you hit the nail on the head already: they sounded better than any other "computer" speakers sold back then! I had a similar experience with them. For a lot of us of a certain age they were our first experience with anything remotely resembling a full-range audio experience. They had dedicated tweeters and a subwoofer.... haha!

They were actually pretty bad, but they were leaps ahead of their contemporaries, and opened our ears and let us know that there were probably even better speakers out there...

Makes me think of Sam Adams beer. It was my first foray back in the late 1980's into beer better than the crap being sold. I haven't bought it in a couple of decades, but thank that company for opening my eyes to what was possible.

 

[Speedskater]

I apologize for not knowing who said this, but my favorite phrase (paraphrase) is "If it measures badly it will sound badly. If it measures well, it might just sound good."

Not necessarily, some audiophiles are always searching for a better sound like the Golden Ears & reviewers do. So if it measures badly it may well sound different, and for those audiophiles that are searching, different is close enough to better for them.

 

[pozz]

The most common criticism of ASR, and one I agree with, is that Amir handed out a *lot* of negative reviews to amps/DACs that "measured badly" ...in ways that were just literally inaudible, even by Amir's own admission.

This doesn't affect my enjoyment of ASR, because I generally know enough to look at the numbers and know what differences might be audible and which aren't.

But it's understandably upsetting for manufacturers (particularly those who cater to the same niche market that reads ASR) to see their products slammed for inaudible flaws. I have also seen a lot of newbies confused by ASR's numbers, thinking that they really need a DAC with 120dB+ of SINAD or whatever if they actually want something that sounds good.

Manufacturers were used to a certain environment and certain tactics, and really had a controlling hand overall for dictating information and sentiment through all of their commercial relationships, one small example of which are reviewers. They never expected consumers to use their own niche products against them (I mean analyzers like the AP, which aren't mainstream in terms of availability or price, and were known just to insiders). What's really new is a systematic way to compare products against each other.

I think the other thing to notice is that the flipside to manufacturer control of information is the uninformed consumer who has no idea how to judge the quality of what he's buying once advertising is taken out of the picture (advertising is meant to inform as much as it is meant to persuade; that first function has just weakened given the strength of marketing studies, which are the business equivalent of applying an APx555 to consumer sentiment and market movements).

 

[Soniclife]

The most common criticism of ASR, and one I agree with, is that Amir handed out a *lot* of negative reviews to amps/DACs that "measured badly" ...in ways that were just literally inaudible, even by Amir's own admission.

This doesn't affect my enjoyment of ASR, because I generally know enough to look at the numbers and know what differences might be audible and which aren't.

But it's understandably upsetting for manufacturers (particularly those who cater to the same niche market that reads ASR) to see their products slammed for inaudible flaws. I have also seen a lot of newbies confused by ASR's numbers, thinking that they really need a DAC with 120dB+ of SINAD or whatever if they actually want something that sounds good.

I don't agree that he was too negative, as in most cases of things measuring badly they came with stories from the manufacturer of how great they were, and were charging for great. On the audibility point I doubt anyone else online has put as much effort into explaining this, and the only reason most of us knew it was below audible thresholds were he had explained it to us in the first place.

 

[pozz]

On the audibility point I doubt anyone else online has put as much effort into explaining this, and the only reason most of us knew it was below audible thresholds were he had explained it to us in the first place.

+1

All of this is marching us headlong towards the Audible Precision analyzer of the future.

 

[Sgt. Ear Ache]

If the thing measuring poorly costs $2000 and the thing measuring excellently costs $200, I think some negativity is warranted whether the issues are audible or not.

 

[Racheski]

People are having arguments about ASR on other forums all the time. Just saw one Gearslutz (worth reading the whole page): https://www.gearslutz.com/board/product-alerts-older-than-2-months/1301562-aps-klasik-2020-a-3.html

Threads like that frustrate me because it reveals the misinformation floating out there. I think folks who dismiss Amir's measurements fall into a couple of buckets, ordered from subjectivist to objectivist:

(1) They are anti-measurement (fundamental subjectivist) and do not want to discuss measurements whatsoever, and believe the only important aspect of evaluating audio equipment is how it sounds, e.g. "The only measurement device I trust are my ears."
(2) They believe measurements are irrelevant and are more interested in discussing listening impressions, and will openly assert they only care about what they hear, e.g. "Measurements do not explain how audio equipment sounds."
(3) They are agnostic about the measurements and are more interested in discussing listening impressions, so they ignore them and are not interested in learning about their significance, e.g. "Yeah those measurements look interesting, but how does Y sound compared to X?"
(4) They believe measurements are somewhat important to understand for audio gear, but believe most of ASR's measurements are inaudible anyways, e.g. "I don't care about this measurement because any difference above this number is inaudible and/or it's unnecessarily hurting the image of manufacturers."
(5) They believe measurements are important to understand for audio gear, but believe ASR's measurements are biased and/or incompetent, e.g. "I think measurements are important, but I don't trust that ASR guy because (insert idiotic reason)."
(6) They believe measurements are the most important (fundamental objectivist) aspect for audio gear, but believe ASR's measurements are biased and/or incompetent, e.g. "Measurements are the most important factor, but I don't trust that ASR guy because (insert idiotic reason)."

Of course these will vary based on the type of equipment measured, DAC/Amp/Speaker etc...
I actually don't think I have read any posts that resemble #6, but maybe more experienced members have.