• WANTED: Happy members who like to discuss audio and other topics related to our interest. Desire to learn and share knowledge of science required. There are many reviews of audio hardware and expert members to help answer your questions. Click here to have your audio equipment measured for free!

Why Audio Science Review's measurements are so different compared to other sources?

Sgt. Ear Ache

Major Contributor
Joined
Jun 18, 2019
Messages
1,894
Likes
4,151
Location
Winnipeg Canada
Exactly - assuming all things are equal...

Definitely. There's often a complex stew of factors that goes into any buying decision. Measurements are one factor. Features, build quality, appearance...those are factors as well. oh, and price of course.
 
Last edited:

pozz

Слава Україні
Forum Donor
Editor
Joined
May 21, 2019
Messages
4,036
Likes
6,827
Apologies. I should have been more clear. I've seen it elsewhere, certainly not on ASR! I'll edit my post for clarity.

I've been a moderator of r/BudgetAudiophile for a few years. I've seen hundreds and hundreds of posts from newcomers (and the many replies to each of them...) wanting advice on piecing together their first steps into hi-fi.

The trend of newcomers having mistaken spending priorities is pretty real; we've seen it over and over.

It's particularly wacky with fans of a certain YouTuber whose moniker begins with a "Z" for reasons I don't quite understand. He doesn't actively recommend those kinds of odd priorities, but his fans seem the most likely to have them.
I still get your main point. We are as responsible for the misunderstandings and impressions and hyperSINADitis prescriptions. I'm just not sure how to address that in a larger way beyond answering questions as they come in.
 

majingotan

Major Contributor
Forum Donor
Joined
Feb 13, 2018
Messages
1,511
Likes
1,781
Location
Laguna, Philippines
I still get your main point. We are as responsible for the misunderstandings and impressions and hyperSINADitis prescriptions. I'm just not sure how to address that in a larger way beyond answering questions as they come in.

I can see the reason for that as newcomers don't really want to spend time to read as much technical info and/or user experiences (in terms of reliability, build quality, features, drivers/firmware etc.) in ASR and just see the highest numbers as a quick way of determining their chosen equipment for their budget
 

Jimbob54

Grand Contributor
Forum Donor
Joined
Oct 25, 2019
Messages
11,066
Likes
14,699
No. A lot of serious musicians and engineers are on that forum, and a lot of amateurs.

The science is less emphasized than here, but there is a good collection of experience that's worth reading as well. You end up having to do a lot of sifting. Plus they have no Thomas to moderate. Massive disadvantage.
Moderating Thomas seems like a challenge in and of itself to me ;-)
 

Sgt. Ear Ache

Major Contributor
Joined
Jun 18, 2019
Messages
1,894
Likes
4,151
Location
Winnipeg Canada
One final thing I'd add, though, is that my concern with audibility is not abstract. I have seen newcomers to the hobby putting together systems where they spend like $200 on the DAC, $200 on an amp, and $79 on some Pioneer speakers and things like that. (edit for clarity: I've seen that elsewhere, not on ASR)

And really, you hate to see that. I hate to see that nearly as much as seeing folks squander money on pure snake oil like "audiophile" USB cables. I want to see them succeed and get the best result for their money and effort. While ASR is pretty explicitly not a beginner-oriented site, beginners do read it and jump to some wacky conclusions due to the figures published here.

I agree with the gist of what you're saying, but I think speakers present a really interesting conundrum. First, they are the element in the chain that is probably the most constrained by our physical situation. Am I buying for my office or bedroom? My living room? House or apartment? My specific situation sort of dictates what I need or really what I can even use in terms of speakers. Secondly, with speakers I think it's possible to get decent performance at almost every price point and the difference between decent and very good is pretty...errr...what's the word I'm looking for?? Elusive? I think it's entirely possible to find $200 speakers that sound every bit as good as decent $400 speakers once EQ'd in a given room. Audio is such a strange realm really. I honestly believe that using digital files from a laptop as a source, one could totally put together a $1000 system that would be essentially indistinguishable from say a $5000 ($10,000?) system in a volume-matched blind listening comparison.
 

raistlin65

Major Contributor
Forum Donor
Joined
Nov 13, 2019
Messages
2,279
Likes
3,421
Location
Grand Rapids, MI
Apologies. I should have been more clear. I've seen it elsewhere, certainly not on ASR! I'll edit my post for clarity.

I've been a moderator of r/BudgetAudiophile for a few years. I've seen hundreds and hundreds of posts from newcomers (and the many replies to each of them...) wanting advice on piecing together their first steps into hi-fi.

The trend of newcomers having mistaken spending priorities is pretty real; we've seen it over and over.

It's particularly wacky with fans of a certain YouTuber whose moniker begins with a "Z" for reasons I don't quite understand. He doesn't actively recommend those kinds of odd priorities, but his fans seem the most likely to have them.

One of the big issues with this is that DACs are often being evaluated in many Internet reviews using headphones. Or even with only measurements, in terms of what is theoretically accurate for the dynamic range of the media or within the range of human hearing.

So let's assume for a moment that DAC A, with lower SINAD, is audible when wearing headphones compared to DAC B, which would be considered transparent by any metric. And let's assume the difference is significant (since to a certain extent that difference is subjective).

With the noise floor of a room when used with speakers, both will sound the same. And how often does any Internet review ever make the case that in a room that difference won't matter?

Consequently, it would be great if there as a little chart at the end of ASR reviews that described each of the color codings for SINAD. Something like "Likely transparent for CD audio in a room at X volume. May be audible with headphones." Or some such thing.
 

pozz

Слава Україні
Forum Donor
Editor
Joined
May 21, 2019
Messages
4,036
Likes
6,827
I can see the reason for that as newcomers don't really want to spend time to read as much technical info and/or user experiences (in terms of reliability, build quality, features, drivers/firmware etc.) in ASR and just see the highest numbers as a quick way of determining their chosen equipment for their budget
There's definitely a contingent of buyers who want to go for the top of the list, no matter the reasoning.

Maybe an article on the limits of what you can expect from stereo is needed to address what some may think is an infinity of improvements.
 
OP
KeithPhantom

KeithPhantom

Addicted to Fun and Learning
Forum Donor
Joined
May 8, 2020
Messages
641
Likes
658
I still get your main point. We are as responsible for the misunderstandings and impressions and hyperSINADitis prescriptions. I'm just not sure how to address that in a larger way beyond answering questions as they come in.
Personally, I use measurements up to a point, after I make sure the device is pretty much transparent, I start looking into design and features.
 

Victor Martell

Active Member
Joined
Mar 9, 2018
Messages
178
Likes
217
It is the largest forum for pros in the music recording business. Back in the beginning it was mostly full time pros only. Very respected and rightly so at the time as a forum. True not all pros are big on science, but they make their living with what they do. And yes OTOH some are very big on science.

In the business with so many new never before possible products due to digital, and gear lust for older gear too, GearSLUTZ was very appropriate. You've done a big time award winning platinum recording people want to know, "what mic pre was that, what compressor did you use, what mike on the vocals". Of course it was a bit of a joke name too. They regularly told people, it is more about know how and who does a recording than what gear being used. Real pros make good recordings, and wankers can't get a good jingle out of Abbey Road.

Now with the influx of affordable quality recording interfaces, affordable good microphones (do you realize how expensive condenser mikes of any kind once were?), and the music business being in decline leading to lots of amateurs and self recorded bands topics have lower Signal to noise than once was the case. Still lots of good info if you pay attention.

Ah indeed - the one thing I at the same time dread and find funny in GearSlutz and similar sites is the cliche - just before someone is about to drop some "wisdom" on you, they always start: "I am a PRODUCER and a DJ...."

:D
 

Racheski

Major Contributor
Forum Donor
Joined
Apr 20, 2020
Messages
1,116
Likes
1,699
Location
Chicago
I still get your main point. We are as responsible for the misunderstandings and impressions and hyperSINADitis prescriptions. I'm just not sure how to address that in a larger way beyond answering questions as they come in.
My suggestion for the SINAD Wars/hyperSINADitis (that's gold btw) would be to leverage the work that @flipflop has done on the Audibility thresholds of amp and DAC measurements and add benchmarks to the SINAD histogram that is currently color-coded by quartile. The quartiles are arbitrary, and what I have seen happen is a new user asks for a DAC recommendation and someone suggests to pick any DAC in the blue range of the SINAD graph. Then another user will chime in that any DAC in the green and above is audibly transparent, and the debate about the SINAD level for audible transparency ensues. The chart also gives newbies the impression that higher SINAD is always better, and if they don't end up taking the time to dive a little bit deeper into the user experience that SINAD actually translates to, I think they draw a conclusion that most members of ASR would not want. I also think (and this is obviously debatable), that little SINAD histogram has influenced manufacturer's product development for DACs to top the chart with the higher SINAD, i.e. hyperSINADitis. Personally I would prefer manufacturers to compete on price and other features that very few budget DACs have like EQ and streaming capabilities, instead of spending time integrating the latest and greatest D/S chip.

My specific suggestion is to pick a SINAD threshold for audible transparency where the average listener will practically never experience any distortion, and color code DACs above that threshold green and DACs below that threshold as red. I manage an operational analytics team so I am hyper-sensitive to the design of visualizations because I understand the influence on the conclusions drawn and behavior changes they can have on end-users. You wouldn't believe the minutiae I get into about seemingly simple choices such as the color of an icon, but when standardizing scorecards across hundreds of people very small changes can have large downstream effects.

Your idea to write an article on the limits of stereo could help, but I think that will be preaching to the choir because the folks taking the time to thoroughly read through it are not the ones I'm concerned about.
 

JohnBooty

Addicted to Fun and Learning
Forum Donor
Joined
Jul 24, 2018
Messages
632
Likes
1,555
Location
Philadelphia area
Consequently, it would be great if there as a little chart at the end of ASR reviews that described each of the color codings for SINAD. Something like "Likely transparent for CD audio in a room at X volume. May be audible with headphones." Or some such thing.

I absolutely love this idea!

I tried something different, but in the same spirit with a speaker review I posted a few weeks back. The speakers sounded quite nice to me despite measuring poorly. In a search for answers I realized it was because they measured accurately in the octaves where the fundamental frequencies of most music happens.

I made this super rudimentary (ugly, but sort of functional) graphic in addition to the frequency response graph:
1593627931570.png


If I do another speaker review, I'd probably come up with a way to overlay this graphic directly onto the speaker's frequency response chart. As well as perhaps highlighting the typical male and female vocal frequencies.

In general, I think anything that helps to tie objective measurements back to reality (as we experience it in a practical sense) is a good thing.
 
Last edited:

Jimbob54

Grand Contributor
Forum Donor
Joined
Oct 25, 2019
Messages
11,066
Likes
14,699
My suggestion for the SINAD Wars/hyperSINADitis (that's gold btw) would be to leverage the work that @flipflop has done on the Audibility thresholds of amp and DAC measurements and add benchmarks to the SINAD histogram that is currently color-coded by quartile. The quartiles are arbitrary, and what I have seen happen is a new user asks for a DAC recommendation and someone suggests to pick any DAC in the blue range of the SINAD graph. Then another user will chime in that any DAC in the green and above is audibly transparent, and the debate about the SINAD level for audible transparency ensues. The chart also gives newbies the impression that higher SINAD is always better, and if they don't end up taking the time to dive a little bit deeper into the user experience that SINAD actually translates to, I think they draw a conclusion that most members of ASR would not want. I also think (and this is obviously debatable), that little SINAD histogram has influenced manufacturer's product development for DACs to top the chart with the higher SINAD, i.e. hyperSINADitis. Personally I would prefer manufacturers to compete on price and other features that very few budget DACs have like EQ and streaming capabilities, instead of spending time integrating the latest and greatest D/S chip.

My specific suggestion is to pick a SINAD threshold for audible transparency where the average listener will practically never experience any distortion, and color code DACs above that threshold green and DACs below that threshold as red. I manage an operational analytics team so I am hyper-sensitive to the design of visualizations because I understand the influence on the conclusions drawn and behavior changes they can have on end-users. You wouldn't believe the minutiae I get into about seemingly simple choices such as the color of an icon, but when standardizing scorecards across hundreds of people very small changes can have large downstream effects.

Your idea to write an article on the limits of stereo could help, but I think that will be preaching to the choir because the folks taking the time to thoroughly read through it are not the ones I'm concerned about.

I fear your green/ red split may have worse unintended consequences than the colour quartiles ever could. Can't put my finger on why quite yet but it makes me uneasy. Also, the "line of audibility" would be the subject of more endless debates than we currently have.
 

JohnBooty

Addicted to Fun and Learning
Forum Donor
Joined
Jul 24, 2018
Messages
632
Likes
1,555
Location
Philadelphia area
My specific suggestion is to pick a SINAD threshold for audible transparency where the average listener will practically never experience any distortion, and color code DACs above that threshold green and DACs below that threshold as red. I manage an operational analytics team so I am hyper-sensitive to the design of visualizations because I understand the influence on the conclusions drawn and behavior changes they can have on end-users. You wouldn't believe the minutiae I get into about seemingly simple choices such as the color of an icon, but when standardizing scorecards across hundreds of people very small changes can have large downstream effects.

Your idea to write an article on the limits of stereo could help, but I think that will be preaching to the choir because the folks taking the time to thoroughly read through it are not the ones I'm concerned about.
I don't know if I like the idea of a specific hard cutoff point between "green" and "red" but generally I like this idea a lot - the key to me would be integrating this sorts of information into graphs themselves if it is to have an impact.

(One challenge among many with a hard "audibility cutoff point" for DACs would be the fact that distortion in an audio chain tends to be cumulative, right? Each component in the audio chain would need to be above the transparency threshold in order for the entire chain to be...)
 

Racheski

Major Contributor
Forum Donor
Joined
Apr 20, 2020
Messages
1,116
Likes
1,699
Location
Chicago
I fear your green/ red split may have worse unintended consequences than the colour quartiles ever could. Can't put my finger on why quite yet but it makes me uneasy. Also, the "line of audibility" would be the subject of more endless debates than we currently have.
If folks are uncomfortable with only a R/G split, the typical alternative is to add a yellow - just not sure where the cut-off between yellow and red would be. Yes it could lead to endless debates on the thresholds for each color, but if our fearless leader put his foot down after an initial spirited discussion, they would be kept to a minimum.
 

Jimbob54

Grand Contributor
Forum Donor
Joined
Oct 25, 2019
Messages
11,066
Likes
14,699
If folks are uncomfortable with only a R/G split, the typical alternative is to add a yellow - just not sure where the cut-off between yellow and red would be. Yes it could lead to endless debates on the thresholds for each color, but if our fearless leader put his foot down after an initial spirited discussion, they would be kept to a minimum.

I think its the fact you're trying to get away from SINAD being everything (correct), yet at the sametime reinforcing that below a certain SINAD is not worth it. More a line problem than a colour problem I think.
 

Racheski

Major Contributor
Forum Donor
Joined
Apr 20, 2020
Messages
1,116
Likes
1,699
Location
Chicago
I think its the fact you're trying to get away from SINAD being everything (correct), yet at the sametime reinforcing that below a certain SINAD is not worth it. More a line problem than a colour problem I think.
More trying to communicate that above a specific threshold, SINAD has no practical benefit because the DAC is audibly transparent, so all the DACs above this threshold should be the same color. In other words, trying to classify SINAD more as a Categorical variable instead of a Continuous variable.
Green = Audibly Transparent and highly unlikely to introduce any distortion or noise in an audio signal chain.
Yellow = could indicate that the SINAD level is above audible transparency, but there is a decent probability that in the average audio-chain the DAC will introduce distortion due to cumulative effects - not sure if that makes any sense - I'm running with the idea made in recent posts here.
Red = The SINAD level of the DAC indicates audible distortion.
 

Jimbob54

Grand Contributor
Forum Donor
Joined
Oct 25, 2019
Messages
11,066
Likes
14,699
More trying to communicate that above a specific threshold, SINAD has no practical benefit because the DAC is audibly transparent, so all the DACs above this threshold should be the same color. In other words, trying to classify SINAD more as a Categorical variable instead of a Continuous variable.
Green = Audibly Transparent and highly unlikely to introduce any distortion or noise in an audio signal chain.
Yellow = could indicate that the SINAD level is above audible transparency, but there is a decent probability that in the average audio-chain the DAC will introduce distortion due to cumulative effects - not sure if that makes any sense - I'm running with the idea made in recent posts here.
Red = The SINAD level of the DAC indicates audible distortion.

Ha, regardless of any change or not, I think we all agree reading the full review and having at least a passing understanding of other measures is preferable than rocking up and asking "which DAC should I buy?"
 

JohnBooty

Addicted to Fun and Learning
Forum Donor
Joined
Jul 24, 2018
Messages
632
Likes
1,555
Location
Philadelphia area
Ha, regardless of any change or not, I think we all agree reading the full review and having at least a passing understanding of other measures is preferable than rocking up and asking "which DAC should I buy?"
Absolutely, BUT....

In the name of pragmatism, let's also consider that the future of our hobby depends on attracting newcomers, who by definition start out completely uninformed.

Most will not be lazy "tell me what to buy" types, but all will be uninformed. :)

(Also, a lot of the thoughtful and polite ones, who don't ask such obnoxious and lazy questions, will nonetheless read the replies to those lazy "tell me what to buy posts" as they search for newbie-friendly information...)
 

krabapple

Major Contributor
Forum Donor
Joined
Apr 15, 2016
Messages
3,169
Likes
3,717
Maybe I'm not giving it a fare shake by this, but the name is enough to completely turn me off. The audiophile community (ASR much less than other places, just speaking more broadly) as a whole has a serious exclusivity problem about a broad swath of issues; age, gender, race, class, etc. Anything that walls off something like the joy of listening to and making music that should be open to all is not a good thing. And people will complain about this sector dying due to 'lack of young people' picking up the hobby. It's kind of mind boggling. Maybe the musician and engineering side is more open, but with a forum name like that, I doubt it.

Gearslutz has been around since 2002, for heaven's sake. You sound like you don't know anything about it.
 
Top Bottom