-
WANTED: Happy members who like to discuss audio and other topics related to our interest. Desire to learn and share knowledge of science required. There are many reviews of audio hardware and expert members to help answer your questions. Click here to have your audio equipment measured for free!
Standards for Audio Electronics Measurements?
- Thread starter amirm
- Start date
DanielT
Major Contributor
Unlike this fiasco. Plastic bicycle. It broke pretty much immediately. That the manufacturer did not test them before they started mass production is a mystery. As many as 30,000 absolute junk bikes were made.
There, a bicycle test standard on durability could have avoided the total fiasco.
....Boxes of bicycles piled up in shops. Among them were returns, mostly with broken parts. Where metal bends, plastic breaks. Getting replacements was not simple as the parts were incompatible with other bikes. Attempts to save the project were unsuccessful....
Plastic bicycle - Wikipedia
Edit:
OT
Incidentally, one of, perhaps the biggest, manufacturing fiasco in Sweden. The only advantage is that you can now visit the ship in a museum.
Vasa (ship) - Wikipedia
Last edited:
I'm all for documenting the exact conditions of the tests, as well as, the AP project files in one, easy to find, place on ASR.
Beyond that, defining some levels that a device must meet to be on the "recommended list" is likely to be counterproductive and will be misused. A recommendation is a subjective call, and just because something meets your minimum set of measurements doesn't mean it'll be good or bad for someone else. A 60dB SINAD that's dominated by distortion just might be perfectly fine for some, while others may prefer 120dB, even if they can't hear the difference. Certainly post the comparison/example charts for already measured devices, so the manufacturers can gauge where their device might fall relative to others. But drawing a line in the sand between recommended and not recommended is... not recommended
Beyond that, defining some levels that a device must meet to be on the "recommended list" is likely to be counterproductive and will be misused. A recommendation is a subjective call, and just because something meets your minimum set of measurements doesn't mean it'll be good or bad for someone else. A 60dB SINAD that's dominated by distortion just might be perfectly fine for some, while others may prefer 120dB, even if they can't hear the difference. Certainly post the comparison/example charts for already measured devices, so the manufacturers can gauge where their device might fall relative to others. But drawing a line in the sand between recommended and not recommended is... not recommended
mikitm
Member
- Joined
- Sep 9, 2021
- Messages
- 6
- Likes
- 5
I would say something like TFT Central does for their reviews.
[There’s the top ‘Recommended’ award, where a monitor is excellent and highly recommended by us. There is also an ‘Approved’ award for a very good screen which may not be perfect, but is still a very good display. These awards won’t be given out every time, but look out for the logo at the bottom of the conclusion section.]
We could also add the "Great Value" award if a product it's the best product in a price/performance ratio, like a DAC or AMP that costs $99 MSRP and performs so well it needs to be acknowledged by all.
[There’s the top ‘Recommended’ award, where a monitor is excellent and highly recommended by us. There is also an ‘Approved’ award for a very good screen which may not be perfect, but is still a very good display. These awards won’t be given out every time, but look out for the logo at the bottom of the conclusion section.]
We could also add the "Great Value" award if a product it's the best product in a price/performance ratio, like a DAC or AMP that costs $99 MSRP and performs so well it needs to be acknowledged by all.
beagleman
Major Contributor
I love the measurements. Mostly of the speakers though.
I will say this and surely get a lot of flack, but I think we as a forum have pushed the SINAD thing a bit too far.
As to it's actual importance, I feel it matters objectively, and in Design and Engineering and yes to show how well a product was conceived etc.
I see not a lot of correlation in SINAD in real life listening though. I have over the last few years, heard great, poor and just mediocre measuring devices in respects to SINAD, and never once felt one was far better or worse, or even in most cases even audibly different for sure.
But I have found, other forums in general tend to bash US here about the importance put on 2 things.
SINAD of electronics and Directivity of speakers.
My point, not sure how well some things are received by the audio community in "general".
Meaning, at least to me, do we focus on what matters to most people into audio, or what matters to "us" in here first and foremost?
I almost avoided even buying an AVR based on the bashing of AVRs in general and in respect to SINAD etc.
After now purchasing literally 3 on the used market, I found my fears almost completely unfounded.
I will say this and surely get a lot of flack, but I think we as a forum have pushed the SINAD thing a bit too far.
As to it's actual importance, I feel it matters objectively, and in Design and Engineering and yes to show how well a product was conceived etc.
I see not a lot of correlation in SINAD in real life listening though. I have over the last few years, heard great, poor and just mediocre measuring devices in respects to SINAD, and never once felt one was far better or worse, or even in most cases even audibly different for sure.
But I have found, other forums in general tend to bash US here about the importance put on 2 things.
SINAD of electronics and Directivity of speakers.
My point, not sure how well some things are received by the audio community in "general".
Meaning, at least to me, do we focus on what matters to most people into audio, or what matters to "us" in here first and foremost?
I almost avoided even buying an AVR based on the bashing of AVRs in general and in respect to SINAD etc.
After now purchasing literally 3 on the used market, I found my fears almost completely unfounded.
computer-audiophile
Major Contributor
Just another aspect, hopefully not taken in a negative light.
Audio Science Review seems to be closely tied to its founder as an individual. With all due respect! Truly!
What happens if he departs? Is there a plan in place for sustainability? Who will carry on with his approach?
Audio Science Review seems to be closely tied to its founder as an individual. With all due respect! Truly!
What happens if he departs? Is there a plan in place for sustainability? Who will carry on with his approach?
- Joined
- Nov 3, 2020
- Messages
- 627
- Likes
- 1,047
What about (desktop) DACs and AVRs? Are those in scope as well? For the latter it’s sadly needed!
- Joined
- Nov 3, 2020
- Messages
- 627
- Likes
- 1,047
I would recommend a heavy consideration on what others out there are measuring and pick one or two measurements that might help “draw in the crowd.” More specifically and importantly though, evaluate what ASR critics are posting/sharing in terms of measurements, not just expensive flowery words, see if there’s any true/objective merit there and include that in the minimum requirement and subsequent reviews. I know Amir has been adding a select few new tests here and there over time, so I think he has the right idea. We just need to make sure it provides value as it adds cost (time and money, even if volunteered by Amir) to every review!
mash
Active Member
- Joined
- Oct 8, 2020
- Messages
- 130
- Likes
- 274
One slight, change around the marketing of the concept....rather than presenting them to manufacturers as general standards (which sounds a little pretentious). Use them as "minimum" (not guaranteed) specs for a golfing panther recommendation. i.e. they are the standards you are using for your recommendations...not standards that you are suggesting the industry needs to follow.
The two challenges will be getting enough of them together for each category that they can reliably predict overall performance and figuring out the categories...as john pointed out, there are 1,001 categories these days related to the grouping of functions where a typical question will be........should the numbers for a small integrated desktop amp costing $150 be the same as a $2000 standalone?
The two challenges will be getting enough of them together for each category that they can reliably predict overall performance and figuring out the categories...as john pointed out, there are 1,001 categories these days related to the grouping of functions where a typical question will be........should the numbers for a small integrated desktop amp costing $150 be the same as a $2000 standalone?
Back in the day I longed for a HiFi satisfying DIN45500. My DUAL HS31 turntable/amp/speaker combo was 2W short of the 6W continuous required. Not sure about distortion. Discussed upgrading the power supply with my physics teacher. Must habe been 1970 or so.
The following may be self evident, but for clarity, there seems to be 3 separate categories in the specs discussed above:
1.minimum specs to allow standardization of measurement: you need every measured DAC (or preamp ‘out’) to have a minimum unbalanced output of 2V to fairly compare them. same for a std gain for an amp (not fair to compare the sinad for a 30dB gain amp vs a 17dB gain amp). These specs are driven by the test equipment and what have generally become industry standards (although Amir rightly points out standards are either not clearly established or followed). These specs would remain constant for the foreseeable future.
This is beneficial to the industry too. We see AVRs with preamp outs that putout less than 1V and distort badly above 0.8 V whereas the next piece of equipment may have an input sensitivity of >2V. So you need to crank the volume up if you want it loud enough, but then you’ll hear bad distortion.
2.criteria that you consider as desirable or a min benchmark (crosstalk sep>70dB, channel balance <0.5dB) etc. These benchmarks are more subjective. They are based on Amir’s experience, forum members input, on what good manufacturers have achieved at reasonable costs. I personally think these criteria are sometimes more stringent than needed (I sometimes enjoy listening to a vinyl, although based on Sinad, crosstalk, channel balance etc vinyl should be utterly intolerable). So although subjective, it’s very useful info to have for consumers as expensive equipment from well known manufacturer sometimes fall way short. The benchmarks in this category may change in the long term: the SINAD DACs can achieve now-days is way beyond what we had 15y ago so the benchmark has shifted.
3. Criteria that make equipment distortion and noise below audible threshold. These are absolute benchmarks. If a machine meets this, there is no way under any circumstance there will be any artifact introduced by the equipment. This is for the consumer ‘s peace of mind. No more urge to purchase new gear.
In reality, given that amps, speakers and room acoustics are orders of magnitude worse than a Dac, for most of us, SINADs do not need to be 120dB. Exceeding these specs (in my eyes at least) just gets you bragging rights.
The benchmarks in group 2 (ie these are the minimum specs you should meet) should be clearly below and distinct from group 3.
1.minimum specs to allow standardization of measurement: you need every measured DAC (or preamp ‘out’) to have a minimum unbalanced output of 2V to fairly compare them. same for a std gain for an amp (not fair to compare the sinad for a 30dB gain amp vs a 17dB gain amp). These specs are driven by the test equipment and what have generally become industry standards (although Amir rightly points out standards are either not clearly established or followed). These specs would remain constant for the foreseeable future.
This is beneficial to the industry too. We see AVRs with preamp outs that putout less than 1V and distort badly above 0.8 V whereas the next piece of equipment may have an input sensitivity of >2V. So you need to crank the volume up if you want it loud enough, but then you’ll hear bad distortion.
2.criteria that you consider as desirable or a min benchmark (crosstalk sep>70dB, channel balance <0.5dB) etc. These benchmarks are more subjective. They are based on Amir’s experience, forum members input, on what good manufacturers have achieved at reasonable costs. I personally think these criteria are sometimes more stringent than needed (I sometimes enjoy listening to a vinyl, although based on Sinad, crosstalk, channel balance etc vinyl should be utterly intolerable). So although subjective, it’s very useful info to have for consumers as expensive equipment from well known manufacturer sometimes fall way short. The benchmarks in this category may change in the long term: the SINAD DACs can achieve now-days is way beyond what we had 15y ago so the benchmark has shifted.
3. Criteria that make equipment distortion and noise below audible threshold. These are absolute benchmarks. If a machine meets this, there is no way under any circumstance there will be any artifact introduced by the equipment. This is for the consumer ‘s peace of mind. No more urge to purchase new gear.
In reality, given that amps, speakers and room acoustics are orders of magnitude worse than a Dac, for most of us, SINADs do not need to be 120dB. Exceeding these specs (in my eyes at least) just gets you bragging rights.
The benchmarks in group 2 (ie these are the minimum specs you should meet) should be clearly below and distinct from group 3.
beagleman
Major Contributor
But "Audible" can be measured in a dozen ways.
1. Audible with headphones cranked up in a sound proof room. That is often the standard for actual audibility.
2. Audible in a normal living room with very mild noise present.....probably what matters to 90% of people......
I call one "Real world audible" and the first "Lab scientific limit of audibility"
Last edited:
- Joined
- Apr 30, 2023
- Messages
- 1,654
- Likes
- 2,511
Sorry Amir, getting confused from what I am reading. Are you saying what the required spec of the product needs such that ASR can do a standardize measurement or are you saying what is the minimum measured performance ASR are expecting for a product to be an adequate and competently designed product?
The counter to this is that Amir's recommendations are mostly objective, based on measurements. The cleanest way to keep recommendations from being messy is to base them ONLY on measurements. Electrical & audio science standards that are met or exceeded. Standards tend to fail as soon as subjective thinking muddies the water. If someone likes the way a unit looks or is a fan of the company or price, or willing to overlook a standard not being met, they'll buy it. ASR's role, in my opinion, is to give some concrete scientific basis to those making purchase, and manufacturing, decisions. And by concrete, I mean the stuff that Amir's expensive equipment measures. No listening. No admiring the art. Just the facts.I'm all for documenting the exact conditions of the tests, as well as, the AP project files in one, easy to find, place on ASR.
Beyond that, defining some levels that a device must meet to be on the "recommended list" is likely to be counterproductive and will be misused. A recommendation is a subjective call, and just because something meets your minimum set of measurements doesn't mean it'll be good or bad for someone else. A 60dB SINAD that's dominated by distortion just might be perfectly fine for some, while others may prefer 120dB, even if they can't hear the difference. Certainly post the comparison/example charts for already measured devices, so the manufacturers can gauge where their device might fall relative to others. But drawing a line in the sand between recommended and not recommended is... not recommended
JayGilb
Major Contributor
I would like to see all of Amir's subjective comments be hidden under a "Spoiler" button in the review. Only objective measurements should be visible upon page load.
Another thing I would like to see is a chart or meter that shows the device's likelihood that it is producing audible distortion based on properly designed tests that
indicate the average person's audible thresholds.
Another thing I would like to see is a chart or meter that shows the device's likelihood that it is producing audible distortion based on properly designed tests that
indicate the average person's audible thresholds.
Hi Amir,
I'm wondering how, and to which manufacturers, you communicate your "minimum" requirements too?
What products are you thinking ... will you include say streamers, speakers and subs?
- Joined
- May 8, 2022
- Messages
- 448
- Likes
- 584
I sincerely hope that this does not happen for a long, long time. Given that ASR has now set the standard for excellence in Audio - Journalism, I suspect that they do have a plan of succession, either formally or informally. Your post reminds me of an old French saying : 'The graveyards of the world are littered with the bones of irreplacable men'.
It really doesn’t make sense for a forum to set the standards so much as record them. More data extrapolation could be very cool! Describing the differences and compromises between varying systems while using empirical data and pricing to evaluate the state of device categories makes a lot more sense than trying to define minimums. Realistically, we don’t want to be the new hi-res audio sticker, imo. It sets an unenforceable minimum standard and does not raise the bar. Having the review database put in work to determine extremely basic evaluations might be a cool way to continue to watch the bar raise.
I know that I have personally been mixed in my support for the way ASR does reviews. The data is awesome, but the valuations beyond the empirical haven’t been my favorite. There’s mixed long term testing, so like crinacle and oratory reviews, judgements on durability and long term use often have to be gathered from other users or reviews. Sometimes I disagree on the way value is interpreted, or aesthetic choices. Rubber stamping everything above the established lines may lead ASR to “recommend” things it otherwise wouldn’t or shouldn’t, perhaps.
It doesn’t matter what I want in this position, but ideally the role of ASR is to make itself irrelevant. To reveal manufacturing and design flaws that could legitimately be improved and have the value of the device itself increase without necessarily increasing the cost. When that work is complete, the only relevant facts about a device would be the price, function, reliability and aesthetics. Setting minimums may not accomplish that.
How would this data come across to manufacturers? Would it be a finish line to cross, like a goal or the start of another race? How would it be communicated and when? As far as I understand, the manufacturer will hand ASR a finished device for objective review and measurement more or less to promote their product, right? We on this end don’t really hear about Amir evaluating devices before launch, because it wouldn’t end up in the review pile, wouldn’t drive traffic, and probably would cost the manufacturer for use. It’s likewise the problem I see with assuming manufacturers even noticing an ASR standard in the first place. Hence the suggestion to find a more empirical way to evaluate recommendations and mean/median/mode the cost vs performance for people who care. Even something as simple as ranking the colored categories by price would be a step in the right direction, as any manufacturer who cares enough could use the tool to evaluate their market (us).
I know that I have personally been mixed in my support for the way ASR does reviews. The data is awesome, but the valuations beyond the empirical haven’t been my favorite. There’s mixed long term testing, so like crinacle and oratory reviews, judgements on durability and long term use often have to be gathered from other users or reviews. Sometimes I disagree on the way value is interpreted, or aesthetic choices. Rubber stamping everything above the established lines may lead ASR to “recommend” things it otherwise wouldn’t or shouldn’t, perhaps.
It doesn’t matter what I want in this position, but ideally the role of ASR is to make itself irrelevant. To reveal manufacturing and design flaws that could legitimately be improved and have the value of the device itself increase without necessarily increasing the cost. When that work is complete, the only relevant facts about a device would be the price, function, reliability and aesthetics. Setting minimums may not accomplish that.
How would this data come across to manufacturers? Would it be a finish line to cross, like a goal or the start of another race? How would it be communicated and when? As far as I understand, the manufacturer will hand ASR a finished device for objective review and measurement more or less to promote their product, right? We on this end don’t really hear about Amir evaluating devices before launch, because it wouldn’t end up in the review pile, wouldn’t drive traffic, and probably would cost the manufacturer for use. It’s likewise the problem I see with assuming manufacturers even noticing an ASR standard in the first place. Hence the suggestion to find a more empirical way to evaluate recommendations and mean/median/mode the cost vs performance for people who care. Even something as simple as ranking the colored categories by price would be a step in the right direction, as any manufacturer who cares enough could use the tool to evaluate their market (us).
Similar threads
