Standards for Audio Electronics Measurements?

[amirm]

You wonder if it's time to publish guidelines, but in practise you're already applying them in your reviews. So it would indeed be more transparent to summarise and publish them. That would also support the public discussion about the validity of the criteria. As such it would also be nice to document the rationale with each criterium.

I only have one justification for such specs: there are already products that meet these specs and exceed them in many case.

Let me give another example. I reviewed a dongle that got great recommendation from me. Company then makes a "Pro" version which output half as much power!!! They were quite surprised when I told them they went backward. Even after seeing the review, they had not internalized what was it that was great about their own products.

 

[testp]

i vote, excellent idea!

 

[gsp1971]

Amir, great idea.
My humble opinion is that:
(a) the minimum thresholds / benchmarks should be set to establish only whether ASR "places the product on the recommended list", and
(b) shouldn't the the audibility threshold come into play in setting those minimum benchmarks?

Thanks,
GS

 

[SSS]

There existed and exist audio equipment standards since long time ago. In Germany started with DIN 45xxx and today there are standards IEC 60268-3:2001 as an example. I think there are also standards for radio station equiment in order to make sure that these can work together. All this might not be good enough for hi-end audio. But just to establish an arbitrary ASR standard can be done but there is low chance that the audio manufacturers will adopt it.
Last not least some of these standards are also for safety tests in terms of electricity and fire hazard which is also in the CE standard. Does amir conduct some safety tests for electricity ?

 

[audio_tony]

For years, manufacturers have had standards by which they can measure, such as IHF, EIAJ, DIN etc.

I guess not many of them are that bothered about measurements, so long as they can continue to churn out products and profit from them.

So what makes anyone think they are suddenly going to adhere to an ASR 'standard' for measurements?

I suspect that the majority of the buying public have not even heard of ASR, and that segment of the public will buy based on subjective reviews, not on an ASR recommendation.

I feel that the idea put forward here by Amir is laudable, but unlikely to gain much traction from manufacturers, in particular the larger ones.

 

[Thomas_A]

It is a good idea. However, th

Haven't thought through all the details for different device categories. Thinking out loud, it would be kind of like this:

DAC:
Output level: Minimum of 2 volts for unbalanced, 4 volts for balanced.
SINAD: >= 110 dB
DNR: >= 115 dB
Jitter (all inputs): < -115 dB (in FFT spectrum)
Filter attenuation 44.1 kHz sampling at 24 kHz: better than -100 dB
Frequency Response flatness: Less than 0.2 dB @ 20 kHz

Don't know yet how to include responses for the sweeps such as IMD. And for multitone, need to do some research to see what is a reasonable minimum.

AVRs:
Output level: Minimum of (undistorted) 2 volts for unbalanced, 4 volts for balanced.
DAC section:
SINAD: AVRs, better than 95 dB. AVPs, better than 100 dB
DNR: >= 115 dB
Jitter (all inputs): < -115 dB (in FFT spectrum)
Filter attenuation 44.1 kHz sampling at 24 kHz: better than -100 dB
Frequency Response flatness: Less than 0.2 dB @ 20 kHz

Amplifier Section:
SINAD with digital or analog input > 80 dB
SNR: ???

You get the idea.

I think it is a good basic idea. However, the AVRs are quite complicated vs. DACs and amps. And generally, there is a price-expectation factor for both functions and performance. While DACs and amps have limited function differences, AVRs can vary quite much. Is it expected that a 10 x difference in price would only be correlated to functionality and performance be static irrespectively of price?

 

[Geert]

I only have one justification for such specs: there are already products that meet these specs and exceed them in many case.

I considered that, but it's different approach than "I am thinking about the fewest key criteria that passes the acceptable mark". Then the guidelines become more of an overview of where the best in class are on specific criteria, with the risk that for certain criteria it's not clear if the level to where they are raising the bar matters (from the perspective of audibility).

 

[LiviuTM]

What a great idea!
The way I see it, it's about setting the expectations for what makes a decent product (not necessarily the greatest), from a technical point of view. "Low-hanging fruit", as Amir nicely put it.
It seems that there are companies willing to do better, but they don't know what to aim for exactly. These "standards" could provide guidance and help them build better products.
What's not to like?

It would be really nice to have more and more gear paying attention to the basics.
And there will always be room for differentiation on industrial design, features, build quality, user experience and so on.

 

[SSS]

So I keep running into companies who send me products with best intentions but miss key aspects that cause me to not recommend. And example is a very nice dongle I recently received that had balanced output but was limited to just 2 volts out. If you don't know, I like to see minimum of 4 volts out from such a port as otherwise, you can find unbalanced dongles at lower cost that do that.

Another example is channel balance issue with had with an AIYIMA amp where there was almost 1 dB differential.

Yet another is expected SINAD for an amplifier. Yes, we don't rate amps on that one number but if SINAD is say, 60 dB, the rest are going to follow.

In many cases decision makers are not knowledgeable in these companies so proudly show me the gear, only then realizing they could have built something better, but didn't.

Note that my focus here is for electronics only. And I am thinking about the fewest key criteria that passes the "acceptable" mark from me, and by implication, from you all. They can do better of course to get higher praise but want to establish what we like to see.

An example for a dongle:

Output voltage: at least 2 volts on unbalanced, 4 volts on balanced.
SINAD: 100 dB or better, 1 kHz, 22.4 kHz bandwidth
SNR at 50mv: 85 dB (?)
SNR at full 2/4 volt output: 110 dB (?)
Output impedance < 1 ohm

Example for Amplifier:
SINAD >= 80 dB
SNR >= 110 dB (?)
Channel balance < 0.5 dB
Crosstalk > 70 dB @20 kHz


This would be presented as general guidelines for companies to adopt (or not). The point of this thread is not to discuss the specifics although you can, but determine if it is time for us to do this. Hate to have companies ready to produce performant products based on objective measurements but not know clearly what those measurements should be.

What say you?

4 Volt balanced is not necessary to my opinion. The problem is with power amps which need higher input voltages today than in former times where 100 mV was good enough or later 0 dBm. When more insensitive a power amplifer is the better the noise/signal ratio which is good for sale.

 

[Torphie]

Given the example that Amir just mentioned (dongle) doesn’t that suggest that the manufacturers should be pushing this forward (as well). There are a few manufacturers (and ex manufacturers) that frequent ASR so it would be good to hear from them what might be helpful in this respect or indeed what has helped in the past perhaps following a product review.

 

[Kimbrough Xu]

Good idea, maybe you could do a certification, like the HI-RES, named ASR certified, that would be so fun!!!

 

[restorer-john]

I feel that the idea put forward here by Amir is laudable, but unlikely to gain much traction from manufacturers, in particular the larger ones.

Maybe not...

If anything, it might result in more accurate, measured and guaranteed specifications, but that has to be accompanied by calls to account when the advertised specifications are not met.

As it stands, there are no public "rakings over the coals" when manufacturers deliberately 'inflate' or otherwise deceptively claim numbers that are never achieved. They have nothing to lose or anyone to answer to. In short, they don't care.

It has to go both ways. If a manufacturer claims 300wpc and the device achieves 85wpc- it should be publicly roasted. If the manufacture claims 0.05% THD+N and it hits 0.003% THD+N from 20Hz-20kHz, it should be praised as being way better than spec.

 

[SSS]

I agree that a minimum requirement should exist. For a combined audio system it is just only neccesary that all components have specs that allow a good working together. For future changes of the system it is good when the components have specs with a positive margin that benefit this.

 

[DanielT]

Standard in HiFi. It's a dear subject. Seems to be difficult for the industry to agree on.

I don't know what happened to Polk's ideas?:

Subwoofer Shopping Made Easier With The CEA 2010 Standard

Subwoofer Performance: How to Know When You Have It (or Why Inches and Watts Tell You Nothing of Value Whatsoever)?

www.ecoustics.com

Standards are considered here:

FTC May Kill Amplifier Rule: Help Us Keep Consumer Protections in Audio!

Your country needs YOU to help protect consumers from false advertising in audio. The Federal Trade Commission (FTC) is reviewing its "Amplifier Rule", help maintain truth in amplifier ratings today!

www.audioholics.com

It seems messy:

What about the norms or standards?​

German standard for HiFi is DIN45500 and the EU norm for HiFi is EN61305. There might be some more, like the IEEE or others, all being of no realistic help for non-technicians who don’t have detailed, specific knowledge and access to measurement systems. This gets a bit complicated as the DIN was replaced by the EN and the EN61305 is not in use anymore, it became kind of irrelevant nowadays. Thus this leaves us with no standard, respectively no available label for HiFi on consumer audio systems.

However, wiki is not wrong. DIN, EN, IEEE and other include frequency range, output power, distortions, signal to noise ratio and several other criteria. While some standards still set thresholds for their criteria to be reached, the EN only describes the measurement procedures for such criteria. All in all, there is currently no standard in use that is commonly accepted and defines technically HiFi.

TestHiFi App HiFi test, sound test or audio test of a music system.

TestHiFi App allows you to test audio quality, to test HiFi quality, to test sound quality or to test audio quality. sound test; audio test; HiFi test.

testhifi.com

 

[audio_tony]

Maybe not...

If anything, it might result in more accurate, measured and guaranteed specifications, but that has to be accompanied by calls to account when the advertised specifications are not met.

As it stands, there are no public "rakings over the coals" when manufacturers deliberately 'inflate' or otherwise deceptively claim numbers that are never achieved. They have nothing to lose or anyone to answer to. In short, they don't care.

It has to go both ways. If a manufacturer claims 300wpc and the device achieves 85wpc- it should be publicly roasted. If the manufacture claims 0.05% THD+N and it hits 0.003% THD+N from 20Hz-20kHz, it should be praised as being way better than spec.

If manufacturers are not that bothered about meeting internationally set specifications, why would they bother adhering to specifications as set out by a relatively small minority forum?

 

[MaxwellsEq]

This would be presented as general guidelines for companies to adopt (or not). The point of this thread is not to discuss the specifics although you can, but determine if it is time for us to do this. Hate to have companies ready to produce performant products based on objective measurements but not know clearly what those measurements should be.

What say you?

I agree, it's a good Idea.

It might also benefit new members who join to ask whether their 5 year old product needs replacing with the latest thing. If they can see either the old product reviewed here or (trustworthy) manufacturer's specifications are still holding up, they can save their money.

I also like the Idea of posting the annual mean of devices tested against these measurements. I'm not sure what it would prove, but over time it might demonstrate a positive journey for the industry.

 

[computer-audiophile]

Just a few fuzzy thoughts on this.

Who actually establishes new standards? Is it individuals? Industry representatives? Politicians? What form of organisation should such an institution have?

In Germany, for example, the 'Stiftung Warentest' was established to provide customers with guidance on the quality of consumer products and is now widely accepted by the population.

In 1964, the Bundestag voted in favour of establishing a Stiftung (foundation) Warentest (merchandise test), and a group of economists, journalists, researchers and engineers pondered over test procedures, test regulations and organisational forms.

 

[Somafunk]

Let me put on my consultant hat

There is a model called Kano model, named after Noriaki Kano, a Japanese educator who it as a way to evaluate customer requirements in 1990's which has been simplified and adopted widely by many companies through their Lean or SixSigma programs.

In that model, there are 3 types of customer requirements; "must have" , "the more the better" and "delight factors".

There is a similar maxim in the bicycle world from one of the greatest designers of the modern bicycle,

Keith Bontrager is from a great biking nation : California , USA. His name appears on most things that go with Trek bikes. This legend in modern bike world created some of the most innovative designs in mountain bikes. During the late eighties and early nineties his Santa Cruz made bikes were highly desired due to their quick handling and unique designs. Holed-up in his garage, he worked on broken cycle parts for hours and hours to learn the defects and to develop some of the best components and frames for the bike-world.

Keith Bontrager’s famous aphorism: “Strong. Light. Cheap. Pick Two” stands the test of times.

Yes, I can pick a strong and light bicycle, and then I cannot get it cheap. I can pick a light cycle and which is cheap, then I cannot have it strong. I can pick a strong cycle which is also cheap, if so, I cannot get it light !

 

[CedarX]

I’m not sure it’s a good idea…. I always took Amir’s tests & associated results as a (strong!) indicator of “good overall design”, not an exhaustive evaluation of every characteristic that matters for audio. I also like that Amir has the freedom to add measurements as needed, covering other specific aspects.
As soon as you define minimum thresholds for a small set of objective measurements, manufacturers will make sure their product (barely) meet these thresholds. Great !!?? Not so sure… Manufacturers are competitive and creative: they will find ways to ‘optimize’ what is not defined—for example, to lower the cost. We’ll end up loosing this correlation with “good overall design”, and still generate incomprehension if Amir adds a few relevant extra tests: “but why do I get a bad review if I meet each one of ASR published ‘standards’?”

 

[Sokel]

Thinking from the manufacturers side:

Say that I want to offer a status item,hire the best designer in the world (and pay dearly for it),choose the best materials available (also pay) ,establish the best costumer pre and after sales service,an established recognition factor etc.,the whole "costumer must feel like a king" thing in other words.
It will of course be priced accordingly and as the market is tiny it will be expensive.

And let's say that the above has a medium/fair performance,let's assume 85db SINAD for an integrated.
Will it be recommended or not?Will the "nice" factor count for anything as in every other item in the world or will be judged purely by it's perfomance and the price premium will make it a target for not-so-nice comments as it usually happens?

If we want to be fair technical performance has to be separated from the price as there are many things at play.
Also,what confuses sometimes is the adoption of pro standards in consumer products,like the 4V balanced.
Market was unbalanced for decades and there's millions of devices out there who will keep working for a long time (the nice ones),it's a little anti economical to through them away as long as they work perfectly and their performance is inaudible (either from the 0.1%dbr THD+N pro standard or the 0.01%dbr THD+N really hard to achieve in a Klippel test) .

Edit: Something that it seems it must be standardized is amp's gain as it leaves a lot of room for playing with numbers and the low ones get an unfair lead.