What should the reference(s) be in measurements?

[amirm]

I have been using the same ad-hoc references in measurements for quite a while. Example:

I suspect most people don't know who or what Khadas is anymore. So what should we use as reference here and in other measurements? Here is another example:

I like the two references but again, Razor is an unknown product here.

 

[Sokel]

What would be real world useful would be the worst and the best heavy populated Klippel test results (where applicable)
As the number of tests are very high we can safely relate to them.

 

[bachatero]

How about Apple Dongle instead of Razer Adapter? "Everybody" knows what that dongle is like and "everybody" can afford one but most people don't even know that Razer is primarily one of the biggest mouse makers, much less that they have audio adapters.

 

[staticV3]

I'd nominate the Topping L30II and E30II.

No-nonsense, high performance and high value benchmarks in their respective category.

 

[Gary_G]

How often could the reference change. Maybe it could be the best in each class. If something is better then it's the new reference.

 

[Matias]

IMO reference should be an excellent and cheap product, yet popular.

 

[kemmler3D]

I suppose the use of the references is to give readers an idea of "is this good, bad, or just OK?" if they don't know just from the numbers.

Some ideas to help with that:

• The best SINAD-per-dollar unit in the class
• The best, worst, and median units in the class
• The best-selling unit in the class (check Amazon rankings?)
• The cheapest unit that achieves full transparency

 

[Bob from Florida]

I have been using the same ad-hoc references in measurements for quite a while. Example:

I suspect most people don't know who or what Khadas is anymore. So what should we use as reference here and in other measurements? Here is another example:

I like the two references but again, Razor is an unknown product here.

Instead of only referencing another product, what about referencing the limits of human hearing? A number that represents the beginning of the “it just doesn’t matter” because you won’t hear any difference for better measurements.

 

[unpluggged]

I suspect most people don't know who or what Khadas is anymore.

Well, it deserves a place in our hall of fame I still have it lying around in a drawer in a DIY case, reminding me of the days when it was a breakthrough DAC at this price point back in 2018.

I think we can do away with any specific product, just drawing a line representing some level of "acceptable" performance. Or, as it had been suggested here, use the Apple dongle as such a reference.

 

[JSmith]

Maybe have the worst and the best in that product class, but a fairly recent and known product.

Instead of only referencing another product, what about referencing the limits of human hearing?

Amir tried that;

I did. There were so many complaints that I didn't want to deal with it.

JSmith

 

[MaxwellsEq]

I searched best selling and popular DAC via Google and the Ifi Zen range popped up a lot. I feel the reference should be something affordable and reasonably common such that people might see it in a store.

 

[JeremyFife]

Picking any one 'reference' product is problematic where there's no clear basis for that choice.
Best selling (is that data available, which market)?
First product (lowest rank) reviewed here rated as "good" ... bit self-referential and subject to change
Threshold of audibility... contentious, but it makes good sense and is kind of what this site is about

I can't see a simple rationale for choosing another product vs the Khadas. Just stick?

I like the audibility threshold though... and put a hard hat on!

 

[solderdude]

Why not draw in lines for 'minimal requirement' and 'measurement equipment floor' ?

 

[Bob from Florida]

Maybe have the worst and the best in that product class, but a fairly recent and known product.

Amir tried that;

View attachment 411316




JSmith

I think the threshold reference is nothing to complain about and I would applaud its inclusion. A lot of folks seem to get caught up in "absolutes" and need to complain about something. I like your best and worst idea as well.

 

[192kbps]

I'd nominate the Topping L30II and E30II.

No-nonsense, high performance and high value benchmarks in their respective category.

DAC: SMSL PS200
AMP: TOPPING L30II

 

[Fredygump]

I have been using the same ad-hoc references in measurements for quite a while. Example:

I suspect most people don't know who or what Khadas is anymore. So what should we use as reference here and in other measurements? Here is another example:

I like the two references but again, Razor is an unknown product here.

Maybe we don't need a specific reference?

Instead, maybe you could overlay colored bands that correspond to your device rankings?

Interpreting lines on a graph requires some knowledge and familiarity with testing metrics and how they correlate to real world performance. But if the measurements are running through the middle of the green band, for example, then a person doesn't need to have training or experience to say with confidence that they don't have to worry about performance with that particular product. And in the same way, if the measurements spend a lot of time in the red zone, that is also a clear indication.