• WANTED: Happy members who like to discuss audio and other topics related to our interest. Desire to learn and share knowledge of science required as is 20 years of participation in forums (not all true). Come here to have fun, be ready to be teased and not take online life too seriously. We now measure and review equipment for free! Click here for details.

Focus on SINAD Gives Vendors Hiding Space

DDF

Addicted to Fun and Learning
Joined
Dec 31, 2018
Messages
565
Likes
1,042
#1
I brought this up in an earlier thread but wanted to raise up to start a conversation on it.

I still wonder what all the focus on SINAD and THD/noise comes to? The measurements are very appreciated and great for filtering out complete junk, but reasonable vendors are now chasing meaninglessly low noise and distortion numbers (performance well below audibility or applicable in only very niche situations) but the stuff that really matters now gets nowhere near the attention it needs from suppliers. Too many products are still "junky" in these areas that greatly reduce enjoyment more than an extra shifted digit on the SINAD could ever compensate for, and they remain unspeced and except for channel balance, not 3rd party measured:
  • channel balance vs volume (see nasty result for L30 just posted by Wolf)
  • ground isolation (risk of ground loops)
  • for DACs, driver quality and robustness (custom driver? what are benefits? Standard xmos driver?....)
  • how it deals with dropped samples (overshoot/undershoot and how audible). I've had dacs sound very different in how the deal with this.
  • reliability cuts corners (what brand caps and what are temp ratings?)
I've had issues with all the above with various dac and HD amp equipment (all well reviewed). A read on the threads also show these issues pop up on highly reviewed product with too much regularity.

I think the extreme focus on thd/noise unintentionally diverts the consumer's gaze and gives vendors a place to hide behind and ignore scrutiny on these things that really matter.
 
Last edited:

amirm

Founder/Admin
Staff Member
CFO (Chief Fun Officer)
Joined
Feb 13, 2016
Messages
32,916
Likes
110,630
Location
Seattle Area
#2
channel balance vs volume (see nasty result for L30 just posted by Wolf)
I measure this in every headphone amp review.
 

amirm

Founder/Admin
Staff Member
CFO (Chief Fun Officer)
Joined
Feb 13, 2016
Messages
32,916
Likes
110,630
Location
Seattle Area
#3
ground isolation (risk of ground loops)
I love to test for this but right now, I don't have a fixture to make it happen. If someone can figure out a design for this, i can put it in place as I too agree it is very important.
 

amirm

Founder/Admin
Staff Member
CFO (Chief Fun Officer)
Joined
Feb 13, 2016
Messages
32,916
Likes
110,630
Location
Seattle Area
#4
for DACs, driver quality and robustness (custom driver? what are benefits? Standard xmos driver?....)
Every DAC today is driverless and that is how I test them. The only reason to install a driver is to get direct ASIO support and/or play high rate DSD.
 
OP
DDF

DDF

Addicted to Fun and Learning
Joined
Dec 31, 2018
Messages
565
Likes
1,042
Thread Starter #5
I measure this in every headphone amp review.
Yes, I (maybe indirectly) acknowledged that in my post, and its a very important measurement. Thanks for doing these. I once had a HD amp bad enough that the vendor agreed to send me a new pot to reinstall myself. Appreciated but what a PITA!
 
OP
DDF

DDF

Addicted to Fun and Learning
Joined
Dec 31, 2018
Messages
565
Likes
1,042
Thread Starter #6
Every DAC today is driverless and that is how I test them. The only reason to install a driver is to get direct ASIO support and/or play high rate DSD.
I've had very different performance from a DAC (frequency of drop outs) using it's drivers (not in ASIO mode) or the stock Windows drivers. Its not a perfect world and different PCs are more or less susceptible to these issues. There's no way to predict this but one thing I've learned is that "custom" drivers may not perform the same as standard drivers.

These things do matter. Look at it another way: testing the ultimate limits of SINAD may have a very niche application for a very small number of users. These effects are more prevalent and impactful IME.

Years ago (dating myself, mid 90s) I was an Ethernet hardware designer in telecom (millions of units shipped, getting it wrong not an option) and used a Smartsbits to lab test Phy/MACs for their robustness to various eye/Q degradations and also their recovery. Its a digital world now, I think vendors could/should spec the performance of their digital links, not just analog. Getting that wrong sounds worse than adding a tiny bit of noise.

I'm not asking anyone to test this stuff and I'm fully appreciated of all the time and measurements everyone provides (I add my own knowledge and measurements into the mix here as a small show of gratitude).

I think filtering out the garbage with standard audio measures has been a very effective step 1, but the points I raised are a sorely needed next step industry evolution IMO.

Not asking anyone to make the sacrifice to do so and these aren't a critique of anyone's outcomes but given the frequency of issues I'm seeing (and read of others seeing) think its food for thought worth discussing.
 
Last edited:

amirm

Founder/Admin
Staff Member
CFO (Chief Fun Officer)
Joined
Feb 13, 2016
Messages
32,916
Likes
110,630
Location
Seattle Area
#7
I am not following you. I perform all of my tests without a driver unless noted. So the performance i report, is the performance you get without a driver.

If by "windows" you mean the shared access mode instead of WASAPI exclusive, then uses, the OS audio pipeline will transform the audio sample. This is not a problem with the hardware though.

If you mean you used WASAPI exclusive and still had issues, then that goes counter to my testing.
 

Speedskater

Addicted to Fun and Learning
Joined
Mar 5, 2016
Messages
900
Likes
637
Location
Cleveland, Ohio USA
#8
I love to test for this but right now, I don't have a fixture to make it happen. If someone can figure out a design for this, i can put it in place as I too agree it is very important.
This is a job for the John Windt "Hummer Tester".
Complete instructions in the Audio Engineering Society Journal, 1995 June issue.
Same issue as the Neil Muncy (RIP) Pin 1 Problem paper.
 

pozz

Data Ordinator
Forum Donor
Editor
Joined
May 21, 2019
Messages
3,426
Likes
5,386
#9
How it deals with dropped samples (overshoot/undershoot and how audible). I've had dacs sound very different in how the deal with this.
Really. I've never thought that this would be a problem. Have you seen measurements of that behaviour?
 
OP
DDF

DDF

Addicted to Fun and Learning
Joined
Dec 31, 2018
Messages
565
Likes
1,042
Thread Starter #10
Really. I've never thought that this would be a problem. Have you seen measurements of that behaviour?
Same here but it was obvious in the audible overshoot on the packet drop edge. I didn't measure this because its not a DBT level phenomena.

I've been a hardware and/or systems engineer (specializing somewhat in algorithms and QoS) in telecom (data and/or audio or the confluence of both) for 30 years. Not making this stuff up.
 

pozz

Data Ordinator
Forum Donor
Editor
Joined
May 21, 2019
Messages
3,426
Likes
5,386
#11
Same here but it was obvious in the audible overshoot on the packet drop edge. I didn't measure this because its not a DBT level phenomena.

I've been a hardware and/or systems engineer (specializing somewhat in algorithms and QoS) in telecom (data and/or audio or the confluence of both) for 30 years. Not making this stuff up.
Not saying you are. I'm curious about the extent of the problem and how widespread it is. Which DAC was it? It could be like clipping behaviour in amplifiers, though a rarer occurrence compared to digital clipping from recordings.

Regarding channel matching, you're just talking about headphone amps right? I could try combing the reviews and putting together a comparion of when each hit a 0.5dB imbalance. It would be very rough though, say +/-2dB.
 
Last edited:

maverickronin

Major Contributor
Forum Donor
Joined
Jul 19, 2018
Messages
1,796
Likes
2,056
Location
Midwest, USA
#12
Pots are just too random. One sample doesn't doesn't tell you anything useful about what kind or tracking you're likely to receive.

There's nothing Amir, or any other reviewer, can do about it without testing multiple units.

It's still worth mentioning in a review though since with enough data points a brand or product can at least get a reputation for high standards.
 
OP
DDF

DDF

Addicted to Fun and Learning
Joined
Dec 31, 2018
Messages
565
Likes
1,042
Thread Starter #13
Pots are just too random. One sample doesn't doesn't tell you anything useful about what kind or tracking you're likely to receive.

There's nothing Amir, or any other reviewer, can do about it without testing multiple units.

It's still worth mentioning in a review though since with enough data points a brand or product can at least get a reputation for high standards.
A sample of one is no indication of the performance due to pot tolerance but there are things that can be done. Pop the lid, look at the pot brand or request it from the vendor. Pots come with specs for this. Alternatively, ask the supplier.
 

maverickronin

Major Contributor
Forum Donor
Joined
Jul 19, 2018
Messages
1,796
Likes
2,056
Location
Midwest, USA
#14
A sample of one is no indication of the performance due to pot tolerance but there are things that can be done. Pop the lid, look at the pot brand or request it from the vendor. Pots come with specs for this. Alternatively, ask the supplier.
That can help a some. The spec'ed tolerances on most pots are terrible though, so hopefully the manufacturer does a little bit of grading/QC on their own end too, which is where multiple data points come in handy.
 

amirm

Founder/Admin
Staff Member
CFO (Chief Fun Officer)
Joined
Feb 13, 2016
Messages
32,916
Likes
110,630
Location
Seattle Area
#15
I have started putting disclosures in my reviews that pots and vary and performance I show may not be typical. If pot variation is an issue for someone, best approach is to get a DAC+Amp combo where the attenuation is digital.
 

Matias

Major Contributor
Forum Donor
Joined
Jan 1, 2019
Messages
2,207
Likes
3,268
Location
São Paulo, Brazil
#16
If pot variation is an issue for someone, best approach is to get a DAC+Amp combo where the attenuation is digital.
Or when using a computer as source, as most do, just use the pot at maximum volume or close to it, and attenuate further on software.
 
OP
DDF

DDF

Addicted to Fun and Learning
Joined
Dec 31, 2018
Messages
565
Likes
1,042
Thread Starter #17
Not saying you are. I'm curious about the extent of the problem and how widespread it is. Which DAC was it? It could be like clipping behaviour in amplifiers, though a rarer occurrence compared to digital clipping from recordings.

Regarding channel matching, you're just talking about headphone amps right? I could try combing the reviews and putting together a comparion of when each hit a 0.5dB imbalance. It would be very rough though, say +/-2dB.
I misunderstood "Really", thanks for clarifying. For channel matching, it would be anything with a potentiometer. Not looking to add to anyone's plate, but over-all gain at which imbalance ~ 1 dB is an important metric to enjoyment of an amp.

You asked for examples, and I like some aspects of my Nuprime Dac-10H, but its USB i/f performance is bad: ground loops, adverse reaction to drop outs (whereas for example my Arcam dac behaves much better), and a driver that has more drop outs than just using the vanilla driver from Windows. I've had bad pot mismatch on a Gilmore HD amp ("highly engineered"), capacitor failures on a $1.5k dac integrated, on a sub and on a Hypex power supply due to cheapo caps; driver problems with an ESI dac as well ("pro" gear). Most people just sell their stuff and move on but there are many user reports of similar issues and usability issues have been widespread in gear I've bought. I have a 30 year old Bryston and a NAD 3020 I bought in the early 80s that are generally faultless.

Not putting the burden on ASR but these quality issues need much more attention and I fear we're in another senseless "spec race", like the amplifier power wars of the 1970s, that is focusing too much attention in the wrong place.
 

pozz

Data Ordinator
Forum Donor
Editor
Joined
May 21, 2019
Messages
3,426
Likes
5,386
#18
I misunderstood "Really", thanks for clarifying. For channel matching, it would be anything with a potentiometer. Not looking to add to anyone's plate, but over-all gain at which imbalance ~ 1 dB is an important metric to enjoyment of an amp.

You asked for examples, and I like some aspects of my Nuprime Dac-10H, but its USB i/f performance is bad: ground loops, adverse reaction to drop outs (whereas for example my Arcam dac behaves much better), and a driver that has more drop outs than just using the vanilla driver from Windows. I've had bad pot mismatch on a Gilmore HD amp ("highly engineered"), capacitor failures on a $1.5k dac integrated, on a sub and on a Hypex power supply due to cheapo caps; driver problems with an ESI dac as well ("pro" gear). Most people just sell their stuff and move on but there are many user reports of similar issues and usability issues have been widespread in gear I've bought. I have a 30 year old Bryston and a NAD 3020 I bought in the early 80s that are generally faultless.

Not putting the burden on ASR but these quality issues need much more attention and I fear we're in another senseless "spec race", like the amplifier power wars of the 1970s, that is focusing too much attention in the wrong place.
The best way to supplement the measurements would seem to be user reports and teardowns. Some way to gather them or ask the membership for their experience would be really valuable. That's how these things come to light most of the time.

Maybe we could fund the ground loop susceptibility test rig for Amir as well.
 

Vini darko

Major Contributor
Joined
Jun 1, 2020
Messages
1,558
Likes
1,991
Location
Dorset England
#19
I'm not sure your right. We're currently crawling up toppings butt in the l30 thread, because of a bit of channel imbalance and low output impedance. Dispite the unit measuring superb in sinad.
 
OP
DDF

DDF

Addicted to Fun and Learning
Joined
Dec 31, 2018
Messages
565
Likes
1,042
Thread Starter #20
Some way to gather them or ask the membership for their experience would be really valuable.
An idea would be to structure it similar to a vote survey appended to the review for buyers to log issues. General categories could be created and refined over time. Difficulty would be avoiding abuse, so it probably shouldn't be anonymous.

Maybe we could fund the ground loop susceptibility test rig for Amir as well.
A first step would be to define use cases. This is really an interop test so the jig would have to emulate some good and not so good grounding practices. Rane used to publish some decent white papers, there's also Bruno Putzey's papers, others to pull from.
 
Top Bottom