AES Paper Presentation on DAC Measurements (Video)

[amirm]

As some of you know, I presented a paper at this years Audio Engineering Society on measuring over 450 DACs. All the tests were detailed in the paper together with motivations and learnings from the multi-year project. The paper is only available for a fee (link below). AES asked for a short presentation at the conference itself. In this video, I go through that presentation with more detail than I intended to cover at the conference. This way, you get a bit more flavor of what I had written in the paper.

Here is the link to the paper and where you can download it: https://www.aes.org/e-lib/browse.cfm?elib=22278

It is free for members and costs $33 otherwise.

 

[Curvature]

Thank you for your effort @amirm in documenting your work and, of course, for continuing to do the work of measuring devices so consistently.

Thank you to all the members who contribute their devices.

 

[sweetchaos]

Amir, in the video, said:

By the way, if i were giving the presentation at AES, I wouldn't have mentioned any of these brands or model numbers.
Their protocol is to be polite and not get specific.
But for you guys, you're getting that bonus information.

We get the unfiltered version with all the dirty details!

 

[Curvature]

Are other papers planned?

 

[amirm]

Are other papers planned?

Yes, for next year. This one we decided to do at the last minute and there just wasn't enough time to do more than one. I originally wanted to cover other areas but was glad I didn't as the work involved was 10X what I had estimated!

 

[amirm]

On that front, I have been remiss in not giving major credit to member @Pe8er. He provided incredible amount of editorial/review help going way beyond anything I could ever ask someone to do. I would have surely failed to produce a good paper without him!

 

[PlasticDoc]

"In Science, one’s private property is established by giving its substance away", Robert Merton - The Matthew Effect in Science(*), II

So, I assume that it is not a scientific paper. Could you give us an abstract?

 

[sweetchaos]

Let's analyze this video with ChatGPT 3.5 (the free one), shall we?

Summary:

Amir from Audio Science Review presented a paper at the Audio Engineering Society (AES) on the objective performance of digital-to-analog converters (DACs) in both consumer and professional spaces. The paper, titled "Objective Performance of DACs: Consumer and Pro Audio," discusses the history of his DAC testing, the significance of AES, and the extensive library of DAC measurements he has conducted.

Amir emphasizes the importance of presenting data to help buyers quickly assess a product's performance. He introduces the concept of "Signad," a simplified positive number derived from THD+N measurements, making it easier for users to understand and compare performance. The presentation includes insights into the performance of various DACs, comparisons between consumer and professional audio devices, and the impact of filters and interfaces on measurements.

He discusses issues such as USB Jitter, the influence of different attack filters, and the challenges with multi-tone testing. Amir also addresses the lack of standards for output voltage in DACs and the variability in gain among amplifiers. He points out the lack of correlation between DAC price and performance and highlights the dominance of Chinese manufacturers in the high-performance DAC market.

Amir concludes by urging the audio industry to focus on objective performance, adopt standards, and improve transparency. He emphasizes the need for companies to measure and publish objective data for their products, promoting a shift back to objective engineering practices. The presentation is intended to educate the audio community and encourage a move towards better transparency and performance standards in the industry.

Actionable Steps:

1. **Standardization of Output Voltage and Gain:**
- Encourage the establishment of industry standards for DAC output voltages and amplifier gains to ensure compatibility and consistency across devices.

2. **Objective Measurement and Reporting:**
- Advocate for companies to adopt a closed-loop design approach, emphasizing the measurement and publication of objective performance data for their audio products.

3. **Education and Awareness:**
- Promote educational initiatives within the audio industry to increase awareness of the importance of objective performance metrics. Encourage manufacturers, engineers, and consumers to prioritize measurable data over marketing claims.

4. **Collaboration with Audio Engineering Societies:**
- Foster collaboration between industry professionals, manufacturers, and organizations like the Audio Engineering Society (AES) to create and endorse standardized testing methodologies. This could involve creating working groups or committees dedicated to defining and refining measurement standards.

5. **Encouraging Transparency:**
- Advocate for greater transparency in product marketing and documentation. Encourage manufacturers to provide comprehensive information about their products' performance characteristics, enabling consumers to make informed decisions.

6. **Industry-Wide Adoption of Objective Metrics:**
- Encourage industry-wide adoption of standardized metrics, such as Signad, to simplify performance comparisons and make it easier for consumers to understand the capabilities of audio devices.

7. **Support for Open Measurement Tools:**
- Support the development and use of open-source or widely accessible measurement tools that enable users to independently assess the performance of audio devices. This can empower consumers to make informed choices and contribute to a culture of transparency.

8. **Advocacy for Consumer Rights:**
- Advocate for consumer rights in the audio industry, encouraging manufacturers to disclose essential performance information. This could involve working with consumer protection agencies and organizations to establish guidelines for transparent product disclosure.

9. **Encouraging Responsible Journalism:**
- Encourage audio journalists and reviewers to prioritize objective measurements in their assessments and reviews. This can contribute to a shift in industry focus towards objective performance rather than subjective opinions.

10. **Industry Events and Conferences:**
- Use industry events and conferences as platforms to discuss and promote the importance of objective measurements. Encourage panel discussions, workshops, and presentations that emphasize the benefits of standardized testing and transparency.

By taking these actionable steps, the audio industry can move towards a more transparent and objective approach to product development, marketing, and consumer education.

 

[renaudrenaud]

Thanks @amirm. Pretty sure your video will become the standard about DACs, far far away the romantic approach.

 

[restorer-john]

Let's analyze this video with ChatGPT 3.5 (the free one), shall we?

Why bother? It doesn't add anything useful.

Just watch Amir's video instead.

 

[sweetchaos]

Why bother? It doesn't add anything useful.

Not everyone can watch a 32 min video? Some people don't have the time.
Not everyone is a visual learner? Some people like text instead.

This summary is a TL;DW.

 

[voodooless]

Nice video @amirm!

Would love to see the $ vs SINAD graph with the $ in log scale though.

 

[amirm]

I can do that....

 

[amirm]

So, I assume that it is not a scientific paper. Could you give us an abstract?

The abstract was at the link posted in the OP: https://www.aes.org/e-lib/browse.cfm?elib=22278

 

[cheapmessiah]

Great work Amir, very informative video as usual.

To Amir and all the people who contibuted, contribute and will contribute in the future to endeavours like this, thank you all for making this hobby better and factual.

 

[AdamG]

Great video Amir. Thanks for sharing this and all the lessons you have learned along the way. Keep up what you are doing because the industry is taking note and making changes to remain competitive. That’s due to your work here. Take credit where credit is due and pat yourself on the back. You deserve it. Bravo Zulu Sir.

 

[AdamG]

Why bother? It doesn't add anything useful.

Just watch Amir's video instead.

They are both useful actually. At least for me they were. Thanks for taking the initiative @sweetchaos. I found the Actionable Steps though provoking and groundwork for a map ahead plan. All of these are objectives that I think Amir and the ASR Team are all aiming to accomplish.

 

[sweetchaos]

@192kbps wrong thread

 

[IAtaman]

Thank you for sharing Amir.

Spending $10K on a device with a DAC in it, does not even guarantee you will get a product with SINAD of 85dB at 1KHz. I don't know what more data people need to see something is rotten in the state of audio market.

Maybe also good to see a graph showing bit converted SNR vs price?

For number of DACs in Excellent Category by Country of Origin graph, I think using absolute numbers do introduce sampling bias as there are a lot more China origin DACs tested as opposed to others. Maybe a better representation would be % of excellent DACs per CoO?

 

[dzerig]

edit: I have not read the paper, which Amir points out covers some of my complaints. I am referring to the charts in the video, and have not read the paper, so I jumped to some conclusions. I leave my comments here so people who don't read the paper will be aware of what is not covered in the charts, and that you need this information to come to any conclusions about the market as a whole. Especially the conclusion about no price correlation, and country of origin, it's hard to say how meaningful it is. Certainly China has a lead in performant dacs, but does it really have 115 to 1 advantage over Japan? :end edit

You need to tell the reader how many dacs you tested from each country. And, how many dacs are on the market, at least do a rough survey, so people will know if your selection of products is representative of the general market. Only then does your chart showing the countries the best performing dacs come from make any sense.

Also if the products are new, or now. You have released reviews on products that are already replaced by the manufacturer, which doesn't truly represent today's market. Indeed, this list of dacs includes some from over over 6 years ago. Including these in your analysis confuses whether your data represnts today's market, or the market over a period of ten years. Or more, depending on how many vintage dacs you've tested. We don't know because you don't say.

So far as the price performance comparison, I expect if it was not linear but logarithmic, or the dacs were compared at prices from 100 to 1000 only, then you might see that relationship. At the highest sinad, there seem to be less dacs at the bottom of the price axis than there are at slightly lower sinad. So maybe the conclusion is that you get a little performance at the margins up to a few hundred dollars. At least, that would explain no apparent $100 - $300 dacs at the high end of the sinad chart. Surely that indicates something if you don't skew data by including super high prices well over $1000..