The Backstory
So... In the last couple of days I've been messing around with some Game Boy Advance (GBA) emulators, and I've found the experience to be very frustrating. I was getting some very annoying audible crackling, and I soon gave up trying to make heads or tails of it for a couple of reasons:
Audio on the GBA sucks to begin with (have a read), but most importantly, I realized that I have no comparison point and that I have no idea how accurate or inaccurate a given emulator is regarding audio performance.
In practice, most of us rely heavily on developers' claims regarding audio accuracy.
At best, channels like Digital Foundry have posted back-to-back sound clips comparing real consoles with specialized clones like the Analogue suite of FPGA consoles. And while there usually are some differences that we can pick out by ear, we ASR forum members know very well how poor a measuring tool our ears really are.
The Idea
In my struggle, I was reminded of an excellent piece of research in video form:
This is from a company called Plogue that aims to develop bit-accurate VST emulators of retro audio hardware. In this video, they tell the story of their Super Nintendo Entertainment System (SNES) plug-in development.
I really love this measurement-based approach, and I believe it to be a perfect fit for ASR.
Watching this documentary for the nth time, I thought... wouldn't it be great for our community to keep the spirit of this analysis going by using our combined effort and skills to compare the audio accuracy of modern digital emulators with our old hardware?
Anyone in this thread will be able to contribute however they wish: with opinions, feedback, articles, resources, audio clips, experiments, code, measurements, etc.
The Challenges
Of course, this is easier said than done. Doing this sort of measurement presents a few challenges. Here are some that I can think of:
Access to Retro Hardware
This research project requires access to the pieces of retro hardware that we want to measure and compare against, and preferably more than one unit of a given console to rule out the possibility of hardware faults or simple wear and tear (some of these things are old!).
Understanding of the Audio Pipeline
We need to have some general knowledge of how real hardware produces sound in order to be able to test edge cases and hardware quirks.
We also need to make sure that the only variable we're measuring is the audio quality of the output of the hardware/software under scrutiny. For example, setting our OS's sample rate to the wrong value (e.g., one that is different from the emulator's own sample rate) can invalidate test results.
Special attention needs to be paid to emulator settings. Audio performance can be affected by many factors, including (but not limited to) video settings such as V-Sync.
The Tools
This is probably the biggest hurdle to our objective. This kind of testing requires specialized tools like the above-mentioned MDFourier. This means we will need to find, write, and utilize code and audio analysis software for comparisons and null tests.
Defining What "Accuracy" Really Means
The reality of the situation is that many old consoles produce highly variable results when rendering audio. For example, many SNES revisions make use of a ceramic resonator that almost never produces the same sample rate twice (yikes!).
Another element not to be ignored is the presence of old, rudimentary DACs in vintage hardware. Sample-and-hold and PWM DACs are common, as are end-of-chain analog low-pass reconstruction filters. This means that recreating the "pure" digital output of an audio processing chip is not necessarily the end goal as far as accuracy is concerned.
There is also the fact that many retro hardware sample rates do not align with modern sample-rate standards. For example, the GBA outputs 9-bit audio at 32.768 kHz in one of its modes. This means resampling is almost certainly unavoidable inside an emulator.
All of these elements make defining "accuracy" a bit of a messy endeavor. That said, well-taken measurements can speak for themselves, and we can all have slightly different thresholds for what we consider acceptable or "close enough."
Conclusion
My own understanding and skills are fairly limited for the scope of this project, so I'm hoping that together we will be able to get some results, whatever they might be!
I do own some retro consoles like a Nintendo DS Lite and a DSi among others and an audio interface so I will contribute in any way I can in case the thread takes off.
Thank you for reading, and let me know what you think if you like!
