• WANTED: Happy members who like to discuss audio and other topics related to our interest. Desire to learn and share knowledge of science required. There are many reviews of audio hardware and expert members to help answer your questions. Click here to have your audio equipment measured for free!

AES Paper Digest: Do Audio Op-amps Sound Different?

amirm

Founder/Admin
Staff Member
CFO (Chief Fun Officer)
Joined
Feb 13, 2016
Messages
44,376
Likes
234,559
Location
Seattle Area
This is a digest of the Audio Engineering Society (AES) conference paper titled, “Distortions in Audio Op-Amps and Their Effect on Listener Perception of Character and Quality.” (http://www.aes.org/e-lib/browse.cfm?elib=16029) As the name indicates, it is a controlled listening test to see if there is audible difference between Op-amps (integrated small amplifiers) when operated below their clipping point.

In recent years, it has become quite fashionable to talk about what Op-amp are used in audio products or swap one for the other in the same vein. While subjective outcomes abound on efficacy of such differences, question becomes if there is any formal, controlled listening tests that can give us reliable indication of audible differences in Op-amps.

The paper in question aims to answer this question. It has good pedigree when it comes to its authors which includes the famous mastering engineer, George Massenburg, and other researchers from various departments of McGill University. If you have not heard of McGill, it is the same college that gave us Dr. Sean Olive of Harman/NRC fame, and is the closest we have to a school contributing to research in sound reproduction.

For the experiment, twenty (20) Op-amps were utilized, seven (7) of which were discrete and fourteen (14) which were Integrated Circuits (ICs). The circuit used for measurements and listening tests is as follows:


upload_2018-4-13_10-54-7.png


Starting from the middle, they use two of the same op-amps, both programmed to have 40 dB of gain. The second one however is in inverting configuration so backs out the actual gain of the first one, resulting in no gain at all.

To compensate for inverting nature of the two-amps in the middle, an inverting "buffer" is used up front. For some reason I don't understand, that component has 9 dB of gain (in which case it is not a "buffer"). They had to then back out that gain with the output "Pad" by the negative -9 dB.

Net, net, the whole thing has "unity gain" meaning it doesn't increase the signal level but launders the audio signal through two of the same op-amp as to obtain their distortion profile.

The resultant circuit resulted in slightly different high-frequency response:

upload_2018-4-13_10-58-10.png


I wish they had compensated for this so that the difference was not there. I assume they wanted to keep all the circuits identical which resulted up to half a dB or so of difference at 20 kHz.

Levels were set such that clipping did not occur:

upload_2018-4-13_11-0-26.png


The full combination of 20 different op-amps would be huge so they made some choices in pairing some op-amps with each other as noted earlier.

For material, being a music school, the had access to raw recordings:

upload_2018-4-13_11-3-50.png


Notice that they ran the signal through two of the fixtures shown before. So the total picture is the op-amp being in the signal path four times. It is not explained why they had to do this.

Here is the listening test protocol:

upload_2018-4-13_11-5-43.png


So it is a double blind, AB preference test.

There is a second phase but that did not generate significant results so I won't go into that.

Here is who took the tests:

upload_2018-4-13_11-8-47.png


Let's see the results of the test:

Opamp Listening Results.PNG

upload_2018-4-13_11-14-22.png



Anywhere you see a "P" value less than 0.05, the results are "statistically valid." See my article on this: https://audiosciencereview.com/forum/index.php?threads/statistics-of-abx-testing.170/

Nine of the outcomes reached this level of statistics. So it seems that differences could be heard in this test under double blind test conditions.

Alas, making sense out of why is complicated. There is little correlation between measurements and listening test results. Where there is some correlation according to the authors (I did not check), higher distortion seems to get the vote.

They also found counter to common understanding, that spectrum analysis of the distortion was not helpful in predicting the outcome:

upload_2018-4-13_11-18-0.png


Likely cause of this is that while later harmonics are more audible (less masked), they are also of much lower amplitude so that takes away audibility.

Sadly they never identify the op-amps so there is no way to do further research on their work, or confirm the results.

Here is their conclusion:

upload_2018-4-13_11-19-16.png

upload_2018-4-13_11-19-45.png


Conclusions
This study is significant in the way it shows audible differences between op-amps at distortion levels that are fairly low. There is a lot of fine print here of course with quadrupling the use of op-amps, and lack of more solid investigation and documentation of the work. It makes for good foundational work for follow on research.
 
Last edited:

Krunok

Major Contributor
Joined
Mar 25, 2018
Messages
4,600
Likes
3,065
Location
Zg, Cro
Extremely interesting!

But.. What I find confusing here on the first sight is that I cannot relate the cases where column 1 indicates a statistically valid result (value less than 0.05) and the values presented in columns 2 and 3. Shouldn't this be ran through some correlation test to see if those cases are showing significant (co)relation between the THD+N figures expressed in columns 2 and 3?

Am I missing something?
 
Last edited:

Krunok

Major Contributor
Joined
Mar 25, 2018
Messages
4,600
Likes
3,065
Location
Zg, Cro
What also seems confusing is a steep roll-off after 10kHz. How comes this doesn't extend after 20kHz to ensure it is out of the hearing range? Modern opamps should be able to ensure that without sweat..
 

FunctionalDoc

Member
Joined
Mar 28, 2018
Messages
66
Likes
39
Thanks for sharing this paper. I see one of the authors is Robert-Eric Gaskell is the lead designer on the ORA graphene drive headphone that I am backing on Indiegogo campaign.

Looks like nice controlled study . I remember a older video lecture from RMAF playing tracks of music with various amounts of distortion and that most people preferred some distortion to their sound.
 
OP
amirm

amirm

Founder/Admin
Staff Member
CFO (Chief Fun Officer)
Joined
Feb 13, 2016
Messages
44,376
Likes
234,559
Location
Seattle Area
What also seems confusing is a steep roll-off after 10kHz. How comes this doesn't extend after 20kHz to ensure it is out of the hearing range? Modern opamps should be able to ensure that without sweat..
Yeh, I was disappointed in that too. They say they took the circuit from Jung's great book on Opamps. I looked through there but could not find anything like it. Sadly they don't mention chapter/section of the book (which is massive at some 700 pages).
 

RayDunzl

Grand Contributor
Central Scrutinizer
Joined
Mar 9, 2016
Messages
13,204
Likes
16,986
Location
Riverview FL
Do the results show any distinction between the discrete and integrated devices?
 
OP
amirm

amirm

Founder/Admin
Staff Member
CFO (Chief Fun Officer)
Joined
Feb 13, 2016
Messages
44,376
Likes
234,559
Location
Seattle Area
Do the results show any distinction between the discrete and integrated devices?
The made no indication of that. And as I mentioned, they don't list the op-amps so no way to figure it out from the data.

I think their email addresses are in the paper so someone less lazy than me can contact them and ask. :)
 

oivavoi

Major Contributor
Forum Donor
Joined
Jan 12, 2017
Messages
1,721
Likes
1,935
Location
Oslo, Norway
I read this paper some time ago. They may have answered this in the paper, but wouldn't the most likely explanation for the perceived differences be the difference in high frequency roll of?
 

Krunok

Major Contributor
Joined
Mar 25, 2018
Messages
4,600
Likes
3,065
Location
Zg, Cro
I read this paper some time ago. They may have answered this in the paper, but wouldn't the most likely explanation for the perceived differences be the difference in high frequency roll of?

As all of them were students (thus quite young) it seems very probable - young people can hear 19kHz without any problems.

Btw, look at row 8: THD of 0,0045 paired with 0,87???

I can't find any correlation between the rows in which they have spotted the difference and the values in columns 2 and 3.
 

Krunok

Major Contributor
Joined
Mar 25, 2018
Messages
4,600
Likes
3,065
Location
Zg, Cro
Btw, I remembered that the lead auhtor (Gaskell) wrote a follow-up article, with additional listening tests: http://www.aes.org/e-lib/online/browse.cfm?elib=17529

It seems to confirm that it's the distorsion characteristics which is indeed causing the listeners to perceive them as different.

Well, there's the F-test which can determine if all those samples belong to the same group and it's quite easy to make it even with Excel, but somehow I believe there will be no correlation.

Btw, how exactly are the values in columns 2 & 3 defined?
 
OP
amirm

amirm

Founder/Admin
Staff Member
CFO (Chief Fun Officer)
Joined
Feb 13, 2016
Messages
44,376
Likes
234,559
Location
Seattle Area
Btw, I remembered that the lead auhtor (Gaskell) wrote a follow-up article, with additional listening tests: http://www.aes.org/e-lib/online/browse.cfm?elib=17529
Ah, I had seen that before and thought it was just a modeling exercise. Didn't realize it had real listening test results. I will dig into it later but for now, it spills the beans on the identity of some of the opamps used in this study:

upload_2018-4-13_13-8-46.png
 

Krunok

Major Contributor
Joined
Mar 25, 2018
Messages
4,600
Likes
3,065
Location
Zg, Cro
Btw, they also don't mention IM distortion at all, which I believe can play even more important role in hearing tests than THD.

Beside not taking into account IM distortion, IMHO this test also lacks a serious statistical analysis which would emphasize the conclusions. Presented in this way it more looks like a bunch of raw data than like a serious scientific paper. @amirm This of course is in no way aimed toward your effort to publish this paper but only toward it's authors.
 

Krunok

Major Contributor
Joined
Mar 25, 2018
Messages
4,600
Likes
3,065
Location
Zg, Cro
And what exactly is their definition of THD when they have such steep roll-off after 10kHz? For example what happens with the distortion of the 3rd harmonic of the base tone of 2kHz when it is so sharply attenuated???
 

junki

Member
Joined
Mar 24, 2018
Messages
55
Likes
43
Nice paper! A scientific, methodical process to minimize bias, hold variables constant, and confirm what audio device manufacturers have already known. Why do people say some DACs sound different when there is literally a single digit number of popular D/A chips of the even smaller number of D/A topologies out there?

The D/A "algorithm" (i.e. delta-sigma, R2R, FPGA) a DAC device uses to reconstruct amplitude and time-based frequency samples to an analog signal plays only a small role in the audible characteristics of the final output. What really matters in terms of audible signature is the how the reconstructed analog signal is subsequently antialiased, filtered, and amplified in the analog output stage (where opamps reside).

The DIY community will tell you that it's not even op-amps alone; it's actually the overall circuit combined with opamps that really determine the sound character. So OPA2134 may more commonly sound darker and thicker than some other opamps, but its actual impact will be different depending on the circuit it's used in. This paper held the circuit variable constant, as it should.

Psychoacoustics is interesting.
 
Last edited:

Krunok

Major Contributor
Joined
Mar 25, 2018
Messages
4,600
Likes
3,065
Location
Zg, Cro
So OPA1612 will sound different depending on the circuit its used in.

Sure it will. But only if you are from Krypton and posses super hearing. But to us, earthlings, OPA1612 can either sound neutral, if surrounding circuitry is designed properly, or it can sound awful, if surrounding circuitry is designed poorly.

P.S. Stay away from kryptonite if you can hear the difference between opamps. ;)
 

oivavoi

Major Contributor
Forum Donor
Joined
Jan 12, 2017
Messages
1,721
Likes
1,935
Location
Oslo, Norway
I do think this test showed differences between the opamps in question. What hasn't been shown is that this is due to distortion, as I think tbey didn't hold the frequency response constant. Need to look closer at the 2nd paper to see if they did that then or not in that 2nd study. I'm not an engineer, but I would guess that competent amp designs attempt to compensate for any deviations from linearity in the frequenycy response of the opamps?
 

Krunok

Major Contributor
Joined
Mar 25, 2018
Messages
4,600
Likes
3,065
Location
Zg, Cro
If the frequency response is not "constant" throughout the hearing frequency range how can we even speak about "linearity"???

Non-linearity is actually the cause of the distortion. The rest of the garbage you can hear is noise. And that pretty much covers it..
 

oivavoi

Major Contributor
Forum Donor
Joined
Jan 12, 2017
Messages
1,721
Likes
1,935
Location
Oslo, Norway
I'm not sure I understand the last comment. Isn't it common to distinguish between linear and non-linear distortion? As I understand it they are investigating non-linear distortion here.
 

Krunok

Major Contributor
Joined
Mar 25, 2018
Messages
4,600
Likes
3,065
Location
Zg, Cro
Heh.. As I said, distortion is actually caused by non-linearity, but it might not be that easy to explain.

Let's try it this way: if you imagine that various instruments try to play C3 tone you would of course notice that none of them sounds the same although they are playing the same C3 tone. The secret is that although all of them are playing the same C3 tone, this having the same frequency of the BASE tone, the shape of their waves are different. While the simplest C3 tone would be a simple sine wave playing at the frequency of the C3 tone, some of them would have more complex waveform. For example, violin playing the C3 tone would have a wave form that resembles triangular shape while other instruments would take different wave form. That is where "harmonics" come into play. There was a guy called Fourier who proved that every periodic wave can be represented as a sum of "harmonic" waves. That means that, in addition to "base" C3 tone, a number of additional tones which frequencies are multipliers of base tone frequency (x1, x2, x3 and so on) are added to the base tone to form a complex waveform specific to that instrument. Those additional tones are called "harmonics", first harmonic is the base tone, 2nd harmonic is having doubled the frequency of the base tone etc. The way that we differentiate violin playing the C3 tone from the guitar playing the same tone is that the harmonics of the violin are having different loudness than the harmonics of the guitar. Say the violin has 20% of the 2nd harmonic, 14% of the 3rd harmonic 10% of the 4th harmonic etc but the guitar has 14% of the 2nd harmonic, 16% of the 3rd harmonic etc. When you look at he geometric shape of their tones let's say violin will look more triangular while guitar will have more round top (just for example).

Ok, now we come to the linearity part of the story..

Let's assume that an instrument is playing a base tone of 2kHz. 2nd harmonic would be 4kHz, 3rd harmonic would be 8kHz, 4th harmonic at 16khz and so on. Now look at the frequency curve of those opamps and you will notice that it falls off sharply after 10kHz. That means it cannot play frequencies higher than 10kHz in a LINEAR fashion because the frequencies higher than 10kHz will be attenuated (muted) in a non linear fashion, because the curve is not a flat horizontal line. That means the 4th harmonic of the 2kHz base tone will not be played at the correct amplitude but it will be MUTED and that will result in a serious DISTORTION of the 4th harmonic which will result in an incorrect waveform which will be heard. That is the reason why I said that non linearity is the cause of the distortion. In this case we are speaking of the HARMONIC distortion. There is also another type of the distortion that I mentioned (intermodulation) but let's put that aside for now. The fact that frequency curve is falling off after 10kHz implicates a serious non linearity in the hearing specter, so I cannot possibly imagine that Total Harmonic Distortion (which is a sum of the distortions of all harmonic across the hearing specter) will be low enough not to be heard, in which case that test doesn't make sense at all.

Hopefully this makes sense to you.. :)
 
Top Bottom