RTINGS study on distortion audibility in headphones and IEMs

[aatoma]

Dear community of addicted to fun and learning, I just found this on RTINGS.com and thought it could be of general interest: The Surprising Truth About Headphone Distortion

 

[voodooless]

Great that they did all the work to do a thorough investigation! The results, though, are not very surprising At least it is another good reference to add to the list.

 

[PierreRtings]

I'm here if you have any question on this. Thanks for the share.

 

[voodooless]

I'm here if you have any question on this. Thanks for the share.

It would be interesting to see if there is a correlation between age and distortion detection ability. Maybe the sample size is too small for anything conclusive, but still

 

[Matias]

L12 is that golden ear audiophile.

Nice study, thanks for sharing!

 

[PierreRtings]

It would be interesting to see if there is a correlation between age and distortion detection ability. Maybe the sample size is too small for anything conclusive, but still

Our group was young in general. very week correlation... I'm the dot at the end at 51....

 

[TurtlePaul]

Cool study. Interesting that Tracy Chapman - Fast Car finds a standout flaw in the Truthear Zero: Red.

 

[EERecordist]

Thanks for posting it. Rtings is the source I use for ASR-like measurements of TVs and usually I focus on gamut which continues to improve. So placing the study there will reach that audience. Let's hope these studies get an even wider audience. And since many consumers use Bluetooth IEMs, that would be a good addition for the future, IMO.

 

[kemmler3D]

Great test and write-up, a real contribution to audio science IMO.

Reading the conclusion I might ask whether there's a caveat worth adding - namely that distortion isn't something to worry about - except maybe if you're applying a ton of EQ? Pushing an actual headphone (as opposed to a recording of the same) by +10-12dB in the bass region seems like a good way to create detectable distortion with some models.

Is phase distortion / group delay covered in the uncorrelated distortion test?

 

[PierreRtings]

Great test and write-up, a real contribution to audio science IMO.

Reading the conclusion I might ask whether there's a caveat worth adding - namely that distortion isn't something to worry about - except maybe if you're applying a ton of EQ?

Fair enough, we did something on EQ as well where we did remeasurements, including distortion and it was not an issue in that particular study. (I'll let you google it would not be good practice to link to our site on ASR). If you are referring specifically to the LCD-S20 case that we discuss in the article it's not so much that EQ made distortion audible but noise present in the recording in the bass. it's a different scenario than simply equalizing your headphones which don't affect the environment noise.

 

[b73i57dc3f]

Using binaural recording, we captured everything the headphones produced: both linear (frequency-response-related ) and nonlinear distortions. Then we removed the frequency response, leaving only nonlinear distortion and level differences between headphones. After this, we normalized all these recordings at -16 LUFS , so that listeners heard distortion, not frequency response or level differences.

Do I understand correctly that no equalization was applied during the recording, only afterwards? Unlike in this previous study:

3. For each headphone, the source tracks were pre-equalized using the correction filter for that headphone. Additionally, overall gain differences between the headphones were measured and accounted for. Playback of each headphone and its correction filter thus resulted in nearly identical acoustical output levels and spectral balance for the recordings.

the LCD-S20 have lower sensitivity, requiring more amplification to reach the same SPL, which could raise the digital noise floor

I don't understand this part. What is meant by digital noise floor, and how would it be raised by more amplification?

 

[phoenixdogfan]

Would be interesting to see the same test run on high end speakers, see how audible distortion is at higher volume levels.

 

[PierreRtings]

Do I understand correctly that no equalization was applied during the recording, only afterwards? Unlike in this previous study:



I don't understand this part. What is meant by digital noise floor, and how would it be raised by more amplification?

This is correct that equalization was applied to the captured audio, but in essence it is essentially the same thing. It was applied in the digital domain... If that matters, actually before exporting to an actual wav file so it was all part of the recording sequence.

The Temme and Olive Study that you refer to, I'd probably let Sean explain it better but they where going for a target response for the replicator headphone, it was different in this case, the idea was to remove the response so it can be compared to the original untouched recording, we where not trying to alter the response of the headphone used for playback, the importance was to not have headphone transfer function twice, from the recording plus from the headphone used to listen.

for the misunderstood part, keep in mind our recordings had to cover a wide range of amplitude, without clipping digitally, a less sensitive headphone, to get to the same spl as the others , require more voltage gain, effectively the noise floor is closer to the 0 dBFS or in other words, having less effective bits. that said, it is a small part and only one Hypotheses. As we showed, it is much more about the environmental noise, that we had to Increase by EQ.

 

[b73i57dc3f]

in essence it is essentially the same thing

It's not, because the distortion of the headphone depends on playback level, and equalization would effectively change playback level, at least for part of the spectrum. The SNR of the recording depends on playback level, too; if the LCD-S20 was equalized during recording, it could have hugely improved the SNR of the recording in the low frequencies, though it would likely also have increased the distortion of the headphone, possibly by quite a lot.

a less sensitive headphone, to get to the same spl as the others , require more voltage gain, effectively the noise floor is closer to the 0 dBFS or in other words, having less effective bits

I still don't understand. As far as I know, in the digital domain, there's only quantization noise, and a higher signal level actually improves SNR, not the other way around. In the analog domain, a higher signal level typically also improves SNR, as seen in many of the measurements here on ASR. That is, as long as the same DAC/amp is used; if a different DAC/amp with hugely inferior SNR was used, that would of course explain it.

 

[PierreRtings]

It's not, because the distortion of the headphone depends on playback level, and equalization would effectively change playback level, at least for part of the spectrum. The SNR of the recording depends on playback level, too; if the LCD-S20 was equalized during recording, it could have hugely improved the SNR of the recording in the low frequencies, though it would likely also have increased the distortion of the headphone, possibly by quite a lot.


I still don't understand. As far as I know, in the digital domain, there's only quantization noise, and a higher signal level actually improves SNR, not the other way around. In the analog domain, a higher signal level typically also improves SNR, as seen in many of the measurements here on ASR. That is, as long as the same DAC/amp is used; if a different DAC/amp with hugely inferior SNR was used, that would of course explain it.

Sorry I don't understand what you mean in your distinction, during recording vs after. We cannot have that degree of precision in equalization in the analog domain, we capture a recording, we apply a EQ curve to it, and save it to wave. Did you mean "before recording"? During recording have no meaning, unless you are talking the analog eq in a recording console, whatewver we do it will be done after the analog to digital conversion or if you prefer, after capturing the pressure variation in the HATS. and the EQ applied will change the level of what is captured, but yes we could have pre equalized the file before playing it. My guess it that there might be more artifact, since the measurement that we take, to make the EQ curve from is taken from the response that we measure and we can only measure a response, obviously, after recording it. So the measurement and the recording are done together, in the same go.

You are correct that "raising the digital noise floor" is technically imprecise. And my explanation didn't help. Sorry. In a calibrated system we cannot each time adjust the level of the amplifier unless we had a way to automate it And in all cases, we need 45 dBs of headroom to reach 120 dB SPL and start at 85 dB SPL. The music program, before playbackhas a Fix FS. Think of a gain increase as Normalizing a file, It require more dBs of gain, you bring everything up, digitally. I agree that my expression was not correct, you bring the noise from the original recording up. Not just the Digital floor. The signal Was Listen Sound check analyser software - RME Fireface UFX 2 - Headphone amplifier - B&K5128 - Some mic preamps/conditioners - Firteface UFX2 - and back in the analyzer software.

 

[PierreRtings]

Great test and write-up, a real contribution to audio science IMO.

Reading the conclusion I might ask whether there's a caveat worth adding - namely that distortion isn't something to worry about - except maybe if you're applying a ton of EQ? Pushing an actual headphone (as opposed to a recording of the same) by +10-12dB in the bass region seems like a good way to create detectable distortion with some models.

Is phase distortion / group delay covered in the uncorrelated distortion test?

Yes phase distortion and Group delay is included in non-coherent distortion.

 

[AbstractAlgebra]

I'm here if you have any question on this. Thanks for the share.

It would be also interesting to include some perceptual metrics, such as rnonlin or GM. As far as i remember, you already mentioned them in the first article. Did you evaluate any?

 

[PierreRtings]

It would be also interesting to include some perceptual metrics, such as rnonlin or GM. As far as i remember, you already mentioned them in the first article. Did you evaluate any?

We wanted to actually, early in the project I tried to calculate R NonLin based on our database of measurements of distortion and apply it to music trough a Volterra serie. (My boss probably think I spended way too much time on this ). I did get a bunch of scores for tons of headphones, but the thing with perceptual metrics, they have to be correlated with preference ratings to be meaningful, and from there you can scale the R values to the chances that people would like a headphone over the other. In our test, listeners where struggling to detect any differences between these recordings. Asking them which one they "prefer", is in these situation irrelevent, you need clear difference to prefer one over the other.

 

[b73i57dc3f]

Did you mean "before recording"?

Yes, sorry that wasn't clear.

Think of a gain increase as Normalizing a file, It require more dBs of gain, you bring everything up, digitally. I agree that my expression was not correct, you bring the noise from the original recording up. Not just the Digital floor.

I think I understand what you mean, but in that case, I think there's a misunderstanding on your part: everything means everything of the original recording, not just noise -- so this doesn't change the SNR, provided we ignore the aforementioned quantization noise and analog noise.

 

[Geert]

@PierreRtings . From the test: "Using binaural recording, we captured everything the headphones produced: both linear (frequency-response-related ) and nonlinear distortions. Then we removed the frequency response, leaving only nonlinear distortion and level differences between headphones. After this, we normalized all these recordings at -16 LUFS , so that listeners heard distortion, not frequency response or level differences".

What target did you use for the frequency response correction? An average between all headphones, flat, or the target response you normally use? The reason I ask is that some of the headphones, like the HD650, have a weak bass response and could maybe show noticeable distortion at lower SPL when you eq them towards the target response. (And you would of course need to use some bass heavy music to notice). Just trying to understand if the test already covers this aspect.

(Source https://www.rtings.com/headphones/reviews/sennheiser/hd-650).