SINAD Measurements

[Rja4000]

If the worst sounding cap and best sounding cap (assuming we detect a true difference) do not impact THD or noise (SINAD) in any way, then what is causing the detected timbre shift? The answer, I think, begins with a reflection on transient (one-off) distortions vs. periodic (static) harmonic distortions.

The ultimate goal is to identify a measurement (or a set of measurements) which results correlate with the ABX difference.
I mean for the final product/device.

 

[signalpath]

The ultimate goal is to identify a measurement (or a set of measurement) which results correlate with the ABX difference.

Audio professionals have been chasing the elusive "perception / measurement" correlation for decades. The number of design and operational permutations in circuits vs. myriad performance metrics is, effectively, infinite. I don't think we'll ever see a satisfactory measurement / perception correlation matrix. A lot of work has been done to assess human preference of different harmonic series flavors, such as "tape emulators" and "tape saturation" and "tube distortion" apps. That's a good starting point.

 

[solderdude]

I don't think it is a good thing to mix preference with measurements. In your business there will be a lot of preference going around (along with quality and longevity).

Do you think the differences can be recorded. Given you are in the recording side of things one would think a decent recorder could easily catch these differences when they are that audible.
Of course one can always say... well the playback chain or listener is not good enough.

I can also understand op-amps performing differently with 40dB (or more) in gain where us reproducing folks often need 1x to 5x gain or so (headphones) in which case opamps perform differently.

You still haven't mentioned which op-amps are the good and bad sounding ones.

 

[Rja4000]

A lot of work has been done to assess human preference of different harmonic series flavors

Well, "preference" and "difference" are 2 different things.

Nowadays, we could probably easily recreate artificially what we prefer, if we know it better.
Convolution and other techniques are improving and will allow more flexibility on that (the success of various convolution engines in pro sound is there to prove it).
Also, "preference", or "taste", probably requires less accurate reproduction. You probably mainly need an "amount" knob of some...

But we're still looking for best "transparency", or whatever name we give to the most accurate transcription of the original.
And that's where we want to identify a set of measurement to define what is "transparent enough".

From what you explain about your experience, you think we're not there yet.
So, let's make sure it's true and reproductible (ABX), let's try to identify possible causes, let's try to find a measure (set) that correlates to those.

 

[solderdude]

I think signalpath is very much aware how much the recorded signal from microphones and instruments is altered in studios (in various ways) before it is sold as a product.

I reckon 'good sounding' reproduction would exceed 'the most accurate amplification' ?

 

[scott wurcer]

I was there for Earl's AES paper, and had a lengthy chat with him afterwards, and later via emails.

I've exchanged a few emails with Earl, he's the wrong guy if you want to listen to op-amps. He has been outspoken in public that the transparency of electronics is a done deal.

Input != output is always measurable, there is nothing that excludes transient effects. I'm waiting for the bit perfect files sound different depending on what brand of SD card they are on, but I don't think you will go there (for the record just joking).

 

[restorer-john]

I'm waiting for the bit perfect files sound different depending on what brand of SD card they are on

You mean I wasted my money?

From here:

 

[amirm]

I've exchanged a few emails with Earl, he's the wrong guy if you want to listen to op-amps. He has been outspoken in public that the transparency of electronics is a done deal.

Indeed unless something has changed, he uses a mass market AVR for his amplification.

 

[Xulonn]

I think signalpath is very much aware how much the recorded signal from microphones and instruments is altered in studios (in various ways) before it is sold as a product.

@signalpath also is heavily involved in the recording of live classical music - a not very popular (percentage-wise) set of mostly acoustic genres that has fans who are fanatics about accuracy. Classical recordings are a small percentage of both albums sold and streaming services:

NYT: While 2.5 percent of album sales in the United States are classical music, it accounts for less than 1 percent of total streams, according to Alpha Data, a tracking service.

Classical recordings, to the best of my knowledge, are not typically "enhanced" by "artistic" the application of heavy-handed electronic processing by recording and mastering engineers as with most amplified pop, rock, jazz, hip-hop, etc. In the latter genres, there is often no live analog with which to compare the recording.

I assume that most of us (the ASR "regulars") are devoted to the pursuit of most accurate (lowest possible noise and distortion) possible during the processing of the audio source through our electronics to our loudspeakers (or headphones for many). But only for fans of pure acoustic music - or recordings of amplified music using microphones in front of the performers - can there be a logical desire to accurately reproduce the actual musical event rather then simply enjoy the results of the collaboration of the musicians and engineers.

(And of course, there are genres of computer-generated electronic music that don't even use microphones - and that is a whole 'nuther universe.)

 

[March Audio]

I've made no claims to being an expert listener, or even that my AB testing is accurate. .

Well you have said;

Two IC op-amps, both of similar phase performance (far less than 0.1 degree in audio band), identical FR, similarly low distortion components. Can they sound different? Yes. They can, and often do.

So, is this something you have personally experienced or not? Can you tell the difference or not?

Can you post the measurements of the two devices you have compared and been able to readily blind identify differences.

 

[March Audio]

Audio professionals have been chasing the elusive "perception / measurement" correlation for decades. t.

I think the reality is more aligned with people chasing their own tails due to not running proper controlled subjective tests.

 

[scott wurcer]

As amrim says the switching between samples is very critical, AC coupling and mean (DC) values on the samples can generate device specific initial transients. I pooped an ABX party at a meeting of the Boston chapter of the AES by demonstrating how easy it was to learn the pattern and easily get 10/10 for any two devices always. Letting the user switch rapidly back and forth as they choose only makes this easier.

 

[KSTR]

And this brings up an interesting question. If the worst sounding cap and best sounding cap (assuming we detect a true difference) do not impact THD or noise (SINAD) or FR in any way, then what is causing the detected timbre shift? The answer, I think, begins with a reflection on transient (one-off) distortions vs. periodic (static) harmonic distortions. But that's a conversation for another time.

We may assume any disturbance the cap does to the signal is small and a proper linear part of the transfer function can be derived without problems and uncertainities. To compare caps or similar opamp circuits etc against each other, or to compare a loopback recording to the original, I may suggest again what I wrote before, in another thread, dealing with differential testing with arbitrary signals:

Minor and irrelevant linear response changes must all be fully factored out before subtraction, otherwise they will dominate the residual, masking what we really wanted to see: harmonic and other distortion products (including any kind of "memory distortion"), spuriae, correlated noise, processing artifacts of any kind etc.
A way to do this is making a robust(!) full range transfer function measurement of the DA-->DUT-->AD first, obtaining a precision impulse response. An RME Adi-2 Pro is ideal for this, as it can be modded to be fully DC coupled on the ADC (the DAC already is). Zero caps in the signal path and zero high-pass effect, other than that from the DUT.

Then convolve the original file with this IR (precision required here, time-domain convolution preferred as the kernel is short).
This is the file we have to compare the loop-back recording with, not the original file.
For a DUT A vs B comparison it's a bit more elaborate but no need to get deep in the details right now.

Additionally, several loop-back recordings of the same setup should be made and compared against each other to establish a baseline measurement, with the null depth only limited by drifts, noise etc.

And of course, the residual must be listenened to and rated on a preceptional basis. For example, an undistorted copy of the original, with little frequency response changes, embedded in uncorrelated white and 1/f noise and maybe some hum or other discrete frequency spuriae is completely benign...

 

[pma]

Upload the files, please. The more examples with very low distortion but still audible differences, the better. I assume they are all using the same test signal?

I am taking on the challenge. Below is the link to test files

https://www.dropbox.com/s/y65xwnsfafqbsqj/boston.zip?dl=0

2 files recorded with different opamps, in 96/24 resolution. The files are 32-bit coded wav. When downloading from dropbox, one does not need to sign-up. Click on the arrow top right and choose Direct Download. Do not try to play wav files directly from dropbox, it is impossible, as they are in a zip archive.

2 times 8/10 ABX run would be sufficient. The results are summed as 16/20 if 2 successful runs of 8/10 were performed. It is difficult to keep attention if there are only slight differences. The files are time aligned. Level matched by nature, opamps keep gain.

 

[pkane]

I am taking on the challenge. Below is the link to test files

https://www.dropbox.com/s/y65xwnsfafqbsqj/boston.zip?dl=0

2 files recorded with different opamps, in 96/24 resolution. The files are 32-bit coded wav. When downloading from dropbox, one does not need to sign-up. Click on the arrow top right and choose Direct Download. Do not try to play wav files directly from dropbox, it is impossible, as they are in a zip archive.

2 times 8/10 ABX run would be sufficient. The results are summed as 16/20 if 2 successful runs of 8/10 were performed. It is difficult to keep attention if there are only slight differences. The files are time aligned. Level matched by nature, opamps keep gain.

So can you distinguish these in an ABX test?

 

[pma]

So can you distinguish these in an ABX test?

No, at least not yet. I will send you a disclosure in a PM, as you also play with file analysis methods.

 

[signalpath]

Input != output is always measurable, there is nothing that excludes transient effects.

I'm not convinced that simple, static (periodic) harmonic testing (SINAD / THD) identifies 'one-off' transient induced distortions, "AM vs. FM distortions" (John Curl's terminology), feedback-generated transient phase modulation effects (very possible related to ICs with massive loop gain, like 140dB loop gain in some cases), very-high-order-dominant harmonic series (which would make the harmonic generator app earlier in this conversation a very instructive exercise), conventional distortion measurements at lower-than-full-scale operating levels (i.e., real listening levels far below FS -- we know that THD ratios often worsen at lower levels), the dynamic (vs. static) impact of power supply topology on AB perceptible timbre under a wide range of operating conditions (another AB test we do extensively -- some power supply topologies have detectable artifacts that don't show up in static THD testing -- sometimes as simple as a regulator*), filter performance ('ringing' and time smear fx) and its impact on sonic quality (missed entirely by common THD metrics), and likely many others.

Beyond simple-static THD testing, there are a range of possible distortion vectors, some well-known, some obscure and not widely studied and in some cases (e.g., old Gerzon papers) bizarre and impenetrable -- which could contribute to the empirical reality that some "higher simple-THD" devices sound more transparent than certain "vanishingly low THD" products that have AB-detectable timbre or spatial issues. The IC test paper that Amir shared even noted this: certain ICs with higher single-reading-THD were deemed more accurate than some with lower THD. We have experienced this over and over and over in 30 years of AB characterization, as has anyone else deeply involved in high quality audio design. Simply ask around -- ask any long-experienced audio design engineer if lower simple-THD readings always correlates with "improved perceptual transparency." (not talking 1% to 0.00001%, but common audio comparisons today).

And .... sigh .... this circles back to my concern with ASR forum. Namely, (1) lower THD does not necessarily mean "better", and (2) -120dB SINAD is not a scientifically irrefutable "guarantee of transparency." The site's owners, if they claim to be scientists, should admit both realities, and make the simple adjustments. They are doing the audio and scientific community a disservice by editorializing personal opinions over fundamental peer-review.

Real science. No personal biases or presumptive aspersions. Professional. And way more fun than "my religion v. your religion" which, frankly we get too much of in the "real world" today and is causing grave psychic damage to our nation and world.

* A year ago, we brought in around ten different SMPS +12V "line lumps" for an AB listening trial with a new product -- in fact the product we're announcing at NAMM next week. More than half of the supplies tested were 8/10'ed rejected. Audible issues. Subtle, yes, but audible. Most people probably wouldn't care or hear the subtleties. We care. And, no, none of these power supplies caused any change to THD, IM, noise, slew, FR, PH, etc.. And, to address all those who say "you didn't really hear any difference" -- I would say, you may be very correct. I don't pretend to be gospel. Let's hope others take the same high road. Thanks.

 

[scott wurcer]

I'm not convinced that simple, static (periodic) harmonic testing (SINAD / THD) identifies 'one-off' transient induced distortions, "AM vs. FM distortions" (John Curl's terminology), feedback-generated transient phase modulation effects (very possible related to ICs with massive loop gain, like 140dB loop gain in some cases)

I never said that everything you mention IS measurable 'one-off' or not. Then you descend into stuff that is technically weak, debunked by several folks. Ron Quan published a follow up paper, modern IC amps for the most part were unmeasurable the worst were valve amplifiers (surprise). Bob Cordell has also published some measurements. The high open-loop gain stuff OTOH is total nonsense with no technical basis at all and yes very un-scientific.

If you read my past posts you will see that I have asked for THD to be separated from noise and for measurements of distortion only down to vanishing output levels

 

[RichB]

I'm not convinced that simple, static (periodic) harmonic testing (SINAD / THD) identifies 'one-off' transient induced distortions, "AM vs. FM distortions" (John Curl's terminology), feedback-generated transient phase modulation effects (very possible related to ICs with massive loop gain, like 140dB loop gain in some cases), very-high-order-dominant harmonic series (which would make the harmonic generator app earlier in this conversation a very instructive exercise), conventional distortion measurements at lower-than-full-scale operating levels (i.e., real listening levels far below FS -- we know that THD ratios often worsen at lower levels), the dynamic (vs. static) impact of power supply topology on AB perceptible timbre under a wide range of operating conditions (another AB test we do extensively -- some power supply topologies have detectable artifacts that don't show up in static THD testing -- sometimes as simple as a regulator*), filter performance ('ringing' and time smear fx) and its impact on sonic quality (missed entirely by common THD metrics), and likely many others.

Beyond simple-static THD testing, there are a range of possible distortion vectors, some well-known, some obscure and not widely studied and in some cases (e.g., old Gerzon papers) bizarre and impenetrable -- which could contribute to the empirical reality that some "higher simple-THD" devices sound more transparent, and some "vanishingly low THD" products have AB-detectable timbre or spatial issues. The IC test paper that Amir shared even noted this: certain ICs with higher single-reading-THD were deemed more accurate than some with lower THD. We have experienced this over and over and over in 30 years of AB characterization, as has anyone else deeply involved in high quality audio design. Simply ask around -- ask any long-experienced audio design engineer if lower simple-THD readings always correlates with "improved transparency." (not talking 1% to 0.00001%, but common audio comparisons today).

I agree with the above. I found the AT522NC (NCore) and Benchmark AHB2 amplifiers sounded different when voltage matched in my system when compared to the ATI AT6000 and AT4000 amps. It's an imperfect test because it was SBT and switching was < 10 seconds but not instantaneous. This data along with specifications and measurement were sufficient for me to make an informed buying decision.

There amplifiers are architecturally different than the prior class A/B linear/unregulated power supply amplifiers. Both, implementation include advanced engineering that have produced significant measured improvements in SINAD, especially at the low power levels for typical listening levels where detail is assessed.

SINAD data is the among the best data available to select well designed products. I don't see the need to separate THD from noise since less is better for both but there is no harm in providing both.

- Rich

 

[signalpath]

I never said that everything you mention IS measurable 'one-off' or not. Then you descend into stuff that is technically nonsense, debunked by several folks. Ron Quan published a follow up paper, modern IC amps for the most part were unmeasurable the worst were valve amplifiers (surprise). Bob Cordell has also published some measurements. The high open-loop gain stuff OTOH is total nonsense with no technical basis at all and yes very un-scientific.

So, you agree that "120dB SINAD guarantees transparency" ? You agree that lower THD is always perceptively better? These are my only concerns with this ASR site. If you agree with either of these statements, the onus is on -you- (and ASR) to deliver the abundant peer-reviewed proof of your position. Otherwise, it is opinion. And opinion is GREAT -- we all have opinions based on our personal experience. Just don't represent it as "audio science."

You know, some of the stuff mentioned above has been discussed in audio circles for years, theories, postulates, with some interesting science behind the ideas. Yes, much of it is postulate (none of it is peer-disproved in Journals -- or if it is, show me.). But some vectors (above) are not disputable: (1) generally higher THD at lower operating levels, (2) dominant high-order resulting in perceptively worse performance, though lower THD number, (3) varying perceptual impact of dynamic performance vis a vis power supplies (not measured by THD), and (4) digital filter impact on perceptual performance.

But, alas, this all returns to peer-reviewed AB testing. If there is a preponderance of Journal evidence to support a statement, I'm in. If there's not, it's not science, but opinion.