Why DA/AD diff tests not getting the attention it deserves in measurements

[fpitas]

Don't feel bad. Seems to me those who bring this time domain argument up almost never know what it is they are claiming or anything about time domain issues. I mean just say you believe in golden eared music pros and let it go at that.

 

[dasdoing]

Mastering engineer with 20k credits and a whole shelf full of gold statues is picking a mediocre converter or the measurement can only be a way to find bad converters.

if a mastering engenier uses ADC, it means he using analog gear for analog character. So his preference (even if we assume golden ears (which we can't btw)) is probably driven by the same idea. it has to sound good, not accurate.

 

[Nickerz]

This is what you're really saying.

We've got gene splicing inside cells going on every day, cell phone coverage of the whole planet, MRI that detects every abnormality in a person's head, radar that tells us the location and intensity of tornados before they hit, GPS that delivers artillery shells to within the area of a car tire, satellites that give us pictures of Mars and the outer planets, all by virtue of measurements done by electronic instruments, but when it comes to audio, measurements done by electronic instruments are inferior to someone's ear, right?

Have I got that right?

Jim

No. I'm saying that the measurement isn't a linear indication of conversion quality.

We agree on some things...

1. ADC quality does not determine overall quality of the output material. (Unless a very poor converter is used on material that demands high quality conversion, e.g. a poor computer based soundcard with unbalanced cables for classical etc)
2. The differences are small, or at least not large.
3. People with lots of time in the game tend to cluster around a couple vendors and share similar sentiments about their sound qualities and determine given their large budgets that such investments are prudent.

So a lot of people will stick on points 1 & 2. And this is understandable. For people who are perfectly content with 1 & 2, there really is no reason to care about the measurement at all really. The higher the number, the more sure you can be all is well and at the very least the engineering is good. But truth be told, if the number isn't in the pits, send it through and move on. $\channel or features guide you to salvation.

It's really point number 3 that people have trouble with, myself included. I just never really bought the idea that these people were being bamboozled. But I also don't think these people would really disagree with points 1 & 2 that much. They might say they feel the investment is worth it and it helps their material, but in the scope of things I think if you craft the argument to them well enough they'd probably concede and agree especially given the fact that lots of people have put money on the table for A\B testing over the years. Now I'm not a huge fan of that anyways because that in itself is a marketing gimmick, but I think it is at least compelling enough for everyone to agree the differences are not large.

But what small differences there are, can be important. I see this with guitar amps. You can do back to back blind tests with high gain amps and it's pretty much just as hard to tell apart amps that play very very different and mix very different as well. Truth be told, these amps to players and listeners alike are different, but a\b testing in general just doesn't work well for this. Comes back to the Pepsi challenge bit and all that. It comes down to the fact that a lot of music production is a game of inches, really. But so is great food and a lot of things in life, tldr the details matter. It's just how much does it matter to what person?

So back to measurement. Am I saying measurement is bad? No. What I'm saying is I think this measurement is more like a binary thing. Put mathematically, I would say it is something like "is said ADC conversion at least Median - 1SD?" And if the answer is yes, you can be assured it will meet requirements 1 & 2. But where I think the measurement falls apart is for people in point #3.

That I think is much more about implementation, gain staging, etc etc. And again, very small differences at the margin here, but I reject the idea that this is merely a bunch of rich guys who are too dumb to realize they're being bamboozled. Instead I think that viewpoint is hubris on the part of underfunded, not particularily motivated engineers or amatuers who simply aren't interested in taking their craft to the bleeding edge. I am not saying people who are unwilling to go this deep are unmotivated. I am saying this point in specific is not a motivation of theirs. And so really, why bother discussing the issue with that engineer? On that point, many studio owners and engineers don't even care about the end product all that much. It is really only a small group of engineers and producers who are fanatical and take things to the very edge that become known for this. People who will try hundreds of mic and cabinet combinations. Spend weeks trying different chains in the drum room etc. It's a different mindset from the person who is bothered by anyone not satisfied at point 1 & 2.

Also, I think many of them have a form of intellectual paranoia, ever worried they might make a bad decision because of some bias, even though the only thing they can really lose in that arrangement is some money until you sell said item and maybe confidence in your judgement. Not really all that big of a deal, but some of these people on these forums treat it like their life would end if they made a decision based on their intuition or less than scrupulus scientific method. At the end of the day, I don't think this is as big as a deal as people make it out to be. Making a decision using your own methodology is not obsolete, just frowned upon in these and similar forums.

Basically I think people are more reliant on these specific measurements and more intellectually paranoid than people in slot 3 are willing to make a subjective decision, and again, I think not through error, just because a lack of a better measurement exists.

Science or scientists often try to quantify subjectively appealing traits and eventually do manage to isolate the things that people find appealing. But often in these niche concerns, such an investigation would be unprofitable. If the way we all achieved erections was by way of "musical ADC conversion," make no mistake, we'd have all the measurements available possible.

Frankly I think the insistence that people are entirely bound by whatever measurements are available and if they use any decision authority on their own they are "idiots" or they are destined to fall for their own biases or audiophiles (pajorative) reeks of intellectual insecurity. So long as everyone is motivated to understand truth and does not willingly reject any attempt to quantify these subjective things, I fail to see the issue.

And in fact, let me personalize this even more. For me, that is Motu828ES in standalone mode doing ADAT vs Burl B2 vs Crane Song HEDD. So let's call it $10 a channel vs $2000 vs $1000 a channel. But wait, not that simple. Crane Song comes with a plugin built in I can't use because I don't use ProTools. I don't want to switch to protools. Don't want to buy a license or a computer built for it etc. Too complicated to compare apples to apples, so let's use the B2... Well.. that has a nice trafo on it people's ears seem to like, and a really high quality attenuator for gain staging... Zuul is $650, and you can get an 1176 for $300 these days... So that's about the same cost as the HEDD.... The closest claim I could outright make 1:1 would be something like a UA2192, which is just a straight up nice chipset converter. And even the stingy guys on this forum agree and very much like the chipset, the famous AK5394.

So really... the argument isn't even anymore about some "huge waste of money to audio quakery" or anything like that, it's just simply a cost\channel issue. Top shelf whisky vs Jim Beam. Put even more on point, I am more warm to the idea of a DAC using a chipset many people are fond of, of an implementation people tend to believe in, that many have a subjective opinion on, than I am interested in the raw SINAR at this point. I don't think that is controversial, honestly.

 

[Nickerz]

if a mastering engenier uses ADC, it means he using analog gear for analog character. So his preference (even if we assume golden ears (which we can't btw)) is probably driven by the same idea. it has to sound good, not accurate.

Right, I fully agree here. I think these are analogous, no pun intended. I suppose what I'm saying might also be something like "engineers and producers tend to prefer ADCs that import the material in a way they expect or prefer to hear it." But also, the specific example that broke me here, was a "straight wire" converter, so one that was absolutely not being used for this purpose. Only to find not only did it perform flatly in the middle in the GS measurement, it was near the bottom for DeltaWave.

So for me, it was really a moment where after a couple years of going back and forth between 3 forums and trying to see who is flim flamming who I had to make a decision. And frankly, I am not willing to accept the physiocognitive conclusion that this guy is falling for his own biases, not intellectually rigerous enough and his ears aren't good enough to tell him that this is a mediocre converter. Oh and the company selling it is also flim flamming the guy by overpricing it through their own innept business of spending countless hours developing a mediocre product. And all the other people at that level's opinions are just copy cats of eachothers opinions, not derived by themselves... oh maybe there's even some conspiracy by them with the companies selling the products to keep margins high... comeeee on.

The claim is actually more absurd than the fact the measurement is just not good enough. That is a simple claim. The claim that there is a systematic failure of all parties to be honest or smart enough through the entire chain is the more absurd claim. It's hubris on the part of keyboard warriors and measurement jockies, people not really in that game. And they have every right to do it, and I have every right to reject the idea and accept the fact that an alternative measurement will be discovered in time which is actually a better measurement.

At this point, the conspiracy theory is people insistent strictly on these measurements with zero willingness to offer any regard to people producing records. Again, a lot of hubris in these claims. At the end of the day, I'm going align myself with people producing good work and going to worry less about measurements so long as they don't suck.

 

[Blumlein 88]

Right, I fully agree here. I think these are analogous, no pun intended. I suppose what I'm saying might also be something like "engineers and producers tend to prefer ADCs that import the material in a way they expect or prefer to hear it." But also, the specific example that broke me here, was a "straight wire" converter, so one that was absolutely not being used for this purpose. Only to find not only did it perform flatly in the middle in the GS measurement, it was near the bottom for DeltaWave.

So for me, it was really a moment where after a couple years of going back and forth between 3 forums and trying to see who is flim flamming who I had to make a decision. And frankly, I am not willing to accept the physiocognitive conclusion that this guy is falling for his own biases, not intellectually rigerous enough and his ears aren't good enough to tell him that this is a mediocre converter. Oh and the company selling it is also flim flamming the guy by overpricing it through their own innept business of spending countless hours developing a mediocre product. And all the other people at that level's opinions are just copy cats of eachothers opinions, not derived by themselves... oh maybe there's even some conspiracy by them with the companies selling the products to keep margins high... comeeee on.

The claim is actually more absurd than the fact the measurement is just not good enough. That is a simple claim. The claim that there is a systematic failure of all parties to be honest or smart enough through the entire chain is the more absurd claim. It's hubris on the part of keyboard warriors and measurement jockies, people not really in that game. And they have every right to do it, and I have every right to reject the idea and accept the fact that an alternative measurement will be discovered in time which is actually a better measurement.

At this point, the conspiracy theory is people insistent strictly on these measurements with zero willingness to offer any regard to people producing records. Again, a lot of hubris in these claims. At the end of the day, I'm going align myself with people producing good work and going to worry less about measurements so long as they don't suck.

The main issue with raw Deltawave or GS measurement is phase. Differing phase between converters ruins the null. Some GS listed converters that are very low aren't poor converters. Paul added the ability to correct for FR (ripples in that differ in the top octave with different filters), and phase (differs between filters). That phase difference at high frequencies is not something human ears care much about. It is an unfortunate fact that the decade long thread at GS has created a list of no use due to this error.

Paul has a list of corrected results on his Deltawave web page, and for the most part it makes much more sense. He used all the files from that GS null test thread.

Gearspace.com AD/DA data analysis using DeltaWave | DeltaWave documentation

DeltaWave Analysis of Gearspace AD/DA results

deltaw.org

Almost everything has an EQ and phase corrected null of better than -80 db. He further lists his PQ metric which further weights things for audibility, like A-weighting and more. I also wonder about whether a few results were just dirty. I've got one on that list which does pretty well. Another person submitted the same device twice with some kind of issue as it was much worse. Possibly hum or grounding issues I don't know.

Pro's have a lot of things and are subject to the same bias that we all are. Things like very high output levels, how the controls are arranged and more mean their workflow is good or not. And like anyone when something just is easy to use rather than being a pain to make work you feel better about it. Plus while some top pros tout high dollar gear, others will tell you ADCs are a solved problem and they don't much worry about it. They are not don't careless guys going thru the motions.

 

[dasdoing]

there are actualy lots of "audiofools" even between the best of the pros. this grammy winner swears he hears the diference between 44.1 and 48 (3min 53sec):

people in the coment section theorize that his converters(!) actually have a problem with 48

 

[pkane]

Right, I fully agree here. I think these are analogous, no pun intended. I suppose what I'm saying might also be something like "engineers and producers tend to prefer ADCs that import the material in a way they expect or prefer to hear it." But also, the specific example that broke me here, was a "straight wire" converter, so one that was absolutely not being used for this purpose. Only to find not only did it perform flatly in the middle in the GS measurement, it was near the bottom for DeltaWave.

So for me, it was really a moment where after a couple years of going back and forth between 3 forums and trying to see who is flim flamming who I had to make a decision. And frankly, I am not willing to accept the physiocognitive conclusion that this guy is falling for his own biases, not intellectually rigerous enough and his ears aren't good enough to tell him that this is a mediocre converter. Oh and the company selling it is also flim flamming the guy by overpricing it through their own innept business of spending countless hours developing a mediocre product. And all the other people at that level's opinions are just copy cats of eachothers opinions, not derived by themselves... oh maybe there's even some conspiracy by them with the companies selling the products to keep margins high... comeeee on.

The claim is actually more absurd than the fact the measurement is just not good enough. That is a simple claim. The claim that there is a systematic failure of all parties to be honest or smart enough through the entire chain is the more absurd claim. It's hubris on the part of keyboard warriors and measurement jockies, people not really in that game. And they have every right to do it, and I have every right to reject the idea and accept the fact that an alternative measurement will be discovered in time which is actually a better measurement.

At this point, the conspiracy theory is people insistent strictly on these measurements with zero willingness to offer any regard to people producing records. Again, a lot of hubris in these claims. At the end of the day, I'm going align myself with people producing good work and going to worry less about measurements so long as they don't suck.

You are making a number of assumptions that make the whole decision process suspect.

First, because some great mastering engineer uses a specific ADC model says nothing about how good that ADC really is for anyone but him. You have no idea why he uses it or what it really does for him or why he chose it. Could easily be completely unrelated to sound quality or fidelity and he may not have picked it because of it.

Then, you assume that the high recognition and accolades this engineer has received must reflect on the specific ADC unit he uses. The final product has little to do with the original ADC. A good mastering engineer produces great results because of his skills in mastering, not because he's great at converting A to D.

Then, you make a conclusion about a measurement protocol based on one poor loopback result, which may have all kinds of faults in it due to incorrect configuration during loopback recording. As @Blumlein 88 pointed out, this happens all the time. I've seen a large number of errors in the submitted recordings on the GS thread, some extremely bad and hard to explain. Maybe the wrong settings were used, maybe mismatched input and outputs, maybe bad cables, maybe the person recording it had their cell phone placed on top of the ADC unit. Who knows?

You jump right into conspiracy theory explanation and claiming hubris, but you've not done any of the homework needed to even start down this path.

 

[dasdoing]

for those who did not understand how preamps are actually used as an effect in production:

what makes them all sound diferent is how the react when they are overdriven. they don't care for how transparent it is, actually quite on the opposite. the earlier it distorts, the better

 

[fpitas]

You jump right into conspiracy theory explanation and claiming hubris, but you've not done any of the homework needed to even start down this path.

This. Everybody is welcome to their opinion, no matter how inexplicable it is to others. But we don't have to believe it.

 

[pma]

People in fact do not wish to participate in ABX tests. If you prepare one, which is quite time consuming due to proper recording, time aligning and level matching, almost no one will come with the ABX report. People like to chat, mostly. Not to work, not to learn something.

 

[Blumlein 88]

for those who did not understand how preamps are actually used as an effect in production:

what makes them all sound diferent is how the react when they are overdriven. they don't care for how transparent it is, actually quite on the opposite. the earlier it distorts, the better

And plenty of pros would say this is not smart use of preamps.

Preamps:
Saving the music industry one magic box of pixie dust at a time.

Preamps: Saving the Music Industry, one magic pixie dust box at a time... - Page 81 - Gearspace.com

Quote: Originally Posted by chessparov2.0 ➡️ I like it the best Kenny! You sound like a 'natural' on the Sax also BTW. Chris Thanks Chris. Yeah, I think I do also. And thanks, sax is coming along slowly. Not enough time left on earth to become a Progger, but I'm having fun! So, the question...

gearspace.com

 

[dasdoing]

And plenty of pros would say this is not smart use of preamps.

most will use vintage compressors, sometimes without even using it to compress at all. at the end it's the same goal: create "saturation" (how they call "good distorsion")

 

[Blumlein 88]

most will use vintage compressors, sometimes without even using it to compress at all. at the end it's the same goal: create "saturation" (how they call "good distorsion")

I don't even know if that is true. Many will use vintage compressors, but plenty use modern compressors or DSP. You only have to listen to your average music release to know we are way past good saturation. More like clipped, heavily limited and harsh as hell (but LOUD, oh yes got to have it LOUD) sound. At this point the biggest selling guys in the field would be an example of what not to do if you ask me. In any case even among good pro sound guys in the field there are lots of myths about what they do and why. In candid moments they will tell you as much.

The GS thread I linked to was about the fact even cheap interfaces have pretty good microphone preamps. Some units in the industry have these reputations as being needed for good results and it just is not true. Just like consumer high end audio, there is lots of magic pixie dust that somehow no one can measure and find. Well except for the top pros who can hear it when layered or over time or some such. Just like high end audio at the other end. ADCs aren't a problem. Mic preamps aren't a problem. There is no magic there. The magic is in how someone shapes the sound with compressors, and EQ and delay, and reverb and other more complex things even including vintage gear that has a sound because it isn't transparent like modern gear can be.

 

[Nickerz]

You are making a number of assumptions that make the whole decision process suspect.

First, because some great mastering engineer uses a specific ADC model says nothing about how good that ADC really is for anyone but him. You have no idea why he uses it or what it really does for him or why he chose it. Could easily be completely unrelated to sound quality or fidelity and he may not have picked it because of it.

Then, you assume that the high recognition and accolades this engineer has received must reflect on the specific ADC unit he uses. The final product has little to do with the original ADC. A good mastering engineer produces great results because of his skills in mastering, not because he's great at converting A to D.

Then, you make a conclusion about a measurement protocol based on one poor loopback result, which may have all kinds of faults in it due to incorrect configuration during loopback recording. As @Blumlein 88 pointed out, this happens all the time. I've seen a large number of errors in the submitted recordings on the GS thread, some extremely bad and hard to explain. Maybe the wrong settings were used, maybe mismatched input and outputs, maybe bad cables, maybe the person recording it had their cell phone placed on top of the ADC unit. Who knows?

You jump right into conspiracy theory explanation and claiming hubris, but you've not done any of the homework needed to even start down this path.

Honestly I think it's a larger conspiracy theory at this point to keep the measurements together. It both "validates" the quality, but somehow there is no way to correlate. The measurement can easily discern poor converters, we all agree on that. But after a certain point, lay people can't and don't have a preference. But people with unlimited money absolutely have a preference.

There are more claims used to support the measurement than are to just say "this measurement is limited, you'll have to use your ears and talk to people who have very good ears if you want to eek out the extra 5%." Simple claim that smells fine to me. I don't need paragraphs to explain it. I think the measurement utility is limited.

Still have one of those 245ADs? :drool:

 

[Nickerz]

most will use vintage compressors, sometimes without even using it to compress at all. at the end it's the same goal: create "saturation" (how they call "good distorsion")

Howie is using his converters that way, it's part of it for sure. Dcs is noted as "crunchy" (probably because of the very old tech involved, a sort of hifi old sampler thing going on). But I'm talking about his straight wire straight line capturing converter.

 

[fpitas]

This just in: people making music utilize special effects, which are not applicable to high fidelity reproduction.

 

[pkane]

Honestly I think it's a larger conspiracy theory at this point to keep the measurements together. It both "validates" the quality, but somehow there is no way to correlate. The measurement can easily discern poor converters, we all agree on that. But after a certain point, lay people can't and don't have a preference. But people with unlimited money absolutely have a preference.

There are more claims used to support the measurement than are to just say "this measurement is limited, you'll have to use your ears and talk to people who have very good ears if you want to eek out the extra 5%." Simple claim that smells fine to me. I don't need paragraphs to explain it. I think the measurement utility is limited.

Still have one of those 245ADs? :drool:

Measurements are supported by objective (and often scientific) studies and the ability of others to replicate the results. Because of this, they can be challenged and improved over time. There is no validation possible for someone's preference or opinion, regardless of who holds them. These are only applicable to that one person. There is no known way, other than the Vulcan mind-meld, to transfer them to someone else, so that they can be examined, evaluated, and understood.

 

[Nickerz]

Measurements are supported by objective (and often scientific) studies and the ability of others to replicate the results. Because of this, they can be challenged and improved over time. There is no validation possible for someone's preference or opinion, regardless of who holds them. These are only applicable to that one person. There is no known way, other than the Vulcan mind-meld, to transfer them to someone else, so that they can be examined, evaluated, and understood.

Intuition and trust. Again, I'm not entirely rejecting the measurement, I think it is a valid way to root out bad converters, but you'd have to come up with a more compelling reason to me why very talented individuals with great ears are buying "mediocre" gear for huge cash other than they're gullible and there's a cabal of companies selling hugely overpriced snake oil boxes because a measurement everyone says people can't use to a\b test themselves is the way. The measurement isn't disproveable either, so I don't see how that is "science" and this is bad decision making any more or less.

 

[DonH56]

Measurements to be valid are repeatable and verifiable by others. I am not sure what you think science is if measurements are not part of it.

 

[Nickerz]

Measurements to be valid are repeatable and verifiable by others. I am not sure what you think science is if measurements are not part of it.

The measurement is repeatable. But there's no way in verify it represents "conversion" quality other than measuring DA>AD. The measurement measures the quality of preserving previously digitized files. It is absolutely resonable to assume it is highly predictive of the conversion of actual analog data to digital. But not the end all be all.

We also all agree that AD and DAs lose something. So certainly, what is lost and what people consider to be the better conversion is still ultimately subjective. Again, the measurement isn't useless, that's for sure.