GoldenSounds passes apparently ABX test for DACs (NOT Really)

[Music1969]

I'm not 100% sure I get what you're looking for

So far we only know what the differences between the filters are, in the digital domain

Simply for completeness, we should see the differences between filters in the analogue domain, ie DAC output

Because ultimately he is listening to output of DAC (plus amp plus tranducers).

As far as levels, just let him do his standard DAC measurements, hes been doing this a while.

Its impossible to replicate ALL the music levels and do these measurements listed above

 

[solderdude]

I'm slightly confused here - this being a software ABX, where do you want in camera to avoid a potential visual cue cheat? We can definitely get multiple viewpoints with recording on, I just need to know which ones you need to see.

LOL... why not simply wear a blindfold and use the actual output sent to his headphone as the sound to the video stream ?

And let him do a simple output voltage measurement of a 0dB 100Hz sinewave at the listening volume on the output of the amp so we can calculate peak levels of what he is listening to.

 

[Mad_Economist]

Simply for completeness, we should see the differences between filters in the analogue domain, ie DAC output

Sure, I'll get this in the pipe - my guess is it'll be mid this week at the earliest.

LOL... why not simply wear a blindfold and use the actual output sent to his headphone as the sound to the video stream ?

And let him do a simple output voltage measurement of a 0dB 100Hz sinewave at the listening volume on the output of the amp so we can calculate peak levels of what he is listening to.

Cameron needs to see in order to click the the UI elements. I guess a custom macro board could be set up...

 

[solderdude]

VR classes and stream the video output + real audio output to the headphones ?

 

[Mad_Economist]

VR classes and stream the video output + real audio output to the headphones ?

Hahaha, I love this idea. I'll run it by the management and see if we can expense a Quest for this!

 

[solderdude]

Cameron looks like the kind of guy that already owns some VR gaming gear
He certainly has the age and funds for it ..

 

[MaxwellsEq]

Cameron needs to see in order to click the the UI elements. I guess a custom macro board could be set up

There are many UI automation tools that can reliably and repeatedly "playback" actions. There's an entire class of corporate robotics used to scrape between one system and another (ideal for non API integration)

 

[SIY]

I'm slightly confused here - this being a software ABX, where do you want in camera to avoid a potential visual cue cheat? We can definitely get multiple viewpoints with recording on, I just need to know which ones you need to see.

You'd want coverage of a 360 degree field of vision. I'm not enough of a videography maven to specify that, but there's lots of them around. I could still think of other ways to game it (which would involve a confederate), but that coverage would complicate any malfeasance. If I were putting up money bets, I'd want a mentalist to look the whole thing over; I'm a semi-knowledgeable amateur, but the pros are FAR better equipped to spot the holes in the protocol.

Would him wearing a gopro work well here, as an addition? With that, you should be able to see everything in his POV

Would that interfere with the headphones? I'm not sure what this gets you beyond the external cameras.

 

[mocenigo]

I'd rather see better controls.

Yep. I am always wary of tests that involve sending generically labeled files. If I know that DAC A has measurably more treble than DAC B, even if the difference is not audible, I expect compression algorithms like ALAC or FLAC to deliver a slightly larger file for A. Same if A has more noise. So, just the act of reading the file size may trigger some bias.

Even if everything else is done properly.

So send WAV. But some people by habit compress everything (I, for instance).

 

[krabapple]

My opinion: Whether it's "forensic listening" or focused listening that resembled casual listening, I don't think it's really important. To me this is about establishing a clear line for capital-T Transparency, in the sense of there being no realistic possibility of hearing the electronics, vs. everyday transparency in which listeners are merely very unlikely to hear the electronics in normal use cases with normal music.

In other words, I want to be 100% correct when I say "people can't hear this" or "people can almost never hear this". There's actually a big difference in the two statements. So to me, this type of test where the boundaries are pushed is quite interesting.

Whereas I don't think that's really important. The difference between 100% and 99% actually isn't that big.

It's a psychological thing, deciding what's 'important'. Anxiety over the possibility that I haven't quite reached '100%' transparency seems more an annoyance than useful,
and to me, this type of test simply defines what a 'corner case' can be. That's interesting to a degree. But it has 0% practical bearing on my enjoyment of home audio, and on the ubiquitous, dubious difference claims of golden eared audiophiles and their print/online/corporate enablers. And that last, to me, as a fan of this hobby, seems very important to remain clear about.

 

[krabapple]

I also care about being completely correct. I want to use the word "never", but only when it's actually true. Never means zero, not 0.00001. And so these edge cases are (in a way) the most interesting.

But who was/is saying 'never' about DAC filters? Much less DACs, or amps, or even cables?

I'm sure *someone* has...just as I'm sure DACs/amps/cables can sound different under some circumstances.

But is it a general thing? Not IME. The most vigorous 'objectivists' I've ever known all acknowledged qualifications: that corner cases can exist.

They just aren't that important. And crucially, they aren't dispositive re: the wild claims of difference so typical in this hobby since at least the 1980s.

 

[krabapple]

I think this is where we disagree. I want to know if gear is transparent in all cases, or just most cases. Just for the sake of knowing, not because it actually matters in real life.

As has been pointed out, you'll never know if 'all cases' have been covered, and statistics for this won't offer absolute certainty.

But I agree that limit-testing results are a legitimate topic of audio science discussion. That science also tells us why they aren't likely to matter during normal listening.

 

[Mad_Economist]

There are many UI automation tools that can reliably and repeatedly "playback" actions. There's an entire class of corporate robotics used to scrape between one system and another (ideal for non API integration)

In the case of ABX, the user needs to be able to select the references and X, thus the issue of automating it completely.

You'd want coverage of a 360 degree field of vision. I'm not enough of a videography maven to specify that, but there's lots of them around. I could still think of other ways to game it (which would involve a confederate), but that coverage would complicate any malfeasance. If I were putting up money bets, I'd want a mentalist to look the whole thing over; I'm a semi-knowledgeable amateur, but the pros are FAR better equipped to spot the holes in the protocol.

Just so's I'm clear, the case you want to hedge against is there being some visible cue that the tested is using to differentiate - is it something in the software covertly showing which is playing, or like, an LED off camera? It's meaningfully easier to show that the software isn't cheating or enabling cheating than it is to show all possible cheating tools (e.g. there could be a small remote vibrating device hidden in his pocket) which could be physically present.

Would that interfere with the headphones? I'm not sure what this gets you beyond the external cameras.

With an acoustically low Z design like the Susvara, not substantially. The main merit of a "POV camera" is that you see everything that Cameron can see if it's field of view is wide enough (or, as @solderdude suggests, we limited his FOV to what the camera can see).

Again, this can't preclude cheating. You being there in person wouldn't be able to conclusively preclude that unless you fully controlled the signal chain and switching setup. But there's a point where you have to decide whether you're taking this as an experiment presented in good faith by an earnest (if, IMO, mislead in some of his views) person, or an attempted deception.

 

[kemmler3D]

and on the ubiquitous, dubious difference claims of golden eared audiophiles and their print/online/corporate enablers.

I thought it would go without saying that the ABX here obviously has nothing to do with the kinds of claims subjectivist audiophools make, but maybe not.

 

[MaxwellsEq]

In the case of ABX, the user needs to be able to select the references and X, thus the issue of automating it completely

You map the gui function to keys. Any gamer can operate the ASDZX keys without looking.

 

[danadam]

Cameron needs to see in order to click the the UI elements. I guess a custom macro board could be set up...

Foobar's ABX comparator has keyboard shortcuts. They show up when you press left Alt: a, b, x and y should be obvious, 1 for "X=A, Y=B", 2 for "Y=A, X=B", n for "Next trial".

 

[danadam]

As has been pointed out, you'll never know if 'all cases' have been covered, and statistics for this won't offer absolute certainty.

@kemmler3D would not grant the Nobel prize for discovering the Higgs boson

 

[Music1969]

Sure, I'll get this in the pipe - my guess is it'll be mid this week at the earliest.

thx. Perhaps his usual IMD vs level test might show something interesting also, between these 2 filters

 

[kemmler3D]

@kemmler3D would not grant the Nobel prize for discovering the Higgs boson

Hey, I'm really willing to settle for 2 sigma here, I'm not that much of a doubter!

 

[sonitus mirus]

I took a self-imposed break, but has there been any progress on providing output files from the DACs? Initially, this was supposed to be provided immediately; however, I still haven't seen anything. Did I miss it?