• Welcome to ASR. There are many reviews of audio hardware and expert members to help answer your questions. Click here to have your audio equipment measured for free!

DAC audible differences are finally proven with ABX?

Most work in science doesn’t challenge the established science, it backs up what’s established.
Possibly true, but the work of good scientists is rarely aimed at confirmation of "what's established". The results under discussion here absolutely and irrevocably fail to back up what's established, and fairly cry out for investigation. @pablolie ennui notwithstanding, I find the root cause of the reported observations to be intensely fascinating. Analytical artifact would be (is) my working hypothesis,

Here's a a favorite example of mine, which I use as an example of "what scientists are like" in a grad-level course on glycobiology (my own area of expertise, which turned out to be chock full of the unexpected) I taught for a decade. This is how ahem one Nobel laureate thinks about his life's work, even if only in retrospect. ;)

1785118685022.png


I think it's fairly safe to assume that, e.g., Nobelist Carolyn Bertozzi, who, among many other things in her stellar career, discovered, but did not expect, RNA covalently modified with carbohydrate, feels similarly.

1785119302695.png
 
Last edited:
Yes, the notion that scientists are only interested in confirming existing work is about as ignorant as it gets. In fact, the lack of scientists doing work on replicating existing work is a big problem. Everyone wants to be the one to make the next big discovery that upends the status quo.
 
9 years ago I tried to build something similar, with separated input grounds, but with only analog volume pots. It worked and the result was a series of "null" results when comparing Amps.

P1010406-crop1.JPG

ab_box.png
 
Last edited:
Since the community here doesn’t accept the results, the reaction is to pick the test apart.
Once again you are confused about who we are. It is *our* job to pick the test apart!!! No other forum has the breadth of expertise in this topic. It behooves the broader audiophile community to read what we can assess about validity of the test.

As I noted very early on, to the extent our doubts can be erased, it adds extreme power to results of the test. That the author is not willing, and you are acting as PR person justifying the same, this is not going to happen.

What is your goal anyway? Trust any such tests without scrutiny? Were you there and heard the same differences? What do you know about the Author's hearing ability?

As I mentioned I have defended some of my positive ABX test results for years and across many challenges. So don't pretend the author is singled out.
 
Josh setup a serious test for himself and tried to run it in a serious way. The best he could do.
IMHO, no, he didn't. He could have asked experts, here or elsewhere, publicly or behind the scenes, for comment on setup and procedure before even starting the actual process. I can hardly believe he wasn't aware of the level of scrutiny required and demanded by anyone seriously interested in these matters, given that he decided to publish the story.
Actually, the moment he realized he got those spectacular results he should have asked himself if this is too good to be true and maybe some small detail was missed that affected the results, and check back with an expert he trusts?
 
Let’s say I present a minor paper to a reputable scientific journal. No established consensus is challenged, just some additional slightly different scientific perspective shown, say additional data not previously known.
That's not what happened here.

A proper peer review of my paper would include examining the procedure, the use of statistics, and the citations in the paper, etc. etc. To say that experimental results that don’t challenge an established consensus don’t and aren’t examined, but are just accepted, is false.
I don't think anyone has said results that agree with established consensus "aren't examined" in a peer-review. Although a paper that runs the same experiment and gets the same results is highly likely to be rejected for lack of novelty, assuming it's a reputable journal.

That’s why the perspective here is incorrect.

Josh setup a serious test for himself and tried to run it in a serious way. The best he could do. He made no broad claims for his results and didn’t say they were exhaustive or a last scientific word.
Of course his test can be examined and improvements proposed.

But that isn’t really what’s going on here.
I've seen multiple comments crediting Josh with his approach to the experiment, and now they are asking perfectly valid questions about that approach and the results, so it can be re-run and validated. That is what is going on here.

Since the community here doesn’t accept the results, the reaction is to pick the test apart. Even to the extent of posting false claims and other posts that show the poster hadn’t actually read Josh’s article with any attention, and made basic mistakes about what was said there in his “scrutiny” of the article.
If people are making false claims or lying about his experiment, then call them out individually, but why on earth would you expect the Audio Science community to accept the results without any challenge (i.e. picking the test apart).

Sorry, but be honest with yourselves. If Josh had run the same test and come to the opposite conclusion most of you wouldn’t look at it critically, and wouldn’t be thinking about reasons the test might be flawed. Not for even a second.
And no reason to look at it, as already answered above.

Not the way real science and peer review actually works.
Nonsense.
 
So I bought a digital version of HFN nov 2017 to look at the Matrix Audio X-Sabre Pro review, which should be the closest cousin to the Pro MQA version. There is no evidence of inverted output. Granted, it is still not the MQA version, but it does not support inverted output of the MQA version. So I would not take inverted output as a primary explanation. Since there is no attempt to signals at output (as yet), this thread is a dead-end.

Matrix.png
 
When one finds expected results it is often seen as verification.
Engineers love to find expected results, unexpected results means one has to find out why the measurements are not as expected which can be for a variety of reasons.

In both cases calibration is important as well as verification of the results is important.

In this case the unexpected results are that they can be told apart.
But only in small excerpts of specific recordings, repeated over many (not logged) attempts which may or may not be caused by issues with the used recording, DAC or protocol.

We can only guess but the results are unexpected (for engineer type folks) and not unexpected, in fact they are highly sought after, by listeners that prefer to listen sighted and are never believed by the 'cloth eared measurement guys that only look at numbers and plots and never listen to music'.

For those folks the test is valid and does not need to be scrutinized.
For the measurement folks the test is at least suspect and would like to see the test to be 'checked' if it was properly done.

As @Josh83 said ... posting his results is a no-win situation for him...
 
Last edited:
Sorry gentlemen, where is the post indicating how many listeners did this ABX test ?
 
Lots of fake amateur armchair science being proposed here.
Most work in science doesn’t challenge the established science, it backs up what’s established.

Let’s say I present a minor paper to a reputable scientific journal. No established consensus is challenged, just some additional slightly different scientific perspective shown, say additional data not previously known.

A proper peer review of my paper would include examining the procedure, the use of statistics, and the citations in the paper, etc. etc. To say that experimental results that don’t challenge an established consensus don’t and aren’t examined, but are just accepted, is false.

That’s why the perspective here is incorrect.

Josh setup a serious test for himself and tried to run it in a serious way. The best he could do. He made no broad claims for his results and didn’t say they were exhaustive or a last scientific word.
Of course his test can be examined and improvements proposed.

But that isn’t really what’s going on here.

Since the community here doesn’t accept the results, the reaction is to pick the test apart. Even to the extent of posting false claims and other posts that show the poster hadn’t actually read Josh’s article with any attention, and made basic mistakes about what was said there in his “scrutiny” of the article.

Sorry, but be honest with yourselves. If Josh had run the same test and come to the opposite conclusion most of you wouldn’t look at it critically, and wouldn’t be thinking about reasons the test might be flawed. Not for even a second.

Not the way real science and peer review actually works.

Nah. Admittedly I have merely skimmed trough most of the discussion. Still, to me this is what happened.

An experiment is done and weird is reported.
Weird gathers attention.
Questions arise and potential error sources are suggested.
Some individuals call BS outright.
Some individuals want more information.
Some indivisuals even offer to help give birth to that information by testing the testing gear for example.
Some rush in to defend the experiment :)

Broad generalisations are tempting but it seems obvious ro me that there is no singular "perspective here". Many angles, many motives. Doubt, curiosity, helpfulness, EVEN GAMBLING - a bet has been proposed to do this supervised.

Personally I believe something is wrong with the test setup.

All it takes is a loose connector somewhere to detect superluminal neutrinos after all. And yeah it causes a stir as it should.
 
For those folks the test is valid and does not need to be scrutinized.
For the measurement folks the test is at least suspect and would like to see the test to be 'checked' if it was properly done.

As @Josh83 said ... posting his results is a no-win situation for him...
As it stands right now, the subjective side is happy because the results align with their beliefs and the objective side isn't happy because the results don't align with their beliefs but also - more importantly - don't align with previous results and there are open questions about the test setup.

The subjectivists have nothing to loose by further investigation. They fully believe the results are valid because they always believed they could hear a difference, so why would they oppose checking them in any way? Clearly, any additional check would only strenghten the evidence supporting their point of view. And the objective side would strongly support further testing. Following that logic, no side should reasonably oppose it.

So I don't see the no-win situation here. I recognize that it would be additional work for Josh, though.
 
Archimago's test was different. He wasn't asking if 2 DACs can be differentiated. He polled listeners about which DAC sound, if any, they preferred, and how different they thought the recordings sounded.
Pretty typical design. Preference suggests a difference, and he measured how well they identified it.
 
This was my 1st thought when I saw what it does. All electronics are purpose built. The VanAlstine's ability to switch quickly between low level sources, amps, and speakers is certainly useful, but perhaps not for identifying minute differences in DACs.

Years ago every big audio store's listening rooms had switching devices capable of quickly accessing perhaps a dozen CD players, two dozen amps, and an equal amount of speaker pairs. They were notoriously noisy, and not that reliable, but that was not really important to their purpose. The first ones I used had no way to adjust/equalize gain. As a result, amps with aggressive linear volume controls sold best ("so much power...and its only turned to 3"). Same with efficient speakers. The residual noise made little difference in a busy showroom. Later versions added the ability to individually adjust gain. Perhaps most sellers used it to assist in making objective decisions. Some of the stores I worked at used to to make sure the push items were always louder.

As a multi-purpose device the VanAlstine is probably great for people comparing speakers...not so much for minute differences in near identical sources.
Back when I worked at Atlantis Sound (1978!) they had one of those. I remember thinking the box itself made everything sound worse. I rewired some speakers directly and it felt quite vividly so. Doesn’t prove anything, but should have prompted further investigation.
 
So I bought a digital version of HFN nov 2017 to look at the Matrix Audio X-Sabre Pro review, which should be the closest cousin to the Pro MQA version. There is no evidence of inverted output. Granted, it is still not the MQA version, but it does not support inverted output of the MQA version. So I would not take inverted output as a primary explanation. Since there is no attempt to signals at output (as yet), this thread is a dead-end.

View attachment 547451
On the other hand this suggests…..something unlike fidelity (but is probably just fantasy)

1785153567316.png
 
The subjectivists have nothing to loose by further investigation.
Well, they have a lot to lose. We all want to stop just after the test that confirms our priors.
 
Back
Top Bottom