DAC ABX shootout - unable to distinguish between 10$ and 15k$

[pkane]

If OP is satisfied, why he wanted to do another round?

Aren't we supposed to help OP getting the best he can get? We shouldn't assume all OP will use is just one headphones forever.

Also, there are other questions.
For example, does OP like rocking out sometimes? Would he sense a difference between his 4 different chains using "rocking out" volume?
What type of songs OP likes? Did he already tried all sorts of songs and all made no difference?

Why not asking more questions to help OP getting the best he can get?

You asked me if I wanted to have the OP do more testing. I said no, not if he is satisfied with the results. Not sure why you want me to care more about it than that?

 

[Pdxwayne]

You asked me if I wanted to have the OP do more testing. I said no, not if he is satisfied with the results. Not sure why you want me to care more about it than that.?

Because I want to help OP getting the best he can get. I am fine that you don't care.

 

[pkane]

Because I want to help OP getting the best he can get. I am fine that you don't care.

Go for it. That’s up to him to decide, he’s an adult, and doesn’t need me to convince him to do more tests that are likely to be a waste of his time.

 

[Axo1989]

Per the OP:

We are repeating the test this Friday, I will post update if I can still type though my tears.

So why not?

 

[pkane]

Per the OP:


So why not?

Yes, why not?

 

[DonR]

Just putting your words back in your mouth. As for triple-blind testing here, I don't see how the interpretation of results, in this case, requires a blind assessment. They are pretty straightforward.

 

[Pdxwayne]

Go for it. That’s up to him to decide, he’s an adult, and doesn’t need me to convince him to do more tests that are likely to be a waste of his time.

It is about giving him guidance. It is about not wasting his time the second time around.

Why not ask more questions to make sure he doesn't repeat any error, if there is any during his first round?

Why not provide some helpful suggestions?

Like:
Test at different loudness levels, including rock out level.
Test with more type of songs, including those with strong sub bass, if he listens to those.
Test with different headphones, if he could get something with lower distortions.
Test with headphones that follow the preference curve all the way down to 20hz

Yes, it is up to the OP to decide. Not you and I to decide if it is a waste of his time.

 

[pkane]

It is about giving him guidance. It is about not wasting his time the second time around.

Why not ask more questions to make sure he doesn't repeat any error, if there is any during his first round?

Why not provide some helpful suggestions?

Like:
Test at different loudness levels, including rock out level.
Test with more type of songs, including those with strong sub bass, if he listens to those.
Test with different headphones, if he could get something with lower distortions.

Yes, it is up to the OP to decide. Not you and I to decide if it is a waste of his time.

What are you arguing about? You want to convince me to offer advice where I already said I’m not interested?

If the OP asks for help, I might offer it. So far, I’ve seen nothing to indicate he did anything wrong, so please, offer whatever advice you want but leave me out of it.

 

[Pdxwayne]

What are you arguing about? You want to convince me to offer advice where I already said I’m not interested?

If the OP asks for help, I might offer it. So far, I’ve seen nothing to indicate he did anything wrong, so please, offer whatever advice you want but leave me out of it.

Haha, I thought OP kind of asking for help.

Quote:
.....We are repeating the test this Friday, I will post update if I can still type though my tears.....

 

[Axo1989]

Just putting your words back in your mouth. As for triple-blind testing here, I don't see how the interpretation of results, in this case, requires a blind assessment. They are pretty straightforward.

I know. I'm just maintaining the emoticon sequence. I assume you are observant.

Edit: your original statement was general and unconditional, hence my response (albeit accidentally posted). Your statement above is 'in this case'.

 

[pkane]

Haha, I thought OP kind of asking for help.

Quote:
.....We are repeating the test this Friday, I will post update if I can still type though my tears.....

Fine! My advice: don’t peel onions next to the DAC. There will be fewer tears.

 

[mdsimon2]

I find it interesting that 5 pages in the above quoted post is the only good request of information on the specifics of how @Echoes performedthe testing, and it's currently unanswered. It seems like a blind spot to me on this forum when people post results that confirm the default ASR hypothesis on audibility, and the results are generally gracefully accepted by people who subscribe to the idea, and everybody does victory laps. People say that's what they would have expected, and yell "science" at people without drilling into testing methodology.

People seem to care deeply about the specifics of how testing was performed when the result is contrary to default ASR hypothesis. In science, we must question the methodology of not just the things that don't confirm the hypothesis, but also those that do confirm it. Otherwise it seems to me like people like to use "science" as a sort of sword and shield around here more so than a means to further knowledge and understanding.

Yes, the stated result of the test confirms my own belief that there are likely not audible differences, but that doesn't mean it's a good test that actually helps bolster the hypothesis. It might be, but we actually have no idea (unless it was posted and I missed it?). To know that, we need to know how the test was performed, and ask some more questions. To ignore this part of science is intellectually lazy. Aren't you at least curious people!?

I'd love answers to the above questions that @radix posted, if for no other reason than to simply improve my own knowledge and attempts at doing valid double blind listening tests.

I tend to agree that "science" gets thrown around a bit much on this forum and we would all be better off if folks practiced what they preached so to speak, although I think this goes to both sides of the objectivist / subjectivist debate.

I grew up as a car audio fanatic and there were always threads on various forums discussing the $10K Richard Clark amp challenge which said if you compare two amplifiers and make sure they have identical frequency response in to load and noise / distortion below a certain threshold they will be indistinguishable. The best explanation of the challenge came from Jeff Scott of Crystal / Silicon Labs (@werewolf / @lycan), he recognized the challenge did not claim that all amps sounded the same but rather defined a set of parameters for which it was believed that all amps would sound the same. The challenge was a test of a hypothesis that if you control those parameters then amplifiers will sound the same. If someone passed the test it implied that our understanding was incomplete and there were some other factors that we did not yet understand. To my knowledge no one ever won the challenge and claimed the $10K.

I never understood why folks that believe in profound audible differences between equipment do not have the desire to learn WHY those differences exist. If you don't understand why something sounds different you will not be able to seek out the characteristic in the future. This requires a bit of test equipment but it has never been a better time to be a measuring hobbyist with products like the Cosmos ADC and the QuantAsylum analyzers.

I recently got a great deal on an old Apogee Ensemble audio interface. After listening to it for a while I thought it had a little bit more sparkle than I was accustomed to. I measured the frequency response and found that where all of my other DACs have a slight frequency roll off the Ensemble actually has a slight boost at upper frequencies. I used a miniDSP MCHStreamer to route ADAT output to an optical splitter which then routed to the Ensemble and to a RME Fireface 800. My CamillaDSP configuration was set to level match the DACs (ADAT 1-2 to Fireface, ADAT 3-4 to Ensemble) at 1 kHz and then enlisted my wife to operate a XLR selector in ABX. In 10 trials I was able to get 9 out 10. I then developed some EQ to flatten out the boost and re-ran the trials and got 4 out 10. Not a super rigorous test but to me it proved that out of the box the Ensemble will sound different than other DACs even when level matched at 1 kHz and that the difference was easily explained due to frequency response.

Here is the frequency of the Ensemble compared to a RME Fireface 800 for reference. Also interesting to note that the high frequency differences were much more audible than the low frequency ones (which I did not correct with EQ).

Similarly I've done listening tests with nothing playing through my system with a variety of DACs and documented whether or not I can tell that the DAC is on. I've also measured noise levels at my amplifier terminals so I have an understanding of what noise levels are audible near field. This gives me a good understanding of whether or not a DAC / amp combination will have audible noise based on the noise performance of each.

TLDR: In some cases audible differences are real and you can learn a lot correlating measurements to listening.

Michael

 

[Grooved]

Hello All,

First of all, thank you for the forum contributors who are sharing their knowledge here. I started reading ASR two month ago and just can't get enough of it.
I am sharing my experience with my first ABX testing. Last Friday me, together with a friend performed a double blind test on this systems:

1) Chord Dave + Upscaller from Chord2Go+2U - headphone output
2) Topping D90 + A90 from a laptop
3) Chord Mojo headphone output from Iphone.
4) Apple lightning 3.5mm adapter, output from Iphone.

Volume matching was performed with a dedicated microphone, nothing fancy, but still more reliable that our ears as proven below.
Switching between sources was done though a 4 position 3 pole rotary switch box, unbalanced 6.3mm cables.
Headphones used were the Seinheiser HD650, standard.
Switching was done randomly by my wife, who, with her scientific background, was happy to assist on this.

Result.. We could not tell the difference reliably between the systems. Which is.. proving either that we are both deaf or audio fools. We are repeating the test this Friday, I will post update if I can still type though my tears. Silly enough I can 100% reliably say which one is better when I see what system is connected.

Hi,

did you try to do the exact same test but on speakers?
I know it's not the same than doing ABX on the same device but with different audio files, but when I tested the last time between AAC 320 and FLAC, there was a song I always got 100% right on speakers, while I was never 100% right with headphones (and it was on my worst set of speakers costing less than several headphones I tried)
Just being curious of the results if you can run this test, if you can't, no problem

 

[Axo1989]

In other words, you think you can hear things we can't measure.

Consider these rings:
View attachment 188303
Your brain will tell you the circles move. Your ears, pressure sensors, taste buds and pain receptors are as flawed and can be tricked too. Just like your hearing.

Measurements however, they tell you the truth: the rings don't move.

While this is an aside, it's an interesting effect. If you mask the gif so you don't see the centre arrow graphics, the illusory movement of the circles is maintained (so the arrows are pedagogical, not functional). The inside diameter does change (compared to a fixed reference) marginally but the apparent movement is greater than the actual pixel variation. The inside and outside border condition (colour) does change (to grey in the direction of the arrows) which presumably drives the illusion.

 

[Zensō]

Fine! My advice: don’t peel onions next to the DAC. There will be fewer tears.

My advice: Don’t buy a $15,000 DAC. There will be fewer tears.

 

[Axo1989]

Go ahead and point me to some scientific research that’s been conducted and published on ASR.

Do your own homework.

 

[Jinjuku]

Who claims differences don't exist, ? A lot of folks on this sight. Why they read reviews if they don't; believe any differences I do not understand.

I think manufacturers can make stuff sound different and market that difference. I also think that will survive bias controlled testing.

Just like the Audioquest HDMI demo video where they fabricated the results by a rather hamfisted increasing of the audio amplitude as they switched between better and better cabling.

 

[pkane]

Do your own homework.

I’ve done mine. I’m guessing the dog ate yours?

 

[JSmith]

There will be fewer tears.

Agreed;

JSmith

 

[Blumlein 88]

I have issues with poor volume matching in the op. Suppose the lesser Dac was slightly louder making it sound as good as the better dac. With matching maybe a difference could be heard, but methodology muddied the results. We need to hold people to the same standards whether the results are improbable or expected. Not a complaint against the op, he was trying to blind the listening comparison. Live and learn and do better.