Master Thread: Are measurements Everything or Nothing?

[Lasker]

[L

Absolutely not. ASR does not police your thoughts.

However, most people here will outright dismiss your listening impressions between two nearly identical pieces of equipment like DACs or Amps, if they came about in a sighted, uncontrolled comparison.

That's because reserch has shown these types of comparisons to be fundamentally flawed and extremely unreliable. They're simply unfit as the basis for a meaningful discussion.

We will, however, be very interested in your impressions when the comparison was performed with the appropriate controls in place

Ok but in my case I don't think the problem was a volume matching
questions, I could turn up the volume a little with each other's DAC, sanskrith mk3 was better in all the cases.

 

[BDWoody]

I'll buy that, but don't the analog output and filters matter at all?

The analog output is what we are talking about... That's what is getting measured.

Filters can certainly impact how it will sound to some, especially those who are younger and haven't lost the very high frequencies (yet). Seems that most DACs these days have a few choices, and picking comparable filters would be part of the comparison.

Of course I would like to do a blind test, but no one here Europe is going to offer to do that, and I am not competent enough to do that.

There are ways you can do a pretty well controlled comparison on your own. It wouldn't pass *rigor* tests, but setting some up for myself when I first found this place was enough to convince me.

If I *had* heard the differences I fully expected to hear between the several vastly different priced DACs I had on hand, I would have enlisted help to develop a better protocol, but I didn't, and it is the before/after point in my audio life.

You won't believe it until you've experienced it for yourself.

I used a neutral preamp, matched the output of each DAC with a voltmeter, mixed up the input switcher on the remote so I didn't know what I was listening to, streamed digitally to both, and went back and forth. Couldn't hear a difference. I used the money from the DAC I returned to upgrade my speaker setup.

 

[staticV3]

Ok but in my case I don't think the problem was a volume matching
questions, I could turn up the volume a little with each other's DAC, sanskrith mk3 was better in all the cases.

Volume matching is just one of the controls required for meaningful comparisons.

 

[Ugo Ruffini]

I have owned numerous smsl, sabaj, topping dacs... SINAD, MULTI-TONE AND CHANNEL SEPARATION are a valid element for understanding the quality of the project but above 100db are useful values for bats!!!
My group of friends have esoteric implants worth over 100,000 euros
With machines that don't even reach 100db but sound indisputable... My friend's chain is made up of the Linn Klimax DSM, AudioResearch 6SE, Krell 300W acanal power amplifier and Wilson Sophia.
I have learned that measurements do not give the result, there are many more factors at play.

 

[Ugo Ruffini]

Wilson sasha daw sorry

 

[CleanSound]

Volume matching is just one of the controls required for meaningful comparisons.

That is by far the MOST important factor to ensure an accurate AB comparison. The most accurate way to make sure you level match accurately is to play a 1kHz tone and use a multimeter to measure the voltage coming out of the amp. You could even match the analog output from the DAC but that get's challenging.

Once everything is level matched, make sure you remember what level setting is for each DAC. Then you can do the A/B switching.

But one of the hardest thing to do in a good A/B test is the ability to quickly do back-to-back A/B switching because our long term memory for sound impression is pretty terrible, hence you need to be able to do back-to-back A/B switching quickly. To be able to do quick back-to-back A/B switching is pretty challenging, because the source that goes into the DACs needs to be switched and then the amp needs to be switched, that takes time, unless you have a specialty switch box. Which I believe companies like Harman and Axiom and the NRC have these custom built specialty switch boxes; I tried to build one myself, but it's laborious and time consuming (thought not technically hard, IF you can find quality parts).

So in short, I have to confess fully that I have not done a proper AB of any kind with the exception of equipment that is fairly easy and fast to switch.

 

[Ricardus]

The only thing missing is a shred of evidence supporting the idea that there is more to find. From anyone. Ever.

I will be borrowing this as a quotable quote!

 

[fpitas]

I will be borrowing this as a quotable quote!

It just means we need to get on the stick and find the X Factor!

 

[DonR]

"You just design and build the stuff, you don't know how it REALLY works."

 

[Ricardus]

It just means we need to get on the stick and find the X Factor!

Yeah. We're not working hard enough. We're SLACKERS!

 

[CleanSound]

My group of friends have esoteric implants worth over 100,000 euros
With machines that don't even reach 100db but sound indisputable... My friend's chain is made up of the Linn Klimax DSM, AudioResearch 6SE, Krell 300W acanal power amplifier and Wilson Sophia.

This raises a big red flag with me. You are equating cost and name brand with performance. While it's logical to think that way, equipment that costs more surely have better parts and more R&D and better manufacturing effort. Surely brand names wouldn't risk making cheap stuff to ruin their name.

Why is it that studies show that Toyota are more reliable than a BMW, Mercedes or Audi? Luxury brand car engineer and design their cars to be luxurious like upscale leather, fancy interior, etc. it is design for the buyers, and the buyers are rich people who gets a new car every few years so reliability is not important for them. Luxury brand HiFi, spends a lot of money on casework, expensive and quality parts that sometimes does not affect sound, customer services and marketing.

I have learned that measurements do not give the result, there are many more factors at play.

Well, since that you have learned, can you tell us what those factors are at play?

 

[Ugo Ruffini]

My personal system is much cheaper consisting of the d300 smsl musical fidelity m6s pre and power amplifier hypex nc500 the speakers are polk R700

From personal experience I have learned to simply listen to each component in my chain and then choose.... I have also listened to r2r dacs with a 90db sinad which personally are excellent.
I conclude by saying that the measures are useful but the performance of the project and listening firsthand reward the choice!

 

[voodooless]

From personal experience I have learned to simply listen to each component in my chain and then choose

Well, we already told you that this is the worst and most unreliable way to choose components, so what can I say? … good luck… I guess?

 

[Killingbeans]

I love this forum, but I have a question... to write here I have to think that all the DAC with a sinad over 110db sound the same?

No. SINAD is just a single number. There's room for the designer to have done something silly to "color" the output, like implementing a non-compliant reconstrution filter.

But if you take the time to look at all of the measurements in the reviews, they won't leave much room for interpretation when combined.

And no, you don't have to think anything specific to be allowed to write here. As long as you have good arguments you'll be out of the line of fire. Bad arguments and/or magical thinking will be ridiculed, but you can still present them if you insist to do so.

Any time that people ask here for listening impressions are crucified.

No. Only when they are presented as facts.

Listening impressions of speakers and headphones definitely have some usefulness, but once you move to electronics, it quickly becomes a coin toss between physics and cognitive bias, and the odds only get progressively worse as you go up stream. The users of ASR will rightfully value it as such.

I have learned that measurements do not give the result, there are many more factors at play.

In case of you friend's setup, the things that make them "indisputable" are most likely 99.99% their speakers, the placement of them in the room and the room acoustics itself.

Those things can also be measured. And yes, brain trickery is absolutely a factor at play, but it has nothing to do with sound coming from the gear.

From personal experience I have learned to simply listen to each component in my chain and then choose....

Risky business:

https://upload.wikimedia.org/wikipedia/commons/6/65/Cognitive_bias_codex_en.svg

 

[Greenman]

I am probably beating on a dead horse with this post, which seems disagreeable to most members.

Fist to be clear, I am only asking questions, as I know that I am not competent enough to have an opinion.

This site is well known for saying that a €100 topping Dac is in no way inferior to a €3000 R2R Dac
.
Magazines and you tube reviewers, most of which are charlatans or at best ignorent influencers are of a different opinion.

Putting those people aside, there are a few that have technical knowledge and engineering degrees who can be categorized as credible.

Many of these disagree with consensus on this sight and actually buy these r2r Dacs for their own use with their own money.

I am in no way taking sides on this issue, but I think the debate is still open, and I am not ready to part with 2 or 3 thousand euros until I know more.

You are right in that this isn’t the place for this type of discussion.
I will say I have a s.m.s.l Su1 (DS) -£80 and a GUSTARD x26pro (DS) (£1400) and a GUSTARD R26 (R2R) (£1600). The R26 is the one I have in my main system having spent several weeks listening to each. The R26 is a country mile better than the Su1 in all respects and is significantly better than the x26 in many respects. So you can use a Su1 (or similar) and convince yourself nothing can possibly sound better, or try an R2R and decide for yourself.

 

[ObjectiveSubjectivist]

R2R DACs are like mechanical watches.

Cheap ones are garbage, ludicrously expensive ones can be surprisingly accurate, but anyone claiming their Grand Seiko is more accurate than some G-Shock Quartz watch is a clown in my book.

You buy them for their novelty or because you appreciate the craftsmanship that goes into them. Not because they're better at telling the time.

You buy R2R DACs for their novelty or because you appreciate the craftsmanship that goes into them. Not because they're better at turning Digital into Analog.

Anyone who claims they do hasn't done their research.

Amen

 

[DSJR]

You are right in that this isn’t the place for this type of discussion.
I will say I have a s.m.s.l Su1 (DS) -£80 and a GUSTARD x26pro (DS) (£1400) and a GUSTARD R26 (R2R) (£1600). The R26 is the one I have in my main system having spent several weeks listening to each. The R26 is a country mile better than the Su1 in all respects and is significantly better than the x26 in many respects. So you can use a Su1 (or similar) and convince yourself nothing can possibly sound better, or try an R2R and decide for yourself.

So, how can you prove to us that the R26 really is better? Usual questions, have you level matched and got a pal to switch one to the other without you knowing which is which? if the differences really are consistently audible under such conditions, chances are that the R2R models is screwing things up somewhat.

Now, this is where it gets personal! You may *prefer* the R2R as that's absolutely your choice, but 'significantly better' may not transfer to the bench where so many R2R dacs are found severely wanting and roll off the highs to boot in many cases. I mean, the Rega dac and CD players spoken of and largely annihilated here do make a pretty good job of making the music they process enjoyable to listen to

P.S. Got to say that the MIB Yggi from Schiit may well be the best of the lot using this technology, but it's a huge amount of money to chuck at what most here regard as a 'done' technology now. Since I prefer things simple and have 'aided hearing these days, an SU1 will be perfect for my needs in the second system and as and when, I can directly A-B compare with my ancient, massive but so good 'sounding' TDA 1541 based player in my main rig

 

[Deleted member 21219]

Any time that people ask here for listening impressions are crucified.

"Impressions" are subjective. They can be accurate and repeatable FOR YOU, but are not reliably transferable to other people. IOW, you may hold them as valuable, but no one else can be sure what the heck you're talking about. Scientific data, OTOH, is repeatable, and scientific notation does not change with time.

There's an example of a brief exchange between two members here, talking about Focal speakers. They could have been talking about any speakers, it would have made no difference. The point was that their subjective opinions differed by 180 degrees, and any visitor here would have been enormously confused by their exchange:

I have never liked Focal because I thought they sounded much too bright. Thanks to your reply, I now know the answer.

This cannot be the reason since I have Focals with inverted tweeter and they don't sound too bright.

I firmly believe that the information that we provide to non-member visitors (scanners) is just as important, if not more important, than the back-and-forth that we provide for each other.


Using words to convey auditory information is an extremely poor idea if you desire accuracy.
Even if you could do it, the recipient of the information may use the same words to describe something entirely different, and in the end, may disagree with you totally.

It's better to stick with data. Learn to interpret it, and you'll have a friend for life.

Jim

 

[Bridges]

So, how can you prove to us that the R26 really is better? Usual questions, have you level matched and got a pal to switch one to the other without you knowing which is which? if the differences really are consistently audible under such conditions, chances are that the R2R models is screwing things up somewhat.

Now, this is where it gets personal! You may *prefer* the R2R as that's absolutely your choice, but 'significantly better' may not transfer to the bench where so many R2R dacs are found severely wanting and roll off the highs to boot in many cases. I mean, the Rega dac and CD players spoken of and largely annihilated here do make a pretty good job of making the music they process enjoyable to listen to

P.S. Got to say that the MIB Yggi from Schiit may well be the best of the lot using this technology, but it's a huge amount of money to chuck at what most here regard as a 'done' technology now. Since I prefer things simple and have 'aided hearing these days, an SU1 will be perfect for my needs in the second system and as and when, I can directly A-B compare with my ancient, massive but so good 'sounding' TDA 1541 based player in my main rig

Since I was the one to start this thread, I can only add that I am more confused. I have never heard a R2R Dac, so of course, I don't have an opinion. However, there are so many experienced and qualified engineers reviewers, yes reviewers, the few who are not charlatans or ignorent influencers, who claim and put up their money to make or buy these R2R Dacs, and it looks to be a lot more complicated to manufacturer than a chip Dac, that one has to wonder. As far asking DSGR to prove his findings , he probably won't. Even within the same company (Shiit Audio), the 2 owners have different views on the subject. I am hoping this debate is not of the order of tube vs SS.

 

[IAtaman]

Risky business:

https://upload.wikimedia.org/wikipedia/commons/6/65/Cognitive_bias_codex_en.svg

Oh wow. That is an astonishing visual / tool. I feel like I have more biases than I have brain cells