Master Thread: Are Measurements Everything or Nothing?

[solderdude]

studies finding that people can notice differences with hi res also a fact

It depends... if these differences were of the exact same master (say a 192/24 downsampled to 48/24 using a known good resampler) and shown to exist in scientific valid blind tests or ... do you mean people that listen to possibly different masters and not under controlled conditions but are basically sighted opinions.

Please post the research that shows audible differences between 192/24 and 48/24 that have scientific rigor instead of opinions from audiophiles,

I think that different reconstruction methods might possible make a subtle difference at the margins in ideal listening situations.

Yes, very likely when poor filters are used and are compared to a proper reconstruction filter.
But that is only because one of the filters is poor.

I seem to prefer dynamic driver to planar magnetic headphones. I don’t know why and if they relates to measurable or non measurable factors

These behave measurably different.
There are poor and good dynamics and poor and good planar (magnetic/electrostatic/electret) headphones.
Perfectly fine to have a preference. DAC differences are magnitudes smaller though so a preference for headphones is something entirely different as claiming different DACs (with known proper reconstruction filters) might sound different because some people claim to hear differences but do not provide scientific evidence you seem to demand.

I think people should be open minded and tolerant and curious

Open minded is good as long as it isn't so open that your brain falls out.
I am all for tolerant behavior.
Being curious is great. That does not mean curious people look for the right info and end up well informed.

 

[oleg87]

I think people should be open minded and tolerant and curious.

Unfortunately, this is audio, where such an attitude will quickly take you into very silly waters. I'm very open minded and curious if someone can demonstrate they have rigorously controlled for sound being the *only* thing they can discriminate by, yet a discernible difference still exists where I wouldn't expect it to. This describes somewhere close to 0% of reports of extraordinary feats of perception, so it's really not worth taking the slightest bit seriously for anyone with some experience with the limits of causal listening tests.

 

[krabapple]

I think it holds in general, with any model of reality. First, you need to realise that your model is just that. It contains simplifications, omissions, inaccuracies, etc. that compound to an error or delta in the depiction of reality. Unless you understand (and ideally quantify) that error you cannot make trustworthy predictions using the model.

Do you work as a scientist?

 

[krabapple]

Isn’t Shannon Whittaker also a fact and studies finding that people can notice differences with hi res also a fact

That such studies exist, is a fact. Have you actually looked into them? Many have. Even here on ASR threads. Shall we wait for you to catch up?

 

[TTT15]

I was referring to the queen mary of university of London study which presumably discussed on here before. I’ve no doubt none of this is new and I do use the search function but it does take some trawling and I only look at stuff when on the go. I confess I’m somewhat surprised by the reaction. I rather naively thought music was meant to be about enjoyment and HiFi was just a hobby. I seem to have misjudged. Personally I’m curious given that I’m looking at new HiFi and generally I’m curious by nature but it’s not a subject that I can get agitated about.

 

[krabapple]

I was referring to the queen mary of university of London study which presumably discussed on here before.

That would be the Reiss I mentioned (and I get to ring my timer bell). It was a meta-analysis of results of earlier studies. Meta-analyses are highly influenced by which studies are included in them.

The most generous interpretation of it would be: it doesn't mean *you* hear those high sample rates.

(Ask yourself: why did it take a meta-analysis to uncover the purported 'small but statistically significant effect' if high sample rates were anything near as audible as golden ears routinely claim they are?)

I’ve no doubt none of this is new and I do use the search function but it does take some trawling and I only look at stuff when on the go. I confess I’m somewhat surprised by the reaction. I rather naively thought music was meant to be about enjoyment and HiFi was just a hobby. I seem to have misjudged. Personally I’m curious given that I’m looking at new HiFi and generally I’m curious by nature but it’s not a subject that I can get agitated about.

I wonder then at your persistence.

 

[TTT15]

But then that is what I mean by all sides showing what appears to be classic confirmation bias whereby they refer to sources that support their view and dismiss sources that are contrary to their world views. You see it everywhere everyday without exception .

I would say typically studies from highly respected universities trump you tube and blogs. But as I said I still would not claim anything is definitive either way. I’m entirely open as I’ve no beliefs on the topic and I know you will rarely persuade anyone on either side if they are convinced of their beliefs.

I’m not only trained and paid to seek evidence and curious but to challenge and be persistent until they can really back it up or the cracks appear. Still doesn’t mean you think someone is wrong and still doesn’t mean humans like having their beliefs scrutinised.

 

[krabapple]

The point is even if you wholly accept the premises and results of that JAES article, what do those results mean for your hearing/enjoyment (the thing you keep banging on about mattering more than 'measurements')?

I'll tell you: it means if you want to know if 'hi rez' is making a real contribution to said enjoyment...you'll have to run your own 'measurements' (blind test). Because statistically per Reiss, it's quite unlikely to be the case.

 

[oleg87]

I was referring to the queen mary of university of London study which presumably discussed on here before. I’ve no doubt none of this is new and I do use the search function but it does take some trawling and I only look at stuff when on the go. I confess I’m somewhat surprised by the reaction. I rather naively thought music was meant to be about enjoyment and HiFi was just a hobby. I seem to have misjudged. Personally I’m curious given that I’m looking at new HiFi and generally I’m curious by nature but it’s not a subject that I can get agitated about.

Is that the meta-analysis that showed that the vast majority of people are just randomly guessing, some people are *mostly* guessing, and one result is such an outlier it should probably be ignored?

I do get agitated about misinformation running rampant on the internet. It bothers me when people speak confidently about things they have no understanding of. I don't know if you've noticed, but it's a bit of a societal crisis these days - though audio is perhaps the least significant manifestation of this. It bothers me that people are unwilling to test the fundamental and easily verifiable reality that our ears can "hear" things in casual listening that completely disappear when comparing blind, which I reckon accounts for 99% of strange audio beliefs about electronics.

 

[antcollinet]

Still doesn’t mean you think someone is wrong and still doesn’t mean humans like having their beliefs scrutinised.

Show me some actual solid evidence that disproves the null hypothesis - and I'll be delighted - it'll be interesting as hell.


Sadly though - despite all the golden ears who believe they can hear the difference - they all, consistently, fail to do so.

 

[theREALdotnet]

What about the tests that show inaudibility between Redbook and high res? Since Redbook is <100dB Sinad - there is a pretty good indicator that 100dB is a reasonably sound limit to accept.

I don't think we can substitute the theoretical performance of a specification for the actual performance of a device, when part of the uncertainty is how well SINAD represents the performance (i.e. presence of audible defects) of a DAC.

But here is the point - proving inaudibly is not possible - it is attempting to prove a negative - or at least to prove it would require a test involving everyone on the planet, and probably all future planet dwellers. So normally in a scientific endeavour - we set the improvable but plausible as the null hypothesis, and attempt to disprove it - i.e., in this case to prove audibility. This has been the method for pretty much every audio-related blind test ever carried out. We hypothesise that something is inaudible, then try to disprove that by proving that it is audible.

Like I said earlier, this isn't a binary question, because we would have to ask an infinite number of those: Is SINAD 20 audible? Is SINAD 21 audible?... Is SINAD 80 audible?... Is SINAD 100 audible? Et cetera, et cetera. What we should be looking for is the threshold between inaudibility and audibility**. This threshold can be experimentally determined for a cohort of subjects. We would end up with something like this:

Just like Russell’s Teapot, there is nothing that can be used to brew tea floating around inside the orbit of Mars - it is up to the doubter to come up with the evidence to prove otherwise.

Ah, but that's not the same situation. A valid analogy would be if we knew for a fact that there are teapots around Pluto, with a couple spotted somewhere between Uranus and Neptune, but none so far observed near Mercury or Venus. What does this allow us to conclude about the existence of a teapot around Mars? Is the onus now completely on the teapotters or is an inquiry warranted?


** by "audibility" I mean audible differences between the DUT and the ideal DAC, or for lack of the latter, the best measuring DAC we have

 

[theREALdotnet]

Do you work as a scientist?

I used to, many years ago. I migrated into engineering (better money).

 

[Jim Taylor]

But then that is what I mean by all sides showing what appears to be classic confirmation bias whereby they refer to sources that support their view and dismiss sources that are contrary to their world views. You see it everywhere everyday without exception .

The Scientific Method was created to eliminate or control confirmation bias, and I'd say that for issues in which the Scientific Method can be used, results are not materially affected by it. After all, is it confirmation bias to believe that if you shot yourself in the foot, that it would hurt? I don't think so.

OTOH, when discussing issues based on emotion, I'd say confirmation bias is the common currency. But that's a whole different world.

. I’m entirely open as I’ve no beliefs on the topic

For someone who has no beliefs on the topic, you seem to have a large number of retorts.

... still doesn’t mean humans like having their beliefs scrutinised.

Looking at the world around us, I'd say that we would be far better off if more people had their beliefs scrutinized, whether they like it or not. Just sayin' ...

 

[TTT15]

Absolutely and I’m merely pointing out that we have very little by way of definitive scientific proof either way so we should be wary of extreme absolute maximum positions.

In discussions of most complex topics I usually find that there is very limited absolute hard proven facts as reality tends to be nuanced and there is usually an element of truth on every side of a debate.

Humans tend to prefer the comfort of certainty and they can get very defensive of their beliefs when challenged especially if they are of a particular psychological disposition but that isn’t an argument for not questioning beliefs as many would prefer.

In reality the universe itself tends to disorder and uncertainty and probability so there is rarely the certain absolutes that some like to claim and which are popular with some groups of people.

 

[DavidEdwinAston]

Absolutely and I’m merely pointing out that we have very little by way of definitive scientific proof either way so we should be wary of extreme absolute maximum positions.

In discussions of most complex topics I usually find that there is very limited absolute hard proven facts as reality tends to be nuanced and there is usually an element of truth on every side of a debate.

Humans tend to prefer the comfort of certainty and they can get very defensive of their beliefs when challenged especially if they are of a particular psychological disposition but that isn’t an argument for not questioning beliefs as many would prefer.

In reality the universe itself tends to disorder and uncertainty and probability so there is rarely the certain absolutes that some like to claim and which are popular with some groups of people.

Luckily, this thread is so limited in it's scope, science can "prove" the " facts", it talks about!

 

[TTT15]

I’ve seen some studies which is good but nothing from the likes of the Nature and Science journals which would normally be considered science.

 

[Jim Taylor]

Philosophical musings are nice, but they are really outside the intended scope of ASR.

 

[pkane]

very little by way of definitive scientific proof either way

I assume you've done a thorough investigation of all available research, published study results and methods used to reach this conclusion? Have you attempted to verify any of this yourself, or you're just an armchair philosopher? Run some tests, come up with some interesting results that contradict existing studies and can be duplicated by others, and then we may have something. Hand-waving isn't very interesting.

In discussions of most complex topics I usually find that there is very limited absolute hard proven facts as reality tends to be nuanced and there is usually an element of truth on every side of a debate.

Science is not about proofs. That's mathematics.

 

[TTT15]

I fully agree we are not here to discuss philosophy but our experience, training and education of the broader world view does inform how we view specific topics. And yes it’s generally science research which may or may not lead to proofs. I’m not making any definitive claims just merely pointing out what has been shared with me doesn’t constitute what would normally be considered as firm scientific research. We have some helpful limited reseach into certain specific aspects. I’m just saying we don’t have definite absolute proof of most matters especially when it comes to the human aspects. I don’t see how that is particularly controversial at all. Just a statement of the reality and therefore it is best to be open minded.

 

[solderdude]

I’m just saying we don’t have definite absolute proof of most matters especially when it comes to the human aspects.

That part is why there is no 100% correlation possible with signal fidelity... because of all the variables caused by the human preference and perception part.

This has long been accepted in the ASR community and long before that.
Nor has that ever been disputed.

It's all about audibility thresholds, preferences (taste) and test methodology.
For that reason all the 'but I can hear differences' talk is not anything close to 'definitive' proof warranting 'science' to look deeper and explain to them why they hear what they hear and because all measurements differ that must be where the proof is and where the 'hearing is superior to measurements' comes from.