Why aren’t there more university level controlled tests?

[kemmler3D]

I don’t get it.

It would appear that there are universities across the globe with testing facilities. We’ve all seen academic papers posted here measuring the limits of human hearing, etc., etc.

One has to ask, what are they doing the rest of the time?

As an example, how long would it take to do a proper, double-blind, controlled test on a handful of DACs with varying degrees of SNR, THD+N, jitter, etc.?

How long would it take to test amps with varying degrees of audible measurements?

It’s not as if universities have a shortage of young people with good hearing.

So many audio-world myths could be put to bed so quickly.

And if you had a few places set up to handle these across the world, whilst not everyone would be able to access them, there’s be enough people close enough who could (and gladly would) bring in their gear, and themselves, for free, reducing the cost.

As far as I know most university audio testing facilities are mostly used for audiology studies, studying speech perception and stuff like that. Doing yet another doomed DAC differences DBT in a futile attempt to shut down anti-scientific cranks on the internet doesn't seem to be worthwhile in comparison...

But in all seriousness, I think that basic thresholds of audibility are considered "known" as far as they need to be. I guess they have bigger fish to fry than edge cases of audibility when it comes to music playback.

 

[RayDunzl]

Advertise that the appropriate level of funding is available for the taking and the testing will be done.

After a few preliminary studies to make sure the monetary flow is uninterrupted, of course.

 

[MAB]

Yes, I can. DEET was invented in the ‘50s, and scientists regularly repeat studies into its safety and efficacy.

Would you like me to provide links?

Chemical safety was not extensively studied at the time of DEET's invention. It is a valid topic since new research suggests we die from chemicals that we didn't know about.
Audio was invented IDK long time ago. I pretty much accept that the difference in gear was settled in that AES study, extensively referenced. I have the AES paper and read it and accept the findings, it both aligns with my experience and isn't refuted by any new or old studies that can be considered reputable. Are you suggesting there is new ground-breaking research that suggests it should be revisited? Certainly, if certain DACs turned out to be health risks I am sure funding will flow and the scientific community would get off their duffs and reopen these old studies.

Certainly research on auditory illusions and hallucinations occurs. Like this:

Robotically-induced auditory-verbal hallucinations: combining self-monitoring and strong perceptual priors | Psychological Medicine | Cambridge Core

Robotically-induced auditory-verbal hallucinations: combining self-monitoring and strong perceptual priors - Volume 54 Issue 3

www.cambridge.org

TBH, this is the research you are wondering about. Not "can we hear differences in DACs", but instead "what kind of auditory stimuli can induce significant hallucinations in human subjects". The hallucination that a DAC sounds different is not very interesting to a researcher unless they were obsessed on audiophilia. The ability to get a subject to hallucinate the presence of a person in a room is 100x more relevant to the research community. And actually circles back to Geddes' research on auditory perception. So actually, now that I think about it, the research is being done, but it is building new scope rather than revisiting the old.

This comes to mind:

Amazon.com

Massively surprised at the extremely hostile response from forum members at Audio Science Review, set up and run by a man who’s an expert in audio electrical engineering, and who regularly posts reviews and links to articles on the importance of scientifically validating claims, crutch is ding audio companies by not backing up their figures.

Why so antagonistic?

Sometimes answers seem antagonistic, but I don't get the impression you accept my answers, can't speak for other's impressions. Not really sure what to say, but in return I hope you weren't actually antagonizing me by suggesting scientist have free time to revisit the old. My experience is I don't have free time. And if I did, it would be doing something fun, and ABXing DACs or amps isn't fun for me. The only time I ever had fun blind testing was fooling the entire staff and owner of the audio store I worked at driving a pair of Apogee planar speakers with a Niles amplifier.

 

[krabapple]

Yes, I can. DEET was invented in the ‘50s, and scientists regularly repeat studies into its safety and efficacy.

Would you like me to provide links?

Yes, please do provide them. I want to see exactly what long-settled facts are being reinvestigated regularly for a product designed to ward off mosquitoes, and exactly which scientists are doing this under what funding regime.

As for umbrage about tone, you might not want to stand on that hill when you basically called scientists lazy at the start, from a position of apparent ignorance as to how research actually gets funded and done.

I'd also like to see evidence that you have ever bothered to make yourself aware of extant academic audio/psychoacoustics research, published in for example JAES, emanating from research centers at for example U of Montreal and McGill University in Canada.

 

[Victor Martell]

I don’t feel comparing a couple (or more) DACs or amps would be prolonged or expensive.

It is if it's done the right way Yes - DBT alone IS NOT A complete valid experiment, that is just one component of the whole thing - to do this the right science way, you have to:

First of all, propose and get the go ahead, then

1.- Design the experiment, should include objective, sample size, sample make up, methodology (DBT? other? DBT + something else?) and result analysis methodology.
2.- Have that peer reviewed
3.- Find the subjects as per the sample size and characteristics, find alternates, wrangle and schedule them.
4.- Technical setup; everything from finding the room, gear, etc.
5.- Conduct the experiment
6.- Analyze and go over the results as per methodology.
7.- Write the report
8.- Get that peer reviewed
9.- Publish
10.- ???
11.- PROFIT!

Some of those steps need to be iterated upon. How valuable is the time of how many researchers needed? One thing is to setup a dac in your home, another is this. It is expensive. And as someone mentioned, why do this? We don't spend time and money proving the earth is not flat - I guess that is the argument. I would love to see it though... I would even PAY for it, if I had the bank... IMAGINE - MIT vs. Synergistic Research!

 

[somebodyelse]

Don't think that "Hifi", is a degree course, is it?

"Audio Systems Engineering" was a final year option for an Electrical Engineering degree where I was.

 

[RayDunzl]

"Audio Systems Engineering" was a final year option for an Electrical Engineering degree where I was.

Is it still an option?

 

[somebodyelse]

Is it still an option?

Good question - I hadn't looked. Not any more, so far as I can tell. There's this which wasn't available back then though:

Multimedia Digital Signal Processing
This module aims to provide an understanding of the fundamental concepts, and the applications, of digital signal processing in multimedia communication systems with a focus on music, speech and video processing.

Checking another university that I remember having audio-specific courses:
https://www.southampton.ac.uk/courses/acoustical-engineering-degree-meng
On the electronics side there is a DSP module similar to the one above. Again the analog side is generic analog and mixed signal, not audio specific.

 

[bodhi]

The audiophile myths are not based on lack of knowledge. Subjectivists' have preemptively declared all such experiments as faulty. Even if everything else is accounted for, from proper test subjects to good enough equipment you still have the "stress of test" which invalidates all failed results.

So, only thing I can think of is to secretly change the equipment of some subjectivist back and forth and do this for quite a while. The subject cannot even know they are being evaluated, the stress, remember. Only person who might change their opinion based on this is the test subject, everyone else just shrugs it off because the subject was a cloth ear. And I'm certain that the subject could come up with explanations as well.

Now, this kind of test would be pretty damn hard and expensive to implement with very little value.

 

[Willem]

Doing such DBT is relatively easy and does not require a big grant. However, doing an experiment that has been done quite a few times before, and with invariably the same result, does not seem like a great research or career strategy. I have been in academic research all my adult life, and I can tell you I would always try to find out something that is worth knowing because it is both telling me a lot, and that I did not know before. The best researchers are the best because they have this intuition or acquired skill to choose the right questions. I would always tell my students that there is a hierarchy of questions. You can investigate and document an infinite number of things, but why does it matter to know a particular thing?

 

[oleg87]

Why? Should universities have an astrology and creationism department? Most audiophile crankery flies in the face of fundamental theory in these fields, and is not very interesting to anyone except audiophiles and people looking to market products to them. Professional engineers and scientists have better things to do than look into claims of people who think their headphone amp violates Ohm's law.

 

[Mnyb]

Circling back to grants and funding. Great research into audio has been done historically when it was important.

Think for instance the original voice communication system the telephone ! In the beginning it was much that was unknown...

Or radio and subsequent improvements of these system and digital television and digital phones with research into compression algorithms.

So it get funding when it matters !

Why a minority of middle aged men like thier tube amps is of no concern for the greater good ? And DAC’s are solved they need to be measured as all the good ones are beyond human hearing already , so humans would not suffice as test subjects.

But spin off from other reasearch may spill over to audio as a hobby too , so let’s not get to pessimistic

 

[kemmler3D]

Why? Should universities have an astrology and creationism department? Most audiophile crankery flies in the face of fundamental theory in these fields, and is not very interesting to anyone except audiophiles and people looking to market products to them. Professional engineers and scientists have better things to do than look into claims of people who think their headphone amp violates Ohm's law.

It's a good point. Part of justifying an experiment is showing that you have done the literature review - at least, as a layperson, this is my understanding.

If you propose to test two DACs or amps against each other in an actual academic study, you won't get $$ for it unless you have shown that you might learn something interesting, something that isn't already known / explained. That would be a high bar, since thresholds of audibility are already established, and the performance of these devices can be characterized thoroughly on a lab bench.

Remember that expectation bias and such ARE considered valid explanations for those heard differences, outside of subjectivist audio forums.

So you would need a plausible scientific hypothesis to test... not just a goal of settling an already-settled debate again.

 

[BobbyTimmons]

Yes, I’m not thinking of a standoff between say a Topping DAC and an SMSL DAC. More a question of picking DACs (perhaps more than one) which measure similarly, and putting them in different levels - let’s say SINAD in 10 dB cohorts, then see where people start to hear the difference. That sort of thing.

Scientists do an experiment when there is some hope of a positive outcome worth publishing. All modern DACs sound the same unless there is something wrong with them, they would go into an experiment without hope of a result, wasting participant time playing them identical samples that no-one could tell any difference between.

 

[Chrispy]

I just don't see why this needs testing like that. Doubt they're very interested unless someone funds it, so go for it. What is the basis for the need for such testing? Odd anecdotes from reviewers and some users?

 

[ahofer]

It’s difficult to get money to do obvious things just to settle arguments from flat-earthers. No prizes, no glory, no tenure, just wasted time.

 

[Keith_W]

Why? Should universities have an astrology and creationism department? Most audiophile crankery flies in the face of fundamental theory in these fields, and is not very interesting to anyone except audiophiles and people looking to market products to them. Professional engineers and scientists have better things to do than look into claims of people who think their headphone amp violates Ohm's law.

Actually, universities do regularly debunk snake oil and academic journals publish those papers. There are dozens of papers criticizing medical snake oil, such as chiropractic, acupuncture, herbal remedies, vitamin supplements, and all sorts of other quackery. I remember reading a fun paper about a DBT with "intercessory prayer". I thought this was absolutely harmless (after all, sending prayers is free ... which is why so many people send "thoughts and prayers" instead of actual money). But apparently it isn't always free, some people donate to religious organizations to ... umm, get more powerful prayers. The only people blinded to the intervention were the medical staff and the patient (it was in an ICU setting, so the patients were unconscious anyway). Unsurprisingly, it returned a null result. Criticism of the study from religious medicos included: how do you know that nobody prayed for the control group.

The point I am making is that medical quackery sometimes has a real potential for danger. It can not only harm your wallet, it can also harm your health or even kill you. This is why I think that audio snake oil is relatively benign. Because it is "only" a waste of money, and affects a very small segment of the population, there is no imperative to study and debunk it.

And while I am here, another point: I think it is a waste of time trying to convince people that snake oil makes no difference. Most people aren't stupid - anybody who wants to buy (say) an Uptone Etherregen network "cleaner" will google for reviews. They will find a dozen positive subjective reviews, and a single negative review on ASR - but we are the only review with actual measurements. They can make their minds up for themselves who they want to believe. I think the data by itself is sufficient persuasion. And if it isn't ... they deserve to be separated from their money.

 

[Chrispy]

Actually, universities do regularly debunk snake oil and academic journals publish those papers. There are dozens of papers criticizing medical snake oil, such as chiropractic, acupuncture, herbal remedies, vitamin supplements, and all sorts of other quackery. I remember reading a fun paper about a DBT with "intercessory prayer". I thought this was absolutely harmless (after all, sending prayers is free ... which is why so many people send "thoughts and prayers" instead of actual money). But apparently it isn't always free, some people donate to religious organizations to ... umm, get more powerful prayers. The only people blinded to the intervention were the medical staff and the patient (it was in an ICU setting, so the patients were unconscious anyway). Unsurprisingly, it returned a null result. Criticism of the study from religious medicos included: how do you know that nobody prayed for the control group.

The point I am making is that medical quackery sometimes has a real potential for danger. It can not only harm your wallet, it can also harm your health or even kill you. This is why I think that audio snake oil is relatively benign. Because it is "only" a waste of money, and affects a very small segment of the population, there is no imperative to study and debunk it.

And while I am here, another point: I think it is a waste of time trying to convince people that snake oil makes no difference. Most people aren't stupid - anybody who wants to buy (say) an Uptone Etherregen network "cleaner" will google for reviews. They will find a dozen positive subjective reviews, and a single negative review on ASR - but we are the only review with actual measurements. They can make their minds up for themselves who they want to believe. I think the data by itself is sufficient persuasion. And if it isn't ... they deserve to be separated from their money.

There's generally a lot more money and more humans affected by those subjects....

 

[Yorkshire Mouth]

I just don't see why this needs testing like that. Doubt they're very interested unless someone funds it, so go for it. What is the basis for the need for such testing? Odd anecdotes from reviewers and some users?

As I say, there’s a lot in it for companies who manufacture cheap but good hi-fi.

I’m interested that people think there’s no money. This weekend an old friend is over from Germany to give a recital at the university - experimental contemporary classical.

Him and four playing a piece at the concert. Ten people attended. I won’t tell you what he was paid, but putting the concert on and paying five like him wasn’t cheap.

 

[DanielT]

If we talk about sound and research. Put research money into finding a cure for tinnitus plus develop research in general within the field of:

Psychoacoustics

Psychoacoustics is the branch of psychophysics involving the scientific study of sound perception and audiology—how the human auditory system perceives various sounds. More specifically, it is the branch of science studying the psychological responses associated with sound (including noise, speech, and music). Psychoacoustics is an interdisciplinary field including psychology, acoustics, electronic engineering, physics, biology, physiology, and computer science.[1]

Psychoacoustics - Wikipedia

en.wikipedia.org

There are many interesting areas to research, of which DAC audibility limits are not one of them.
We amateurs can tinker with our own DAC blind tests, if we think it's fun. Such DAC tests are good enough.