Listening test maybe not so usefull

[Mulder]

I don’t know if this text has been discussed earlier, but I couldn’t find anyting. Anyway, I found it to be somewhat interesting. Most of what it says are not new to most members of the forum, I suppose, but he is kind of ditching formal listening tests. https://www.tonestack.net/articles/digital-audio/high-resolution-audio-vs-16bit-44khz.html

 

[fpitas]

Valid listening tests need controls that frankly, most people seem unwilling to follow.

 

[Mulder]

Valid listening tests need controls that frankly, most people seem unwilling to follow.

Yes. But the argument in the linked text is, as I understand it, that formal (controled) listening tests as such are meaningless.
” And what about listening tests? We cannot understand how things work by listening to music and changing components, so if we use listening tests to understand digital signal behaviour or hearing then we just shoot ourselves into the foot.”

 

[fpitas]

Yes. But the argument in the linked text is, as I understand it, that formal (controled) listening tests as such are meaningless.
” And what about listening tests? We cannot understand how things work by listening to music and changing components, so if we use listening tests to understand digital signal behaviour or hearing then we just shoot ourselves into the foot.”

I'm not sure what they mean there, but various people have performed controlled listening tests on hi-res etc. Maybe they don't like the results. Or maybe they are talking about the typical YouTube tests, which are worth what you paid.

 

[danadam]

Monty Montgomery (Xiph.org) wrote a detailed article in 2012 on why 24bit/96kHz and 24bit/192kHz file downloads don't make sense (24/192 Music Downloads ... and why they make no sense) . The article was removed from Xiph.org a few years ago, but it is still available in the web archive

Actually, it's back on xiph.org.

24/192 Music Downloads are Very Silly Indeed

people.xiph.org

 

[Zensō]

Maybe they don't like the results.

That often seems to be the explanation.

 

[DVDdoug]

In the end listening is the ONLY important thing in this "hobby"!

Of course listening tests CAN be valid and useful! But they (usually) have to be done properly, scientifically, and blind. (I don't need a blind listening test to know that my home theater speakers are better than the speakers built into my TV, or to know that there's no sound coming out of the left speaker, etc. And you have to know what you are listening/testing for. Of course, there are limitations and different people have different hearing ability.

MP3 (and other lossy CODECs) were developed & refined through listening tests. You can get a "pretty spectrum" at the expense of sound quality. Compression artifacts are difficult to measure. "Audiophiles" often talk about vague mysterious things that can't be measured, and compression artifacts are one case where it's actually true.

If someone claims they can hear a difference between CD quality and high resolution or a difference in speaker cables, a blind listening test is the ONLY way to prove/demonstrate it.

And for example, the only way we know the general limitations of hearing is through listening tests.

Measurements can be misleading too... Better measurements don't mean better sound once you've exceeded the limits of human hearing.

 

[Curvature]

Did anyone in this thread, including the OP, even read the article?

This isn't the same old topic. The writer said if you designed a controlled worst case scenario listening test, it is far more useful than than a traditional song based test for evaluating digital audio.

That's it. There is no wider claim.

 

[Zensō]

Did anyone in this thread, including the OP, even read the article?

This isn't the same old topic. The writer said if you designed a controlled worst case scenario listening test, it is far more useful than than a traditional song based test for evaluating digital audio.

That's it. There is no wider claim.

You're correct. I quickly skimmed the article, assuming it was the same old, same old, which usually doesn't warrant a careful read.

The bit about the advantages and disadvantages of null tests is also worthwhile.

 

[Mulder]

Did anyone in this thread, including the OP, even read the article?

This isn't the same old topic. The writer said if you designed a controlled worst case scenario listening test, it is far more useful than than a traditional song based test for evaluating digital audio.

That's it. There is no wider claim.

Of course I have read it.
Still. It is true he has this idea about worst case listening tests. But at the same time he ditch the listening test as they are often conducted. ”there is another reason why traditional listening tests, format discrimination studies lead nowhere. We can cover all possibilities by testing the "worst case". Worst-case analysis helps us get out of the hell of statistical mumbo-jumbo filled discrimination studies and metaanalysis kind of creationism science.”

 

[fpitas]

Of course I have read it.
Still. It is true he has this idea about worst case listening tests. But at the same time he ditch the listening test as they are often conducted. ”there is another reason why traditional listening tests, format discrimination studies lead nowhere. We can cover all possibilities by testing the "worst case". Worst-case analysis helps us get out of the hell of statistical mumbo-jumbo filled discrimination studies and metaanalysis kind of creationism science.”

True. But satisfying as that is to an engineer, it will leave most people scratching their heads.

 

[Mulder]

True. But satisfying as that is to an engineer, it will leave most people scratching their heads.

Also true. But also the traditional blind listening tests in many cases leave most people with an all to demanding task to complete in a proper way.

 

[fpitas]

Also true. But also the traditional blind listening tests in many cases leave most people with an all to demanding task to complete in a proper way.

It's true, they're not easy.

 

[Curvature]

It's true, they're not easy.

It's not that they aren't easy. If we're going by the article, they aren't efficient. You can accomplish the same task more quickly by understanding the boundary conditions of the technology. Standard tactics in the sciences.

If there's a broader lesson to be drawn here, it's that if you understand the technology well, you can design specific tests to make it fail. By doing that you will establish what is necessary for an optimum result. How many bits, etc., until no noise floor is heard or other audible artefacts disappear. Then traditional listening tests become pointless, since the principles are known.

 

[fpitas]

It's not that they aren't easy. If we're going by the article, they aren't efficient. You can accomplish the same task more quickly by understanding the boundary conditions of the technology. Standard tactics in the sciences.

If there's a broader lesson to be drawn here, it's that if you understand the technology well, you can design specific tests to make it fail. By doing that you will establish what is necessary for an optimum result. How many bits, etc., until no noise floor is heard or other audible artefacts disappear. Then traditional listening tests become pointless, since the principles are known.

Once again, completely opaque to the non-technical audience. I think engineers were convinced long ago.

 

[Mulder]

It's not that they aren't easy. If we're going by the article, they aren't efficient. You can accomplish the same task more quickly by understanding the boundary conditions of the technology. Standard tactics in the sciences.

If there's a broader lesson to be drawn here, it's that if you understand the technology well, you can design specific tests to make it fail. By doing that you will establish what is necessary for an optimum result. How many bits, etc., until no noise floor is heard or other audible artefacts disappear. Then traditional listening tests become pointless, since the principles are known.

Well, yes. I understand what he means in his text. I thought it was a somewhat different view of the topic of blind listening tests. Thats why I posted the link. What I find harder to figure out by myself is what possible objections, if any, there could be to his arguments.
I think maybe one objection could be that - yes 16 bits are enough if dither is used in a correct manner when 24 bits studio masters are downsampled to 16. But, you can never know if the 16 bit file you buy is treated in best possible way. With 24 bits, a master original, you are free to do your downsampling yourself if you wish, or just leave it, and you don’t have to trust someone else to do the downsampling in the best possible way. But as @fpitas wrote. This is over the head for most people. This tends to be HiFi for engineers.

 

[Curvature]

Well, yes. I understand what he means in his text. I thought it was a somewhat different view of the topic of blind listening tests. Thats why I posted the link. What I find harder to figure out by myself is what possible objections, if any, there could be to his arguments.
I think maybe one objection could be that - yes 16 bits are enough if dither is used in a correct manner when 24 bits studio masters are downsampled to 16. But, you can never know if the 16 bit file you buy is treated in best possible way. With 24 bits, a master original, you are free to do your downsampling yourself if you wish, or just leave it, and you don’t have to trust someone else to do the downsampling in the best possible way. But as @fpitas wrote. This is over the head for most people. This tends to be HiFi for engineers.

You're right. In the "real world" of digital audio there's no way to know if everyone did their job correctly.

Hi-res containers aren't useful other than for archival, creative (like downpitching ultrasound) or scientific purposes. I think it's pretty clear that you don't need listening tests to establish this.

I would prefer FLAC or PCM at 44.1/24, but pretty much all the time I end up working with less since that's what's available.

The problems come about when trying to squeeze the pipeline, and we have the whole history of compression, tranmission and streaming protocols. There traditional listening tests matter because you can make the container fail easily with certain signals. Like MQA or MP3 being unable to handle certain sweeps or transients. Psychoacoustic encoding relies on more finely set boundaries, since you don't have the luxury of full informational bandwidth. Games designers in AAA companies working on sound are often allowed a very small portion of space and processing, and have to make do with serious compromises, often not psychoacoustically informed.

Outside of those circumstances I think the point of good audio reproduction is to eliminate the need for listening tests. We're there with electronics, but not with speakers, although I have many more thoughts on the latter.