MQA Deep Dive - I published music on tidal to test MQA

[amirm]

@amirm. It also was pointed out continuously that what you suggest is NOT how good testing works. Instead, you create test cases that intend to break the system.

You are mistaken on that. The idea is not to "break" a codec but to find its limits of performance. The content needs to be "legal" otherwise. To wit, MPEG, ITU, and other international standards groups have created a set of audio clips used for lossy codecs. Here is the list:

All of these are legal music but will stress codecs due to carefully chosen material that a) pushes the codec to the limit and b) makes it easier to hear the artifacts.

Do you see square wave here? No.

Do you see impulse? No.

Do you see white noise? No.

Do you see ultrasonic content? No.

Do you see random bits to break the encoder/decoder? No.

You see carefully chosen natural recordings that are chosen and standardized for evaluation of such coders. No one can argue that these clips are not valid input to lossy codecs.

OP, not being familiar with this field, threw a bunch of test clips at MQA. MQA explained to him why this was not correct that the very foundation of MQA is the assumption of the music content in music file. He ignored that. Didn't come and ask people like me what we thought. He decided he knew enough and ran with it. The result is that his testing can be and is easily dismissed as being broken.

Please learn to listen to experts and people who do this for a living. Not everything works based on how your audiophile experiences and stuff you read online. Some topics in audio are very complex and this is one of them.

BTW, this is the forth or fifth time I have explained this very thing. I hope we are done and a new challenge doesn't come up yet again.

 

[Raindog123]

Testing to failure in most cases in my line of work (defense) is not needed. Fun though.

...no single ATO is granted without a 'destruction' test. A guy who lives a few miles from Hanscom has to know that... But I'm forgetting I am talking to RTX here, who have long ways catching up with LMs and BAs, the true industry cream-of-the-crop...

 

[Raindog123]

You are mistaken on that. The idea is not to "break" a codec but to find its limits of performance. The content needs to be "legal" otherwise.

There is more than one test. And each has its purpose. And again, you know that already.

 

[amirm]

The real question is, where are the true interested parties are not stepping in to collaborate on the tests and drive them in the 'right' direction?

Well, I am not an interested party so don't look to me. I use MQA but if it went away, c'est la vie! I could have reviewed half a dozen more gear in the time I have spent in this thread. I can't even tell who is really interested in MQA or high-res audio versus making a name for themselves by chanting, "MQA is evil."

 

[amirm]

There is more than one test. And each has its purpose. And again, you know that already.

No I don't. What are you trying to say? More than one test for what?

 

[abdo123]

Let me show you what *is* ironic. OP is a subjectivist and puts his sighted unreliable listening way ahead of any measurements. Just check out this review of Schiit Magnius which he did in February of this year. I gave it a glowing review and this is his take on it. The title is, ready for it? "
Schiit Magnius Review - Measurements don't tell the whole story.....

"the dynamic range feels quite compressed...it doesn't come across in measurements"

"... this is a fatiguing amp to me..."

"there is slight graininess and lack of separation...."

"there is not a lot of texture [in bass]..."

"timbre is not great in this amp..."

"mid texture and detail is not good..."

And this is in the first 7 minutes of this video!

It ends with some nonsense about how Schiit was "forced" into designing Magnius to please people who look for good measurements. And that this can't be their fault, i.e. it is our fault!

Now he says measurements matter with MQA? If measurements are not important, why didn't he do a listening test of MQA instead? Let's see if it has or has not any bass texture and speed. What it is timbre doing. Does it have texture? Or does it not? We need to know!

My suggestion: be very careful who you are holding up as your expert witness. And making snide remarks. When the truth is not on your side, it can backfire big time!

He already shared that school of thought earlier in this forum

https://www.audiosciencereview.com/...-music-on-tidal-to-test-mqa.22549/post-747236

However just like you, he is entitled to his own opinion.
from my perspective you both have a lot of unsubstantiated opinions (MQA marketing claims are not facts either) and you’re both belittling each other so I kindly advise you to stop this back and forth before you further embarrass each other.

 

[DimitryZ]

Or not bad enough to make a real difference, let's not forgot this possibility to stay neutral.

Anyway, my feeling is that I get a different master, because after thinking it twice, the main thing I hear can look like adding a transient exciter, and if you add such a thing to a master, it's not the same master anymore. It can technically be a different thing, but the listening result gives me this feeling. There are other things but it's the main difference I see. And won't say it's better or not, because it should be an artistic choice.
Even subtil, it can be heard, an enough to not match a small difference like we see in null test of the files, that's why I'm not sure anymore there's more change on the file itself than on the rendering. Need more test.

Sure, one can never test too much.

You recording experience is valuable here.

To my ears, MQA sounds like it does have a lower noise floor and very sharp edges.

And I know this is ASR and all, but when MQA works well (and that's not always) I get something I call "performance specificity." A kind of certainty in the back mind that you are listening to a very specific performance/recording. It's an odd phenomena, and I am sure that haters here will have a ball with it.

 

[DimitryZ]

...no single ATO is granted without a 'destruction' test. A guy who lives a few miles from Hanscom has to know that... But I'm forgetting I am talking to RTX here, who have long ways catching up with LMs and BAs, the true industry cream-of-the-crop...

Our hardware is far too expensive to destroy in a test. Our big production runs are a dozen or two.

Why a weird dig at RTX and my place of residence?

Just like Amir is gently suggesting that you don't know much about codec testing, I suggest that you don't know much about dynamics testing. I do vibration test profiles development for a living (one of my many interests as an engineering fellow) and in the context of large and very expensive military systems it's a very specialist field.

And no we don't break our AMDR radar (first shipset delivered) to find out where that may be. Lockmart sure lost that one.

 

[Grooved]

...

Do you see square wave here? No.

Do you see impulse? No.

Do you see white noise? No.

Do you see ultrasonic content? No.

Do you see random bits to break the encoder/decoder? No.

You see carefully chosen natural recordings that are chosen and standardized for evaluation of such coders. No one can argue that these clips are not valid input to lossy codecs.
...

Wouldn't it be good if you create some kind of a sticky thread with this information? Just saying, at least anyone could link to it if needed.

Regarding the OP file, if I'm not wrong, it starts with music. Can this part be considered as usable for test ?
I would say that :
- if the encoder is processing without analyzing the whole file first, yes
- if the encoder analyze the whole file first, no

 

[Raindog123]

No I don't. What are you trying to say? More than one test for what?

To bring a system to the market. And to validate its claims and operational validity.

Google created the entire Google Test ecosystem, methodology and philosophy as an answer to your this very question... And even at Microsoft, I am sure you guys have heard of it.


As for

Please learn to listen to experts and people who do this for a living. Not everything works based on how your audiophile experiences and stuff you read online. Some topics in audio are very complex and this is one of them.

you must have missed my post. Yes you are an undeniable expert here, but you are not the only expert.

 

[mtristand]

Let me show you what *is* ironic. OP is a subjectivist and puts his sighted unreliable listening way ahead of any measurements. Just check out this review of Schiit Magnius which he did in February of this year. I gave it a glowing review and this is his take on it. The title is, ready for it? "
Schiit Magnius Review - Measurements don't tell the whole story.....

"the dynamic range feels quite compressed...it doesn't come across in measurements"

"... this is a fatiguing amp to me..."

"there is slight graininess and lack of separation...."

"there is not a lot of texture [in bass]..."

"timbre is not great in this amp..."

"mid texture and detail is not good..."

And this is in the first 7 minutes of this video!

It ends with some nonsense about how Schiit was "forced" into designing Magnius to please people who look for good measurements. And that this can't be their fault, i.e. it is our fault!

Now he says measurements matter with MQA? If measurements are not important, why didn't he do a listening test of MQA instead? Let's see if it has or has not any bass texture and speed. What it is timbre doing. Does it have texture? Or does it not? We need to know!

My suggestion: be very careful who you are holding up as your expert witness. And making snide remarks. When the truth is not on your side, it can backfire big time!

What?

I'm not really sure what your intent is with this post but it comes off as bullying to me - this is completely unrelated to the topic at hand.

There is a difference between:

1. Evaluating/reviewing something subjectively by giving feedback from listening tests.

and

2. Evaluating a company's hard claim of something being lossless, which is not something you do with listening, but by comparing the original's structure to the thing being compressed/decompressed.

Now he says measurements matter with MQA? If measurements are not important, why didn't he do a listening test of MQA instead? Let's see if it has or has not any bass texture and speed. What it is timbre doing. Does it have texture? Or does it not? We need to know!

Hard measurements matter when evaluating a claim of losslessness -- in such a context you ensure what comes back hasn't lost any information.

You just said this a couple posts back:

No. It is the opposite. It shows that we/I understand how you test each audio system appropriately.

...

Both airplane and cars have wheels but they serve completely different purposes. Don't mix them up together.

It seems like you are now committing the very type of fallacy you just decried by conflating the types of tests used for different types of claims.

 

[Raindog123]

Why a weird dig at RTX and my place of residence?

Are you sure its a dig?

Or maybe because of your posts like this?

 

[Rottmannash]

So this means you don't agree? Or you have a different interpretation of the plot?

Your logo says you are addicted to learning.

Well?

The "addicted to fun and learning" tag is bestowed by Amir to most members, not created by the member.

 

[DimitryZ]

Are you sure its a dig?

Or maybe because of your posts like this?

My employment at RTX has nothing to with my posts here.

And yes it was a dig. Why?

And are you the guy who posted an employment letter for yourself from 20 years ago and won't reveal your current situation except to boast about it?

You know that's weird, right?

 

[DimitryZ]

The "addicted to fun and learning" tag is bestowed by Amir to most members, not created by the member.

Oh cool! I am a newbie here.

I will endeavor to earn it in the future.

 

[amirm]

To bring a system to the market. And to validate its claims and operational validity.

There is no MQA claim regarding encoding square waves, white noise to 44 kHz, etc. Its claims are all about music encoding.

Google created the entire Google Test ecosystem, methodology and philosophy as an answer to your this very question... And even at Microsoft, I am sure you guys have heard of it.

Software testing has little to do with signal processing testing. They are completely different things. Yes, the tool itself shouldn't crash and all but past that, it is all about its intended purpose which is determined with listening tests. In the case of MQA, you would look at what it intends to do and develop valid tests to evaluate it.

Now maybe wherever you learned testing you blindly develop tests with no regard for what something is supposed to do. That is not what we did at Microsoft or all the other places I worked. Nor is it true of Google.

Is this concept that hard to understand? I don't know how many times I have explained yet I get more pushback and snide remarks like this.

 

[DimitryZ]

What?

I'm not really sure what your intent is with this post but it comes off as bullying to me - this is completely unrelated to the topic at hand.

Amir, you claim to be an expert -- surely you understand that there is a difference between:

1. Evaluating/reviewing something subjectively by giving feedback from listening tests.

and

2. Evaluating a company's hard claim of something being lossless, which is not something you do with listening, but by comparing the original's structure to the thing being compressed/decompressed.



Hard measurements matter when evaluating a claim of losslessness... in such a context you ensure what comes back hasn't lost any information.

You just said this a couple posts back:



Like, you just said this. You literally just said this.

So why now are you committing the very fallacy you just decried by mixing them up together and conflating testing methods for different types of claims?

The fact that MQA was not mathematically lossless has always been obvious to all and certainly since Archimago's tests in 2017, MQA confused advertising notwithstanding.

4 years ago he has shown MQA as a bit lossy, but much, much better than the best lossy codecs. And demonstrated that it "works" - i.e. delivers ultrasonic content in a CD container. He also shown that it nulls against the original to a level that is extremely hard to discern for your equipment, much less you. He even commended MQA on its accomplishment.

All of this has been known and publically available for four years. And yet here we are going over the same misconceptions again, every day!

 

[amirm]

What?

I'm not really sure what your intent is with this post but it comes off as bullying to me - this is completely unrelated to the topic at hand.

The intent was quite obvious. The poster asked me why I do one type of testing for hardware and am advocating another for MQA. In other words, he was challenging me to see if I am consistent. So I showed how the OP could care less about measurements yet all of a sudden has gotten religion when it comes to MQA. To the extent his reviews of hardware is all about random listening tests, then that is what he should have done with MQA. If he is not, he is the one that is hugely inconsistent.

As to rest of your post, you are being purely argumentative with no value added to the thread. One more like them and you will get a Reply Ban. Think hard before hitting post.

 

[bboris77]

Let me show you what *is* ironic. OP is a subjectivist and puts his sighted unreliable listening way ahead of any measurements. Just check out this review of Schiit Magnius which he did in February of this year. I gave it a glowing review and this is his take on it. The title is, ready for it? "
Schiit Magnius Review - Measurements don't tell the whole story.....

"the dynamic range feels quite compressed...it doesn't come across in measurements"

"... this is a fatiguing amp to me..."

"there is slight graininess and lack of separation...."

"there is not a lot of texture [in bass]..."

"timbre is not great in this amp..."

"mid texture and detail is not good..."

And this is in the first 7 minutes of this video!

It ends with some nonsense about how Schiit was "forced" into designing Magnius to please people who look for good measurements. And that this can't be their fault, i.e. it is our fault!

Now he says measurements matter with MQA? If measurements are not important, why didn't he do a listening test of MQA instead? Let's see if it has or has not any bass texture and speed. What it is timbre doing. Does it have texture? Or does it not? We need to know!

My suggestion: be very careful who you are holding up as your expert witness. And making snide remarks. When the truth is not on your side, it can backfire big time!

What I find ironic is that you guys are inadvertently helping each other by directing traffic to each other’s accounts. Many SBAF guys are here watching this epic argument and now tons of ASR people are going to watch @GoldenOne’s YouTube videos.

Plus, as the saying goes, there’s no bad publicity. So I’m going to propose a brand new conspiracy theory.....naaaah. I think we’ve all had enough of those.

 

[Raindog123]

You know that's weird, right?

Weird is my middle name.

And stating obvious, if you're not cool with either - my post or my weirdness - there are plenty other places. I am sure the Internet is big enough for the two of us.
Being an optimist I am, I generally would be curious to hear your opinion, but so far have not been too impressed. Let's hope this will change.