MQA Deep Dive - I published music on tidal to test MQA

[amirm]

What I find ironic is that you guys are inadvertently helping each other by directing traffic to each other’s accounts. Many SBAF guys are here watching this epic argument and now tons of ASR people are going to watch @GoldenOne’s YouTube videos.

I am not worried about sending him traffic. If traffic helps advance the topic, so be it. People look to me for consistency, constantly asking why I am not at war with MQA. I want them to know that they themselves are championing a pure subjectivist that puts little to no value to objective audio testing and research.

 

[mtristand]

So I showed how the OP could care less about measurements yet all of a sudden has gotten religion when it comes to MQA.

You're sidestepping the main point I am making in that post:

MQA was making claims about losslessness. GoldenSound set out to test those claims. To test a claim of losslessness, that requires measurements and structural analysis / comparison.

It's not a case of "finding religion when it comes to MQA." It's using the right kind of analysis for the claim. You don't use a "listening test" to evaluate a claim of losslessness. You can use a listening test for other types of claims -- but losslessness is a specific type of claim.

And to address your earlier comments, he tested more than just test sounds - he also tested files containing "natural music" and found the same kinds of results.

Edit: I got banned because of this post, I can't reply anymore.

 

[Grooved]

Sure, one can never test too much.

You recording experience is valuable here.

To my ears, MQA sounds like it does have a lower noise floor and very sharp edges.

And I know this is ASR and all, but when MQA works well (and that's not always) I get something I call "performance specificity." A kind of certainty in the back mind that you are listening to a very specific performance/recording. It's an odd phenomena, and I am sure that haters here will have a ball with it.

I will try to not rely only on my impression, and ask to a friend with impressive earing if he can give his one without telling him anything.
Now, I see what you mean by "sharp edges", the thing that needs to be check is that even if add something interesting, it can be at the cost of loosing soething elsewhere in the mix. At this moment, I don't know if it's the case.
The other thing to be sure is if it's possible or not to achieve the same sound with PCM by using different setting in the mix/master, because, if it's the case, it would mean MQA add an "effect" that was not decided to be used by the creators. If it can't be achieved while keeping PCM, it's a different thing.
I have a transiant exciter that I can put between the DAC and the amp/speakers, maybe it can be interesting to test it to see if it approaches MQA.

I'm not a hater, neither a pro, and I would think it's not what I would use. I will never use "performance" in an case until I'm sure that I get more on one thing without loosing on another

 

[DimitryZ]

There is no MQA claim regarding encoding square waves, white noise to 44 kHz, etc. Its claims are all about music encoding.


Software testing has little to do with signal processing testing. They are completely different things. Yes, the tool itself shouldn't crash and all but past that, it is all about its intended purpose which is determined with listening tests. In the case of MQA, you would look at what it intends to do and develop valid tests to evaluate it.

Now maybe wherever you learned testing you blindly develop tests with no regard for what something is supposed to do. That is not what we did at Microsoft or all the other places I worked. Nor is it true of Google.

Is this concept that hard to understand? I don't know how many times I have explained yet I get more pushback and snide remarks like this.

Strong advocate of functional testing as well.

We actually developed a HALT methodology that used system's environmental envelope (instead of standard profile) and raised it until we got failures. If you are going to break something, makes sense to do it with a profile that's related to its' actual service environment.

 

[mtristand]

The fact that MQA was not mathematically lossless has always been obvious to all and certainly since Archimago's tests in 2017, MQA confused advertising notwithstanding.
...
All of this has been known and publically available for four years. And yet here we are going over the same misconceptions again, every day!

MQA hasn't changed their marketing materials accordingly - only just recently updating their "Is MQA lossless?" page. Everything else I quoted in my previous posts here is recent or still live.

 

[BlackTalon]

What I find ironic is that you guys are inadvertently helping each other by directing traffic to each other’s accounts. Many SBAF guys are here watching this epic argument and now tons of ASR people are going to watch @GoldenOne’s YouTube videos.

Plus, as the saying goes, there’s no bad publicity. So I’m going to propose a brand new conspiracy theory.....naaaah. I think we’ve all had enough of those.

Eh, I'm not going to watch his YT videos. After seeing the crap he posted on the other forum accusing Amir's company of PPP fraud I frankly lost any respect for anything he has to say. Most forum owners would have banned him permanently from their sites; the fact that Amir lets him keep posting says a lot more about Amir than about the OP, who debates here like he did not write any of that crap elsewhere. He apparently thinks on this forum if he just sticks to technical stuff and pretends he did not write any of that than people will look at him seriously. Maybe some do, but I definitely do not.

 

[DimitryZ]

Weird is my middle name.

And stating obvious, if you're not cool with either - my post or my weirdness - there are plenty other places. I am sure the Internet is big enough for the two of us.
Being an optimist I am, I generally would be curious to hear your opinion, but so far have not been too impressed. Let's hope this will change.

Same here.

You sound like a high level manager that insists on participating in a brainstorming session about a really hard technical problem and whom an old engineer has to interrupt with "Do you have any good ideas?"

 

[Raindog123]

All of these are legal music but will stress codecs due to carefully chosen material that a) pushes the codec to the limit and b) makes it easier to hear the artifacts.

Do you see square wave here? No.
Do you see impulse? No.
Do you see white noise? No.
Do you see ultrasonic content? No.
Do you see random bits to break the encoder/decoder? No.

@amirm, next time I am in Kent, I'll sneak you in our lab. To show you how a waveform that 'keeps you and your family (and mine) safe at night' is tested. During the design phase:

Square wave - Absolutely, the bread-and-butter!
Impulse response - Can't do without!
Wide while noise: Yes!
Ultrasonic content: Well, sure, various out-of-band artifacts!
Random bits: This is what numerous protocol testers and error generators are for.

Again, this is the encoder/decoder design and implementation phases. And each of the above test vignettes has it well defined purpose... Now, after those are complete, you do use your 'user application' loads (your test files), to assess the user utility, compare with other codecs, verify integration with other elements, etc. But this comes way later, after you're sure the basics work rock-solid.

 

[Raindog123]

Same here. You sound like a high level manager that insists on participating in a brainstorming session about a really hard technical problem and whom an old engineer has to interrupt with "Do you have any good ideas?"

Be careful! Our boss here on ASR comes from those ranks, if you have not read his Bio yet... Yet, if you ask me, he does pretty okay with his hands/equipment/head! So, not every high level retired CEO or VP is as useless as you paint them...

And I salute your keen sense of 'figuring everything/everyone out' on your second day here. (As for me, I am just 'a dude with a mop.' Good Will... kind, you know. )

 

[ericrosenfield]

Let me show you what *is* ironic. OP is a subjectivist and puts his sighted unreliable listening way ahead of any measurements. Just check out this review of Schiit Magnius which he did in February of this year. I gave it a glowing review and this is his take on it. The title is, ready for it? "
Schiit Magnius Review - Measurements don't tell the whole story.....

"the dynamic range feels quite compressed...it doesn't come across in measurements"

"... this is a fatiguing amp to me..."

"there is slight graininess and lack of separation...."

"there is not a lot of texture [in bass]..."

"timbre is not great in this amp..."

"mid texture and detail is not good..."

And this is in the first 7 minutes of this video!

It ends with some nonsense about how Schiit was "forced" into designing Magnius to please people who look for good measurements. And that this can't be their fault, i.e. it is our fault!

Now he says measurements matter with MQA? If measurements are not important, why didn't he do a listening test of MQA instead? Let's see if it has or has not any bass texture and speed. What it is timbre doing. Does it have texture? Or does it not? We need to know!

My suggestion: be very careful who you are holding up as your expert witness. And making snide remarks. When the truth is not on your side, it can backfire big time!

This particular post is really strange to me. Are you suggesting that it’s not possible for an amp to measure well and sound bad? That there’s nothing an amp does that isn’t accounted for in these measurements?

I’ve never heard the Magnius and can’t speak for it’s qualities, but Golden One is not the only reviewer I’ve seen say that it measures well but doesn’t sound good. If such a thing is possible, I don’t see what’s wrong with a review that points it out.

 

[bboris77]

This particular post is really strange to me. Are you suggesting that it’s not possible for an amp to measure well and sound bad? That there’s nothing an amp does that isn’t accounted for in these measurements?

I’ve never heard the Magnius and can’t speak for it’s qualities, but Golden One is not the only reviewer I’ve seen say that it measures well but doesn’t sound good. If such a thing is possible, I don’t see what’s wrong with a review that points it out.

I think that @GoldenOne would not describe himself as a subjectivist. He obviously does believe in measurements while leaving open the possibility that the currently available set of measurements may not capture absolutely everything that defines how a component performs. We have had this argument here many times. The two sides will never see eye to eye. Most people on these forums believe that it is all placebo and proposed to people that doubt total reliance on measurements to try performing a level-matched double blind test to see if they can distinguish between different components. To be honest, I have not seen one video where anybody has done this and demonstrated that they can perform this feat.

Personally, I do believe that we are not able to measure absolutely everything, but until we figure out a way to be able to do this, I am going with current measurements and science.

 

[amirm]

I think that @GoldenOne would not describe himself as a subjectivist.

If he doesn't then he doesn't know the meaning of the word. The entire video is full of the type of talk that subjectivists use. It is 20 minutes of "my ears say this and that," none of it confirmed with one blind test. He says on dynamic range that the loudest signal is loud but that the softest signals have been raised so that the overall dynamic range is too low. This is absurd talk and something measurements instantly show.

As far as extremism go in audio, he could not be farther from us. A headphone amp that I have given the highest praise for is positioned by him as a mistake for Schiit to have even produced! This is no minor disagreement. This is completely polar opposite of what we are about.

 

[amirm]

@amirm, next time I am in Kent, I'll sneak you in our lab. To show you how a waveform that 'keeps you and your family (and mine) safe at night' is tested. During the design phase:

Square wave - Absolutely, the bread-and-butter!
Impulse response - Can't do without!
Wide while noise: Yes!
Ultrasonic content: Well, sure, various out-of-band artifacts!
Random bits: This is what numerous protocol testers and error generators are for.

What? The fact that you use these test signal for a different technology has nothing to do with the topic at hand. Show me one research paper that uses these signals for lossy audio compression. You won't find one. I remember doing a frequency response test on our codec at Microsoft and finding issues there. My team was surprised as nobody had run even this test! They fixed the problem but it simply was not on their radar. MPEG test clips as I listed rule the world. As do controlled listening tests.

 

[amirm]

This particular post is really strange to me. Are you suggesting that it’s not possible for an amp to measure well and sound bad? That there’s nothing an amp does that isn’t accounted for in these measurements?

I’ve never heard the Magnius and can’t speak for it’s qualities, but Golden One is not the only reviewer I’ve seen say that it measures well but doesn’t sound good. If such a thing is possible, I don’t see what’s wrong with a review that points it out.

There are plenty of things people say online. It doesn't make them correct or true.

So it is clear: listening tests are wonderful. You just need to run the controlled so you don't know what is what. If the bass is not fast, that is cool. Just do a blind test against another that has fast bass and show that you can tell 9 out of 10 times. If it is that obvious that such a flaw exists, then it should be trivial to do this. It is much less work than producing 20 minutes of video.

If you don't do that then what you are saying for the device lacks any foundation to be valid. Indeed, I will donate $1000 to Goldeneye's favorite charity if he can prove any of those observations if he can do so blind and in controlled testing.

And yes, other people who repeat the same things are wrong as well. Plurality of mistakes doesn't make them correct.

In this forum we follow proper science and engineering. There are plenty of other places where they practice what you say. You or OP don't belong here if you don't understand what we are doing.

 

[ericrosenfield]

I think that @GoldenOne would not describe himself as a subjectivist. He obviously does believe in measurements while leaving open the possibility that the currently available set of measurements may not capture absolutely everything that defines how a component performs. We have had this argument here many times. The two sides will never see eye to eye. Most people on these forums believe that it is all placebo and proposed to people that doubt total reliance on measurements to try performing a level-matched double blind test to see if they can distinguish between different components. To be honest, I have not seen one video where anybody has done this and demonstrated that they can perform this feat.

Personally, I do believe that we are not able to measure absolutely everything, but until we figure out a way to be able to do this, I am going with current measurements and science.

I mean, Resolve did a video where he shows he can pick out differences between DACs and amps in blind testing:

But he's not doing any comparison to measurements there. It feels like it wouldn't be easy to set up blind tests between the Magnius and an amp that didn't test as well (the Asgard 3?) and see which one people thought sounded better. "Better" is, of course, entirely subjective, but it's also the only thing that actually matters when you're talking about audio gear, right? What sounds good? Or am I missing something?

But I'm probably wading into a debate that's gone on for a long time around here, so I should probably keep my mouth shut, since I'm obviously the ignorant n00b asking the ignorant n00b questions.

 

[bboris77]

I will donate $1000 to Goldeneye's favorite charity if he can prove any of those observations if he can do so blind and in controlled testing.

While I chuckle every time you call @GoldenOne GoldenEye, I would welcome him or anyone else be able to demonstrate this feat. Obviously, there would have to be some kind of independent oversight to ensure that that both “sides” accept the legitimacy of this experiment.

 

[ericrosenfield]

There are plenty of things people say online. It doesn't make them correct or true.

So it is clear: listening tests are wonderful. You just need to run the controlled so you don't know what is what. If the bass is not fast, that is cool. Just do a blind test against another that has fast bass and show that you can tell 9 out of 10 times. If it is that obvious that such a flaw exists, then it should be trivial to do this. It is much less work than producing 20 minutes of video.

If you don't do that then what you are saying for the device lacks any foundation to be valid. Indeed, I will donate $1000 to Goldeneye's favorite charity if he can prove any of those observations if he can do so blind and in controlled testing.

And yes, other people who repeat the same things are wrong as well. Plurality of mistakes doesn't make them correct.

In this forum we follow proper science and engineering. There are plenty of other places where they practice what you say. You or OP don't belong here if you don't understand what we are doing.

My reply came at the same time as your reply so I missed it. (And I'm sorry I'm taking us so wildly off the topic of MQA.)

Just so I'm clear: your position is that you shouldn't criticize a piece of gear unless the problems show up in measurements or can be determined through blind testing, is that correct?

I can see where you're coming from, but it would make a reviewer's job a lot more difficult if they hear things that aren't represented in the measurements, since they'd have to set up a blind test every time.

 

[DimitryZ]

I will try to not rely only on my impression, and ask to a friend with impressive earing if he can give his one without telling him anything.
Now, I see what you mean by "sharp edges", the thing that needs to be check is that even if add something interesting, it can be at the cost of loosing soething elsewhere in the mix. At this moment, I don't know if it's the case.
The other thing to be sure is if it's possible or not to achieve the same sound with PCM by using different setting in the mix/master, because, if it's the case, it would mean MQA add an "effect" that was not decided to be used by the creators. If it can't be achieved while keeping PCM, it's a different thing.
I have a transiant exciter that I can put between the DAC and the amp/speakers, maybe it can be interesting to test it to see if it approaches MQA.

I'm not a hater, neither a pro, and I would think it's not what I would use. I will never use "performance" in an case until I'm sure that I get more on one thing without loosing on another

Look forward to your new posts.

 

[GoldenOne]

Wouldn't it be good if you create some kind of a sticky thread with this information? Just saying, at least anyone could link to it if needed.

Regarding the OP file, if I'm not wrong, it starts with music. Can this part be considered as usable for test ?
I would say that :
- if the encoder is processing without analyzing the whole file first, yes
- if the encoder analyze the whole file first, no

Also worth noting I did try submitting a file which was just a 1khz -3dBFS sine, but I received a rejection notice saying "The MQA encoder was unable to encode the file"

 

[DimitryZ]

Be careful! Our boss here on ASR comes from those ranks, if you have not read his Bio yet... Yet, if you ask me, he does pretty okay with his hands/equipment/head! So, not every high level retired CEO or VP is as useless as you paint them...

And I salute your keen sense of 'figuring everything/everyone out' on your second day here. (As for me, I am just 'a dude with a mop.' Good Will... kind, you know. )

My VP is excellent. She is the best software engineer anyone has seen East side in some time. She also doesn't post her employment offers on forums.

I was talking about you.