• WANTED: Happy members who like to discuss audio and other topics related to our interest. Desire to learn and share knowledge of science required. There are many reviews of audio hardware and expert members to help answer your questions. Click here to have your audio equipment measured for free!

MQA vs. Non-MQA, Empirical Measurements

Tokyo_John

Active Member
Forum Donor
Joined
Mar 6, 2021
Messages
214
Likes
289
As a newer member of ASR, I've learned a ton from reading posts on a variety of topics that use relevant empirical data in order to cut through the snake oil of audio marketing. This is a great service to audiophiles, a way to provide some ground truth that can focus our energy on getting the best sound quality at a reasonable cost without being distracted by all the gimmicks. Special thanks to @amirm for doing this (I know that you take a lot of flack from people for bursting their bubbles, I completely understand and empathize).

That being said, I've been a bit frustrated by some of the discussions surrounding MQA on this forum, and I've been distracted from ground truth a couple times and followed discussions down a tin foil hat rabbit hole to nowhere. We live in distrustful times, and people are often too quick to question or judge the motives of others, devolving into ad hominem arguments that distract us from the empirical facts. There are many claims about MQA encoded audio, both from MQA and from critics, but the only way to get hard facts is to make measurements...followed up by thoughtful analysis that focuses on those facts alone and doesn't jump to conclusions that aren't supported by data.

I am mostly concerned about the practical end product, which is the sound that actually comes out of my speakers and into my ears. We need to measure this by recording what comes out of the speakers directly, in as uniform a manner as possible. And I would like to encourage other people to do the same thing. When we say that there is "no audible difference" we should back up such claims with empirical data instead of hypothetical arguments. When we say that there is a difference, then let's see the measurement to prove it.

To this end, I did a simple experiment: I compared recorded playback through my loudspeaker system of MQA tracks streamed through the Tidal App vs. CD-derived (AIFF 16/44.1) tracks played using the latest version of Audirvana using its standard (factor of 2x) upsampling algorithms. The audio system is simple: MacBook Pro -> USB -> Topping D90 MQA -> March Audio P252 -> LS50 Meta speakers. The D90 roll-off filter is in mode 3 with typically -20 dB attenuation. The recording is made by a Sony PCM-D100 at 24/192 mounted on a heavy tripod at speaker level with the stereo microphones individually aimed at the speakers' focus points.

All components are kept in the same positions and settings as I play back different tracks. This means that, even if the measurement equipment is imperfect, correlated artifacts and imperfections will cancel out in the relative comparison. In terms of an equation:
Recording_1 = Signal_1 + Correlated_Imperfections_1 + Uncorrelated_Imperfections_1
Recording_2 = Signal_2 + Correlated_Imperfections_2 + Uncorrelated_Imperfections_2
where I define:
Correlated_Imperfections_1 = Correlated_Imperfections_2.
If I subtract these first two equations and use the third to simplify,
Recording_1-Recording_2 = Signal_1 - Signal_2 + (Uncorrelated_Imperfections_1 - Uncorrelated_Imperfections_2)
then the correlated imperfections go away. Correlated imperfections include any artifacts in the sonic character or quality of the entire chain of playback equipment down to the speaker itself. Uncorrelated imperfections, on the other hand, are artifacts that are not the same between recordings, such as stray sounds or noises in the room that appear in one recording but not the other. There are also ways to deal with uncorrelated imperfections, if they cannot be adequately controlled. If one assumes that these are normally distributed about a null mean then we can take many recordings of each signal and then average them together, in which case the amplitude of the uncorrelated imperfections decreases inversely proportional to the square root of the number of averaged samples.

I searched for a variety of tracks for which I have quality CD rips and which also stream as "Master" quality on Tidal and are explicitly recognized as MQA on the display of the D90 during playback. There were plenty of examples from my library (mostly rock music), however, subjectively I could not hear any differences in the tracks I compared. I opened the comparison recordings in Adobe Audition, and aligned the stereo tracks with one another for direct comparison. Here is an example excerpt from the track "Starshine" by Gorillaz, which has a lot of texture in the high frequencies and for which I expected to see some differences, if they exist. The top pair of waveforms (purple-ish) are the MQA/Tidal versions, and the bottom pair (orange-ish) are the CD quality/Audirvana versions:
Stereo_Waveforms_MQA_CD_AIFF_Upsampled.png


The data seems to indicate that, apart from slight over all amplitude differences, the sound that I recorded from both versions is visually the same. At least upon visual inspection, the waveforms appear to be well-matched. (I showed some excerpts to my wife, a seismologist who studies wiggly lines and sniffs out differences, and she had the same impression.)

I scaled and overlaid some recordings on top of one another to see if any difference shows up. Here is an example from the above excerpt:
Stereo_MQA_CD_Overlay_Comparison.png


If you squint, you can see some very subtle differences in the top channel (less so for the bottom). However, it isn't clear if this is due to interpolation anomalies from the Adobe Audition display (which draws the solid line), caused by uncorrelated imperfections, or if it reveals some very slight differences in the MQA vs CD renderings.

I looked all through this track, and I found the same characteristics everywhere I looked. I've also looked at some other tracks (mostly RHCP tunes that have clean CD rips). So far, it looks as if playback (through my system) of Tidal "Master" MQA tracks and upsampled CD quality played through Audirvana are very close to identical.

Caveats: This work so far is done using visual comparisons of the waveforms, I've only looked for obvious differences and I might have missed more subtle cues. Also, any inferences made from this data are only relevant to my specific system and recorder set-up, as well as to the tracks I've tested. Other systems and tracks might reveal differences that my system does not.

Future work: Quantitative analysis is merited, cross-correlation to exactly align the pairs of recordings and normalization of the amplitudes. Once this step is completed, the aligned and scaled tracks can be subtracted from one another to reveal their measured differences. The difference can then be analyzed in order to see if it arises from actual differences in the tracks, or if it smells like uncorrelated noise. This is an obvious next step...it doesn't seem that Audition has the capability to do this, so I may need to write some code and run the analysis myself, which takes a little time (unless somebody else has some suggestions for software that performs this task).

Practical Conclusion: I cannot tell any difference subjectively and no compelling indications (yet) objectively that MQA and upsampled CD-quality tracks sound any different in my system. In this sense, at least for me personally, MQA is no better, nor is it any worse, than listening to CD quality tracks through a quality player.

I look forward to any comments on how to improve my method, if anyone sees any flaws or pitfalls, and discussion of the measurements (and techniques).
 

muslhead

Major Contributor
Forum Donor
Joined
May 28, 2020
Messages
1,560
Likes
1,718
Were you sure the MQA and CD sources were from the same studio master?
 

BlackTalon

Addicted to Fun and Learning
Forum Donor
Joined
Apr 14, 2021
Messages
586
Likes
898
Location
DC
Thank you for running these tests. It is 'real world' to me (engineer, not a scientist -- so real world matters to me).

It would be interesting to see some of the non-Master MQA files compared to original Redbook files, as that seems to be the biggest area of debate, especially if catalogs are being replaced with a lot of MQA files.
 

AdamG

Proving your point makes it “Science”.
Moderator
Forum Donor
Joined
Jan 3, 2021
Messages
4,636
Likes
14,918
Location
Reality

RayDunzl

Grand Contributor
Central Scrutinizer
Joined
Mar 9, 2016
Messages
13,201
Likes
16,983
Location
Riverview FL
I look forward to any comments on how to improve my method, if anyone sees any flaws or pitfalls, and discussion of the measurements (and techniques).

The differences will (maybe) mostly exist at a level not revealed by your wiggles viewed at "full scale".

If the errors are below -80dBFS, for example, you would need to magnify your visual scale by 10,000 to see differences in the waveform.

1619831925326.png
 
Last edited:

amirm

Founder/Admin
Staff Member
CFO (Chief Fun Officer)
Joined
Feb 13, 2016
Messages
44,368
Likes
234,393
Location
Seattle Area
Future work: Quantitative analysis is merited, cross-correlation to exactly align the pairs of recordings and normalization of the amplitudes. Once this step is completed, the aligned and scaled tracks can be subtracted from one another to reveal their measured differences. The difference can then be analyzed in order to see if it arises from actual differences in the tracks, or if it smells like uncorrelated noise. This is an obvious next step...it doesn't seem that Audition has the capability to do this, so I may need to write some code and run the analysis myself, which takes a little time (unless somebody else has some suggestions for software that performs this task).
Deltawave from our member @pkane does all of this: https://www.audiosciencereview.com/...test-deltawave-null-comparison-software.6633/
 
OP
Tokyo_John

Tokyo_John

Active Member
Forum Donor
Joined
Mar 6, 2021
Messages
214
Likes
289
Were you sure the MQA and CD sources were from the same studio master?

Not sure.

Thank you for running these tests. It is 'real world' to me (engineer, not a scientist -- so real world matters to me).

It would be interesting to see some of the non-Master MQA files compared to original Redbook files, as that seems to be the biggest area of debate, especially if catalogs are being replaced with a lot of MQA files.

Agree...will put this in the to-do list. I'm getting more curious, and not just about MQA.

The differences will (maybe) mostly exist at a level not revealed by your wiggles viewed at "full scale".

If the errors are below -80dBFS, for example, you would need to magnify your visual scale by 10,000 to see differences in the waveform.

I don't think that my setup has that much dynamic range anyways. I'll probably stick to order ~1% or greater differences.


Thanks @amirm this looks just right...unfortunately I'm strictly a linux/unix/mac user. Interestingly, there may be some seismic analysis code that does all of this, which I'm looking into.
 

amirm

Founder/Admin
Staff Member
CFO (Chief Fun Officer)
Joined
Feb 13, 2016
Messages
44,368
Likes
234,393
Location
Seattle Area
Thanks @amirm this looks just right...unfortunately I'm strictly a linux/unix/mac user. Interestingly, there may be some seismic analysis code that does all of this, which I'm looking into.
If you post a link to the files, a member can run the tool for you.
 

jokan

Addicted to Fun and Learning
Joined
Feb 19, 2019
Messages
532
Likes
563
Location
Tokyo, Japan.
I keep reading that MQA is a lossy format, is this true?
I'm not sold on the idea of a lossy format, I still prefer reel-reel if everything is properly maintained and recorded.
FLAC in 192/32bit seems to be a good compromise and is readily available.
What am I missing? I know I'm missing something. Japan doesn't have Tidal, we have Spotify but it's hardly high resolution. Even paid Spotify accounts which is why I no longer pay for spotify. Japan is very protective of its own recording industry but that means JPop. Which I don't have any interest in. Nothing but Auto-Tune and 120hz bass notes and manufactured bands. Independent music labels don't survive and there isn't an infrastructure for talented individuals to offer their own music directly to the consumer. The record labels are too greedy to let that happen.
But my original question. Is MQA a leap forward, or is it a gimmick like the Hi-Res format and sticker that Sony licensed out to companies and later sold to make a substantial amount of money?
We also don't have Qobuz.
 
OP
Tokyo_John

Tokyo_John

Active Member
Forum Donor
Joined
Mar 6, 2021
Messages
214
Likes
289
If you post a link to the files, a member can run the tool for you.

Great idea...I uploaded to my Google Drive. To avoid trouble, it is best if any willing takers would send me a PM (sorry for the extra hassle).
 

Zensō

Major Contributor
Joined
Mar 11, 2020
Messages
2,740
Likes
6,740
Location
California
I keep reading that MQA is a lossy format, is this true?
I'm not sold on the idea of a lossy format, I still prefer reel-reel if everything is properly maintained and recorded.
FLAC in 192/32bit seems to be a good compromise and is readily available.
What am I missing? I know I'm missing something. Japan doesn't have Tidal, we have Spotify but it's hardly high resolution. Even paid Spotify accounts which is why I no longer pay for spotify. Japan is very protective of its own recording industry but that means JPop. Which I don't have any interest in. Nothing but Auto-Tune and 120hz bass notes and manufactured bands. Independent music labels don't survive and there isn't an infrastructure for talented individuals to offer their own music directly to the consumer. The record labels are too greedy to let that happen.
But my original question. Is MQA a leap forward, or is it a gimmick like the Hi-Res format and sticker that Sony licensed out to companies and later sold to make a substantial amount of money?
We also don't have Qobuz.
https://www.audiosciencereview.com/...i-published-music-on-tidal-to-test-mqa.22549/
 
OP
Tokyo_John

Tokyo_John

Active Member
Forum Donor
Joined
Mar 6, 2021
Messages
214
Likes
289
...Japan doesn't have Tidal, we have Spotify but it's hardly high resolution. Even paid Spotify accounts which is why I no longer pay for spotify. Japan is very protective of its own recording industry but that means JPop. Which I don't have any interest in. Nothing but Auto-Tune and 120hz bass notes and manufactured bands. Independent music labels don't survive and there isn't an infrastructure for talented individuals to offer their own music directly to the consumer. The record labels are too greedy to let that happen...

I'm based in Japan, and I've been able to subscribe to Tidal and Qobuz in the past year without issues. Give it a try, they have trial periods where you can hear for yourself, and make measurements too.
 

Habu2u

New Member
Joined
Dec 3, 2019
Messages
1
Likes
6
Location
Schertz, TX
Back in Spring 2017 the folks behind MQA stated, "MQA is a new digital music format, designed to deliver high-resolution sound at a much reduced bitrate, typically close to that of regular CD. The name Master Quality Authenticated gives a hint to its second selling point—authenticated, an assurance that what you are hearing is a digital clone of the original encoded file, and not one that has been degraded en route.

During my MQA research I learned of GoldenSound’s “MQA Review” video. After watching it, I realized it’s one of the more balanced, informative, and invective-free discussions of MQA I’ve seen. As of this very moment, that video has 156,538 views (since 15 April 2021), by “GoldenSound’s” channel, which started just 3 months ago.

“MQA Review” -

He time stamps the video so you can jump around.
In our hobby the goal is to put together neutral sounding gear for pure music.

Neutral sounding gear for a “perfectionist” hobby. (You do know the enemy of perfection, right? Answer: Good Enough! To me, MQA is not even, good enough!)

With this in mind, I rewatched GoldenSound’s MQA Review video. I realized time and again MQA “messed up” the audio signal of the music played by getting away from “neutral”. Even to the point where GoldenSound's “fantastic sounding” iFi iDSD Diablo (a full MQA decoder) with it’s GTO filter (27:44 in the video) displayed the same ultrasonic attenuation response curve as the MQA unfold. (Attachment’s #1 & #2)

The ideal ultrasonic attenuation curve is displayed @ 15:33 in the video (Attachment #3). The MQA filter displays extremely poor attenuation plus strange patterns in the ultrasonic range indicating ultrasonic content is added.

Attachment #4 and GoldenSound’s explanation (better than mine) is found at 16:00 in the video. Watch it.

I merely share some of the many examples GoldenSound provides in his analysis video.

All away from “neutral sounding” and definitely away from MQA’s "assurance that what you are hearing is a digital clone of the original encoded file, and not one that has been degraded en route.”

All the best
 

Attachments

  • 1. iFi GTO filter ultrasonic response.jpg
    1. iFi GTO filter ultrasonic response.jpg
    153.4 KB · Views: 154
  • 2. MQA's Unfold.jpg
    2. MQA's Unfold.jpg
    213.2 KB · Views: 154
  • 3. Ultrasonic attenuation White-Noise test.jpg
    3. Ultrasonic attenuation White-Noise test.jpg
    160 KB · Views: 154
  • 4. Ultrasonic attentuation added.jpg
    4. Ultrasonic attentuation added.jpg
    117.7 KB · Views: 141

SIY

Grand Contributor
Technical Expert
Joined
Apr 6, 2018
Messages
10,384
Likes
24,749
Location
Alfred, NY
I keep reading that MQA is a lossy format, is this true?
I'm not sold on the idea of a lossy format, I still prefer reel-reel if everything is properly maintained and recorded.
FLAC in 192/32bit seems to be a good compromise and is readily available.
What am I missing? I know I'm missing something. Japan doesn't have Tidal, we have Spotify but it's hardly high resolution. Even paid Spotify accounts which is why I no longer pay for spotify. Japan is very protective of its own recording industry but that means JPop. Which I don't have any interest in. Nothing but Auto-Tune and 120hz bass notes and manufactured bands. Independent music labels don't survive and there isn't an infrastructure for talented individuals to offer their own music directly to the consumer. The record labels are too greedy to let that happen.
But my original question. Is MQA a leap forward, or is it a gimmick like the Hi-Res format and sticker that Sony licensed out to companies and later sold to make a substantial amount of money?
We also don't have Qobuz.
Reel to reel is a highly lossy format.
 
OP
Tokyo_John

Tokyo_John

Active Member
Forum Donor
Joined
Mar 6, 2021
Messages
214
Likes
289
...During my MQA research I learned of GoldenSound’s “MQA Review” video. After watching it, I realized it’s one of the more balanced, informative, and invective-free discussions of MQA I’ve seen...

...All away from “neutral sounding” and definitely away from MQA’s "assurance that what you are hearing is a digital clone of the original encoded file, and not one that has been degraded en route.”

Thanks, I've seen much of this material in the past. If these claims about MQA are true, then it may constitute evidence of wire and mail fraud, a criminal offense. Tidal and others would be put at legal risk. I'm sure they have lawyers to tell them so.

Still, one thing that is not addressed in this MQA review is how MQA processing affects the sound that actually comes out of my speaker. Any artifacts or flaws in digital MQA that have no influence on the analogue result are interesting, but practically irrelevant. Of course, if MQA does not add any improvement above CD quality, then it would also imply that the marketing claims by MQA are fraudulent in an empirically verifiable way.

I'm still working on analyzing my recordings, and developing some numerical tools to emphasize subtle differences between CD and MQA playback. I've been busy, so it might be a little while longer. I've discovered open source SoX and installed the unix tools, which is going to speed up my work flow quite a lot.
 
OP
Tokyo_John

Tokyo_John

Active Member
Forum Donor
Joined
Mar 6, 2021
Messages
214
Likes
289
MQA was revealed as a scam back in 2014. I thought that 'science' was settled long ago!

Science is never settled. If it were settled, then it wouldn't be science.

Also, the term "scam" is not very scientific. What exactly do you mean when you say "scam?" What is the empirically testable condition for identifying something as a "scam?" How does one measure the "scam-ness" of a thing?
 

Jimbob54

Grand Contributor
Forum Donor
Joined
Oct 25, 2019
Messages
11,066
Likes
14,697
Science is never settled. If it were settled, then it wouldn't be science.

Also, the term "scam" is not very scientific. What exactly do you mean when you say "scam?" What is the empirically testable condition for identifying something as a "scam?" How does one measure the "scam-ness" of a thing?

Dogecoin has a scam factor of 5 (exactly 2/3 of a Madoff in the audible band)

MQA is a solid 3. But if you apply the right filters and pay the Bob tax, it can be as low as 1.2
 

lamode

Member
Joined
Apr 5, 2019
Messages
19
Likes
30
Also, the term "scam" is not very scientific. What exactly do you mean when you say "scam?" What is the empirically testable condition for identifying something as a "scam?" How does one measure the "scam-ness" of a thing?

The definition of the word scam is there in the dictionary for all to see. MQA qualifies as a scam because there is clear evidence that claims made by the company are false.
 
OP
Tokyo_John

Tokyo_John

Active Member
Forum Donor
Joined
Mar 6, 2021
Messages
214
Likes
289
The definition of the word scam is there in the dictionary for all to see. MQA qualifies as a scam because there is clear evidence that claims made by the company are false.

Do you have something useful to add here? Or are you just trying to blow up a thread?

I don’t care about the veracity of MQA claims, that is a legal question. Start your own thread if you’d like to explore that topic.

I just want to know what is different about what comes out of my speakers as I playback different types of audio files, and how to best measure that objectively.
 
Top Bottom