Source: Streaming Services - Detailed Measurement Review

[nawfal07]

Hi @amirm / Reviewers / Everyone,

Hopefully this thread is worth having, if not I fully understand and it can be deleted.

I'm wondering, besides from a blind test, can audio sources i.e. streaming services (Tidal, Apple Music, Qobuz, Amazon Music, Spotify, etc or even radio apps like TuneIn, etc) be scientifically measured for their sound quality?

Even though I am a scientist but I'm from a different field, and home theatre/audio is a very serious hobby of mine, I would rather have the experts in this field do the tests. All the usual test if possible (I have no idea if it's even valid for discussion), or maybe a null test, or anything you can think of.

Using the same audio setup and tracks as reference for each streaming service. It can be one service provider in one review, and then the other a few weeks later, and a final review of a comparison tests (null test, etc).

I know @amirm and other reviewers are very busy, this is not a request but a thought/suggestion and only if you're free and really feel like doing it. And of course if it's possible and making sense that is.

If it's not possible, then I'm hoping this thread can be a discussion on ideas on how to scientifically measure the sound quality of the audio sources instead of just audio equipment, or if it's a waste of time then why.

Apologies if this thinking is not sciencey. Be gentle!

Cheers.
Iswardi

A little bit of my backstory, although this thread should not be about my personal reason but about how we can expand the scientific measurements to source audio.

I have subscriptions to both Apple Music and Tidal.

I have Apple Music because it has the largest library (all lossless, at least my songs), but I can't get lossless audio (and as a bonus, hi-res) without tethering to my DAC and restricts my mobile phone from being mobile (until I bought an Apple TV 4K recently but I don't want to turn on the TV just to listen to music, sold it anyway), and of course also that I'm in the Apple ecosystem.

I have Tidal because as above I would like to get lossless audio, and googling seems that the majority are saying, all track specification being equal, the sound quality is better (not scientific at all, I know, hence this thread, my ears also say it's better but I need numbers to back that up). The only problem is that they're a bit short on the library, out of thousands of my songs in Apple Music, few hundreds of them are not available in Tidal (mostly local songs from my country). Tidal is also priced almost double from Apple Music (triple if subscribed via Apple).

Now that I can stream Apple Music in lossless easily without tethering my phone, I'm debating if I should keep both Apple Music (for library) and Tidal (for audio quality, as mentioned it does sound better to my ears but that is far from scientific so would like to see the numbers), or cancel Tidal (if measurements show no advantage) and save some money.

 

[monx]

Hi, the problem is similar to guessing how a human would rate audio equipment based on measurement.
Of course, the ASR community is famously concerned with this interesting problem and has demonstrated success.

So it could also be done for streaming audio quality: you will need to predict perceptual effects based on measurements, and figure out which type of error is most audible, and thus relevant to preference scores.

But...this is exactly how perceptual audio compression has been advanced for decades. And the problem here is that you would have to somehow come up with a perceptual error prediction that matches or exceeds all the perceptual modeling from Fraunhofer (and everyone who came afterward) that went into developing the compression technology in the first place...thus, for my money, the best tool we have is blind testing.

 

[roladyzator]

There are ways to dump audio from Tidal, Deezer, Qobuz and possibly more. Comparing such files in Foobar2000's ABX comparator would be an easy way to test for preference.

For example, in Tidal when you use HiFi mode (16 bit, 44.1 kHz Flac) to get a file that was encoded using MQA, you get a less than CD quality file, because some MQA Metadata is being written to the least significant 3 bits. Thus, signal noise ratio of the medium is lowered form 96 dB to 78 dB. This makes the Flac version to he bigger (it's hard to compress random noise) and the MP4 version is an encode of that noisy version, not the original CD quality file.

It's very obvious to see when opening such file in an analyzer. See the famous video on MQA by GoldenSound for examples. But can you reliably hear it? And if yes, how much of a degradation is it really? Not much probably since most folks are happy with Tidal's sound quality.

And if you think software side (the client app) may be influencing sound further, in Windows you can use WASAPI loopback to record the output digitally and then compare the results.

The likely result is that same file from same mastering in lossless mode will be indistinguishable between various streaming platforms.

So again, use an audio file analyzer to check for differences. When there is a difference,do ABX tests for preference.