I found an abx test on a website

[ILikeMusicToo]

Hi everyone,
I have wondered, as I'm sure many of you did, if my system could play music well enough to let me distinguish between lossy and lossless streaming music. I'd tried comparing my FLAC CD rips to the same album streaming on Spotify (I don't have Tidal/Qobuz) and honestly couldn't hear the difference, but wasn't sure because I couldn't switch instantly between the sources.

I found this website (http://abx.digitalfeed.net/#) that has an ABX test with five music samples, played five times each, covering a range of music types. Sadly I couldn't tell them apart, scoring 52% and 48%. I intend to try different speakers, tweaks, speaker positions to see if I can find some sonic signature that stands out.

I just noticed that if you dig around they have other tests: like can you identify the "X" when one candidate is lossless and the other is a 96k encoded mp3! I mean...surely...right?

If someone scores much higher than 50% PLEASE reply here and share details of your system!

I'm running an Outlaw RR2160 amp, Mission 774 tower speakers (they were about $US 1200 twenty years ago), generic monoprice cable. I used a laptop as my source, once with USB into the Outlaw using its internal DAC, and a second test using my Dragonfly Black to feed the Outlaw's aux socket. Both times I played the music around 75 db, peaking at 80db.

 

[Tks]

Amir and some others have ABX'd lossless vs 320 MP3. Though he's a trained listener (some Harman program that takes a while to actually be qualified or something along those lines).

There are supposedly giveaways.

Also, I think these kinds of tests were done using headphones and somewhat modern high performance DAC/AMP. I doubt anyone was using gear as old as yours (almost certainly to be quite a bit off of the sort of performance you would need for ABX'ing something like 320kbps MP3's vs FLACs).

Personally, I can't ABX 256kbps vs FLAC when I tested myself a while ago. Anything under that, it depends on the day really. Sometimes I manage to score 90%+ accuracy over the span of 18 tests. Sometimes I fail utterly even with 128kbps (I'm not trained btw).

 

[ILikeMusicToo]

Amir and some others have ABX'd lossless vs 320 MP3. Though he's a trained listener (some Harman program that takes a while to actually be qualified or something along those lines).

There are supposedly giveaways.

Also, I think these kinds of tests were done using headphones and somewhat modern high performance DAC/AMP. I doubt anyone was using gear as old as yours (almost certainly to be quite a bit off of the sort of performance you would need for ABX'ing something like 320kbps MP3's vs FLACs).

Personally, I can't ABX 256kbps vs FLAC when I tested myself a while ago. Anything under that, it depends on the day really. Sometimes I manage to score 90%+ accuracy over the span of 18 tests. Sometimes I fail utterly even with 128kbps (I'm not trained btw).

That's really interesting, thank you for the reply. At what point, dollar-wise, would you say a well set up system (speakers in the right place etc) would be capable of revealing an appreciable difference between lossless and lossy? My system is about $US 2,000 and is in a dedicated room. I need to add room treatment at some point, but the speakers are a generous 3-4 feet from each wall. I also have a more recent set of speakers, Elac Debut 6.2, but as good as they are they probably wouldn't be revealing enough either, right?

Do you think headphones are more likely to reveal these differences? I'm using a set of Thinksound ON2, which were about $200 and bought new 2-3 years ago. I tried the test there too and couldn't hear a difference, but I'm going to give it another try

I'd like to see a video or article about what level of equipment is required to reveal each higher step in audio quality, and perhaps an explanation as to why that is the case.

 

[Rntlee]

Don’t beat yourself up too much, the majority of people can’t discern 320 from lossless. Enjoy your music!

Archimago ran an online blind test a few years ago and I think he got about 150 people to participate. Not the most scientific test, but it was done more as a fun exercise you could try. Those who participated used headphones and various stereo kit from $500 on up to $20000+. An interesting result from the test was that a higher percentage of those who had systems worth <$500 actually preferred the lossless files vs. those who had systems worth $6000 and up. Those with the high priced gear were actually more likely to prefer the lossy file over the lossless!




.

 

[Jimbob54]

We have been conditioned to think the higher the sample rate/ cost of equipment the better the ability to appreciate the source. Bigger numbers is always better , right?

The longer you spend here, the more you will appreciate this isnt (necessarily) true. For the bulk of us mere mortals , lossy encoding and audio electronics to replay them with high fidelity have been a solved issue for years. Dont take the fact you cant tell the difference better than guessing as a critique of your hearing but a compliment to the fine engineering and science that has got us to this point.

Focus any efforts to improve your listening experience on your speakers/ headphones and how to best EQ them to your room/ ears , not on the stuff that comes before them.

 

[Pio2001]

The ability to hear difference between lossless and lossy is not better with high end equipment. High end speakers give you neutral frequency response. Lossy encoding too.
Distortion, signal to noise ratio, linearity are also completely independant from lossy encoding. Therefore better amplifiers or DACs won't help.

Lossy encoding introduces quantization noise where it should be masked. Cheap speakers with awful frequency response may reveal this. The noise introduced by lossy encoding is supposed to be masked with neutral frequency response. Cheap speakers, like the ones included in your laptop may unmask this noise by not reproducing the masking frequencies properly.

Lossy encoding also has problems with transients. In order to hear time spearing on sharp transients, headphones is a must. Transients are completely smeared by the acoustics of our rooms.

Last, the difference between lossless and lossy depends a lot on the musical sample. Some music can be undistinguishable from lossless, while another will fail to be encoded properly.

The site that you link proposes 320 kbps AAC files. Although it is a complete nonsense to have a fixed bitrate with AAC, that is designed for variable bitrate, 320 kbps is enough in 99.9% of situations.
If it is a variable bitrate with an average target of 320 kbps, it is overkill. In my experience with ABX tests, I'd guess that AAC should be transparent in 50% of the cases at 128 kbps, 90 % of the cases at 160 kbps, and 99 % at 200 kbps.
By "50 % of the cases", I mean that for 10 tracks, 5 of them will be undistinguishable from the original, and 5 others will have an audible loss of quality.

If you want to hear the difference, here are some examples encoded at various quality levels : https://www.homecinema-fr.com/forum...3-et-vorbis-exemples-a-ecouter-t30091984.html
You can download all of them with the link in the first message, or pick the ones you'd like to try in the second message.
Among the killer samples, "drone" in mp3 V2 (229 kbps) can easily be distinguished from the original with headphones. There is an audible glitch in the first second in the mp3 (like a scratch on a vinyl record), that is not there in the original.

Edit : according to my messages posted two years ago, I managed to ABX successfully (8/8) all the lossy versions of the killer sample "Second Bad Vilbel", and all the lossy versions of the "Organectomy" sample. Even the Vorbis Q6.78 (217 kbps).
But I was unable to hear the difference between the lossless version and the Vorbis Q4 version (84 kbps !) of Michel Corette.

 

[Tks]

That's really interesting, thank you for the reply. At what point, dollar-wise, would you say a well set up system (speakers in the right place etc) would be capable of revealing an appreciable difference between lossless and lossy? My system is about $US 2,000 and is in a dedicated room. I need to add room treatment at some point, but the speakers are a generous 3-4 feet from each wall. I also have a more recent set of speakers, Elac Debut 6.2, but as good as they are they probably wouldn't be revealing enough either, right?

Do you think headphones are more likely to reveal these differences? I'm using a set of Thinksound ON2, which were about $200 and bought new 2-3 years ago. I tried the test there too and couldn't hear a difference, but I'm going to give it another try

I'd like to see a video or article about what level of equipment is required to reveal each higher step in audio quality, and perhaps an explanation as to why that is the case.

Speakers? I haven't a clue tbh. I think the most important portion of when listening, is just to have a quiet environment (or just loud volume to combat the environment). Your speakers are fine (so as long as you're not bottoming out the woofer and distorting it), and the differences wouldn't reveal themselves due to each speakers particular frequency response though.

If you're really keen on the process someone has to go through in order to be able to ABX lossy/lossless, maybe @amirm can chime in on the specifics of what sort of training, or signs you need to listen to, to catch it.

But I wouldn't assume it's tied much at all to what exact headphone. The only thing to me in headphones that matters is the particular frequency response (how the headphone sounds out of the box), and distortion profile (how much SPL are the drivers capable of, before distortion starts becoming really an audible problem). The cost of a headphone is rarely tied to these things. You could for instance have a relatively higher distortion headphone (at least in the bass region) like the HD800, with a frequency response that only an idiot would have greenlit (as Sennheiser somehow did). But the headphone's construction is really good, so the cost is mostly for that reason.

Amps and Dacs would be something that comes after, but any modern device already bottlenecks the listening device (speakers and headphones will never be as transparent as something like a 24-bit DAC with SNR over 120dB+ or SINAD of 120dB for instance).

All "higher" quality audio means personally to me, is a device that maintains fidelity of a signal (meaning the audio I send out of my source, is as close to the same as the audio eventually played back). So the lower quantities of artifacts (unwanted noise and distortion in the signal) the "higher" the audio quality is. And of course also the frequency response as well needs to be maintained. DACs and AMPs of the modern day have already achieved audible fidelity for physically connected devices. The listening devices themselves (speakers and headphones) are all that's really left to get better (though with some of the floor-standing Revel speakers, I'd argue we're pretty much finished). But since there's a bit more than "distortion and frequency response" to a speaker, there's a bit more to do on that front (see speaker measurements yourself to see what else can be more improved). But for speakers, unlike headphones you also have the added headache of room configuration that you may possibly never "perfect" so to speak.

 

[ILikeMusicToo]

Great responses everyone, thank you I've enjoyed my Spotify account with its 320k mbps format though always felt a bit guilty about it, but no more!

I'm taking delivery of my first subwoofer today, the well reviewed Dayton Sub 1200. I look forward to hearing what it does to the sound of my room, and I'm glad I picked the Outlaw amp because it has two dedicated sub-out ports and bass management features.

@Pio2001 Thank you for the link, I'll check it out. Actually the site I linked has other tests too, not just the 320k vs lossless. I started trying the "lossless vs 96k mp3" one and definitely heard extra noises on the 96k in my headphones, but didn't have a chance to complete all the tests (I have two young kids and something is always on fire!)

 

[Mnyb]

The ability to hear difference between lossless and lossy is not better with high end equipment. High end speakers give you neutral frequency response. Lossy encoding too.
Distortion, signal to noise ratio, linearity are also completely independant from lossy encoding. Therefore better amplifiers or DACs won't help.

Lossy encoding introduces quantization noise where it should be masked. Cheap speakers with awful frequency response may reveal this. The noise introduced by lossy encoding is supposed to be masked with neutral frequency response. Cheap speakers, like the ones included in your laptop may unmask this noise by not reproducing the masking frequencies properly.

Lossy encoding also has problems with transients. In order to hear time spearing on sharp transients, headphones is a must. Transients are completely smeared by the acoustics of our rooms.

Last, the difference between lossless and lossy depends a lot on the musical sample. Some music can be undistinguishable from lossless, while another will fail to be encoded properly.

The site that you link proposes 320 kbps AAC files. Although it is a complete nonsense to have a fixed bitrate with AAC, that is designed for variable bitrate, 320 kbps is enough in 99.9% of situations.
If it is a variable bitrate with an average target of 320 kbps, it is overkill. In my experience with ABX tests, I'd guess that AAC should be transparent in 50% of the cases at 128 kbps, 90 % of the cases at 160 kbps, and 99 % at 200 kbps.
By "50 % of the cases", I mean that for 10 tracks, 5 of them will be undistinguishable from the original, and 5 others will have an audible loss of quality.

If you want to hear the difference, here are some examples encoded at various quality levels : https://www.homecinema-fr.com/forum...3-et-vorbis-exemples-a-ecouter-t30091984.html
You can download all of them with the link in the first message, or pick the ones you'd like to try in the second message.
Among the killer samples, "drone" in mp3 V2 (229 kbps) can easily be distinguished from the original with headphones. There is an audible glitch in the first second in the mp3 (like a scratch on a vinyl record), that is not there in the original.

Edit : according to my messages posted two years ago, I managed to ABX successfully (8/8) all the lossy versions of the killer sample "Second Bad Vilbel", and all the lossy versions of the "Organectomy" sample. Even the Vorbis Q6.78 (217 kbps).
But I was unable to hear the difference between the lossless version and the Vorbis Q4 version (84 kbps !) of Michel Corette.

Is it not so that also somewhat bad hearing can help you with this in some odd cases ? If you don't hear the things that's supposed to mask it would not work for you .

Another give away can be very hot mastered dance or electronic music decoder clipping can be an issue if the decoder is not properly implemented .

I have some old squeezeboxes the mp3 decoder in them is some old integer based library as the puny cpu does not do float.
I had some example track where this could be heard, but not when i moved the decoding to the server .

 

[raistlin65]

That's really interesting, thank you for the reply. At what point, dollar-wise, would you say a well set up system (speakers in the right place etc) would be capable of revealing an appreciable difference between lossless and lossy?

I think you might want to consider another question. If one is immersed in the aesthetic experience of the music, will the difference between high bit rate lossy and lossless ever impact the listening experience?

The answer for me is no. I suspect others may feel the same.

Think of it like this. If I take two slices of pizza and add 5 grains of salt to one and 10 to the other, perhaps I would be able to pick out the difference when comparing them directly in the same eating session. But if eating either of those slices of pizza a week later, then I would enjoy each equally.

That being said, for some people, this hobby is constantly about testing and comparing. In other words, if you are always experimenting with comparing slices of pizza with slightly different amounts of spice, then it matters a lot more. But for many of us, we just want to eat our pizza

 

[Pio2001]

Is it not so that also somewhat bad hearing can help you with this in some odd cases ? If you don't hear the things that's supposed to mask it would not work for you .

Being able to hear above 16 kHz should help, as lossless audio is often lowpassed around this frequency.

Another give away can be very hot mastered dance or electronic music decoder clipping can be an issue if the decoder is not properly implemented .

In the examples that I linked, decoder clipping is removed with replaygain, and the original files are adjusted to match the replaygain correction. All files are processed in 24 bits in order to avoid quality loss during these adjustments.
This way we can compare the quality of mp3 and Vorbis as they should be used : with no decoder clipping.

 

[Wes]

Given the price of storage these days, it's really a non-issue.

I expect the type of music would matter, tho maybe not as much as confirmation bias. But my rule is simple: no lossy stuff if I can avoid it. Makes life simple and doesn't cost me anything.

 

[Mnyb]

Given the price of storage these days, it's really a non-issue.

I expect the type of music would matter, tho maybe not as much as confirmation bias. But my rule is simple: no lossy stuff if I can avoid it. Makes life simple and doesn't cost me anything.

yes use lossles for your own archival of files , lossy is for streaming these days imho

 

[Pio2001]

I always backup all my files.
I have 741 hours of music in my computer.
In lossless format, if I'm not mistaken, it would take about 286 GB of disc space (at 900 kbps).

A backup consists in copying all the data to an external HD, and re-reading all this data in order to check that the backup works. That would be more than 500 GB of data transfer everytime I backup the files.
And that's only for audio. I also backup the whole system and all other files. For that purpose, I clone the disc partitions of the computer in a disc image.
During this operation, the computer remains unusable for hours. That's not very convenient. With lossless audio, I think that a complete backup of my system would take more than 10 hours (depending on the performance of the external HDD).
That's why I still use lossy compression, except for important files (carefully made vinyl copies, music I care about and for which I have no CD...).

I use Vorbis -q6 for lossless sources (one quality level above -q5, that sounds tranparent for me), and Vorbis -q7 for lossy sources (to avoid cumulating losses).

 

[paulraphael]

If you want to make an exercise of it, you can try training yourself to hear the differences the way they would for formal listener training. Make several compressed versions, starting at something ridiculously compressed (like 60 kb/s) and then higher, like 128, 160, 200, 256, 320. Start your ABXing at the low end. You'll almost certainly hear the difference. Spend some time toggling between A and B to get familiar with where the artifacts hide. This will make it easier to hear them on the next round with the higher quality files.

Repeat and see how high you can go. And notice how this doesn't in any way resemble listening to music for pleasure!

 

[Staki]

Great responses everyone, thank you I've enjoyed my Spotify account with its 320k mbps format though always felt a bit guilty about it, but no more!

I'm taking delivery of my first subwoofer today, the well reviewed Dayton Sub 1200. I look forward to hearing what it does to the sound of my room, and I'm glad I picked the Outlaw amp because it has two dedicated sub-out ports and bass management features.

@Pio2001 Thank you for the link, I'll check it out. Actually the site I linked has other tests too, not just the 320k vs lossless. I started trying the "lossless vs 96k mp3" one and definitely heard extra noises on the 96k in my headphones, but didn't have a chance to complete all the tests (I have two young kids and something is always on fire!)

There are reasons not to use Spotify even if sound quality is not among them. Bandcamp is by far the most artist-friendly streaming service. Among others, Qobuz is apparently leading the pack.

 

[j_j]

Listener training is always a big issue. When MP3 came out people thought 64kb/s was good, until they learned the signature errors, then it was 128kb/s, and so on up.

With specialized material, one can almost always distinguish due to algorithm limits, but only when you know what to listen for.

 

[Chrispy]

You can also do your own abx with foobar2000 and the abx component....

 

[hyperplanar]

FWIW I did this test a few years ago (believe it was the one for Spotify HQ, so 320 Kbps Vorbis encoding) and was able to distinguish the two:


Obviously the differences were extremely subtle and I have no qualms continuing to use Spotify to this day. It’s nothing you would possibly notice when you’re actually listening to the music, instead of being anal and looking for microscopic deficiencies. I’m also not sure whether I could pass this test again today as my ears are a few years older. I posted this on Reddit a few years ago, and this is what I had to say about the differences:

The most immediately noticeable difference is in the "airiness" frequencies somewhere above 10 kHz where the lossy version is often brick wall low pass filtered, resulting in a loss of perceived airiness. The filter frequency tends to be set lower with decreasing bitrate so it's going to be much more audible on 128 Kbps than 320 Kbps.
Another difference is in the decay of instrument sounds and reverb tails. Lossy encoders tend to throw away information about quiet sections such as these and make them sound less smooth/natural.
The last difference I noticed were in certain transients, such as some percussive hits where in the lossy encoded versions they would sound more smeared/smoothened over time compared to the lossless original.

 

[Haint]

Hi everyone,
I have wondered, as I'm sure many of you did, if my system could play music well enough to let me distinguish between lossy and lossless streaming music. I'd tried comparing my FLAC CD rips to the same album streaming on Spotify (I don't have Tidal/Qobuz) and honestly couldn't hear the difference, but wasn't sure because I couldn't switch instantly between the sources.

I found this website (http://abx.digitalfeed.net/#) that has an ABX test with five music samples, played five times each, covering a range of music types. Sadly I couldn't tell them apart, scoring 52% and 48%. I intend to try different speakers, tweaks, speaker positions to see if I can find some sonic signature that stands out.

I just noticed that if you dig around they have other tests: like can you identify the "X" when one candidate is lossless and the other is a 96k encoded mp3! I mean...surely...right?

If someone scores much higher than 50% PLEASE reply here and share details of your system!

I'm running an Outlaw RR2160 amp, Mission 774 tower speakers (they were about $US 1200 twenty years ago), generic monoprice cable. I used a laptop as my source, once with USB into the Outlaw using its internal DAC, and a second test using my Dragonfly Black to feed the Outlaw's aux socket. Both times I played the music around 75 db, peaking at 80db.

I could somewhat predictably pick out very minor very high frequency differences in some of the tracks with Harman target EQ'ed triple flange Etymotic ER2XR's jammed next to my ear drum. I don't expect I would ever be able hear it with traditional headphones or speakers due to the comparatively much higher noise floors and reflections. With this test, you are talking about utterly meaningless differences here, like a single lost hair on a dog.