Objectivists vs. Subjectivists - Who's right?

[voodooless]

I am actually a good example for this, haha.

To be perfectly honest, after performing countless ABX tests over a span of ~13 hours, I do believe that my original, untested observation, regarding the difference between lossless and compressed formats, may have been slightly exaggerated, yet not completely unfounded.

The results were pretty much consistent with that.

Actually not quite. You introduced an additional variable: lossy compression. If you were to prove your original claim, you should have compared to a downsamples, noise-shaped dithered 16/44.1 version of the tracks without lossy compression added.

But great progress has been made: from "Absolutely, unequivocally incredible." to (to paraphrase) "sometimes I can hear a difference if I try really hard". To me, that sounds like it was much more than just a slight exaggeration Kudos nevertheless!

 

[Sgt. Ear Ache]

Actually not quite. You introduced an additional variable: lossy compression. If you were to prove your original claim, you should have compared to a downsamples, noise-shaped dithered 16/44.1 version of the tracks without lossy compression added.

But great progress has been made: from "Absolutely, unequivocally incredible." to (to paraphrase) "sometimes I can hear a difference if I try really hard". To me, that sounds like it was much more than just a slight exaggeration Kudos nevertheless!

exactly. The not-so-subtle shifting of goal posts must be noted!

 

[JJB70]

I think codecs and sample frequency/bit depth are an example of where objective evaluation doesn't necessarily mean measurably better.

 

[Sgt. Ear Ache]

Both for Apple and Amazon you pay the same amount, others will follow soon.

Yeah I totally agree that the debate between lossy and lossless is really just an intellectual argument at this point. I always go for lossless if I can, but by no means do I believe i can actually hear a difference between 320kb mp3 and lossless under any sort of normal listening circumstances.

 

[audiofilet]

It's true, my original expectation was blown out of proportion, but I was able to prove that specific EQ profiles, at least in this case, were able to improve my ability to pick correctly.

Then again, after having just completed another set of ABX runs with AAC, I'm more inclined to suggest that this gradually increasing success rate was probably just a result of familiarity over time.

It's extremely difficult to perform these tests in a controlled environment.

 

[BDWoody]

It's true, my original expectation was blown out of proportion

Would you have ever figured that out without the AB/X test?

So, your bias was clearly needing to be dealt with so you could focus on what you were actually hearing.

You are showing yourself why we ask for more than anecdote.

Have you tried this with hi-res vs Redbook as suggested? That IS what you initially claimed after all. Lossy codec's are their own deal.

 

[Killingbeans]

It is my understanding that at the core of it all, the issue lies with one group choosing to approach, interpret and assert claims about audio that the other considers unscientific.

It doesn't matter whether it's scientific or not. It's the reliability that's important.

It's the difference between confirmation and verification. If you want to minimize the risk of reaching a false conclusion, you have look at the methods that has so far been the basis of a known conclusion, and then attack it from a new angle where you eliminate as many factors of uncertainty as possible. Simply repeating the old methods does not work. The error scales proportionally with the sample size (EDIT: ?).

I guess the real question is, how far should we allow graphs, measurements and hard science to dictate the listening experience and how much merit should be attributed to the subjective perception of the individual i.e audiophile?

Zero length and very little.

Hard science gives you the tools to reach a goal without shooting in the dark. Nobody is dictating what that goal should be as an individual.

Subjective perception is crucial for the enjoyment of music. It's important for you and me as an individual, but their value as data points is a crapshoot.

I wouldn't say that subjectivists don't have evidence of any kind, but rather that the kind of evidence they do offer is simply not accepted by objectivists, namely their own, personal interpretation based on years of experience.

Empirical evidence is empirical evidence. You can't substitute it for a popular vote or a belief systems based on personal experience.

You can have had the same experience your whole life, but if it made you reach the same faulty conclusion every time, what good does it do?

 

[JSmith]

my original expectation was blown out of proportion

What you should also try is picking which you prefer, as in not knowing which you are listening to and picking which sounds superior and which sounds inferior. The results can be surprising...

You may find this resource interesting;

ABX High Fidelity Test list

A set of browser-based ABX tests comparing lossless and lossy music samples

abx.digitalfeed.net

This was from 2013;

High Bitrate MP3 Internet Blind Test: Part 2 - RESULTS

A blog for audiophiles about more objective topics. Measurements of audio gear. Reasonable, realistic, no snakeoil assessment of sound, and equipment.

archimago.blogspot.com

Remember that Set B was the MP3, yet for those who picked A or B, most thought A sounded inferior!

JSmith

 

[PierreV]

@audiofilet truly deserves our respect and consideration I think.

He first told us how impressed he was with some Hi-Res downloads.
Then, facing some opposition, he did his best to test his initial impression extensively.

I find this admirable.

How many of us simply have strong opinions they have never tested and will hold on to them like it was some kind of holy truth?

 

[Robin L]

 

[audio2design]

Agreed, it's anecdotal evidence.

For example, an audio engineer with 40 years experience is convinced he can detect an audible difference between FLAC and AAC, yet is unable to reproduce or describe this feat scientifically.
Available tests neither confirm nor refute his claim.

How should a scientifically literate person weigh this?

The scientifically literate would assume that with 40 year experience, perhaps the "engineer" is experiencing mental decay, or they would accept the fact they can't reproduce their claim to mean it is all in their head.

Of course being the internet, the scientifically literate would be going "bullshit he is an engineer", not to mention, if truly scientifically literate, they would know that an "audio engineer" only has engineer in his title, sort of like the ole "Microsoft Certified Systems Engineer", but whose job has sweet diddly to do with the actual professional practice of engineering.

Of course, also knowing it is the internet, the scientifically literate would figure the anecdote communicated was second hand and would assume the person communicating it probably does not know that an "audio engineer" is not a real "engineer" in the typical meaning of the profession.

 

[audio2design]

Even when a subjectivist buys a piece of gear he is satisified with, you can still ask a question how big of a portion of money was wasted bc he didn't do a proper research and informed himself and probably could've get the same result with a better deal.

Subjectivists are never satisfied though ... that is why they are always buying tweaks. If you don't address fundamentals you are doomed to tweaks

 

[Raindog123]

The error scales proportionally with the sample size.

Well, technically it depends on the distribution of this error.

 

[DVDdoug]

I have an engineering background so I'm a specs & measurement guy but I don't have any problem with subjective reviews analysis. I do have a couple of BIG problems with most subjective "audiophile" reviews. They have a whole vocabulary of meaningless words, "It was slightly veiled but musical", etc. And, they don't do blind listening.

Amir listens (non-blind) and reports his subjective opinion but it's always backed-up or compared to the measurements. (I don't expect Amir to do blind listening tests... That would be impractical and we are very fortunate to have someone doing independent measurements.)

HydrogenAudio is different... They don't allow measurements (as "proof of sound quality") but they require scientific-blind listening tests (if you are making a claim about sound quality) and they "discourage" nonsense terminology.

 

[Killingbeans]

Well, technically it depends on the distribution of this error.

Yeah, I'm definitely no expert on statistics, but hope it still made some sort of sense

 

[Frgirard]

I have an engineering background so I'm a specs & measurement guy but I don't have any problem with subjective reviews analysis. I do have a couple of BIG problems with most subjective "audiophile" reviews. They have a whole vocabulary of meaningless words, "It was slightly veiled but musical", etc. And, they don't do blind listening.

Amir listens (non-blind) and reports his subjective opinion but it's always backed-up or compared to the measurements. (I don't expect Amir to do blind listening tests... That would be impractical and we are very fortunate to have someone doing independent measurements.)

HydrogenAudio is different... They don't allow measurements (as "proof of sound quality") but they require scientific-blind listening tests (if you are making a claim about sound quality) and they "discourage" nonsense terminology.

All is not so dark

Matrix-HiFi

New and cutting-edge approach to the Hifi world using a rationalistic tendency. We do not give anything for granted, commercial information does not fit our purposes. It is only valid that is obtained by objective procedures which are those we try to develop.

matrixhifi.com

 

[Gorgonzola]

What you should also try is picking which you prefer, as in not knowing which you are listening to and picking which sounds superior and which sounds inferior. The results can be surprising...

You may find this resource interesting;

ABX High Fidelity Test list

A set of browser-based ABX tests comparing lossless and lossy music samples

abx.digitalfeed.net

This was from 2013;

High Bitrate MP3 Internet Blind Test: Part 2 - RESULTS

A blog for audiophiles about more objective topics. Measurements of audio gear. Reasonable, realistic, no snakeoil assessment of sound, and equipment.

archimago.blogspot.com

JSmith

Who was really surprised that MP3 was preferred over lossless? No me, not any more than I'm not surprised that many audiophiles prefer the sound of tube equipment with quite high distortion over S/S, including class D, with far lower distortion.

I would like to suggest that MP3 and tubes are both preferred because they "deburr" the sound -- they take off the sharp edges so to speak,. To be sure they do it differently one from the other but result is quite similar. So this is pure conjecture on my part, (though the fact that it isn't "proven" experimentally doesn't mean I'm wrong).

 

[mhardy6647]

… shit, now there is an elephant in the room.



JSmith

 

[Killingbeans]

I would like to suggest that MP3 and tubes are both preferred because they "deburr" the sound -- they take off the sharp edges so to speak,.

I think the jury is still out on that one? I've seen people suggest something somewhat opposite. That high levels of harmonic distortion gives the impression of elevated treble and thereby the illusion of added detail.

 

[JSmith]

Who was really surprised that MP3 was preferred over lossless?

A little, but not entirely unexpected;

Though MP3 compression is considered lossy because some data cannot be recovered after compression, the MPEG algorithm can achieve transparent, or perceptually lossless, compression. After testing, it was concluded that expert listeners could not distinguish between coded and original audio clips even with a six to one compression ratio (Pan 3).
MP3 compression is so successful for imitating CD quality music because it utilizes the concept of auditory masking. Basically, this type of masking occurs when the presence of a strong audio signal makes weaker audio signals in the proximity imperceptible. This non-linear and adaptive threshold of hearing (the level below which a sound is not heard) varies with frequency and between individuals. Whether a person hears a sound or not depends on the frequency of the sound and whether the amplitude is above or below that persons hearing threshold at that frequency. For example, in the vicinity of a loud noise such as an airplane passing overhead, it is impossible to hear ordinary conversations due to the distortions present at the hearing thresholds of the individuals. Sounds that are inaudible due to dynamic adaptation of the hearing threshold are said to be masked.

MP3: Concept

JSmith