Why is audio objectivism so frequently focused on all the wrong things?

[Killingbeans]

Might this knowledge function much like the appearance of a DAC in a not-blinded listening test, and, hence, up-bias our impression of that DAC in a listening test. Sure, its an objective measurement, but don't the same prejudicial mechanisms kick in?

Absolutely!

Personally I don't mind any kind of bias as long as it's not exploited for profit.

 

[GrimSurfer]

 

[Killingbeans]

You owe me a new monitor.

 

[MediumRare]

What is your definition of "draw"? How close to 50/50 does it have to be?

Two part answer:
1. Within a statistical error band
2. Doesn't have to be one winner; there could be various segments each with a different preference (I like more bass, you like zero distortion, she likes flat FR plus 2nd order harmonics, etc.)

But none of the above is in conflict with the overarching precept: Use objective data to describe performance.

 

[Shadrach]

That's not true, though; you can do preference tests that end in draws. It's totally fine!

I'm afraid it is true. The logic demands it.
You cannot set a test to establish a preference without first assuming there will be a difference between the tested units/products. If there is no difference how can one reasonably have a preference?

 

[nhunt]

I'm afraid it is true. The logic demands it.
You cannot set a test to establish a preference without first assuming there will be a difference between the tested units/products. If there is no difference how can one reasonably have a preference?

Why does this matter? It's the null hypothesis. You test for both.

 

[Shadrach]

Why does this matter? It's the null hypothesis. You test for both.

I'm not going to clarify further. I don't think you would understand and I think you want a debate/argument and I've got better things to do with my time.
I think you were bored and decided to go and rattle the cages of the objectivists over on ASR.

 

[GrimSurfer]

First of all, a DBT can only result in subjective results. This is because human sensory responses are predominantly subjective. They are definitely not calibrated. They are inconsistent. Their accuracy varies. Even after listener training.

Second, the methodology behind a DBT seeks only to reduce visual bias.

Third, the only thing that larger audiences in a DBT do is to decrease the margin of error of the median and mean human responses. They don't increase the accuracy of the findings. They simply increase the probability that the responses are representative of the species.

Fourth, a DBT has no way for accounting for certain variables like hearing acuity of the test subjects, past influences, critical listening skills, cultural differences, varying responses to musical formats etc. Other tests may be needed to screen test subjects for these but, again, elimination of these variables is virtually impossible in any objective sense.

 

[Shadrach]

First of all, a DBT can only result in subjective results.

Nonsense. It's a subjective assessment that produces empirical results. What you end up with is data. How you interpret the data may well have subjective influences but the data itself is not subjective.
I think that will do for me here.

 

[Island_Kenny]

It is mine as well but that makes you wrong still. The domain matters. Dr. Toole is talking about sound reproduction in a room. There, a single microphone cannot remotely measure what you hear. You have two ears. Each hears differently. The brain interprets that difference. Sometime it ignores it. Sometimes it uses it to detect sense of space, etc.

Move your head an inch and a measurement mic and analyzer show completely different picture. But you don't "hear" a difference as you are moving your head all the time.

This is on top of a ton of folklore created over the years on how to measure rooms and interpret them. This is why Dr. Toole puts down their efforts as do I.

Note that in low frequencies both ears hear the same thing and there measurements are critically important.

In electronics, we don't have these issues. You can move your DAC to any place in the room and unlike a speaker, it makes no difference in its output. You can move around and again, what comes out of the DAC makes no difference. Most importantly, we don't use a microphone to measure a DAC. We use a direct connection with resolution and accuracy that exceeds that of your hearing.

Another point is that speakers have so much frequency response variation from ideal that focusing on much else is futile. Preference for speakers is almost entirely based on that response. We don't have this concern with our electronics anymore since we divorced from analog formats.

So take some insight from Dr. Toole's work but don't generalize too much. The domains are vastly different.

This thread becomes very difficult to follow. And I feel that a lot of posters spoke too fast, I included, without fully understanding what others try to say. As the context often got lost in between, the arguments appear more offensive or defensive than they actually are. I think that most of us agree far more than we disagree, and I will not post in this thread until I have time to write something clear and comprehensive.

 

[GrimSurfer]

Nonsense. It's a subjective assessment that produces empirical results. What you end up with is data. How you interpret the data may well have subjective influences but the data itself is not subjective.
I think that will do for me here.

I didn't say that it wouldn't lead to empirical results. I only said it would be subjective, influenced by many variables that make each response uniquely accurate. Clustering these reduces ambiguity but the results are not objective. They are just an averaged result of a variety of subjective responses.

 

[amirm]

My interpretation is that the measurement thing to some degree started as a way to attract people to the forum.

No. I had an idle audio analyzer so people asked me to measure a few DACs, specifically the $99 ones. I had no idea it would be a popular idea at the time. And the concept of me buying them all then was not something that could scale. I was surprised to see people then offer their products to measure and how fast this ramp up.

That said, I did realize that we were filling a massive vacuum of objective data in a field that was running on online myths and gossip. So I decided to spend a ton of money upgrading my audio analyzer. A bit after that, I decided to make this my part-time job, finishing a review per day.

The appreciation from memberships and visitors alike is like a drug you can't find any other way. So I am going at it as fast as I can.

It is true that without people reading what I do, the ROI will be poor. So I am very much in favor of our crowd size growing and with it, also enabling wider range of products to be tested.

 

[amirm]

thought 1: What do we think about the fact that many read ASR reviews and measurements before ever hearing a particular DAC or amp. Might this knowledge function much like the appearance of a DAC in a not-blinded listening test, and, hence, up-bias our impression of that DAC in a listening test. Sure, its an objective measurement, but don't the same prejudicial mechanisms kick in?

I perform a lot of the listening tests blind. You can tell this when two devices are involved in the listening test section. Levels are also matched although this is not always easy. The rigor is not there in running many trials, with more listeners and such. But measurements do not bias the output.

They do however inform the type of testing I do. With 20 million tracks out there, where do you start your listening test as to be sensitive enough to find any flaws? Answer: you do that based on what measurements show.

There are people who want to maximize not hearing a difference in a listening test. That is not me. I want to know where the weakness is and go after it. If your car breaks, you tell the mechanic what you think is broken. You don't leave him in the dark to find out what is wrong with the car. Same here. We want to increase the chances of problems being found and fixed. Others seem to be more interested in winning some Internet argument.

In blind tests, attempting to find the identity of worse performing product can backfire, and backfire big. I have reported on such many times. So I avoid it and instead try to identify if A or B sounds better. Then after I am confident of the outcome, look at which is which.

 

[tmtomh]

And this is a key thing: It's easy to accept Toole and Olive's research, because it showed what we, as proper sensible audiophiles, already suspected: That anechoic flat response is ideal, and that everyone prefers it if they really listen.

But imagine the thought experiment where it didn't pan out that way, where what people preferred (with the same near-universality) was a V-shaped response with boomier bass and brighter treble than strict anechoic neutrality. How would we react then?

My argument is, science is science. You do the experiment, and you believe the results. If science says that what's perceptually ideal is a V-shaped response, then that's what it is. It's now the job of theory to explain why that's true, but we start from the premise that what's been empirically determined is in fact true. Based on some of the responses here, it seems like there's a mindset that would think that anechoic-neutral was correct even if it were never chosen. And I think that's a big conceptual mistake.

And so just like you're saying: If you do a blind test with Yggdrasil vs. a cheap Topping (or a Modi 3), which (if either) do people actually prefer? A lot of people here seem to believe that the obvious answer is that nobody could tell the difference, but I'm not convinced of that. (There are lots of blind tests out there where people find differences between DACs. Maybe they're all methodologically flawed, but maybe not.) A lot more people are convinced that if people could tell a difference, they'd prefer the better-with-the-measurements-we-have one, and I'm even less convinced of that, because there's no real reason to believe it's true. I'd like to see the actual tests, and how it falls out.

I agree with you that science is science. But your comment here actually undermines the premise with which you started (and named) this thread. The fact that you have to pose the V-shape-response-as-ideal question as a hypoethetical, as a "what if," illustrates the point that anechoic-neutral is in fact the experimentally determined perceptual ideal, and that perceptual ideal corresponds to what we think of as good measurement.

This is where I think arguments like yours fail: while you do acknowledge (at least implicitly) that an experimentally derived human perceptual preference exists, the logic of your argument has to proceed as if the human perceptual preference is unknown and is a toss-up between something that corresponds to good measurements and something that does not. Only with that pretense can you make your argument that ASR's objectivist members might be close-minded because they focus on equipment measurements based on an a priori assumption/value-judgment that neutral=good.

This kind of thought experiment is rather frustrating IMHO, not because thought experiments are valueless (in fact, they often are quite valuable), but rather because this type of thought experiment asks us to pretend that we don't know as much as we actually do - and then implicitly casts anyone who points that out as close-minded.

This scenario plays itself out over and over again online in arguments about science. People who reject Creationism are called close-minded and reminded that evolution is "just a theory." People are called intolerant for rejecting and mocking flat-earth arguments. People who stand up for well-established scientific theories are often told that they're biased and unscientific because they've never observed the relevant phenomena themselves or because they don't understand the complex math the theories are based upon.

I want to be 100% clear: I am not saying you believe any of the above. I am only pointing out that there is a link there to the logic of your argument, and the way (at least as I read your argument) you seem to be asking us to consider a hypothetical that requires us to "un-know" something that's already been experimentally established.

Relying on existing knowledge doesn't make us close-minded. And that, once again, means that the premise of the question, "Why is audio objectivism so frequently focused on all the wrong things?" is false - you have not established that objectivism is in fact focused on the wrong things.

 

[amirm]

This whole thing reminds me of books on how to be a great sailboat racer, which include steps to meticulously wet-sand and prepare boat bottoms before a regatta, even if the same was done days ago. The argument is that entering a race with near-perfect equipment has a strong psychological effect on "no excuses" mentality: its the sailor's responsibility to win the race.

I make this point often: there is a great subjective feeling knowing that you have perfection in your gear. The notion that "it should be transparent" doesn't do it because you don't know. I like to know that my plane pilot knows every trick there is to keep me safe in sky. Not that "he should know."

 

[GrimSurfer]

I make this point often: there is a great subjective feeling knowing that you have perfection in your gear. The notion that "it should be transparent" doesn't do it because you don't know. I like to know that my plane pilot knows every trick there is to keep me safe in sky. Not that "he should know."

There is also training value in this. If all reasonable testing concludes that a DAC and amp are transparent to the limit of human hearing, then you can be confident that over time your ear will adjust... and you'll develop an appreciation for objectively better sound.

This is the inverse of the ludicrously long "burn in" claims from component manufacturers. They want you to keep the gear, regardless of its measurement, long enough for your ear to adjust (and at least until the end of the return period).

If your ears don't adjust, then you can be confident that the fault lies with speakers or room. If you don't wish to address either, equalization can be used to paper over the cracks. If your ears can't acclimatize, then set manual levels.

By selecting objectively better performing gear, one will either train themselves to appreciate good sound, find and fix the next weakest links, or cave in to subjective preference. Any way you cut it, it breaks the circle of confusion by giving listeners a path that ends somewhere other than the audio shop and a credit card statement.

 

[dc655321]

Based on some of the responses here, it seems like there's a mindset that would think that anechoic-neutral was correct even if it were never chosen. And I think that's a big conceptual mistake.

I don't think the "correct" response could be other than anechoic-neutral (as you call it).

If I am tasked with designing a system that takes a signal consisting of one form of energy and converts/reproduces that signal into another form of energy (electrical to acoustic, here), I would want that transformation to be as linear as possible, preserving fidelity across the target bandwidth.

How else might that be accomplished with other than a flat frequency response?

 

[DonH56]

This thread becomes very difficult to follow. And I feel that a lot of posters spoke too fast, I included, without fully understanding what others try to say. As the context often got lost in between, the arguments appear more offensive or defensive than they actually are. I think that most of us agree far more than we disagree, and I will not post in this thread until I have time to write something clear and comprehensive.

I agree entirely, but long ago decided if I rigorously followed the bolded part I would never post again, so I allow myself the benefit of the doubt and hope I don't mess up too often and people have a forgiving nature. Both premises have been wrong at times...

 

[Robin L]

Yes. Here is a 1/8th inch capsule B&K measuring microphone. Very flat response in audible band. Resonance is at 160 khz. However, it is rather insensitive. Plus it has a self noise of 43 dbA. Using it for recording music might result in hiss like reel to reel tape.

https://www.bksv.com/-/media/literature/Product-Data/bp2030.ashx

About the boring and lifeless comment, yes it seems to follow accurate gear. Certainly inaccurate gear adds something to the signal. It might seem like more is there, but it is a distortion. A couple of my favorite microphones I value for their accuracy, and they have better than average measured responses. They have a rep in some circles as being boring, lifeless, and lacking mojo.

One of my favorite recordings uses B&K microphones that have additional phantom power, producing a line level signal to the digital recorder it was attached to. Audible self noise from the microphones on that Lute recording, Astrée / E 8703, but more than enough mojo, with extraordinary capture of the hall acoustic.

 

[Mikey]

I don't think the "correct" response could be other than anechoic-neutral (as you call it).

If I am tasked with designing a system that takes a signal consisting of one form of energy and converts/reproduces that signal into another form of energy (electrical to acoustic, here), I would want that transformation to be as linear as possible, preserving fidelity across the target bandwidth.

How else might that be accomplished with other than a flat frequency response?

I'm not sure it's intuitively obvious whether anechoic-flat or in-room-flat was correct -- the idea that the recording already contains the sounds of the recording room, and so you'd want to remove the listening room from the playback doesn't seem obviously crazy. Empirical testing clears that up.

But there's more than just that, too. Their experiments also showed that off-axis response was important (which some speaker manufacturers still argue about to this day), and that time and phase coherence were not important. None of those things are obvious without doing the test.

When it comes to electronics, I think we can all fully expect that flat frequency response is going to be important. But are there other things that are important? Are there things that we think are important that really aren't? My sense is that while we don't know nothing, we also don't have anything like the empirical answers that Toole and Olive's work has given us with speakers.