• WANTED: Happy members who like to discuss audio and other topics related to our interest. Desire to learn and share knowledge of science required. There are many reviews of audio hardware and expert members to help answer your questions. Click here to have your audio equipment measured for free!

Science, Measurements, and Commoditization leads to FUD

LeftCoastTim

Senior Member
Forum Donor
Joined
Apr 15, 2019
Messages
375
Likes
757
I was looking at some headphone measurements [1] done by a reddit user, using very high end equipment. He even rated them based on Harman target curves. One of the best measuring headphone was the NAD HP50 [2], which is $150 new on Amazon. Yes, those were designed by ex-Harman people, but what struck me was the price point. That $150 headphone trounces all InnerFidelity Wall-o-Fame cans. And then there is Samsung Galaxy Buds [3], which for $120 shames most other IEMs ten times the price. I don't own either of those, but I did try the suggested EQ settings for my Bose QC20 [4]. These target curves work!

But someone will eventually come along and trash these headphones and applaud the $5000 ones.

I have some JBL 308's. They sound better than many speakers ten times the price. But In a thread [5], someone trashed "cheap class D amps", because of the hiss. The hiss is only audible with my ears 50cm from the tweeters, and only when there is no music. The noise it's making is below the threshold of hearing at listening distance.

What I'm trying to say is that the research has shown how to build great audio equipment, and the market will try to provide them at decent prices. Science and measurements lead to commoditization of good sound. I welcome this!

But for many in the audio business, the commoditization is a threat to be squashed. There is an entire business model of selling junk at ridiculously high prices. And to justify the high prices, it is Fear, Uncertainty and Doubt every day. Every thread on this very forum there is someone recommending high priced junk. Maybe they are a paid shill or just someone who fell for it. And elsewhere on the internet, it is much worse. People overpay for silly things and stake their reputations on their purchases. The shills disguise as helpful fellow consumers.

There is a very easy way to fight snakeoil: good measurements and comparison rankings. Sadly, there isn't much money to be made there, so consumers suffer.

I think speakers and headphones should be measured and ranked like DACs and amps. But I think this opinion will be out-shouted by the shills.

[1] https://www.reddit.com/r/oratory1990/wiki/index
[2] https://www.dropbox.com/s/mjbp2dau2o3m680/NAD Viso HP50.pdf?dl=0
[3] https://www.dropbox.com/s/buxclwpmt02c215/Samsung Galaxy Buds.pdf?dl=0
[4] https://www.dropbox.com/s/96123l8n2o5vqq6/Bose QC20.pdf?dl=0
[5] https://www.audiosciencereview.com/...8-inch-2-way-monitors-first-impressions.7977/
 

RayDunzl

Grand Contributor
Central Scrutinizer
Joined
Mar 9, 2016
Messages
13,201
Likes
16,983
Location
Riverview FL
I have some JBL 308's. They sound better than many speakers ten times the price.

I have a pair too.

They're fine till played too loudly, then audibly distort. YMMV.

Nevertheless, they're my daily drivers now, set up alongside the "ten twenty five times the price" stuff.
 

Ron Texas

Master Contributor
Joined
Jun 10, 2018
Messages
6,078
Likes
8,914
LS50's distort if played too loudly as nearly everything. It's just that some speakers can play louder than others.
 

617

Major Contributor
Forum Donor
Joined
Mar 18, 2019
Messages
2,404
Likes
5,296
Location
Somerville, MA
One significant factor with inexpensive equipment is variation between examples of the same product. A big part of why expensive speaker drivers are costly is consistency between units - this requires really crazy tight tolerances on things like how much glue is used, the weight of the cones and magnetic flux and all that. My JBL 305s measure very poorly, and I believe it is due to a lack of testing and consistency.

As far as headphone measurements go, we've been discussing that recently and I don't believe there is a standard for meaningful headphone measurement.
 
OP
L

LeftCoastTim

Senior Member
Forum Donor
Joined
Apr 15, 2019
Messages
375
Likes
757
One significant factor with inexpensive equipment is variation between examples of the same product. A big part of why expensive speaker drivers are costly is consistency between units - this requires really crazy tight tolerances on things like how much glue is used, the weight of the cones and magnetic flux and all that. My JBL 305s measure very poorly, and I believe it is due to a lack of testing and consistency.

As far as headphone measurements go, we've been discussing that recently and I don't believe there is a standard for meaningful headphone measurement.

Does your pair measure very differently from each other? How about "renting" another pair and testing those.

I think the debate about meaningful headphone measurements is another FUD driven drivel. Yes, it's not perfect, but it [one that Harman used] is certainly good enough to predict listener preference with good accuracy and is so much better than what other review sites use, and it certainly beats the pants off subjective "art appreciation" we get from all other reviews.
 

amirm

Founder/Admin
Staff Member
CFO (Chief Fun Officer)
Joined
Feb 13, 2016
Messages
44,370
Likes
234,397
Location
Seattle Area
From the title, I thought you were going to say what we did was FUD. :)

But yes, FUD is the biggest hurdle for us. While its level is very low here, in other forums people immediately raise random objections which can cloud the perception of a new reader. It is a disservice to the community to do such things.
 

617

Major Contributor
Forum Donor
Joined
Mar 18, 2019
Messages
2,404
Likes
5,296
Location
Somerville, MA
Does your pair measure very differently from each other? How about "renting" another pair and testing those.

I think the debate about meaningful headphone measurements is another FUD driven drivel. Yes, it's not perfect, but it [one that Harman used] is certainly good enough to predict listener preference with good accuracy and is so much better than what other review sites use, and it certainly beats the pants off subjective "art appreciation" we get from all other reviews.

My pair of lsr 305 mkii vary from each other dramatically. No reference curve was published for this speaker but the idea of using these as a reference speaker is absurd - I published the measurements here and you can see for yourself. I wouldn't even consider that response hifi. I correct mine digitally and they're fine.

Regarding headphone measurement, I don't think it's FUD when headphones are varying 5db+ when taken off and then put back on the test jig.

I started this discussion to learn more and it was eye opening:
https://www.audiosciencereview.com/...ngful-headphone-measurement.7919/#post-191727
 

andreasmaaan

Master Contributor
Forum Donor
Joined
Jun 19, 2018
Messages
6,652
Likes
9,399
I was looking at some headphone measurements [1] done by a reddit user, using very high end equipment. He even rated them based on Harman target curves. One of the best measuring headphone was the NAD HP50 [2], which is $150 new on Amazon. Yes, those were designed by ex-Harman people, but what struck me was the price point.

DIY measurements are not really comparable to the Harman target as each measurement rig has its own distinct transfer function. Not to dispute your general point, but rather noting that if a measurement shows the headphones match a target based on a different rig, that doesn't necessarily tell you a lot.
 

617

Major Contributor
Forum Donor
Joined
Mar 18, 2019
Messages
2,404
Likes
5,296
Location
Somerville, MA
OP
L

LeftCoastTim

Senior Member
Forum Donor
Joined
Apr 15, 2019
Messages
375
Likes
757
DIY measurements are not really comparable to the Harman target as each measurement rig has its own distinct transfer function. Not to dispute your general point, but rather noting that if a measurement shows the headphones match a target based on a different rig, that doesn't necessarily tell you a lot.

The reddit guy is not doing DIY measurements. I believe he is a graduate student using a university lab off-hours.

This is his setup (copied from his results sheet)

measured on GRAS43AC/43AG coupler with anthropometric pinna
BruelKjaer Nexus Type 2960 Low-Noise Signal Conditioner
APx515 or APx526 Signal Generator/Analyzer
APx1701 or GRAS 12AU amplifier
 

andreasmaaan

Master Contributor
Forum Donor
Joined
Jun 19, 2018
Messages
6,652
Likes
9,399
The reddit guy is not doing DIY measurements. I believe he is a graduate student using a university lab off-hours.

This is his setup (copied from his results sheet)

measured on GRAS43AC/43AG coupler with anthropometric pinna
BruelKjaer Nexus Type 2960 Low-Noise Signal Conditioner
APx515 or APx526 Signal Generator/Analyzer
APx1701 or GRAS 12AU amplifier

I see. Using the same GRAS is a good start. But the pinnate are critical - if they differ from those used by Harman, the possibility for meaningful comparison is going to be limited.
 
OP
L

LeftCoastTim

Senior Member
Forum Donor
Joined
Apr 15, 2019
Messages
375
Likes
757
I see. Using the same GRAS is a good start. But the pinnate are critical - if they differ from those used by Harman, the possibility for meaningful comparison is going to be limited.

Well, apparently he knows what he's doing:
(edit, apparently reddit links don't work like they should. Here is the body text from https://www.reddit.com/r/headphones/comments/7szpqm/_/dt8pt9p ):


I'm going to talk a lot about acoustic impedance, so if you're not really familiar with this term I suggest reading this short article before continuing.

This is a comparison of the same headphone (my HD800S) measured on 3 different setups.

The Orange graph is the industry-standard GRAS 43AG, an artificial ear with an anatomically correct pinna and ear canal, that has the same acoustic impedance as the human ear. This is the measurement that you can rely on, this is the sound pressure that arrives at your ear when you listen to this headphone. This is also the measurement that you can safely compare against the Harman Target, or the Diffuse-Field Target, or any other Target response.

The Blue graph is the same headphone measured on the miniDSP EARS. You can clearly see the effect of an acoustic impedance mismatch - there is a very strong resnance at 4.5 kHz, and the region from 1-3 kHz is lacking the boost it gets from a real human ear.
You can also see a drop-off above 10 kHz, likely because the microphone capsules that were used in the EARS aren't very good and only go up to 10 kHz reliably. Then again, the whole setup is incredibly inexpensive.
You can also see a misrepresented tonality from the bass to the mids - the miniDSP EARS shows a slight downward slope, which would point to a warm sounding, slightly bass-boosted headphone, when in reality the HD800S is almost perfectly linear between 10 Hz and 1 kHz. This too is a result of the acoustic impedance of the EARS not being the same as in a human ear. And the difference will be different for other sets of headphones.

Lastly the Grey graph is the same headphone measured on a Neumann KU100, which is probably the cheapest way to get a dummy head (it's still around 5000 €). But the KU100 is not designed for measurement applications, it is a binaural microphone intended for dummy head recordings of, say, an orchestra. It too does not have the acoustic impedance of a human ear, plus it has a built-in, fixed EQ. The goal was that this binaural microphone would produce a flat, linear response when placed in a diffuse field, so that when you listen to recordings made with this microphone on a diffuse-field equalized headphone (a headphone that conforms to the DF target) you will head it exactly as if your head would stand where the binaural microphone stood during the recording session.
I included the KU100 because some people use it to measure headphones. It's more reliable than the miniDSP EARS, but still has a slight impedance mismatch. All further discussion will only include the EARS and the 43AG coupler.

Now, what is the difference between measurements with the EARS and with the 43AG?
First off: It doesn't have to be the 43AG, it could be a 45CA or a KEMAR or other rigs that are compliant with the relevant IEC-norms, they will all deliver the same results (within margin of error, up to 8 kHz).
Now, I measured an HD800, an AKG K601 and a CX2.00 (insert-earphone) on the 43AG and on the EARS, this is the difference.
These are graphs obtained by subtracting the EARS measurement from the 43AG measurement. If the EARS would be as reliable as the 43AG then the result would be virtually identical for every headphone.
While the various measurements of the HD800 and HD800S produce relatively the same difference, the AKG K601 has a different "difference". It's even worse with the CX2.00 insert earphone, which has a very different acoustic impedance as over-ear headphones, and therefore interacts very differently with the acoustic impedance of the ear (or in this case: of the EARS).
The average looks like this, also shown is the deviation (grey) from the average.
The deviation is very high in the sub-bass (due to non-reliable seal), around 3 kHz (due to the wrong acoustic impedance of the pinna), and in the frequency regions 5 kHz and upwards, where deviations are much to high to obtain reliable measurements from the EARS.

The goal of all of this was to find out whether I could simply add a "compensation curve" to a measurement made with the EARS, and the result would be identical to a measurement made with the 43AG.
The answer is NO, because this compensation curve would have to be different for every headphone (depending on the acoustic impedance of the headphone).

I still tried to calculate an average of just the Over-Ear headphones - This is the compensation curve that you would have to add on average.
When you do a measurement with this compensation curve (called "calibration curve" in REW) and you see a flat linear response, then the headphone would be very close to the Harman Target.
The red graph shows the compensation curve that miniDSP provides along with every EARS rig. It looks similar to the compensation that I calculated, but it actually differs quite a lot.
 
OP
L

LeftCoastTim

Senior Member
Forum Donor
Joined
Apr 15, 2019
Messages
375
Likes
757
I like this guy a lot. But I hope he's wrong about "spending that kind of money just to see the frequency response".

"The big problem with such a highly accurate rig obviously is the cost. A complete rig with dummy head, dummy ears, ear canal simulators and measurement microphones together with the necessary signal conditioners (basically microphone preamps) and necessary AD/DA converters and measurement amplifiers (so you can measure signal voltage and current draw simultaneously while measuring the acoustic response) will easily set you back 50 to 100 grands.

I do this for a living, which is why have such a setup in the lab (or rather: the company that I work for put such a setup in the lab), but obviously most (virtually all) people won't spend that kind of money just to see the frequency response or other measurements of their headphone."


https://www.reddit.com/r/DIEMs/comments/75pfaq/_/do7wka1
 
OP
L

LeftCoastTim

Senior Member
Forum Donor
Joined
Apr 15, 2019
Messages
375
Likes
757
I think Harman use a G.R.A.S. 45 AG, so they won't conform exactly to what Harman are doing.
He says in a different post, and I quote:
"First off: It doesn't have to be the 43AG, it could be a 45CA or a KEMAR or other rigs that are compliant with the relevant IEC-norms, they will all deliver the same results (within margin of error, up to 8 kHz)."
 

andreasmaaan

Master Contributor
Forum Donor
Joined
Jun 19, 2018
Messages
6,652
Likes
9,399
I think Harman use a G.R.A.S. 45 AG, so they won't conform exactly to what Harman are doing.

It’s all very confusing! I do think they use (or at least used) the 43 but someone with more expert knowledge will need to chip in here methinks.

Something I find problematic about the level of confidence in these measurements is that Soundstage!’s measurements using apparently the same 43AG rig show the HP50 to deviate quite significantly from the Harman target. Both can’t be right...
 
OP
L

LeftCoastTim

Senior Member
Forum Donor
Joined
Apr 15, 2019
Messages
375
Likes
757
It’s all very confusing! I do think they use (or at least used) the 43 but someone with more expert knowledge will need to chip in here methinks.

Something I find problematic about the level of confidence in these measurements is that Soundstage!’s measurements using apparently the same 43AG rig show the HP50 to deviate quite significantly from the Harman target. Both can’t be right...
Do you have a link please?
 
Top Bottom