• WANTED: Happy members who like to discuss audio and other topics related to our interest. Desire to learn and share knowledge of science required. There are many reviews of audio hardware and expert members to help answer your questions. Click here to have your audio equipment measured for free!

Poll: Should We Get Into Testing Headphones or Speakers?

Should ASR get into testing speakers, headphones, or neither for now.

  • Speakers

    Votes: 145 56.0%
  • Headphones

    Votes: 77 29.7%
  • Neither. Can look again in a year or two.

    Votes: 35 13.5%
  • Never

    Votes: 2 0.8%

  • Total voters
    259

amirm

Founder/Admin
Staff Member
CFO (Chief Fun Officer)
Joined
Feb 13, 2016
Messages
44,368
Likes
234,385
Location
Seattle Area
I created a thread on motivation to tackle speaker testing: https://www.audiosciencereview.com/...-i-get-into-speaker-testing-measurement.7904/

Proper measurement techniques are fairly well-known and decades of work has been performed to correlate them with preference. Alas, the issue is high cost of state-of-the-art measurement gear.

In contrast, headphone testing is cheaper although not cheap. I don't have a quote in front of me but I am thinking it will be between $10K to $20K (I am investigating some cheaper options). This is one third of the bill for speaker testing.

The challenge with headphone testing is that the test fixture matters a ton. Repeatability is a challenge too. So while many measure headphones, correlating them with each other is hard, and decide what is correct, challenging. I have some ideas around this but at the end, it is still a less precise world than speakers.

Of course headphone testing is much easier than speakers. They are cheap and easy to ship around and I don't need a giant room as I do with speakers to test them.

At the end, we may do both but financial constraints mandate that we do one before the other right now.

The option also exists to do neither and keep testing electronics.

So share any comments, feedback, etc. in the thread and click on the poll options.

For those of you who are visitors, this is a good reason to sign up as a member so you can vote and shape what we are doing.
 

boXem

Major Contributor
Audio Company
Joined
Jun 19, 2019
Messages
2,014
Likes
4,852
Location
Europe
Headphones. Didn't change my mind during the last 12 hours, which is good news.

You have done an excellent work finding good electronics for headphones, now proper stuff needs to be plugged on these!

Concerning the issue of variability of the results pending the geometry of the dummy head, we consider that a good amplifier is not too sensitive to the load, a good speaker not too sensitive to the room, so in my mind a good headphone should not be too sensitive to the head/ear geometry.
But I am more of an electronics guy, and it's late here so I may be completely out of subject.
 

andreasmaaan

Master Contributor
Forum Donor
Joined
Jun 19, 2018
Messages
6,652
Likes
9,399
In my view, headphones should not be measured for two primary reasons:
  1. Headphone measurements already abound online. Rarely do I google a pair of headphones and not find at least one - and often two or more - sources of measurements.
  2. Headphone measurements are inherently unreliable due to the geometrical differences between listeners’ ears/heads. This has been discussed at length in the other thread, so no need to rehash all that here.
Loudspeakers, OTOH, are measured less frequently and less well. The Klippel system being proposed would provide measurements that are close to as accurate as possible. Moreover, some very knowledgeable members here could have a say in what parameters to measure. This is important, since there is currently no online source (of which I’m aware) that provides comprehensive enough measurements to properly judge a loudspeaker’s performance. ASR could be the first to do this.
 

Eurasian

Active Member
Forum Donor
Joined
Nov 2, 2018
Messages
253
Likes
214
Hello Amir,

I have a minidsp Ears set up NIB that I will be happy to send to you. I bought it with the intention of measuring my collection of cans, but never got around to it. Please let me know if you are interested.

Best wishes to all,
Eurasian
 

Mad_Economist

Addicted to Fun and Learning
Audio Company
Joined
Nov 29, 2017
Messages
543
Likes
1,618
What sort of system would you be using for measuring headphones? I know of Klippel's work in speaker characterization, but I haven't seen much from them regarding headphones. If it's just your bog standard GRAS 43AG/45CA, this probably wouldn't add much other than another source of measurements for the community - which would be a plus (there's still plenty out there that hasn't been measured on an industry standard system), but not nearly so distinctive as a regularly updated database of affordable speakers.

Having spent quite a lot of time measuring headphones, I can tell you that it's a lot more comparatively annoying than it might seem on the face of it - while you do away with the practical difficulties of quasianechioc speaker measurement, issues like what constitutes an accurate and average coupling and placement of the headphone are subjective and vexing, and the comparatively lower distortions of headphones are more troublesome to measure than those of most speakers (as an example, much of Innerfidelity's measurements of THD+N were N dominated).

I'd certainly be more interested in results from headphone measurements, however, as a primary headphone user, and given the popularity of this site with some other headphone-centric communities that may be a majority position among lurkers (although I think frequent posters are dominantly speaker listeners).

Regardless of which direction you went, IMO speakers/headphones/electroacoustic transducers in general is a markedly more interesting thing to be measuring from an end user's perspective, simply because it's comparatively easy to find DACs and amplifiers with inaudible nonlinearities, but functionally impossible to do so with speakers or headphones, and both are quite variable in quality even at similar price points. Thus speaker and headphone measurements are a lot more meaningful in their implications for what we're hearing.
 

DuxServit

Senior Member
Forum Donor
Joined
Apr 21, 2018
Messages
428
Likes
508
Millennials and Gen-Z mostly live in urban/city in smaller spaces, and thus most likely be interested in headphones.

I’m guessing the speaker-buying population size is several magnitudes smaller than headphone-buying one (I don’t have any stats to prove this).
 
OP
amirm

amirm

Founder/Admin
Staff Member
CFO (Chief Fun Officer)
Joined
Feb 13, 2016
Messages
44,368
Likes
234,385
Location
Seattle Area
I have a minidsp Ears set up NIB that I will be happy to send to you.
Hello @Eurasian. Thank you for the kind offer. I bought one of those as soon as they came out. Unfortunately I could not generate a measurement with it that remotely came close to anything that others have published. Wish it worked as we would have no cost involved.

One of the issues with it is the pinnae is hard so doesn't flex like the ones that Harman made, or the new one that GRAS has.

Thanks again.
 
OP
amirm

amirm

Founder/Admin
Staff Member
CFO (Chief Fun Officer)
Joined
Feb 13, 2016
Messages
44,368
Likes
234,385
Location
Seattle Area
If it's just your bog standard GRAS 43AG/45CA, this probably wouldn't add much other than another source of measurements for the community - which would be a plus (there's still plenty out there that hasn't been measured on an industry standard system), but not nearly so distinctive as a regularly updated database of affordable speakers.
That would be it and the point you make is right.
 

pozz

Слава Україні
Forum Donor
Editor
Joined
May 21, 2019
Messages
4,036
Likes
6,825
I'll echo what andreasmaaan has said.

This is a unique opportunity in view of what ASR has already become.
 

Tks

Major Contributor
Joined
Apr 1, 2019
Messages
3,221
Likes
5,494
While speakers could be easier to measure repeatably.. they are a nightmare logistically, so a no vote for that impracticality. Also costs of testing and gear for testing seems prohibitive.

While headphones are plagued with issues like "differences between people's heads".. that's irrelevant as long as testing can be reproduced within acceptable margins of error. So if I had to vote either, it'd be headphones to be honest. But neither in actuality.
 

RayDunzl

Grand Contributor
Central Scrutinizer
Joined
Mar 9, 2016
Messages
13,198
Likes
16,981
Location
Riverview FL
*votes "Neither. Can look again in a year or two."

Even so, I wouldn't mind seeing some speaker measures - whatever test you do, they would rank relatively to each other in the same environment.

Maybe work on Part 3 of "Room Measurement Tutorial for Dummies".
 
Last edited:

Wombat

Master Contributor
Joined
Nov 5, 2017
Messages
6,722
Likes
6,459
Location
Australia
Cost, time and practicability. Then there is credibility(specialised knowledge, and experience), environment and standardised process for the results to be widely accepted technically. Buying some hardware and software is not enough, although many amateurs believe this makes them instant experts.
 
Last edited:
OP
amirm

amirm

Founder/Admin
Staff Member
CFO (Chief Fun Officer)
Joined
Feb 13, 2016
Messages
44,368
Likes
234,385
Location
Seattle Area
Cost, time and practicability. Then there is credibility(specialised knowledge, and experience), environment and standardised process for the results to be widely accepted technically. Buying some hardware and software is not enough, although many amateurs believe this makes them instant experts.
Headphone testing will be new for me but not the rest of the audio science involved. Surprised that you think my only value would be said hardware and software. :( Regardless, I have access to a lot of industry resources from research to people who perform them.

As I said somewhere though, this will be a slow ramp where we all build-up the relevant knowledge together. So not setting high expectations at the start.
 

Wombat

Master Contributor
Joined
Nov 5, 2017
Messages
6,722
Likes
6,459
Location
Australia
Headphone testing will be new for me but not the rest of the audio science involved. Surprised that you think my only value would be said hardware and software. :( Regardless, I have access to a lot of industry resources from research to people who perform them.

As I said somewhere though, this will be a slow ramp where we all build-up the relevant knowledge together. So not setting high expectations at the start.

How much direct involvement in testing(not listening) and hands-on practise have you done? If you wish to get into the testing of transducers, that is fine. Sometimes a cautionary input is warranted

Maybe you could become the new Hirsch-Houck of the industry. That would be good. ;)
 

oivavoi

Major Contributor
Forum Donor
Joined
Jan 12, 2017
Messages
1,721
Likes
1,934
Location
Oslo, Norway
In my view, headphones should not be measured for two primary reasons:
  1. Headphone measurements already abound online. Rarely do I google a pair of headphones and not find at least one - and often two or more - sources of measurements.
  2. Headphone measurements are inherently unreliable due to the geometrical differences between listeners’ ears/heads. This has been discussed at length in the other thread, so no need to rehash all that here.
Loudspeakers, OTOH, are measured less frequently and less well. The Klippel system being proposed would provide measurements that are close to as accurate as possible. Moreover, some very knowledgeable members here could have a say in what parameters to measure. This is important, since there is currently no online source (of which I’m aware) that provides comprehensive enough measurements to properly judge a loudspeaker’s performance. ASR could be the first to do this.

Good and measured post, Andreas. You got me curious: What is it that you find lacking in the Soundstage or Stereophile measurements?
 

Krunok

Major Contributor
Joined
Mar 25, 2018
Messages
4,600
Likes
3,065
Location
Zg, Cro
How much direct involvement in testing(not listening) and hands-on practise have you done? If you wish to get into the testing of transducers, that is fine. Sometimes a cautionary input is warranted

That sophisticated machine seem to have automated measurement process comparable to the AP analyzer, except for the moving arm with microphone. I don't think interpreting it's spinorama-alike results would present challenge for Amir. What I think should be part of the testing is a subjective listening test, but as he is a level 7 (?) trained listener that also shouldn't be a problem. I'm sure he would put aditional effort to move to a higher level if he decides to go with loudspeaker testing.
 

nemesisrobot

Member
Forum Donor
Joined
Jun 20, 2019
Messages
38
Likes
38
Location
Seattle, WA
I voted speakers. They have the potential to be much more of an investment than headphones, and having Amir's objective and scientific measurements will have a big influence on my speaker purchasing decisions personally!
 

soundwave76

Addicted to Fun and Learning
Forum Donor
Joined
Dec 28, 2018
Messages
730
Likes
1,364
Location
Finland
Speakers, since headphone measurements already exist, as pointed out in the thread earlier.
 

andreasmaaan

Master Contributor
Forum Donor
Joined
Jun 19, 2018
Messages
6,652
Likes
9,399
Good and measured post, Andreas. You got me curious: What is it that you find lacking in the Soundstage or Stereophile measurements?

Thanks @oivavoi :) In Stereophile, it's mostly distortion measurements and measurements at multiple SPLs that are lacking. In Soundstage, it's mostly vertical polar response measurements.

Overall, I'd also like to see (in addition to what Soundstage and Stereophile measure):
  • distortion measured at multiple SPLs
  • vertical polar responses over a wide(er) window
  • SPL limits
...and probably one or two other measurements if I thought it through more.

German mag Sound & Recording does a good suite of measurements, come to think of it.
 

Thomas savage

Grand Contributor
The Watchman
Forum Donor
Joined
Feb 24, 2016
Messages
10,260
Likes
16,298
Location
uk, taunton
Headphones, test headphones. The market is bigger for us and it represents much better value.

Grow the forum with this , maybe try and get other outlets involved to creat a standardized method for testing .

Then in 18 months or so look at where we are and if the ( hopefully larger ) community can support a move to speakers .

Headphones also pair nice with the electronic you have been testing , mostly DACs and headphone amps.

Makes more sense to do this imo.
 
Top Bottom