• WANTED: Happy members who like to discuss audio and other topics related to our interest. Desire to learn and share knowledge of science required as is 20 years of participation in forums (not all true). Come here to have fun, be ready to be teased and not take online life too seriously. We now measure and review equipment for free! Click here for details.

Poll: Should We Get Into Testing Headphones or Speakers?

Should ASR get into testing speakers, headphones, or neither for now.

  • Speakers

    Votes: 133 56.8%
  • Headphones

    Votes: 63 26.9%
  • Neither. Can look again in a year or two.

    Votes: 36 15.4%
  • Never

    Votes: 2 0.9%

  • Total voters
    234

amirm

Founder/Admin
Staff Member
CFO (Chief Fun Officer)
Joined
Feb 13, 2016
Messages
22,650
Likes
35,025
Location
Seattle Area
#1
I created a thread on motivation to tackle speaker testing: https://www.audiosciencereview.com/...-i-get-into-speaker-testing-measurement.7904/

Proper measurement techniques are fairly well-known and decades of work has been performed to correlate them with preference. Alas, the issue is high cost of state-of-the-art measurement gear.

In contrast, headphone testing is cheaper although not cheap. I don't have a quote in front of me but I am thinking it will be between $10K to $20K (I am investigating some cheaper options). This is one third of the bill for speaker testing.

The challenge with headphone testing is that the test fixture matters a ton. Repeatability is a challenge too. So while many measure headphones, correlating them with each other is hard, and decide what is correct, challenging. I have some ideas around this but at the end, it is still a less precise world than speakers.

Of course headphone testing is much easier than speakers. They are cheap and easy to ship around and I don't need a giant room as I do with speakers to test them.

At the end, we may do both but financial constraints mandate that we do one before the other right now.

The option also exists to do neither and keep testing electronics.

So share any comments, feedback, etc. in the thread and click on the poll options.

For those of you who are visitors, this is a good reason to sign up as a member so you can vote and shape what we are doing.
 

Fred Jacquot

Active Member
Patreon Donor
Joined
Jun 19, 2019
Messages
228
Likes
242
Location
Europe
#2
Headphones. Didn't change my mind during the last 12 hours, which is good news.

You have done an excellent work finding good electronics for headphones, now proper stuff needs to be plugged on these!

Concerning the issue of variability of the results pending the geometry of the dummy head, we consider that a good amplifier is not too sensitive to the load, a good speaker not too sensitive to the room, so in my mind a good headphone should not be too sensitive to the head/ear geometry.
But I am more of an electronics guy, and it's late here so I may be completely out of subject.
 

andreasmaaan

Major Contributor
Joined
Jun 19, 2018
Messages
3,517
Likes
2,801
#3
In my view, headphones should not be measured for two primary reasons:
  1. Headphone measurements already abound online. Rarely do I google a pair of headphones and not find at least one - and often two or more - sources of measurements.
  2. Headphone measurements are inherently unreliable due to the geometrical differences between listeners’ ears/heads. This has been discussed at length in the other thread, so no need to rehash all that here.
Loudspeakers, OTOH, are measured less frequently and less well. The Klippel system being proposed would provide measurements that are close to as accurate as possible. Moreover, some very knowledgeable members here could have a say in what parameters to measure. This is important, since there is currently no online source (of which I’m aware) that provides comprehensive enough measurements to properly judge a loudspeaker’s performance. ASR could be the first to do this.
 

Eurasian

Active Member
Patreon Donor
Joined
Nov 2, 2018
Messages
111
Likes
65
#4
Hello Amir,

I have a minidsp Ears set up NIB that I will be happy to send to you. I bought it with the intention of measuring my collection of cans, but never got around to it. Please let me know if you are interested.

Best wishes to all,
Eurasian
 

Mad_Economist

Active Member
Joined
Nov 29, 2017
Messages
103
Likes
147
#5
What sort of system would you be using for measuring headphones? I know of Klippel's work in speaker characterization, but I haven't seen much from them regarding headphones. If it's just your bog standard GRAS 43AG/45CA, this probably wouldn't add much other than another source of measurements for the community - which would be a plus (there's still plenty out there that hasn't been measured on an industry standard system), but not nearly so distinctive as a regularly updated database of affordable speakers.

Having spent quite a lot of time measuring headphones, I can tell you that it's a lot more comparatively annoying than it might seem on the face of it - while you do away with the practical difficulties of quasianechioc speaker measurement, issues like what constitutes an accurate and average coupling and placement of the headphone are subjective and vexing, and the comparatively lower distortions of headphones are more troublesome to measure than those of most speakers (as an example, much of Innerfidelity's measurements of THD+N were N dominated).

I'd certainly be more interested in results from headphone measurements, however, as a primary headphone user, and given the popularity of this site with some other headphone-centric communities that may be a majority position among lurkers (although I think frequent posters are dominantly speaker listeners).

Regardless of which direction you went, IMO speakers/headphones/electroacoustic transducers in general is a markedly more interesting thing to be measuring from an end user's perspective, simply because it's comparatively easy to find DACs and amplifiers with inaudible nonlinearities, but functionally impossible to do so with speakers or headphones, and both are quite variable in quality even at similar price points. Thus speaker and headphone measurements are a lot more meaningful in their implications for what we're hearing.
 

DuxServit

Active Member
Patreon Donor
Joined
Apr 21, 2018
Messages
211
Likes
180
#6
Millennials and Gen-Z mostly live in urban/city in smaller spaces, and thus most likely be interested in headphones.

I’m guessing the speaker-buying population size is several magnitudes smaller than headphone-buying one (I don’t have any stats to prove this).
 

amirm

Founder/Admin
Staff Member
CFO (Chief Fun Officer)
Joined
Feb 13, 2016
Messages
22,650
Likes
35,025
Location
Seattle Area
#7
I have a minidsp Ears set up NIB that I will be happy to send to you.
Hello @Eurasian. Thank you for the kind offer. I bought one of those as soon as they came out. Unfortunately I could not generate a measurement with it that remotely came close to anything that others have published. Wish it worked as we would have no cost involved.

One of the issues with it is the pinnae is hard so doesn't flex like the ones that Harman made, or the new one that GRAS has.

Thanks again.
 

amirm

Founder/Admin
Staff Member
CFO (Chief Fun Officer)
Joined
Feb 13, 2016
Messages
22,650
Likes
35,025
Location
Seattle Area
#8
If it's just your bog standard GRAS 43AG/45CA, this probably wouldn't add much other than another source of measurements for the community - which would be a plus (there's still plenty out there that hasn't been measured on an industry standard system), but not nearly so distinctive as a regularly updated database of affordable speakers.
That would be it and the point you make is right.
 

Tks

Addicted to Fun and Learning
Joined
Apr 1, 2019
Messages
794
Likes
762
Location
NYC
#10
While speakers could be easier to measure repeatably.. they are a nightmare logistically, so a no vote for that impracticality. Also costs of testing and gear for testing seems prohibitive.

While headphones are plagued with issues like "differences between people's heads".. that's irrelevant as long as testing can be reproduced within acceptable margins of error. So if I had to vote either, it'd be headphones to be honest. But neither in actuality.
 

RayDunzl

Major Contributor
Central Scrutinizer
Joined
Mar 9, 2016
Messages
7,702
Likes
4,552
Location
Riverview FL
#11
*votes "Neither. Can look again in a year or two."

Even so, I wouldn't mind seeing some speaker measures - whatever test you do, they would rank relatively to each other in the same environment.

Maybe work on Part 3 of "Room Measurement Tutorial for Dummies".
 
Last edited:

Wombat

Major Contributor
Joined
Nov 5, 2017
Messages
4,553
Likes
2,927
Location
Australia
#12
Cost, time and practicability. Then there is credibility(specialised knowledge, and experience), environment and standardised process for the results to be widely accepted technically. Buying some hardware and software is not enough, although many amateurs believe this makes them instant experts.
 
Last edited:

amirm

Founder/Admin
Staff Member
CFO (Chief Fun Officer)
Joined
Feb 13, 2016
Messages
22,650
Likes
35,025
Location
Seattle Area
#13
Cost, time and practicability. Then there is credibility(specialised knowledge, and experience), environment and standardised process for the results to be widely accepted technically. Buying some hardware and software is not enough, although many amateurs believe this makes them instant experts.
Headphone testing will be new for me but not the rest of the audio science involved. Surprised that you think my only value would be said hardware and software. :( Regardless, I have access to a lot of industry resources from research to people who perform them.

As I said somewhere though, this will be a slow ramp where we all build-up the relevant knowledge together. So not setting high expectations at the start.
 

Wombat

Major Contributor
Joined
Nov 5, 2017
Messages
4,553
Likes
2,927
Location
Australia
#14
Headphone testing will be new for me but not the rest of the audio science involved. Surprised that you think my only value would be said hardware and software. :( Regardless, I have access to a lot of industry resources from research to people who perform them.

As I said somewhere though, this will be a slow ramp where we all build-up the relevant knowledge together. So not setting high expectations at the start.
How much direct involvement in testing(not listening) and hands-on practise have you done? If you wish to get into the testing of transducers, that is fine. Sometimes a cautionary input is warranted

Maybe you could become the new Hirsch-Houck of the industry. That would be good. ;)
 

oivavoi

Major Contributor
Joined
Jan 12, 2017
Messages
1,393
Likes
881
Location
Oslo, Norway
#15
In my view, headphones should not be measured for two primary reasons:
  1. Headphone measurements already abound online. Rarely do I google a pair of headphones and not find at least one - and often two or more - sources of measurements.
  2. Headphone measurements are inherently unreliable due to the geometrical differences between listeners’ ears/heads. This has been discussed at length in the other thread, so no need to rehash all that here.
Loudspeakers, OTOH, are measured less frequently and less well. The Klippel system being proposed would provide measurements that are close to as accurate as possible. Moreover, some very knowledgeable members here could have a say in what parameters to measure. This is important, since there is currently no online source (of which I’m aware) that provides comprehensive enough measurements to properly judge a loudspeaker’s performance. ASR could be the first to do this.
Good and measured post, Andreas. You got me curious: What is it that you find lacking in the Soundstage or Stereophile measurements?
 

Krunok

Major Contributor
Joined
Mar 25, 2018
Messages
4,190
Likes
2,319
Location
Zg, Cro
#16
How much direct involvement in testing(not listening) and hands-on practise have you done? If you wish to get into the testing of transducers, that is fine. Sometimes a cautionary input is warranted
That sophisticated machine seem to have automated measurement process comparable to the AP analyzer, except for the moving arm with microphone. I don't think interpreting it's spinorama-alike results would present challenge for Amir. What I think should be part of the testing is a subjective listening test, but as he is a level 7 (?) trained listener that also shouldn't be a problem. I'm sure he would put aditional effort to move to a higher level if he decides to go with loudspeaker testing.
 

nemesisrobot

Member
Patreon Donor
Joined
Jun 20, 2019
Messages
34
Likes
31
Location
Seattle, WA
#17
I voted speakers. They have the potential to be much more of an investment than headphones, and having Amir's objective and scientific measurements will have a big influence on my speaker purchasing decisions personally!
 

andreasmaaan

Major Contributor
Joined
Jun 19, 2018
Messages
3,517
Likes
2,801
#19
Good and measured post, Andreas. You got me curious: What is it that you find lacking in the Soundstage or Stereophile measurements?
Thanks @oivavoi :) In Stereophile, it's mostly distortion measurements and measurements at multiple SPLs that are lacking. In Soundstage, it's mostly vertical polar response measurements.

Overall, I'd also like to see (in addition to what Soundstage and Stereophile measure):
  • distortion measured at multiple SPLs
  • vertical polar responses over a wide(er) window
  • SPL limits
...and probably one or two other measurements if I thought it through more.

German mag Sound & Recording does a good suite of measurements, come to think of it.
 

Thomas savage

Power hungry desperado
Moderator
The Watchman
Patreon Donor
Joined
Feb 24, 2016
Messages
7,547
Likes
6,394
Location
uk, taunton
#20
Headphones, test headphones. The market is bigger for us and it represents much better value.

Grow the forum with this , maybe try and get other outlets involved to creat a standardized method for testing .

Then in 18 months or so look at where we are and if the ( hopefully larger ) community can support a move to speakers .

Headphones also pair nice with the electronic you have been testing , mostly DACs and headphone amps.

Makes more sense to do this imo.
 

Similar threads

Top Bottom