Master Thread: Are measurements Everything or Nothing?

[Blumlein 88]

Ah, yes I should note that I performed this with headphones and simply placing my iPhone mic near the cup while playing a 1Khz tone from both amps until matched. Imperfect? Perhaps... but I could not tell any difference.

To do this to the level of useful accuracy requires dead steady positioning of the measurement mic and the DUT whether speakers or headphones. Then you need to use a reasonably high resolution FFT and average it over a few seconds to get a steady result not upset by environmental noise. This will get you within .1 db if done with care. I have tried it after a discussion with someone else saying it could be done and they were doing it with headphones.

 

[rsoffer]

To do this to the level of useful accuracy requires dead steady positioning of the measurement mic and the DUT whether speakers or headphones. Then you need to use a reasonably high resolution FFT and average it over a few seconds to get a steady result not upset by environmental noise. This will get you within .1 db if done with care.

Yes, check out this study.

The Accuracy of iPhone Applications to Monitor Environmental Noise Levels - PubMed

NA Laryngoscope, 131:E59-E62, 2021.

pubmed.ncbi.nlm.nih.gov

The headphone never needs to move. It's actually pretty simple.

 

[Blumlein 88]

Yes, actually within .1 db. (At least according to the iPhone)
The mic is surprisingly good.

Try it, I think you'll be surprised. Is it enough to prove anything? No. But can be a useful first step for someone.

Edit:

The Accuracy of iPhone Applications to Monitor Environmental Noise Levels - PubMed

NA Laryngoscope, 131:E59-E62, 2021.

pubmed.ncbi.nlm.nih.gov

I added a study outlining the accuracy of the iPhone's mic for SPL testing.

NO, it can also be a confounding result making someone mismatch level as a first step and then say, "hey, I matched levels, and I still hear a difference".

 

[pkane]

To do this to the level of useful accuracy requires dead steady positioning of the measurement mic and the DUT whether speakers or headphones.

And the person doing the set up. Just moving a few inches in the room can result in a change in level, even if the mic remains stationary and is very precise.

 

[Blumlein 88]

Yes, check out this study.

The Accuracy of iPhone Applications to Monitor Environmental Noise Levels - PubMed

NA Laryngoscope, 131:E59-E62, 2021.

pubmed.ncbi.nlm.nih.gov

The headphone never needs to move. It's actually pretty simple.

Apples and oranges application in level matching phones or speakers vs measuring environmental noise.

 

[Blumlein 88]

And the person doing the set up. Just moving a few inches in the room can result in a change in level, even if the mic remains stationary and is very precise.

Yes, this is absolutely correct. Thanks for bringing it up.

 

[rsoffer]

NO, it can also be a confounding result making someone mismatch level as a first step and then say, "hey, I matched levels, and no there is a difference".

And that's ok, now you can move to precision volume matching methodologies. They are now in the game of validating their experience. The iPhone test is more of a initial screen, rather than something you would publish in a study haha.

I personally could not tell amps apart while volume matching with my iPhone. So for me it was already pretty convincing.

(With headphones specifically, would not use this for speakers)

 

[HarmonicTHD]

Yes, actually within .1 db. (At least according to the iPhone)
The mic is surprisingly good.

Try it, I think you'll be surprised. Is it enough to prove anything? No. But can be a useful first step for someone.

Edit:

The Accuracy of iPhone Applications to Monitor Environmental Noise Levels - PubMed

NA Laryngoscope, 131:E59-E62, 2021.

pubmed.ncbi.nlm.nih.gov

I added a study outlining the accuracy of the iPhone's mic for SPL testing.

Nope. You don’t even know what the accuracy of the mic is plus moving it within the ear pads already will have a significant effect. You claim to be scientific? So what’s so hard getting a voltmeter out - costs 20bucks at most.

 

[rsoffer]

Nope. You don’t even know what the accuracy of the mic is plus moving it within the ear pads already will have a significant effect. You claim to be scientific? So what’s so hard getting a voltmeter out - costs 20bucks at most.

I don't understand the concern. The mic is in the same exact position, near the headphone cup for both amps. What is moving in this scenario? Nothing is moving.

I'm all for getting a voltmeter. I'm only saying this is still worthwhile as a first step, and can be illuminating. Not everyone has a volt meter on hand, and when the curiosity strikes, it's nice to be able to do something.

 

[DonR]

I don't understand the concern. The mic is in the same exact position, near the headphone cup for both amps. What is moving in this scenario? Nothing is moving.

I'm all for getting a voltmeter. I'm only saying this is still worthwhile as a first step, and can be illuminating. Not everyone has a volt meter on hand, and when the curiosity strikes, it's nice to be able to do something.

It still cannot account for ambient noise fluctuation in the room. This can be several dB even if you perceive the background noise remaining constant.

 

[BDWoody]

And that's ok, now you can move to precision volume matching methodologies. They are now in the game of validating their experience. The iPhone test is more of a initial screen, rather than something you would publish in a study haha.

I personally could not tell amps apart while volume matching with my iPhone. So for me it was already pretty convincing.

(With headphones specifically, would not use this for speakers)

I'm only saying this is still worthwhile as a first step, and can be illuminating.

I understand what you are saying, but what if when you listened, you heard differences? This is often what happens, and when those very well meaning folks are challenged to use a more precise method of measurement (multimeter), we get the 'Nothing is ever good enough' line or something similar, which tends to discourage much further.

This just reduces the likelihood of disappointment when results are inevitably questioned.

It really is better to start with an easier, much more accurate method that then removes any doubt about that variable at least. When I tested myself single blind, I figured that if I 'passed' I would need to enlist the help of someone to be able to make it a double blind test, but it never got that far. Because the level matching is SO important, it really needs to be the first thing on the list to get right.

 

[rsoffer]

I understand what you are saying, but what if when you listened, you heard differences? This is often what happens, and when those very well meaning folks are challenged to use a more precise method of measurement (multimeter), we get the 'Nothing is ever good enough' line or something similar, which tends to discourage much further.

This just reduces the likelihood of disappointment when results are inevitably questioned.

It really is better to start with an easier, much more accurate method that then removes any doubt about that variable at least. When I tested myself, I figured that if I 'passed' I would need to enlist the help of someone to be able to make it a double blind test, but it never got that far. Because the level matching is SO important, it really needs to be the first thing on the list to get right.

Fair enough, I understand this concern as well.

 

[Blumlein 88]

Things that you need to know about level matching with a voltmeter:
With cheap meters don't use over 400 hz for the tone.
Match voltage within 1.5%.

Things you need to know about level matching with a microphone:
Microphone and device and people in room need to remain in constant position.
Match within .1 db SPL
Did a truck drive by your house during the measurement? Oops no good.
Did a jet fly over your area without you noticing?
Did your fridge turn on and off even at the other end of the house?
Did your HVAC come on or turn off during the two measurements?
Did a train come down tracks within 5 miles of your house?
Is there a distant lawn mower running outside?
Is one of your neighbors using a leaf blower?
Was a dog barking during a measurement?
Use averaging over several seconds with an FFT (what's an FFT?).
Oh you might need at least some understanding of an FFT.

 

[ahofer]

It would be fun to set up an ”ASR blind testing day” somewhere. A bunch of members in a particular city could bring their amps, cables, DACs and APs and we could agree on proper procedure. More complicated - who brings the KEF blades (or similarly objectivist/subjectivist preferred speakers) and how would you attract any subjectivists motivated to actually demonstrate their abilities?

 

[rsoffer]

It would be fun to set up an ”ASR blind testing day” somewhere. A bunch of members in a particular city could bring their amps, cables, DACs and APs and we could agree on proper procedure. More complicated - who brings the KEF blades (or similarly objectivist/subjectivist preferred speakers) and how would you attract any subjectivists motivated to actually demonstrate their abilities?

Canjam NYC is coming up. Would be cool to set something like that up.

 

[HarmonicTHD]

I don't understand the concern. The mic is in the same exact position, near the headphone cup for both amps. What is moving in this scenario? Nothing is moving.

I'm all for getting a voltmeter. I'm only saying this is still worthwhile as a first step, and can be illuminating. Not everyone has a volt meter on hand, and when the curiosity strikes, it's nice to be able to do something.

Did you never notice that the sound changes when you move your headphones? If yes, how can you expect that the iPhone mic measures accurately? Did you ever look into how headphones are measured and how particular one has to be about the fit to the measuring device?

Take the voltmeter.

 

[rsoffer]

Did you never notice that the sound changes when you move your headphones? If yes, how can you expect that the iPhone mic measures accurately? Did you ever look into how headphones are measured and how particular one has to be about the fit to the measuring device?

Take the voltmeter.

I'm not moving the headphone or the mic at any time during this level matching. But given the amount of confusion my description has caused, yes - stick with the voltmeter.

 

[HarmonicTHD]

I'm not moving the headphone or the mic at any time during this level matching. But given the amount of confusion my description has caused, yes - stick with the voltmeter.

You moving it when you switch to the other DAC and then level match it. It will never be in the same position again. Yes take the voltmeter

 

[Chagall]

I'm not moving the headphone or the mic at any time during this level matching. But given the amount of confusion my description has caused, yes - stick with the voltmeter.

Funny how even when you detect no differences between amps and keep saying preliminary, people just see potential problems with the suboptimal test.
But you have to admit, that zeal for accuracy is actually quite admirable. It's nothing personal, just pure science and no half-measures.

 

[ahofer]

Funny how even when you detect no differences between amps and keep saying preliminary, people just see potential problems with the suboptimal test.
But you have to admit, that zeal for accuracy is actually quite admirable. It's nothing personal, just pure science and no half-measures.

The trouble is that those of us who’ve been through this are now predisposed to hear no differences, and also know that volume is the usually the culprit when you do (See the quotes in my signature). I guess we’re just anxious to get past that and truly “control” for other variables.