Volume Matching JND

[John Kenny]

I know that Fletcher-Munson is used quite a lot on audio forums & taken to be gospel according to many but this forum is hoping to rise above the normal forum standards & apply some real science to what may appear simple questions. Sometimes, by questioning the "received wisdom", we discover & uncover new information.

With that in mind can I introduce a recent (2014) paper "Development and Current Status of the “Cambridge” Loudness Models"
that opens up the whole area of loudness perception & the various models that are current, their advantage, shortcomings & future work needed.

Here are the conclusions "This article has reviewed the development of a series of loudness models developed in Cambridge. The models draw on earlier work of Fletcher, Zwicker, and their coworkers (Fletcher & Munson, 1933; Zwicker & Scharf, 1965). The models have been developed over time to improve their accuracy and applicability, to take into account new data on loudness perception, and to allow the models to predict loudness perception for listeners with hearing loss. The models have found wide applicability in industry for prediction of the loudness of everyday sounds such as noise from aircraft, traffic, inside cars, heating and ventilation systems, wind turbines, and many other situations. They have also been used by government organizations, for example, in predicting the loudness and audibility of potential warning sounds for electric vehicles (Hastings, Pollard, Garay-Vega, Stearns, & Guthy, 2011). The models applicable to impaired hearing have been used in the development of widely used methods for fitting hearing aids (Byrne et al., 2001; Keidser et al., 2011)."

It has some interesting areas that we should all look into such as
- models which incorporate time varying sound (such as speech or music)
- models which account for binaural loudness summation (for stereo sound)

I would welcome other people's view on this paper & the information within it

 

[AJ Soundfield]

With that in mind can I introduce a recent (2014) paper "Development and Current Status of the “Cambridge” Loudness Models"

That doesn't look (preferred??) anecdotal-Pink fish media, etc. based at all. It reeks of audio science, with some rather alarming words:

The models presented here can be considered as characterizing loudness perception under conditions where context and visual cues play a minimal role.

How would that support not level matching an electronic component like an amplifier or DAC with (presumably) flat response 20-20k +/- 0.5db???
And instead doing zero level matching and including conditions where context and visual cues play a minimal maximal role.?

 

[SoundAndMotion]

That appears to be a acoustics science journal, not an audiophile one. Are the results you're hanging your hat on, anecdotal in nature? Or where they obtained via controlled listening tests?
Not a member like you, so please clarify what methods they used, what were the quality checks, how they were proctored, etc, so that we know the results are rigorously science valid.

Hi AJ, I hope no one accepts your invitation to jump in the mud.
I'm here to invite you out! Yes, the Joshi paper in JASA uses controlled listening tests. FYI JASA has a higher JCR impact factor than JAES (only slightly, but it's not a rinky-dink journal) They have many auditory neuroscience and psychoacoustic articles. Check it out.
Three experiments are described, each using classic psychophysical methods. The first 2 use 2IFC and the third is a matching protocol. For Expt 1 they use the method of constant stimuli (meaning a fixed set, in this case 10 blocks of 100 trials). Expt. 2 uses an adaptive method (up-down, apparently 1 up 1 down; I prefer other more robust patterns) to fit a psychometric function. And Expt. 3 uses the matching protocol I mentioned. Perhaps I can produce a digest later this week, if there is interest. I hesitate to include too much copyrighted material, but here is a sample from the methods of Expt. 1:
C. Methods

Listeners were seated in a double-walled sound attenuat- ing booth. Stimuli were generated using 24-bit digital-to- analog convertors (Digital Audio Labs, CardDeluxe) and presented monaurally to the left ear through Sennheiser 250- HD headphones.

In a two-interval forced choice (2IFC) task, listeners were asked to pick the interval that was louder. For the “signal” interval, the mean of the level distribution for the individual components was 48 dB SPL (Overall mean level of 58dB); for the “non-signal” interval it was 45dB SPL (Overall mean level of 55 dB). A visual description of a sam- ple trial is shown in Fig. 1(B). The two intervals, each 500- ms long, were separated by a 500-ms inter-stimulus-interval. Each trial began with a 500-ms light emitting diode (LED) light on a response box indicating the beginning of the trial. Separate lights to cue the listener highlighted each interval. There was no gap between the initial warning light offset and the onset of the light indicating the beginning of the first interval. Stimulus onset was simultaneous with the onset of the interval lights. Although each trial could be marked with a “correct” and “incorrect” interval, listeners received no feedback regarding their response. Data were collected in blocks of 100 trials and appropriate breaks were given to the listeners. Each listener completed 1000 trials for each of the two ten-tone complexes. Blocks for the two ten-tone com- plexes were run in a random order.

Cheers,
SAM

 

[AJ Soundfield]

Hi AJ, I hope no one accepts your invitation to jump in the mud.

Yet you keep mentioning mud, not I...

Yes, the Joshi paper in JASA uses controlled listening tests.
The first 2 use 2IFC and the third is a matching protocol.
Listeners were seated in a double-walled sound attenuat- ing booth.
etc, etc.

So how are we to reconcile hanging ones hat on this type of rigorous audio science test results, with an openly stated preference for pure anecdote?
How would these sort of JND results support dismissing any kind of voltage measurement (using wideband vs single tone if desired) for say, a DAC...and instead prefer/rely on zero controls anecdotal internets "reports"??

 

[SoundAndMotion]

Yet you keep mentioning mud, not I...

So how are we to reconcile hanging ones hat on this type of rigorous audio science test results, with an openly stated preference for pure anecdote?
How would these sort of JND results support dismissing any kind of voltage measurement (using wideband vs single tone if desired) for say, a DAC...and instead prefer/rely on zero controls anecdotal internets "reports"??

Oh well... I was hoping to get you to discuss science or audio or audio science instead of people... didn't work. When you engage in personal stuff (or being "rude and snide" as you seem to defend in another forum), I call it mud wrestling and I'm not interested.
I haven't yet read the Brian CJ Moore article a couple of posts up. Maybe we can discuss it after we've both read it. He has been quite prolific in the area of loudness perception modeling, writing many articles about it...looks interesting, perfect for this thread.

 

[NorthSky]

I know the "received wisdom" is that blind tests require volume matching to within 0.1dB for a valid test otherwise the "louder" device/track will be perceived as of better quality & not perceived as louder. My question is where are the blind test results which support this claim? Are there any blind test results with music as the signal (this is what we are talking about in most blind testing) that show level differences > 0.1dB are perceived differentially?

Now in my researching this topic I came across this Audioholics article which seems quite sensible in it's approach & does as survey of the research papers on the topic

In it a chart of research results over 60 years is given & the lowest JND is 0.25dB F.E. Toole and S. Olive, "The Modification of Timbre by Resonances: Perception and Measurements", JAES vol 36, # 3, March 1988, pp 122-142

The author states "Toole and Olive, on the other hand, in their 1988 study used pink noise for their acoustic signal source and determined that a 5 kHz resonance, with Q = 1 was just detectable at .25 dB." But later "The .25 dB figure quoted from the Toole & Olive research seems to contradict this (his JND of 0.75dB or 1dB), but consider the filter Q = 1. That's a pretty broad chunk of the audible spectrum over which that resonance exists. With the ear-brain combo performing an integration across that broad a portion of the audible spectrum than its easy to see how a large amount of acoustical energy is captured, leaving a change that small noticeable. However, pink noise is not real world and one thing my research has shown is that the hearing process reacts very differently to different types of sound; a .25 dB detectable difference using real music just isn't plausible and the research supports that."

He goes on to state "In this particular article, I settled on a minimum discernable difference dB value of .75 - 1.0. My experience has shown that this is what the average listener, under average listening conditions, listening to music played back through typical consumer-grade audio gear will be able to clearly identify - and do so repeatedly."

Now loudness is a psychoacoustic phenomena i.e intensity + auditory processing -> perception of loudness so some factors relating to this. The frequency filterbank (ERBs) that we split the sound into & analyse over is instrumental in how we perceive loudness. In other words, a single tone whose frequency falls into one filterbank, will be perceived as quieter than a wider band signal that spans more than one frequency filterbank, given that both are the same intensity (dB).

A consequence of this is that we are more sensitive to broadband noise intensity than to single tones & hence there is a curve of noise sensitivity called ITU-R 468 which defines our sensitivity to noise intensity & it is different to the Fletcher-Munson curves. here's the comparison which shows that we are more sensitive to noise @ 6KHz frequency than to a single tone at this frequency by > 12dB as can be seen here:

So, my questions again
- what evidence do we have that intensity mismatches of > 0.1dB will be perceived as quality issues with music signal
- what is the JND for amplitude differences with music signal?

This ↑ is the topic, and not the people.

Methinks that a preamp with a good volume control, and a good amp too, and the speakers; if they all can play beautifully @ low volume level, with almost zero distortion, is a great great audio attribute. ...And then we climb the ladder in raising the volume @ the best listening level where we're comfy with.
And it depends...of the music recordings, and our room's dimension.

Small increments from a good pot volume control is good, it allows for more precision play.
I don't know our perception in 0.1dB level increments though; if they affect in the sense of better sound reproduction...more pleasing.
I guess it's different for each individual, and not easy to define generally.
And the music genre playing is even more affecting than the small increments difference in volume level...I think.

Still, my first thought above remains. ...Good sound reproduction @ low volume listening levels.
Can a quality exist in that range of say between 0.001 and 0.01 watt? ...And then from 0.01 to 0.1 and 1 watt?

 

[AJ Soundfield]

How would these sort of JND results support dismissing any kind of voltage measurement (using wideband vs single tone if desired) for say, a DAC...and instead prefer/rely on zero controls anecdotal internets "reports"

Oh well... I was hoping to get you to discuss science or audio or audio science instead of people.

Yeah I get so confused especially when I don't follow the conversation, like how this thread sprung directly from what I said
http://audiosciencereview.com/forum...hat-is-measurable-thread.214/page-5#post-6152
regarding
http://audiosciencereview.com/forum...fshoot-from-the-what-is-measurable-thread.214
Silly me.
So SAM, please pull us back into discussing audio by explaining how the results of these non-anecdotal research papers, would negate the need for level matching in audio difference (preference?) "tests" like the above cited one for unmeasurable "soundstage depth".
TIA

 

[Blumlein 88]

I know that Fletcher-Munson is used quite a lot on audio forums & taken to be gospel according to many but this forum is hoping to rise above the normal forum standards & apply some real science to what may appear simple questions. Sometimes, by questioning the "received wisdom", we discover & uncover new information.

With that in mind can I introduce a recent (2014) paper "Development and Current Status of the “Cambridge” Loudness Models"
that opens up the whole area of loudness perception & the various models that are current, their advantage, shortcomings & future work needed.

Here are the conclusions "This article has reviewed the development of a series of loudness models developed in Cambridge. The models draw on earlier work of Fletcher, Zwicker, and their coworkers (Fletcher & Munson, 1933; Zwicker & Scharf, 1965). The models have been developed over time to improve their accuracy and applicability, to take into account new data on loudness perception, and to allow the models to predict loudness perception for listeners with hearing loss. The models have found wide applicability in industry for prediction of the loudness of everyday sounds such as noise from aircraft, traffic, inside cars, heating and ventilation systems, wind turbines, and many other situations. They have also been used by government organizations, for example, in predicting the loudness and audibility of potential warning sounds for electric vehicles (Hastings, Pollard, Garay-Vega, Stearns, & Guthy, 2011). The models applicable to impaired hearing have been used in the development of widely used methods for fitting hearing aids (Byrne et al., 2001; Keidser et al., 2011)."

It has some interesting areas that we should all look into such as
- models which incorporate time varying sound (such as speech or music)
- models which account for binaural loudness summation (for stereo sound)

I would welcome other people's view on this paper & the information within it

I haven't read it all. I am familiar with some fair bit of the background they are using. I too wonder, what part of this relates to your idea sloppy volume matching doesn't interfere with comparative listening for differences?

There is much interesting in it. Like where they show our ear only somewhat incidentally responds to sounds below 50 hz and above 15 khz.

Again, where does this fit with the idea of 2-3 db matches are good enough with music? The paper would be a good jumping off point for a new thread about what is loudness in subjective perception.

 

[amirm]

Yeah I get so confused especially when I don't follow the conversation, like how this thread sprung directly from what I said
http://audiosciencereview.com/forum...hat-is-measurable-thread.214/page-5#post-6152
regarding
http://audiosciencereview.com/forum...fshoot-from-the-what-is-measurable-thread.214
Silly me.
So SAM, please pull us back into discussing audio by explaining how the results of these non-anecdotal research papers, would negate the need for level matching in audio difference (preference?) "tests" like the above cited one for unmeasurable "soundstage depth".
TIA

He has done that AJ. I am at a loss to see your contributions in reverse. Level matching is a complex field when it comes to speakers and room acoustics. Do you have any technical information you want to contribute?

 

[AJ Soundfield]

He has done that AJ.

Nothing of the sort. The only thing SAM did was confirm the cited paper was anything but anecdote, the exact opposite of audiophile preference.

I am at a loss to see your contributions in reverse. Level matching is a complex field when it comes to speakers and room acoustics.

This

has zero to do with speaker level matching. It's the same speaker!
What I said had to be matched, was the T+A voltage outputs for inputs A and B. John, like you at one time, insists level matching is non-critical to this sort of audio "test", especially with music.
Hence here we are.....

 

[Blumlein 88]

Here are some more articles for you.

http://www.ingentaconnect.com/content/ince/ncej/2008/00000056/00000006/art00001

I had this complete paper and will try to find the link to it again. In this case the issue was JND's for various aspects of refrigerator noise. They determine with a 2ifc methods that for loudness it was .5 db using 40 test subjects.

More interesting are other aspects of noise much based on work by Zwicker as well.
Vacil is a measure of roughness.
Basic explanation here:
http://www.salford.ac.uk/computing-...uality-testing/roughness-fluctuation-strength

Two others are sharpness and booming measured in acum and asper. Basic explanation here:
http://www.salford.ac.uk/computing-...etter/sound-quality-testing/sharpness-booming

Lest you think this not a big deal, a friend was a compressor and cooling system engineer for a large fridge company a decade or so ago. He was tasked with and succeeded in making the most efficient refrigerator compressor on the market at the time. They declined to make it because it failed their noise testing slightly. The fridge maker had a well instrumented anechoic chamber most audio companies would be highly envious of having. With some mods and testing he ended up with the second most efficient compressor which also was the second quietest and that design they used.

I am sorry Jk none of this will lend any support to sloppy level matching.

https://www.head-acoustics.de/downloads/eng/application_notes/Psychoacoustic_Analyses_II_e.pdf
A paper on a software suite that can use Zwicker modeling to predict issues with sounds amongst noise. You can look for support of sloppy matching. Might work if you want your hifi in a convertible with the top down.

 

[amirm]

Nothing of the sort. The only thing SAM did was confirm the cited paper was anything but anecdote, the exact opposite of audiophile preference.

What plot are you on AJ? A paper was presented on the topic of this thread. You said you didn't have a subscription to ASA (shame on you ), and so Sam provided a synopsis of the methodology:

Hi AJ, I hope no one accepts your invitation to jump in the mud.
I'm here to invite you out! Yes, the Joshi paper in JASA uses controlled listening tests. FYI JASA has a higher JCR impact factor than JAES (only slightly, but it's not a rinky-dink journal) They have many auditory neuroscience and psychoacoustic articles. Check it out.
Three experiments are described, each using classic psychophysical methods. The first 2 use 2IFC and the third is a matching protocol. For Expt 1 they use the method of constant stimuli (meaning a fixed set, in this case 10 blocks of 100 trials). Expt. 2 uses an adaptive method (up-down, apparently 1 up 1 down; I prefer other more robust patterns) to fit a psychometric function. And Expt. 3 uses the matching protocol I mentioned. Perhaps I can produce a digest later this week, if there is interest. I hesitate to include too much copyrighted material, but here is a sample from the methods of Expt. 1:
C. Methods

Listeners were seated in a double-walled sound attenuat- ing booth. Stimuli were generated using 24-bit digital-to- analog convertors (Digital Audio Labs, CardDeluxe) and presented monaurally to the left ear through Sennheiser 250- HD headphones.

In a two-interval forced choice (2IFC) task, listeners were asked to pick the interval that was louder. For the “signal” interval, the mean of the level distribution for the individual components was 48 dB SPL (Overall mean level of 58dB); for the “non-signal” interval it was 45dB SPL (Overall mean level of 55 dB). A visual description of a sam- ple trial is shown in Fig. 1(B). The two intervals, each 500- ms long, were separated by a 500-ms inter-stimulus-interval. Each trial began with a 500-ms light emitting diode (LED) light on a response box indicating the beginning of the trial. Separate lights to cue the listener highlighted each interval. There was no gap between the initial warning light offset and the onset of the light indicating the beginning of the first interval. Stimulus onset was simultaneous with the onset of the interval lights. Although each trial could be marked with a “correct” and “incorrect” interval, listeners received no feedback regarding their response. Data were collected in blocks of 100 trials and appropriate breaks were given to the listeners. Each listener completed 1000 trials for each of the two ten-tone complexes. Blocks for the two ten-tone com- plexes were run in a random order.

Cheers,
SAM

WHat do you mean "nothing?" If you want to know more about the paper, ask him or I politely and we can quote more. Don't keep battling something against "audiophiles." Posts should contribute to the topic of the threshold of detection with respect to level differences. Level matching is the heart of blind testing so we need to increase our knowledge there, rather than fighting personal battles.

 

[John Kenny]

I haven't read it all. I am familiar with some fair bit of the background they are using. I too wonder, what part of this relates to your idea sloppy volume matching doesn't interfere with comparative listening for differences?

I'm sorry you have to personalise discussions in this way - it's not my "idea sloppy volume matching" - I was summarising the posts up to that point & the views expressed in them. As I said at the beginning, I'm interested in the studies that establish the JND for volume matching. There are some good reports from people here who report having done some of this JND testing & that is great info.

As Amir has mentioned already on this forum, it's in trying to answer questions for oneself that research uncovers some info that one wasn't aware of before & this is what I'm discovering here. I knew & mentioned before that using tones for establishing listening curves/thresholds was probably not appropriate to use as the bible as far as real-world listening to real-world sound. I already posted a different threshold curve for noise audibility & was aware that binaural or dichotic listening may well provide different results. A new aspect I wasn't aware of was the possible differences in perception of loudness with amplitude modulating signals or time-varying signals as they're called in this paper

There is much interesting in it. Like where they show our ear only somewhat incidentally responds to sounds below 50 hz and above 15 khz.

Again, where does this fit with the idea of 2-3 db matches are good enough with music? The paper would be a good jumping off point for a new thread about what is loudness in subjective perception.

Don't get hung up on 2-3dB matches, please - it was just my summary of posts as I said above.
Yes, please start another thread along those lines you mention

 

[John Kenny]

Here are some more articles for you.

http://www.ingentaconnect.com/content/ince/ncej/2008/00000056/00000006/art00001

I had this complete paper and will try to find the link to it again. In this case the issue was JND's for various aspects of refrigerator noise. They determine with a 2ifc methods that for loudness it was .5 db using 40 test subjects.

Yes, I posted the noise threshold curve ITU-R 468 before which shows that @ about 6KHz it has a lower threshold by about 12dB "The CCIR curve differs greatly from A-weighting in the 5 to 8 kHz region where it peaks to +12.2 dB at 6.3 kHz, the region in which we appear to be extremely sensitive to noise. "

More interesting are other aspects of noise much based on work by Zwicker as well.
Vacil is a measure of roughness.
Basic explanation here:
http://www.salford.ac.uk/computing-...uality-testing/roughness-fluctuation-strength

Two others are sharpness and booming measured in acum and asper. Basic explanation here:
http://www.salford.ac.uk/computing-...etter/sound-quality-testing/sharpness-booming

Thanks I came across the Vacil link before but not the other link

Lest you think this not a big deal, a friend was a compressor and cooling system engineer for a large fridge company a decade or so ago. He was tasked with and succeeded in making the most efficient refrigerator compressor on the market at the time. They declined to make it because it failed their noise testing slightly. The fridge maker had a well instrumented anechoic chamber most audio companies would be highly envious of having. With some mods and testing he ended up with the second most efficient compressor which also was the second quietest and that design they used.

I am sorry Jk none of this will lend any support to sloppy level matching.

Can you leave out this personalisation of "sloppy level matching"?

https://www.head-acoustics.de/downloads/eng/application_notes/Psychoacoustic_Analyses_II_e.pdf
A paper on a software suite that can use Zwicker modeling to predict issues with sounds amongst noise. You can look for support of sloppy matching. Might work if you want your hifi in a convertible with the top down.

And again please leave out the sloppy matching comments.

 

[AJ Soundfield]

What plot are you on AJ? A paper was presented on the topic of this thread.

Right, a(nother) paper that did zero to support Johns argument that level/volume matching is irrelevant, more so with music signals
Actually quite the opposite, much like the very funny timbre theory.

Level matching is the heart of blind testing so we need to increase our knowledge there, rather than fighting personal battles.

We???
Some of us are quite clear on that. Others, not so much.

 

[John Kenny]

I do seem to remember Amir using a technique where he didn't level match two sources but did a blind preference test & then turned up the "worse" one to compensate for any bias that it might have been "worse" because it was lower in volume.

Is there anything wrong in this approach?

 

[Blumlein 88]

So JK what are trying to learn? Do you need matching to .1 db? No one has seen indications if you do that you will have problems with the match corrupting results.

If you leave a room and come back later maybe 2-3 db will be required to be JND with music. Of course that kind of testing has numerous other issues even beyond the level match to be relied upon. Echoic memory means listening comparisons done without relatively quick switching suffer in discernment levels. Are we trying to determine if people listen over longer time with big level differences and varying background noise levels they can hear important differences? If they think so, the obvious next step is to do a bit more disciplined listening to see if it holds up.

So what is your goal here in this thread?

 

[amirm]

We???
Some of us are quite clear on that. Others, not so much.

No wonder there is so much confusion then if you think it is that clear. Here is the frequency response of half a dozen Room EQ systems per Harman:

Tell us how you would level match these to compare them to each other.

 

[John Kenny]

So JK what are trying to learn? Do you need matching to .1 db? No one has seen indications if you do that you will have problems with the match corrupting results.

If you leave a room and come back later maybe 2-3 db will be required to be JND with music. Of course that kind of testing has numerous other issues even beyond the level match to be relied upon. Echoic memory means listening comparisons done without relatively quick switching suffer in discernment levels. Are we trying to determine if people listen over longer time with big level differences and varying background noise levels they can hear important differences? If they think so, the obvious next step is to do a bit more disciplined listening to see if it holds up.

So what is your goal here in this thread?

You only have to go back to the o/p to see two questions I posed:

So, my questions again​

- what evidence do we have that intensity mismatches of > 0.1dB will be perceived as quality issues with music signal
- what is the JND for amplitude differences with music signal?

These were the two questions to get the ball rolling but I'm happy to discuss all issues related to the subject except the ones where I'm being put on trial as "the accused"

 

[Blumlein 88]

You only have to go back to the o/p to see two questions I posed:

So, my questions again​

- what evidence do we have that intensity mismatches of > 0.1dB will be perceived as quality issues with music signal
- what is the JND for amplitude differences with music signal?

These were the two questions to get the ball rolling but I'm happy to discuss all issues related to the subject except the ones where I'm being put on trial as "the accused"

Okay, you have Don's input on the matter. I have done my own tests that might not make it to .2 db, they do make it to .25 db.

There is this:
https://archive.org/stream/pdfy-ZGJ4B3_VGwVs9CCt/Can-You-Trust-Your-Ears_djvu.txt

One of the items is a test with 31 people half of two 30 second bits of music are level matched, and half aren't. The unmatched ones are 1 db louder. No one picked up on a loudness difference with music. 75% described a quality difference.

So is it .1 db or is it 1 db? It is somewhere in that range. Since there are now a few instances relayed to you how people hear a difference, ascribe it to quality and don't perceive the difference is loudness the case is on solid ground that at some point below where listeners hear a loudness difference they do hear a quality difference. Maybe you don't have certainty it is just above .1 db, but it is at or below 1 db. It indicates matching by ear to perceived volume equality leaves a gap of some size where you still hear quality differences even if none are there. So that should make one wary of level matching by ear. As you at that point have to do some kind of measuring even if only with a multi-meter you might as well do it to .1 db. If someone only does it to a .25 db it might be worth looking into. I wouldn't give much credibility to more than that myself as some tests of JND with complex signals show a threshold of only .5 db. So that question is pretty well answered for you.

The JND for music is not quite as firmly established. The general trend in audio testing is listening tests with appropriate artificial signals is more discerning than with music. If you are striving for much rigor you wouldn't use the loosest criteria. So if the JND for music is 3 db, I wouldn't accept listening tests about much using matching no better.