Announcement. AES Presentation on “What is Accuracy” by our very own member @j_j_ or James D. (jj) Johnston - Chief Scientist - Immersion Networks

[j_j]

I have a big question! I can’t believe I missed this on the live stream.

You talked briefly about measuring accuracy of DACs, amps and all things that handle an audio signal where we can directly compare the input to the output.

But then you dropped a couple bombs! You seemed to say that while there are thresholds of objective measurements that can tell us a component is unequivocally transparent the reality is such components that measure to that degree of accuracy are in the minority to a degree that would surprise many of us.

And you seemed to say that typical measurements are not a good measure of accuracy. Something to the effect that we should burn the paper they are written on?

I say “seemed” because I don’t want to misrepresent what you said.

But my questions! (Yeah more than one). My amps, DACs, processors, ADCs, am I believing incorrectly that as modern devices designed and built for transparency that they are achieving transparency? Or….that there may be audible colorations that I don’t know about?

How do we know? How do we find out? Can we do the proper measurements ourselves?

If I am incorrect about the transparency about my audio signal chain I want to know about it and I want to know how to correct it.

Plot the ERROR SPECTRUM and compare it, using the response of the rest of your system past that device, to the actual absolute threshold.

There are a lot of amplifiers that still choke on tricky loads.
There are a lot of devices that have bad noise rejection.
A pure ThD rating at 1kHz and nothing else really, really misses a lot.

Just for example.

 

[voodooless]

Thanks @j_j, a really succinct.. dare I say accurate..? talk!

One thing related to the above though which I found a tad questionable: you said you could come up with certain IM signals that would be clearly audible, and therefore make a certain set of measurements not good enough. The issue I have with this is that for a certain set of equipment you know what kind of errors you may expect, so that is what you typically test for. But is it then realistic to expect the kind of error signals that you can artificially produce that are known to be audible?

Oh: what the ladder for?

@amirm maybe have the video on the front page?

 

[pma]

@amirm maybe have the video on the front page?

Yeah, much more important than many other videos.

 

[antcollinet]

Recording is available now there. Report will appear in a week, take or give, at www.aes.org/sections/pnw

Brilliant - will look forward to watching that later.


Now: Bike ride.

 

[j_j]

Thanks @j_j, a really succinct.. dare I say accurate..? talk!

One thing related to the above though which I found a tad questionable: you said you could come up with certain IM signals that would be clearly audible, and therefore make a certain set of measurements not good enough. The issue I have with this is that for a certain set of equipment you know what kind of errors you may expect, so that is what you typically test for. But is it then realistic to expect the kind of error signals that you can artificially produce that are known to be audible?

Oh: what the ladder for?

@amirm maybe have the video on the front page?

Well, to your first question, some IM signals at higher frequencies can lead to products right at the ear canal resonance, which makes them annoyingly audible. While this is not always a problem, it's a very real problem for some kinds of percussion and bells.

To the ladder? I really don't know. It was there in the lecture room behind the ping pong table. You might as well ask why that was there.

 

[voodooless]

To the ladder? I really don't know. It was there in the lecture room behind the ping pong table. You might as well ask why that was there.

I found it rather amusing to replace the real background with something like this . It's totally unnecessary and distracting. Or was it part of some elaborate sound experiment ?

 

[j_j]

I found it rather amusing to replace the real background with something like this . It's totally unnecessary and distracting. Or was it part of some elaborate sound experiment ?

I gave the talk in a classroom at Digipen. What you see is what you get, as it were.

 

[voodooless]

I gave the talk in a classroom at Digipen. What you see is what you get, as it were.

Nope, it was a Zoom background replacement You actually had a whiteboard in your background, as was later clear from the different camera angles.

 

[pma]

I found it rather amusing to replace the real background with something like this . It's totally unnecessary and distracting. Or was it part of some elaborate sound experiment ?

Irrelevant comments?

 

[voodooless]

Irrelevant comments?

Captain obvious?

 

[j_j]

Nope, it was a Zoom background replacement You actually had a whiteboard in your background, as was later clear from the different camera angles.

No, the white board and the camera on me went into a switcher. It was not a background replacement. Trust me on this, I was, after all, there. There were a couple of cameras, a direct feed from my laptop, and a camera that sometimes got me and the screen for the local people in it, and sometimes just me because I tend to walk about while talking. There was no background replacement.

When I've done other talks, the background has appeared to be Milford Sound, NZ. Now THAT was a replacement.

 

[voodooless]

No, the white board and the camera on me went into a switcher. It was not a background replacement. Trust me on this, I was, after all, there.

Ah, indeed, I see it now These blue side lights made for a good illusion though. Let's not distract more from the excellent talk

 

[Rick Sykora]

Thanks @j_j you confirmed what I already suspected about speakers. As we downsize, was planning to get rid of most of them anyway. Except the speakers I build of course! My wife will retire happier when I switch to headphones or a VR set. At least until she figures out I have really escaped into my own little concert.

 

[ahofer]

The story queued up here, about the 10/10 test based on JJ's hand movement, is worth mentioning to people who don't believe they are influenced by non-audible factors

 

[DDF]

I committed my career at one time to the scientific pursuit of audio fidelity but came to an aha moment decades ago that there is no absolute sound, no attainable reference.

I posted the following to rec.audio in ~ 1993. It's quoted from the brilliant late Richard Heyser who summed it up best:

"The physical soundfield which a modern sound reproduction system creates is definitely not congruent with the apparent soundfield which we hope the listener perceives. What a listener 'hears' is not a reconstructed hologram of a live performance. Instead he is subjected to a carefully contrived sound field which is intended to stimulate a a specific type of perception. Perceptual structuring is based on physical and emotional experience and is such as to align the majority of sensory experience at any moment with a consistent world-picture in our minds. The perception of sound involves much more than what we "hear'. It is a holistic experience that involves not only the senses, but past experience and present emotional state as well. I submit that what we are trying to do in today's technology is provide a particular type of listening experience under the limitations imposed by our ability to recreate a physical sound field...Once we recognize that the actual sound field in a listening environment is not identical to the sound field which we may perceive, we get a whole new perspective on the problem of being able to measure what we hear. IT IS THE ILLUSION OF REALITY, NOT THE REALITY ITSELF THAT WE MUST MEASURE"

I then went on to relate instances of how playback distortions can fool perceptions to believe the auditory experience is one of heightened reality and, critically, therefore can be good things. Examples included cartridge crosstalk enhancing soundfield perception, distortion providing perceptual clues that seem like enhanced dynamic range etc.

Everyone at that time thought this perspective was nuts. Including jj who vocalized perhaps the strongest disagreement.

Out of all the years and all the posts, I never forgot that thread.

 

[j_j]

I committed my career at one time to the scientific pursuit of audio fidelity but came to an aha moment decades ago that there is no absolute sound, no attainable reference.

I posted the following to rec.audio in ~ 1993. It's quoted from the brilliant late Richard Heyser who summed it up best:

"The physical soundfield which a modern sound reproduction system creates is definitely not congruent with the apparent soundfield which we hope the listener perceives. What a listener 'hears' is not a reconstructed hologram of a live performance. Instead he is subjected to a carefully contrived sound field which is intended to stimulate a a specific type of perception. Perceptual structuring is based on physical and emotional experience and is such as to align the majority of sensory experience at any moment with a consistent world-picture in our minds. The perception of sound involves much more than what we "hear'. It is a holistic experience that involves not only the senses, but past experience and present emotional state as well. I submit that what we are trying to do in today's technology is provide a particular type of listening experience under the limitations imposed by our ability to recreate a physical sound field...Once we recognize that the actual sound field in a listening environment is not identical to the sound field which we may perceive, we get a whole new perspective on the problem of being able to measure what we hear. IT IS THE ILLUSION OF REALITY, NOT THE REALITY ITSELF THAT WE MUST MEASURE"

I then went on to relate instances of how playback distortions can fool perceptions to believe the auditory experience is one of heightened reality and, critically, therefore can be good things. Examples included cartridge crosstalk enhancing soundfield perception, distortion providing perceptual clues that seem like enhanced dynamic range etc.

Everyone at that time thought this perspective was nuts. Including jj who vocalized perhaps the strongest disagreement.

Out of all the years and all the posts, I never forgot that thread.

Well, I would like to see exactly what parts of that I disagreed with. For some parts of the system, there is "absolute sound", in the electronic sense, in particular. However, even way back then, I will point out perceptual coding was a real thing (dating back to 1983/84) and the need was, and is, to learn what kind of reproductive system was needed to provide those perceptual cues.

PSR, in particular, was being born at that time. This was also written at a time when people were insisting vocally that we could not measure errors in a PCM system, amplifier errors, etc, and were mythologizing endlessly about the parts that absolutely CAN be measured. So some context is necessary here.

I also must point out that cartridge performances, etc, were absolutely things I was mentioning then in terms of euphonic distortions, so I'm a bit curious as to the actual reference, if you don't mind. In particular, it is absolutely false that I was arguing against the euphonic issues when I was in fact pointing out repeatedly, since long before whenever this was, that some people may prefer various distortions, and that was fine until presented as "the right way". So, something is off here.

 

[Sal1950]

I'm a bit curious as to the actual reference, if you don't mind. In particular, it is absolutely false that I was arguing against the euphonic issues when I was in fact pointing out repeatedly, since long before whenever this was, that some people may prefer various distortions, and that was fine until presented as "the right way".

Amen. !

 

[DDF]

Well, I would like to see exactly what parts of that I disagreed with. For some parts of the system, there is "absolute sound", in the electronic sense, in particular. However, even way back then, I will point out perceptual coding was a real thing (dating back to 1983/84) and the need was, and is, to learn what kind of reproductive system was needed to provide those perceptual cues.

PSR, in particular, was being born at that time. This was also written at a time when people were insisting vocally that we could not measure errors in a PCM system, amplifier errors, etc, and were mythologizing endlessly about the parts that absolutely CAN be measured. So some context is necessary here.

I also must point out that cartridge performances, etc, were absolutely things I was mentioning then in terms of euphonic distortions, so I'm a bit curious as to the actual reference, if you don't mind. In particular, it is absolutely false that I was arguing against the euphonic issues when I was in fact pointing out repeatedly, since long before whenever this was, that some people may prefer various distortions, and that was fine until presented as "the right way". So, something is off here.

The Heyser quotes come from from "Hearing vs. Measurement", Audio, March 1978. He may have published an AES paper on the topic.

I was involved in the work on the first perceptual codecs for wireless, this is something different. This concept also differs from euphonic distortion.

Your and other's disagreement then (and seemingly in this forum now) was that no reduction in objective performance was worthwhile, unless it was merely to target an individual preference. This perspective misses one crucial dimension but requires a bit of explanation.

Objective measures look for added impurities and so are useful. But they ignore the circle of confusion.

So we introduce preference ("I like it") such as in the Harman headphone and in-room loudspeaker response targets, for which objective measures may be derived. Statistically broad subjective mean opinion score results, for example, are used to determine these targets. Good so far.

The next evolution that I've been long advocating for (and that Heyser was on the nose about) is to measure the illusion of reality: "this sounds more real". This differs from preference: one may prefer a more realistic illusion, one may not. This would entail asking different questions than MOS, such as the famous "they are here" or "I am there" questions. I'm sure with some thought, better ones can be determined.

Preference explains some of why subjectivists and objectivists differ and bicker, but ignores the opinion of a large swath of well meaning subjectivists whose metric is instead the illusion of reality. Same as preference, such targets aren't "objectively" valid. But as human consumers of a delivered experience, such a goal, the heightened illusion of reality, is highly rational and desirable.

I suspect this is why Harman's in room response target curves differ for trained listeners vs. untrained listeners who desire more bass. Training confers the ability to detect with higher resolution and consistency. It doesn't necessarily confer preference. But I suspect those that undergo training have far more experience with live sound compared to untrained listeners, and so prefer less of the listening room bass boom in the presentation that untrained listeners are used to.

I don't think we can arrive at a simple set of design targets for optimizing the illusion of reality, as can sometimes be done for preference. I agree that our typical objective goals are a worthy starting point, as are the preference targets as secondary targets, but I posit they are not enough.

My point here is that we need to leave room for added deviations from our objective targets which may enhance the illusion of reality, on an individual basis. I pointed a couple out earlier, there are more. Again, this is different than preference. One may prefer an illusion of reality, or may not, as evidenced in Harman's preference curves for untrained listeners.

Of course, the next step is "what do you do with that"? That's where the fun starts: crossfeed algorithms, room mode cancellation algorithms, dispersion targets, DSP "enhancements" of all sorts such as used on the production side, etc. Lots of fertile feeding ground to explore.

I think this industry is stuck in a rut and I think chasing the illussion of reality can not only make the experience far more worthwhile, but bring a lot of fun back to this space. The illusion of reality is what I think we as engaged consumers of hifi are really after. And there's a big, necessary role for the objective community in all this, an opportunity that they're unfortunately ignoring on the playback side.

I predict that this perspective will continue to be a controversial, heretical viewpoint. It shouldn't be. I think it's rational.

 

[MattHooper]

Amen. !

To which I also say "Amen."

 

[j_j]

The Heyser quotes come from from "Hearing vs. Measurement", Audio, March 1978. He may have published an AES paper on the topic.

That's not what I asked for, I asked for the nutnews citation, as you well know. You've asserted that I am completely opposed to Heyser's work, which is flat-out false, although since he did not have a couple of decades of learning about the auditory system in hand, I rather think I have reason to disagree with some of the proposed sequence, etc.

I was involved in the work on the first perceptual codecs for wireless, this is something different. This concept also differs from euphonic distortion.

Really citations, please. If you've worked on this subject, who wrote "Pychoacoustic Model 2" in the MPEG-1 texts? I think I rather do know all about the difference between euphonic distortions and auditory masking, and since I'm quite sure I haven't confused them, what you're doing here is the "straw man" fallacy, putting words I did not use in my mouth.

Your and other's disagreement then (and seemingly in this forum now) was that no reduction in objective performance was worthwhile, unless it was merely to target an individual preference. This perspective misses one crucial dimension but requires a bit of explanation.

So in other words, I did not actually disagree in any fashion with Heyser. Yeah. Kind of figured how this would work out. There is no point to reducing the accuracy of measurements, simply one must be aware of WHAT they are measuring. Furthermore, your use of "merely" is inappropriate, and my position on what people prefer has been clear for many years now, and is exactly contrary to what you claim about my position. More straw man. Various of my objections are to processes that do not allow PROPER INVESTIGATION of those effects, to put it another way, to ACCURATELY understand the euphonic and preferential effects, so that they can be provided to a variety of preferences and listening practices. Again, you're making a straw-man claim that appears based on an agenda.

Objective measures look for added impurities and so are useful. But they ignore the circle of confusion.

You can't claim that until you know what the confusion is UNTIL YOU MEASURE IT. What's more, the confusion you allege most often is across listeners. You might more ACCURATELY claim that the variance in listener preference must be accommodated. Yeah, which must be investigated in order to be able to accommodate, and the investigation, again, MUST BE ACCURATE.

Preference explains some of why subjectivists and objectivists differ and bicker, but ignores the opinion of a large swath of well meaning subjectivists whose metric is instead the illusion of reality. Same as preference, such targets aren't "objectively" valid. But as human consumers of a delivered experience, such a goal, the heightened illusion of reality, is highly rational and desirable.

Yeah, like my current work on what some people call "immersive" audio, you mean.

But I suspect those that undergo training have far more experience with live sound compared to untrained listeners, and so prefer less of the listening room bass boom in the presentation that untrained listeners are used to.

I have some news for you. Live sound people listen differently than recording engineers who have tons of experience, too. Been there, measured those.

I don't think we can arrive at a simple set of design targets for optimizing the illusion of reality, as can sometimes be done for preference. I agree that our typical objective goals are a worthy starting point, as are the preference targets as secondary targets, but I posit they are not enough.

Since different people very obviously listen to different aspects of the sound in a room (or headphones), I think the way to go about it is to ensure that the perceptual cues are there for anyone who's listening, and let their own brain cope further. And they do. Note, this does not mean "accurate" in the LTI sense. How one measures is key, and in modern times, not that hard for most examples.

And, yes, the industry itself has been in a rut since the (cough) astoundingly brilliant (cough) <that's sarcasm markers> of using the rather much worse 4 channel setup of L/R/RL/RR for "quad" (as opposed to L/C/R/Back> despite the evidence from Steinburg and Snow's 1933 work. Yes, really. The move from mono to stereo ignored complaints about distance perception that are well understood in the present day. The move to 4 channel maintaining the "phantom center" ignored that, and the understanding of spatial perception (which was rather primitive at that time due to adherence to the thoughts of the day) was both completely missing AND the mixes, which removed a very, very important cue from the end result, created the misbegotten idea that "there is only one listening position" which is demonstrably false given a proper mixing algorithm. (even for quad, but we need to dump that idea forever, please!).

So, why your whole miscostrued attack on me with that claim I vocally rejected Heyser's words?

Yes, you CAN measure an individual's preference, given enough work, although the learning involved may change an individual's preference, and the kind of coffee they had in the morning, likewise, can change preference.

What you can also do is ensure that the various cues provided to a listener are delivered to the listener are ACCURATELY ENOUGH DELIVERED to provide those cues, make them work appropriately so that the periphery can capture the cues at appropriate levels. Then you let the brain do its work. Yes, it's another form of 'coding', if you will.