Research Project: Infinity IL10 Speaker Review & Measurements

[echopraxia]

But here an assumption is made before the data was gathered bases only on an alleged skill of th listener.

Can you show where anyone here is making an assumption (that THD/IMD is to blame), as opposed to a hypothesis? The distinction between assumption and hypothesis is very important -- and I've only observed the latter here so far. For example, right from the very beginning in Amir's first post:

Now, there are two possibilities here:

1. Placebo effect and the M16 being similar to IL10.

2. My brain is forever transformed to listen to distortions in small notes. Right after tonality, this is what I listen for in speakers.

Conclusions
There are none as yet. I expect this to be a living research thread where we discuss what we have found here, and whether we can better rationalize speaker preference from measurements. The preference score for this speaker will be high (@MZKM will post shortly) putting me once again at odds with it.

Amir is NOT just making assumptions here before additional data is gathered. He is making a list of hypotheses (with a '...' at the end, inviting us to discuss other ideas), where the first hypothesis he states is placebo!

So I do not understand where you are getting this idea that assumptions are being made about distortion.

The only practical assumption we are making here is that we should not outright reject empirical data just because it disagrees with existing models: Discarding all data that does not fit your expectation is a deeply unscientific approach/perspective.

 

[Newman]

This entire thread / discussion is operating on the assumption that @amirm's results are correct (that one speaker sounds much worse than the other), which therefore implies that the speaker test protocol is not adequate when used as intended (unless we can formulate a new preference model from the existing measurements that is a better overall fit to Amir's subjective preference results).

If you wish to continue to debate whether or not we should use this operating assumption (that Amir's subjective results are trustworthy), I would ask that you kindly take that debate elsewhere. That debate has consumed the last 20 or so pages of this thread, and has lead to virtually no progress here other than the same disagreements repeated over and over and over and over. We are all exhaustively aware now of the opinions on both sides of that debate. Please, let's move on.

The 'debate' which you want to shoo away, in favour of an 'operating assumption' that the least likely explanation of an anomaly is the true one, is not a debate: it is a denial that the most likely explanation is just that. The only reason it has taken 20 pages so far is down to the deniers.

And how you managed to package my comment #574 into that debate, is most spurious. You raised an hypothesis, and I said it contains a logical flaw. Totally on topic.

But I'll suggest this: when this thread is completely played out, long after all the 'debate' of the last 20 pages has calmed down and those wishing to explore the 'operating assumption' have explored it over and over until they are in fact the ones going in circles, I have a very strong expectation that this is what you will have to show for it: nothing. Or, to put it in the preferred terms of such investigations: "further research is needed".

And when you all get to that point, I hope one of you has the intellectual honesty to admit that the 'operating assumption' was more of a 'suspension of disbelief', really, and that the early doubters were really invoking Occam's Razor. Namely, when one detects an anomaly between a gold standard measurement and a bronze standard listening test, the most likely explanation -- and the first thing that needs to be addressed -- is validation of the listening test with a gold standard listening test. Bronze standard is good -- it's not rubbish by any means -- but it isn't the gold standard and that is the obvious first port of call.

cheers

 

[echopraxia]

There's only so many times you can repeat over and over your doubts of the trustworthiness of Amir's test; therefore, this is why I claim we need to make a simplifying assumption to make any progress here. It's clear there will not be a binary consensus on whether we should (0) disregard Amir's subjective evaluation or (1) trust Amir's subjective evaluation. Since there will not be a binary consensus here, we need to qualify our discussions within the context of operating assumptions.

I have no problem if you wish to discuss topics under operating assumption (0), so long as you have something new or noteworthy to say there. I just think all that can be said on that topic, has been said. For example, your post above is just one more useless repetition of what has already been said 100 times in this thread + lots of ad hominem (accusing me of being a science denier, accusing me and many others here of intellectual dishonesty, etc.).

My post here was made after checking all 30 pages of this thread to make sure nobody brought up the topic of audio compression,as an analogy to the existence of nonlinear distortions (or as an example to illustrate how even THD and IMD are insufficient in some cases, like the AAC codec). I think this is an interesting point to explore, e.g. is such a case known to be theoretically possible in physical systems? If so, why? If not, why not? Etc.

Can you say your posts do the same due diligence to avoid repetition? Whatever operating assumption you wish to go on about Amir's subjective results, please at least do your part to keep us from going in circles endlessly debating something that there will clearly be no consensus on without further data.

So I propose we focus on discussing what kind of data we can capture, how we can capture it realistically, and how we can analyze it. For example, is there a good way I can measure IMD on my speakers? Is this something I can do with REW? I'm not aware of such a feature. What is the easiest / most accessible way for people to perform such tests?

 

[edechamps]

How can we measure all the physically possible distortions that are audible, e.g. is THD + IMD enough, or do we need more than that (as is clearly the case with lossy compression algorithms, for example)?

You are correct that the problem of arbitrary (non-linear) distortion is not measurable in the general case. That is, one could very well design an active speaker with a very clever DSP which looks for a very specific pattern in the input signal, and if it sees it, substitutes the input signal with a copy of Justin Bieber's Baby stored in the DSP's memory. An infinitely pedantic person will successfully argue that this does count as distortion, because it's an alteration of the signal that is related to the input (since it was triggered by it). I'm sure everyone here will agree that would be distortion of the most evil, most offensive kind, yet you'd be hard-pressed to find any kind of standard measurement suite that will detect it. What you showed with that lossy digital compression demo (which I found quite neat, by the way) illustrates the same basic problem.

This issue arises because we treat the Device Under Test as a complete "black box" with no known properties. In order to make progress we need to understand the kind of distortion the DUT is expected to produce (which requires relaxing the "black box" assumption) and test specifically for these. So, for example, in the case of speakers, that would be stuff like force factor, suspension stiffness, thermal effects, Doppler effect, turbulence, rub & buzz, etc. basically, we deduce the kinds of distortion we should expect by working from first principles and looking at the physical behaviour of the speaker, and/or audible issues that people typically object to in listening tests. (Note that this is not an exact science, sadly, because it's impossible to know with 100% accuracy what exactly is physically going on inside the speaker in a particular situation, nor to design listening tests that will shine light on every potential problem.)

Once we know what kinds of distortion we should test for, then we can come up with stimuli specifically designed to trigger such distortion, and we measure the response to these stimuli. (This part is not necessarily trivial either. For example, measurement of impulsive distortion such as rub & buzz requires very specific kinds of stimuli and analysis.)

Klippel wrote a large number of papers that go into the various (known) kinds of speaker non-linear distortion in a truly excruciating amount of detail. There are also freely available webinars that are being organized as we speak where Wolfgang Klippel himself gives lectures on these topics. The papers are very technical, so it's not for the faint of heart. The webinars are more accessible and quite interesting but they go on for hours and hours. I'll admit I don't understand most of these issues myself, and I wouldn't expect anyone to unless they're very familiar with speaker design. To give you an idea of what to expect, this is just a summary slide before a grinning Wolfgang Klippel proceeds to drown you in a sea of technical detail:

So the only question that we can meaningfully discuss (until we gather more data, e.g. IMD measurements or beyond) is whether it is theoretically possible for a speaker driver to have greater IMD than its THD chart would suggest.

Yes. For example Doppler effect induced distortion cannot be induced from a single-tone (THD) stimulus, you need at least two tones. If I remember correctly there are few other cases as well. Let's bring Wolfgang here again:

In fact, on the topic of non-linear distortion, I remember at some point (not sure when) Wolfgang said something along the lines of "if you're only going to do one test, it should be Multi-Tone Distortion, that will give you the most information" (as opposed to a single-tone THD test which is what @amirm is currently doing). Of course, how to relate the results of such a test to perception of sound quality is another matter entirely.

The webinars are structured around a new international standard for the measurement of speakers, IEC 60268-21, published in 2018, that seems to cover a wide variety of stuff, including linear/small-signal and non-linear/large-signal measurements. That's probably the way to go for consistent, reproducible non-linear distortion measurements. I haven't looked at the standard myself though.

 

[QMuse]

My point would still stand. I claim no more than this: We have trustworthy empirical data (from multiple sources now) indicating that there is something the Olive Preference Score (and even the CEA-2034) is missing.

I'm not sure that can be called trustworthy empirical data, but certainly some suspicions have been raised, not only on behalf of role of the distorion but on the role of greater importance of the horizontal reflections. If possible, both of those things should be addressed. But again, I really suggest no claims are made before other things are rulled out.

My way of contributing here is via the blind tests I've performed [1] [2]. I understand the (perpetual) criticism that the sample size is too small. But you have to understand, when all you have to offer is such criticism, you are creating a self-fulfilling prophecy: If we never perform any experiments, out of fear of the sample size being too small, we are only ensuring the sample size will perpetually remain small

It is not my intention to offer criticism but to offer an opinion. And in my opinion, before making a distortion the primary suspect for addressing Amir's dislike of IL-10 following things should be considered:

- sighted bias
- there was another speaker (it was mentioned by another member feew pages ago) with similar distortion peak with different listening impressions
- personal preference
- some other factors like tonality, directivity etc. should be eliminated
- ... (insert whatever I failed to mention this moment)

With regards to the personal preference I mentioned we can for example imagine that Amir indeed has developed extreme sensitivity toward distortion, but if that same distortion level is ok for 99% of the folks than statistically speaking it again comes to personal preference. But then again, there was that other speaker that had similar distorion spike and there were no such complaints. So, as a conclusion, if you really want to dig into this some serious work is needed and there can be no shortcuts.

 

[QMuse]

Can you show where anyone here is making an assumption (that THD/IMD is to blame), as opposed to a hypothesis? The distinction between assumption and hypothesis is very important -- and I've only observed the latter here so far. For example, right from the very beginning in Amir's first post:





Amir is NOT just making assumptions here before additional data is gathered. He is making a list of hypotheses (with a '...' at the end, inviting us to discuss other ideas), where the first hypothesis he states is placebo!

So I do not understand where you are getting this idea that assumptions are being made about distortion.

The only practical assumption we are making here is that we should not outright reject empirical data just because it disagrees with existing models: Discarding all data that does not fit your expectation is a deeply unscientific approach/perspective.

Sure, let's stick with hypothesis as that is a much better word. However, the main point of discussion was Amir's attitude where he claimed, opposed to Olive's paper, that he, as a trained listener is immune to sighted bias, which IMO was totally unnecessary and inappropriate. It would be much better if he admitted that as a fact and start evaluating speakers without seeing the measurements first. Also, IMO opinion a proper room EQ filters should also be setup and used for every listening as that wide filter at 120Hz he is currently using is simply wrong and is contributing to wrong impressions about LF response. As we all know LF response is the single most important factor and now some speakers are making better use of room modes than others do.

 

[TimVG]

Guys - the woofer is breaking up.

This also shows up in the spin

Also I mentioned the tonality difference vs the M16

These are obvious differences and flaws. Why must there be another, possibly mystifying, explanation?

 

[QMuse]

Guys - the woofer is breaking up.

View attachment 72286

This also shows up in the spin

View attachment 72287

Also I mentioned the tonality difference vs the M16

View attachment 72288

These are obvious differences and flaws. Why must there be another, possibly mystifying, explanation?

I fully agree - obvious diffferences should be eliminated first before suspecting distortion. As you may remember I already proposed EQ filter to adjust tonality of the IL-10 to M16 for the most part of it but they were never compared like that.

 

[amirm]

Sure, let's stick with hypothesis as that is a much better word. However, the main point of discussion was Amir's attitude where he claimed, opposed to Olive's paper, that he, as a trained listener is immune to sighted bias, which IMO was totally unnecessary and inappropriate.

You were just reminded of what I said in the original article:

Now, there are two possibilities here:

1. Placebo effect and the M16 being similar to IL10.

2. My brain is forever transformed to listen to distortions in small notes. Right after tonality, this is what I listen for in speakers.

And yet continue to keep misstating my position?

Trained listeners are NOT immune to bias.

What I have said is that they are less likely to be affected as such because their hearing acuity is far better than the general public and they are routinely used in sighted evaluations as their job. We would never use them as such if they were like general public and routinely provided biased opinions. You can see their level of acuity here in Sean Olive's test:

Their much superior acuity allows them to get such strong queues from what they are listening that it is able to overcome many superficial bias sources. Not always. Not all the time. But many times.

In contrast the average person who thinks most any speaker sounds good regardless of how flawed technically, is obviously far more biased by looks, brand, size, cost, etc. of the speakers.

Put yourself in the shoes of a trained listener who hears an impairment in the speaker for which he is trained to hear. Do you think the fact that the speaker is tall or short would be a stronger bias for him?

Now, there is a distinction here with respect to training for distortion and tonality. Harman listeners are trained in the latter. There, their preference is no different than the general public although they are far more critical.

With respect to training for non-linearity however, trained listeners of this type routinely hear artifact that vast majority of people including audiophiles cannot hear. This is demonstrated countless times and by me. Here is one of many double blind tests I have run which are next to impossible for most people to pass:

foo_abx 1.3.4 report
foobar2000 v1.3.2
2014/07/11 06:18:47

File A: C:\Users\Amir\Music\AIX AVS Test files\Mosaic_A2.wav
File B: C:\Users\Amir\Music\AIX AVS Test files\Mosaic_B2.wav

06:18:47 : Test started.
06:19:38 : 00/01 100.0%
06:20:15 : 00/02 100.0%
06:20:47 : 01/03 87.5%
06:21:01 : 01/04 93.8%
06:21:20 : 02/05 81.3%
06:21:32 : 03/06 65.6%
06:21:48 : 04/07 50.0%
06:22:01 : 04/08 63.7%
06:22:15 : 05/09 50.0%
06:22:24 : 05/10 62.3%
06:23:15 : 06/11 50.0%
06:23:27 : 07/12 38.7%
06:23:36 : 08/13 29.1%
06:23:49 : 09/14 21.2%
06:24:02 : 10/15 15.1%
06:24:10 : 11/16 10.5%
06:24:20 : 12/17 7.2%
06:24:27 : 13/18 4.8%
06:24:35 : 14/19 3.2%
06:24:40 : 15/20 2.1%
06:24:46 : 16/21 1.3%
06:24:56 : 17/22 0.8%
06:25:04 : 18/23 0.5%
06:25:13 : 19/24 0.3%
06:25:25 : 20/25 0.2%
06:25:32 : 21/26 0.1%
06:25:38 : 22/27 0.1%
06:25:45 : 23/28 0.0%
06:25:51 : 24/29 0.0%
06:25:58 : 25/30 0.0%
06:26:24 : Test finished.

----------
Total: 25/30 (0.0%)

Above was a public challenge to see if anyone could hear difference between an original high res track and downsampled version of it. As you see, there is 0.0% chance that I was guessing (25 out of 30 right).

So please don't keep mixing domains here. When discussing non-linear artifacts, this is my domain. It is not something is researched by Harman because frequency response errors are dominant in speakers.

 

[amirm]

Klippel wrote a large number of papers that go into the various (known) kinds of speaker non-linear distortion in a truly excruciating amount of detail.

Oh gosh, you just reminded me that one of their blind tests of distortion is online and many of us took it: https://www.audiosciencereview.com/forum/index.php?threads/distortion-listening-test.8152/

Despite some quirks in the test protocol which makes it hard to take, many of us had no trouble hearing distortions specific to speakers. These were my results for example:

As it happens, Tracy Chapman is also used by Harmah for their listening tests. As you see, I was able to get to -27 dB and could have done more if the test had not stopped me.

Indeed a number of people finished it to the end (-45 dB):

This is why I draw the -50 dB line in my distortion measurement graphs:

And for this speaker:

Those peaks are below 32 dB around 1 to 2 kHz.

As I said, the notion that speaker distortion is never audible is just preposterous. In comparison to other speakers it may not stand out. But by itself compared to original content, it definitely changes the sound and derates it.

 

[amirm]

Guys - the woofer is breaking up.

You are saying woofer breakup doesn't generate distortion???

 

[TimVG]

You are saying woofer breakup doesn't generate distortion???

Nope. In fact I'm saying this particular distortion element is likely to be audible.

 

[amirm]

Nope. In fact I'm saying this particular distortion element is likely to be audible.

Ah, OK. At first it read like friendly fire.

 

[QMuse]

And for this speaker:

Those peaks are below 32 dB around 1 to 2 kHz.

As I said, the notion that speaker distortion is never audible is just preposterous. In comparison to other speakers it may not stand out. But by itself compared to original content, it definitely changes the sound and derates it.

Toole didn't say "never", he used the term "very rare":

"In the general population of consumer loudspeakers, it
has been very rare for distortion to be identified as a factor in the overall
subjective ratings. "

Let's take a look at another speaker (JBL HD-1600) you reviewed with very similar distortion. It peaks at the same level, 2% which equals to -34dB.

Oddly enough, although both of these speakers are showing the same level of distortion at practically the same spot one of them gets "the sound was so good I sat there listening to track after track" and the other one got "there is this grunginess and lack of clarity to everything it played ".

So different impressions for so similar distortion. And yet you keep on thinking it's about distortion..

 

[QMuse]

Nope. In fact I'm saying this particular distortion element is likely to be audible.

Maybe so, but I can hardly imagine woofer cone breakup at 4.5kHz bringing "grunginess and lack of clarity to everything it played". Hearing such description I would certainly be looking for something else.

Looking at measurements I can find traces of that cone breakup at FR and it maybe also be responisble for that small THD spike near to 9kHz (2nd harmonic of the breakup frequency), but I can hardly imagine it would be responsible for the overal listening impression in a way that was described.

 

[amirm]

So different impressions for so similar distortion.

What are you talking about? Here is IL10:

Look at the low frequency distortion. It is off the chart.

Here is the JBL:

Here, worst case distortion is still below 4.5% in low frequencies. We are talking tons less distortion here. The JBL has less distortion in low frequencies at 96 dB that the IL10 has at 86 dB.

 

[amirm]

Also, keep in mind the order of the distortion in higher frequencies. It is a completely different spectrum in JBL, being comprised of 2nd harmonic with the rest in the noise. In contrast with IL10 third harmonic is also quite strong. Due to perceptual masking, this is a big difference from audibility point of view.

 

[amirm]

Toole didn't say "never", he used the term "very rare":

"In the general population of consumer loudspeakers, it
has been very rare for distortion to be identified as a factor in the overall
subjective ratings. "

What is very rare for general public, can be very common for me. So if you accept that, I suggest cutting back on the protests regarding distortion being a factor.

 

[TimVG]

Maybe so, but I can hardly imagine woofer cone breakup at 4.5kHz bringing "grunginess and lack of clarity to everything it played". Hearing such description I would certainly be looking for something else.

Looking at measurements I can find traces of that cone breakup at FR and it maybe also be responisble for that small THD spike near to 9kHz (2nd harmonic of the breakup frequency), but I can hardly imagine it would be responsible for the overal listening impression in a way that was described.

I'm not saying it is the definitive answer, as much as a mere possibility. If it weren't so expensive in terms of shipping and import I'd grab the pair on ebay USA to test them for myself.. I'm tempted

 

[QMuse]

What are you talking about? Here is IL10:

Look at the low frequency distortion. It is off the chart.

Here is the JBL:

Here, worst case distortion is still below 4.5% in low frequencies. We are talking tons less distortion here. The JBL has less distortion in low frequencies at 96 dB that the IL10 has at 86 dB.

JBL distortion graph doesn't even show distortion at 50Hz. In any case distortion at such low frequencies are hardly a factor as our ears tolerate much much more there than higher in the spectrum.