TRUTHEAR x Crinacle Zero Red

[amirm]

As Sharur has pointed out in a follow up video, the Harman group itself EQed up to 15 kHz in: Todd Welti, Sean E. Olive, and Omid Khonsaripour - "Validation of a Virtual In-ear Headphone Listening Test Method" (2016).

And as I explained to him privately, what works for Harman is not going to necessarily work for him. Harman verified that the emulation using EQ with *their* measurements and *their* surrogate IEM and with the set of headphones/IEMs worked up to a statistical limit. None of that is true of what Sharur is doing. He grabbed a random IEM to use this way. And he is trusting random measurements. His claim is that if the measurement rig is the same, the comparative measurements must also be correct. This is flatly wrong with it comes to high frequencies. Breath on the IEM and measurements change especially in that region.

The biggest issue I have, and again, I explained this to him in private, that he is being unethical with his viewers. Other than flashing the first page of that study, he runs off talking as if he is listening to the RED headphone when in reality he is not. As a minimum, he needs to repeat and consistently state that he doesn't have this IEM. And that he is relying on an emulation.

Note also that spatial qualities, distortion, etc. cannot be emulated in this manner. Yes, there is less of the former for IEMs but still, it would be another reason to provide caution.

I realize it is incredibly convenient for a youtuber to use just one IEM and keep EQing it to this that. He wouldn't have to buy or beg to have products to send to him. But his convenience comes at a cost of uncertainty. To not understand this factor means there are more problems here than the one or two reviews.

 

[julian_hughes]

And as I explained to him privately, what works for Harman is not going to necessarily work for him. Harman verified that the emulation using EQ with *their* measurements and *their* surrogate IEM and with the set of headphones/IEMs worked up to a statistical limit. None of that is true of what Sharur is doing. He grabbed a random IEM to use this way. And he is trusting random measurements. His claim is that if the measurement rig is the same, the comparative measurements must also be correct. This is flatly wrong with it comes to high frequencies. Breath on the IEM and measurements change especially in that region.

The biggest issue I have, and again, I explained this to him in private, that he is being unethical with his viewers. Other than flashing the first page of that study, he runs off talking as if he is listening to the RED headphone when in reality he is not. As a minimum, he needs to repeat and consistently state that he doesn't have this IEM. And that he is relying on an emulation.

Note also that spatial qualities, distortion, etc. cannot be emulated in this manner. Yes, there is less of the former for IEMs but still, it would be another reason to provide caution.

I realize it is incredibly convenient for a youtuber to use just one IEM and keep EQing it to this that. He wouldn't have to buy or beg to have products to send to him. But his convenience comes at a cost of uncertainty. To not understand this factor means there are more problems here than the one or two reviews.

I watched his video. He is not actually reviewing the IEM but instead he is reviewing an EQ curve that supposedly matches his (different) IEM to the product he claims to be reviewing! Is this really true? Did I miss something? This guy is really pretending to review IEMs without ever having heard or measured them by using other people's EQ curves on some other IEM? I hope I'm wrong and I misunderstood but if not then he is mindblowingly stupid.

 

[amirm]

I think he leaves it very clear that he is reviewing the EQed version of the Red. He has done that several times for other models in the past and later on got the real IEM and made a follow up review ("Quarks DSP is the best IEM" - > "Quarks DSP is trash"), so everybody following him should understand what is going on.

He absolutely does not. I watched the whole video and was the reason I reached out to him yesterday because I couldn't believe how misleading it was. He is listening to these white IEMs but keeps talking about "here is red without the adapter, here it is with the adapter." He does this for track after track. After I told him this is wrong and misleading, he gave me your excuse. That his regular viewers know this. Well, many people watch these videos who are not regular viewers. People search for reviews, land on his and watch it with no idea that it is an emulation.

Ok, but this would also partially discredit the Harman research itself. For headphones, you simply cannot EQ one headphone to another one to have the soundstage of the HD 800, but still they work with the assumption that frequency response dominates listeners preference (which I agree with, but just only up to spatial effects).

It does. But they had no choice because early on they realized that people could tell what headphone they were listening to, destroying the blind aspects of the testing. So they came up with this emulation strategy which kind of works but with at times, significant limitations.

It is these kinds of issues that originally almost stopped me from testing headphones. But I realized that if we get 50 to 60% right with measurements based on Harman target, we are still way ahead of the game compared to doing nothing. This is why I will not release a measurement without listening tests. This is why I develop individual EQ filters for each region that is out of compliance to hear if the measurements are valid or not.

Pushing all of this aside and running with measurements as if it is 100% accurate is just wrong. You must understand the limitations of any research and once you have enough experience under your belt to know what you are talking about, allow for these level of inaccuracy.

Really, the first signal that you should ignore anyone's opinion of headphone science and engineering is if they are too sure of the measurements. Run, and run fast....

 

[amirm]

I watched his video. He is not actually reviewing the IEM but instead he is reviewing an EQ curve that supposedly matches his (different) IEM to the product he claims to be reviewing! Is this really true? Did I miss something? This guy is really pretending to review IEMs without ever having heard or measured them by using other people's EQ curves on some other IEM? I hope I'm wrong and I misunderstood but if not then he is a mindblowingly stupid ar**hole.

That is *exactly* what is going on! I read comment after comment after his video and folks are oblivious to this because he does just a poor job of saying what he is doing. They all think he is actually listening to said IEM when he says the highs are bad, etc., etc. When in reality he has entirely different IEM in his ears and just applying this or other EQ to it.

If people really realized this is what he is doing, they would roast him alive. The reason they are not is that he is not giving proper notice that this is what he is doing. Worse yet, he is not willing to listen when you tell him all of this as I tried to do so extensively in messages to him. He thinks he has figured all this out and is running 100 miles an hour...

 

[julian_hughes]

I looked at some of his other reviews/polemics/rants. He is truly mad. Real click bait stuff.

 

[julian_hughes]

Once again, this is the whole assumption of the Harman research, so it is absolutely nothing crazy, just has it's limitation. If you really think this is so absurd to do, please say the same thing about the Harman research.

I don't know of any examples of Harman or AKG or Samsung or Dr Olive pretending to review and describe headphones or IEMs of which they have zero actual experience by using other people's measurements applied to an entirely different listening device. The Harman curve is merely a description of a preference. It is not a substitute for listening or measuring, and Harman/Dr. Olive have never presented it as such.

 

[TonyJZX]

i think this 'sharur' guy actually bothered to register here and "like" his own cooked content!

i feel like discussing his brain damaged reviews on a product he couldnt be bothered spend $50 and making really brain dead assumptions does the product or Crinacle some huge disservice.

The basis of his 'opinion' is that if you add an EQ on another product that has its own EQ curve then what assumption can you reliability glean from that?

Even if you start with a known product and you know its own curve you still havent learnt anything at the end.

I put this guy at or underneath the same level of your friends Jay and McGowan Robinson... the list goes on.

Further if he's able to make such a brain conclusion then nothing will change his mind and is in the end do you care about the opinions of those suitably brain damaged.

I may not agree with some of the conclusions that Crinacle makes but as least he gives a rebuttal for his decisions and you can take it from there.

 

[thecheapseats]

He already did a video response to Amir‘s complaints.
I’m getting some popcorn

there's popcorn?... gimme'...

 

[phoenixdogfan]

That is *exactly* what is going on! I read comment after comment after his video and folks are oblivious to this because he does just a poor job of saying what he is doing. They all think he is actually listening to said IEM when he says the highs are bad, etc., etc. When in reality he has entirely different IEM in his ears and just applying this or other EQ to it.

If people really realized this is what he is doing, they would roast him alive. The reason they are not is that he is not giving proper notice that this is what he is doing. Worse yet, he is not willing to listen when you tell him all of this as I tried to do so extensively in messages to him. He thinks he has figured all this out and is running 100 miles an hour...

It's so obvious on the Red review because the IEM he's wearing looked nothing like a pair of red cuff links, as they should. I was watching him saying they had no highs, and at first I thought he was just relating a listening experience he had at some earlier, off-camera point in time. But then I realized he wasn't, that he was offering contemporaneous commentary on the device he was listening to on camera, and, as I looked at the IEM in his ear, I saw it was silver, not red and had my WTF moment right then and there. Looks to me like he's wearing the Moondrop Variations. Can see that very clearly at 2:47 of his review.

And, of course, absolutely unacceptable method for any reviewer though he did mention he "EQ'd to the Project Red" in the review, but that statement was a little opaque, and I think most viewers would not have caught on at first to what he was exactly doing without a more detailed explanation. Moreover, I think most would find the method dubious, if not outright invalid--just as Amir does.

My guess is he couln't lay his hands on a pair of the RED, and wanted to do a review in the worst way. He did and hurt the hell out of his credibility in the process.

 

[amirm]

Maybe @amirm could provide some of the original works?

As an AES member, I cannot share the papers outside of snippets here and there. It would also violate our terms as far as copying copyrighted material. Wish it was not so but it is.

 

[amirm]

as I trust Amir that he can place the Truthear Red in the G.R.A.S.

You should not trust me to that level. I make no representation of the accuracy of my measurements at > 7 or 8 kHz. And further, I make no assurance down to 1 or 2 dB in the rest of the region.

 

[amirm]

I see. But could I some papers I have freely share here somewhere?

No. If they are copyrighted material, the fact that someone else has distributed them illegally, doesn't make it legal for us to do so. You are welcome within bounds of fair uses to quote sections as we do but not wholesale copies. You could contact the author(s) and see if they give you a copy independent of AES and whether you can share that. Some of the Harman speaker research papers were provided this way. AES also has an open source project so you could look in there and see if there is a copy there.

 

[MayaTlab]

The correlation between listener preference ratings of real over-ear headphones virtual over-ear headphones is only r = 0.85 while the correlation between real and virtual IEMs is r = 0.98. One of the biggest reason behind them would be leakage control. "To our knowledge, this is the first published study where listening tests on IE headphones were reported and leakage was not a factor that corrupted the results. For that reason, we believe these results are more reliable and valid than those from previous studies. It also explains why the correlations between the actual and virtualized headphones were so high and consistent between listeners."

The correlation is higher mainly because the IE virtualisation methodology, unlike the OE one, never used the actual IEMs in the listeners' ears, but rather compared recordings of the IEMs to an EQ profile that matched these IEMs' FR, both played back through the replicator IEMs (Sennheiser Momentum in-ear iei). The actual IEMs were never inserted in the listeners' ears. So all notions of insertion depth / IEMs interacting differently with different loads / etc. were nullified, other than when it comes to the FR delivered by the replicator IEM, for which a serious attempt was made to control for leakage at lower frequencies.

E-library page - AES

AES E-Library ← Back to search

www.aes.org

For the OE virtualisation, Harman used large, open over-ears which tend to be more resilient to leakage in the first place.

The r numbers shouldn't be compared between the two methods as they are simply too radically different. It's also an insufficient metric IMO, but that's another subject.

 

[Lunafag]

A fat L for audiosciencereview, if you don't believe virtualized headphone method works why even bother using Harman target that was developed using it.

 

[well...]

So all notions of insertion depth / IEMs interacting differently with different loads / etc. were nullified

It was recorded, yes, but the frequency response of the replicator headphones was equalized to flat based on the masurement from GRAS RA0045, instead of the in-ear response of real listeners. So the notions of insertion depth / IEMs interacting differently with different loads / etc. would still be there, since Olive only controlled the bass leakage, and the insertion depth in each listener's ears would be very likely different, as well as the differences in ear canals.

The r numbers shouldn't be compared between the two methods as they are simply too radically different.

I agree. But it is Olive who compare them first, so please blame him.

 

[MayaTlab]

It was recorded, yes, but the frequency response of the replicator headphones was equalized to flat based on the masurement from GRAS RA0045, instead of the in-ear response of real listeners. So the notions of insertion depth / IEMs interacting differently with different loads / etc. would still be there, since Olive only controlled the bass leakage, and the insertion depth in each listener's ears would be very likely different.

The only interaction you're left with, other than leakage, is how the Sennheiser Momentum replicator IEM interacted with the test subjects. For each individual, this was a constant across all comparisons.

If the real IEMs had been used (not a practical solution for that test as you would have needed to mod all of them with MEMS mics), you would have been comparing how IEM A and B differ when subjected to the ears of listener X (real IEMs) vs how they differ when subjected to the ear simulator (virtual). The methodology used mean that in both cases ("real" and virtual), you're only comparing how they differ when subjected to the ear simulator.

I agree. But it is Olive who compare them first, so please blame him.

Well on that subject I do.

 

[thecheapseats]

...He thinks he has figured all this out and is running 100 miles an hour...

irresponsible... truly...

 

[MayaTlab]

A fat L for audiosciencereview, if you don't believe virtualized headphone method works why even bother using Harman target that was developed using it.

They way Harman researched their target means that you don't need to conceptualise it as a comparison of one or several target candidates to other headphones, but merely alternative curves. So whether the virtualisation was successful or not is a bit irrelevant, what really matters is that the in-situ response of the replicator headphones was known to a good enough degree, and Harman spent quite a bit of time to make sure that it was, as practically feasible in the context of these studies.

 

[well...]

For the OE virtualisation, Harman used large, open over-ears which tend to be more resilient to leakage in the first place.

The replicator headphone was large, open over-ears, yes, but the real headphones were not all open backs. One of them was Beats by Dre Studio Limited Edition, which was a pair of closed back on-ear headphones. Another one was AKG K550, which was closed back around-ear headphones. So there would still be leakage differences on each listeners' ears. As I quote from Olive, "Headphone leakage effects for (HP4) were a factor in the standard test (see Fig. 9 of [1]) but not the virtual tests. This would explain its higher ratings in the virtual test."

 

[amirm]

A fat L for audiosciencereview, if you don't believe virtualized headphone method works why even bother using Harman target that was developed using it.

A few of you still think this is a black and white thing. Until you learn that it is not, you will continue to be lost in this discussion.

The research used a technique to solve the blinding problem. That is an approximation. An approximation can still be quite useful. You simply need to understand where its accuracy ends and don't go past that. If I give you a ruler with just inch markings, you can sort of guess where 1/4 inch might be. But it won't be as accurate as the inch markers on the ruler. Go to 1/16 of an inch and then the error can be become quite large and attempting to make such measurements with an inch ruler, would be very problematic.

The target curve is also heavily smoothed and has little bearing in higher frequencies to what we measure. This again attempts to give a high level answer, not a detailed one.

My use of the target is an approximate one and one that you could easily deviate a few dBs here and there and still be good. And extreme care in interpreting anything above 5 or 6 kHz.

Learn to deal with shades of gray in headphone measurements or don't bother.