ZMF Caldera Headphone Review

[amirm]

This is a very interesting discussion, so much so that I finally decided to actually register in order to inquire.

I'm extremely curious as to how many participants on the discussion, and especially in the poll, have heard (or owned) the headphones?

Heard how? With or without their eyes?

 

[renaudrenaud]

This is a very interesting discussion, so much so that I finally decided to actually register in order to inquire.

I'm extremely curious as to how many participants on the discussion, and especially in the poll, have heard (or owned) the headphones?

We don't care at all.

Measurements tell us where to go. Obviously not in that direction. And we sleep well, with no question, just data, no romantic explanations, no bs . "When you want to know, you measure" as my wife says all the time.

 

[asrUser]

We don't care at all.

Measurements tell us where to go. Obviously not in that direction. And we sleep well, with no question, just data, no romantic explanations, no bs . "When you want to know, you measure" as my wife says all the time.

You sound like Sharur.

 

[Chagall]

We don't care at all.

Measurements tell us where to go. Obviously not in that direction. And we sleep well, with no question, just data, no romantic explanations, no bs . "When you want to know, you measure" as my wife says all the time.

@DenverW has a legitimate question.

I'm unsure why you said, "We don't care at all."

I can understand if someone likes these headphones, the same goes for any other headphones reviewed here - the majority of them need more or less EQ.
Amir recommends them after EQ, he questions the price - but that's for the listeners to decide after having listened to them.

We should not dismiss listening to headphones and then judging, headphones aren't DACs.

 

[NorthernSoul73]

I don't post here that often but do look at the site daily ... Maybe the later has to change.

From Amir himself:

So going forward, these are the new guidelines:

• Comments in review threads need to be specific to the product being discussed. If your comment is generic in nature, “who needs a DAC since they all sound the same,” it does not belong in the review thread.
  
• The review thread is not where you come and take on your opponents, whether it is me or someone else. Please don’t drag your grudges into review thread. And oh, don’t think getting a “like” means you did good. Gladiators received a lot of praise from the audiences right until they got their heads chopped off!
  
• Not all rules apply to all people equally. Yes, you read that right. We have some very experienced members who bring their objections in the most constructive and friendly way and their feedback will be welcome far more than one of you just throwing out one liner jabs. So please don’t point to someone saying this being OK and yours not. You are not them but you could learn from them how to express disagreement properly and constructively.
  
• Your conduct must be of highest levels of professionalism in review threads. This is the formal part of our site and requires members act responsibly and to highest regards. If you are nasty and obnoxious, expect a harsh treatment from management in return. Your posting history and status in the forum will not protect you.

Violating these rules will be dealt with simply. The first action would be getting a “reply ban” to the thread, stopping your ability to post. Your post may or may not be deleted as well. Show disregard for these rules again and you will be given a temporary ban. Do it again, and you will be banned longer. And repeat once more and you will receive a permanent ban.

Seems like all of the above has been totally abandoned.

Very poor show from some people on this thread.

Adios!

PS - When it comes to headphone measurements and the subsequent eq that is suggested it would be useful if the headphones were measured again to better understand how they then comply to the preferred target. Just a thought ...

 

[solderdude]

PS - When it comes to headphone measurements and the subsequent eq that is suggested it would be useful if the headphones were measured again to better understand how they then comply to the preferred target. Just a thought ...

That is fun to see but not really needed for 2 reasons.

1: The outcome is totally predicatable and follows the EQ exactly.
caveat A)... when there is a high distortion (not the case here) and there is compression then the bass correction might still be incorrect.
caveat B)... when a dip is caused by a null that null will still be there and could be inaudible/harmless.

2: The the resultant measurement will only be 'correct' for that fixture with that particular EQ and may still be 'off' in...
A) different copies of the same model (production spread)
B) on other test fixtures that follow a differing standard
C) when a different target is used
D) on someones actual head the result may differ from the plot yet again.
E) when re-seated on the same fixture

So while there is science and a standard being followed and EQ created (whatever way it is done on that measurement) a re-measurement with EQ is fun to see but not really needed.

 

[oleg87]

This is a very interesting discussion, so much so that I finally decided to actually register in order to inquire.

I'm extremely curious as to how many participants on the discussion, and especially in the poll, have heard (or owned) the headphones?

Most of the votes are just evaluating the measurements, I’m sure.

I've listened to the Calderas a few times. I think they are solid headphones if you can borrow them from a buddy. But unfortunately since I’m not made of money, I have to factor in the value proposition, so these are a nonstarter for me.

 

[Mulder]

Most of the votes are just evaluating the measurements, I’m sure.

I've listened to the Calderas a few times. I think they are solid headphones if you can borrow them from a buddy. But unfortunately since I’m not made of money, I have to factor in the value proposition, so these are a nonstarter for me.

If the Caldera had been a headphone costing like a pair of Sennheiser HD600 or something similar, it would probably have been regarded as a very competitive headphone by most people here at ASR. But at US $3,500 I guess most people will not accept anything that can be measured as inadequate. I suppose those who staunchly defends it are those who already own it. A sort of bias of course.

 

[renaudrenaud]

@DenverW has a legitimate question.

I'm unsure why you said, "We don't care at all."

I can understand if someone likes these headphones, the same goes for any other headphones reviewed here - the majority of them need more or less EQ.
Amir recommends them after EQ, he questions the price - but that's for the listeners to decide after having listened to them.

We should not dismiss listening to headphones and then judging, headphones aren't DACs.

I mean, if the measurements are not ok, I am not interested in a listening session. I am not interested at all, this is not for me and I think not for objectivists.

If you like it, enjoy it, but don't come to me saying you have to listen. In one side there is some facts, in the other side there is your feelings. Enjoy your fellings and let me enjoy the measurements!

 

[Grobbelboy]

It's funny how in practically every instance of an "outside" party trying to make their case here, be it the HP.com guys for their new measurement rig or a company/individual for their product, I come away with renewed respect for ASR and what it stands for without exception. It's also weird how in almost each of the ensuing discussions pragmatic evidence is the foundation for every argument coming from Amir, but the other party sooner or later inevitably gets Very Offended and throws their toys out of the pram.

 

[L0rdGwyn]

Has Harman's preference curve data been reproduced or validated by a third party study? I tried to find an answer to this on my own but could not find any information.

 

[Benesyed]

it is actually very important to stress the above point.

the harmon studies are very valuable data but it is primarily done by one group that has a clear financial interest. in the world of medical research which is more my expertise this would cast a lot of doubt on the validity of the work until it was reproduced by at least a few other parties.

that being said some data that has objective measurements behind it and is well designed is still better than no data or purely subjective data where a lot of bias can come through.

as an aside I do think it's kind of pointless to have a a voting option on this site for whether or not a headphone is good or bad. I'm not really sure what it means when someone just agrees very aggressively with an objective measurement. pull data doesn't really seem to provide much helpful additional information.

 

[IAtaman]

Has Harman's preference curve data been reproduced or validated by a third party study? I tried to find an answer to this on my own but could not find any information.

I always found it astonishing, when you tell someone a headphone they like is not tuned optimally, they question the research. Nothing is wrong with that of course; I think everybody should check the research and be convinced themselves, but you never see people checking aerodynamic research before getting on a plane, or glancing at medical research before going into a major surgery. Life and death science is taken for granted but headphone research, I want to see proof!

For me, I do like the super low distortion, flat frequency response and the ability to have a variety of tuning options. I would not buy it without hearing it first, but I would definitely like to hear it if I get the opportunity.

 

[solderdude]

Has Harman's preference curve data been reproduced or validated by a third party study? I tried to find an answer to this on my own but could not find any information.

Nope.... but if one doubts the data someone should. So far no one (including Zach) has shown us the Harman research is (somewhat or totally) flawed.
You just can read some dislike and others like that specific target. I assume you are talking about the validity of the target.
Harman themselves also have researched what percentage prefers that target so it is clear not everyone agrees.

 

[amirm]

the harmon studies are very valuable data but it is primarily done by one group that has a clear financial interest.

It is actually the opposite. They have a disinterest in publishing their research, letting their competitors use it at will. This rarely happens when the research is so practical and easy to deploy. We are so lucky that management has allowed this to happen and continue to go on.

 

[Benesyed]

It is actually the opposite. They have a disinterest in publishing their research, letting their competitors use it at will. This rarely happens when the research is so practical and easy to deploy. We are so lucky that management has allowed this to happen and continue to go on.

respectfully I have to disagree strongly with this sentiment.

first purely at the level of marketing across any field all companies will publish research that shows that their reference standard or intervention or treatment is better than those done by others. Even if others ape their success it's still gives them the clout to say that we established this and we did this better. that is seen across all forms of marketing that is done with research. for example spine instrumentation companies will publish all sorts of data about animal model studies showing that their hardware or biological agent etc is superior to XYZ competitor.

but at the second level of bias in data anyone conducting a study has biased towards wanting that study to show a reproducible meaningful effect. it is often in medical research and other function search that repeated measurements of a study don't actually end up showing a statistical significant effect and further skepticism shows that there was perhaps play with statistics or study design post-hawk that created an illusion of significant findings. perhaps the most significant version of this is when you look at registry data that's collected by a unbiased source for example CMS tracking of outcomes all of a sudden many interventions that seem to be beneficial no longer appear to be so when a third party with no interest is doing the outcomes assessment and data analysis

 

[Benesyed]

All this being said, subjective listeners are being hypocritical and ignoring nuances and an even bigger source of bias when they argue against the much better quality evidence being used by amir.


It is completely reasonable for amir to say that in a blinded setting, most people would prefer a Harmon tuning to that of the Calderas. How big that group is and how dramatic the effect size would need to be studied in a separate study.

 

[Mulder]

Nope.... but if one doubts the data someone should. So far no one (including Zach) has shown us the Harman research is (somewhat or totally) flawed.
You just can read some dislike and others like that specific target. I assume you are talking about the validity of the target.
Harman themselves also have researched what percentage prefers that target so it is clear not everyone agrees.

I have tried to find any number in the Harman research for the level of statistical significance, i.e. the statistical significance of the 64 percent who prefer the Harman curve. I suppose these number is to be found somewhere, but I have not found them.

 

[L0rdGwyn]

I always found it astonishing, when you tell someone a headphone they like is not tuned optimally, they question the research. Nothing is wrong with that of course; I think everybody should check the research and be convinced themselves, but you never see people checking aerodynamic research before getting on a plane, or glancing at medical research before going into a major surgery. Life and death science is taken for granted but headphone research, I want to see proof!

For me, I do like the super low distortion, flat frequency response and the ability to have a variety of tuning options. I would not buy it without hearing it first, but I would definitely like to hear it if I get the opportunity.

I work in the medical field actually, in cardiothoracic critical care, so I do know a thing or two about major surgeries

The reason these arguments are being had is because compared to matters of life and death, headphone measurements and audio preference are small potatoes, this is a hobby after all. Abandonment of the research data will not have significant negative ramifications. Most people will not question data in matters of life and death. In the medical field, patients put their trust in their providers and assume they have reviewed the literature and are practicing evidence-based medicine. Sometimes they do (at tertiary, academic medical centers), sometimes they don't (think small, for-profit regional hospitals). Medical information is highly specialized and requires years of training to understand, so patients have no choice but to trust their providers, whereas a high school education allows one to read a paper on headphone frequency response preferences. An introductory statistics course could help one interpret the data presented.

I am extrapolating what ASR is doing with the Harman data to the medical field. In medicine, if there is a sufficient body of evidence, medical organizations will release clinical practice guidelines based on the results of those studies such that members of their organization might implement them in to clinical practice. The strength of the evidence to support these clinical practice guidelines is rated in some way, usually based on the type and number of study/studies performed (e.g., meta-analysis, systematic review, RCT, cohort study, etc.).

What ASR is trying to do is, in essence, create a clinical practice guideline and encourage the headphone industry to adapt the Harman curve. But what is the strength of the evidence? It is a single study, corporately funded, and as such the strength of the evidence is moderate at best.

So, with that being said, if ASR wants to advance the field and create industry guidelines, perhaps Amir should do some science and show us the research is in fact reproducible. ASR has a large enough readership that a Harman target study could be conducted. Perhaps even remotely - there are some talented engineers on this site, maybe someone could write program to EQ a headphone (based on model) with the user rating their subjective response numerically. Interestingly, it seems there is already some discrepancy among the readership from the Harman data in regard to the midrange:

Nope.... but if one doubts the data someone should. So far no one (including Zach) has shown us the Harman research is (somewhat or totally) flawed.
You just can read some dislike and others like that specific target. I assume you are talking about the validity of the target.
Harman themselves also have researched what percentage prefers that target so it is clear not everyone agrees.

Keep in mind Harman is a >$8 billion annual revenue corporation with an abundance of resources. Any time someone has an alternative opinion, the retort is "show me the data!" Realistically, no one can, because random guys on a forum do not have those resources at their disposal, let alone the time. Attempting to reproduce Harman research even for a small business with tight margins would be taxing. Maybe some data could be collected at a CanJam event, but even that is not an ideal environment due to show noise. I think ASR could conduct a study, and if the evidence is repeatable and compelling, would add validity to the grading system being used in its reviews. But right now the "we know what you like better than you do" message being propagated by some members of this forum based on the Harman studies is not going to convince people, even those of us who understand the research.

 

[Mulder]

I work in the medical field actually, in cardiothoracic critical care, so I do know a thing or two about major surgeries

The reason these arguments are being had is because compared to matters of life and death, headphone measurements and audio preference are small potatoes, this is a hobby after all. Abandonment of the research data will not have significant negative ramifications. Most people will not question data in matters of life and death. In the medical field, patients put their trust in their providers and assume they have reviewed the literature and are practicing evidence-based medicine. Sometimes they do (at tertiary, academic medical centers), sometimes they don't (think small, for-profit regional hospitals). Medical information is highly specialized and requires years of training to understand, so patients have no choice but to trust their providers, whereas a high school education allows one to read a paper on headphone frequency response preferences. An introductory statistics course could help one interpret the data presented.

I am extrapolating what ASR is doing with the Harman data to the medical field. In medicine, if there is a sufficient body of evidence, medical organizations will release clinical practice guidelines based on the results of those studies such that members of their organization might implement them in to clinical practice. The strength of the evidence to support these clinical practice guidelines is rated in some way, usually based on the type and number of study/studies performed (e.g., meta-analysis, systematic review, RCT, cohort study, etc.).

What ASR is trying to do is, in essence, create a clinical practice guideline and encourage the headphone industry to adapt the Harman curve. But what is the strength of the evidence? It is a single study, corporately funded, and as such the strength of the evidence is moderate at best.

So, with that being said, if ASR wants to advance the field and create industry guidelines, perhaps Amir should do some science and show us the research is in fact reproducible. ASR has a large enough readership that a Harman target study could be conducted. Perhaps even remotely - there are some talented engineers on this site, maybe someone could write program to EQ a headphone (based on model) with the user rating their subjective response numerically. Interestingly, it seems there is already some discrepancy among the readership from the Harman data in regard to the midrange:

View attachment 333362



Keep in mind Harman is a >$8 billion annual revenue corporation with an abundance of resources. Any time someone has an alternative opinion, the retort is "show me the data!" Realistically, no one can, because random guys on a forum do not have those resources at their disposal, let alone the time. Attempting to reproduce Harman research even for a small business with tight margins would be taxing. Maybe some data could be collected at a CanJam event, but even that is not an ideal environment due to show noise. I think ASR could conduct a study, and if the evidence is repeatable and compelling, would add validity to the grading system being used in its reviews. But right now the "we know what you like better than you do" message based on a single paper is not going to convince

I agree that a study of the kind conducted by Harman would only be considered indicative at most in many contexts where the demands on reliability are much more critical. But as you say, HiFi headphones are a gadget for hobby use (although many people feel that music is vital). In practice, the Harman is an unusually solid study, for HiFi - where the absence of solid data in the mainstream is striking. It is even the case that the manufacturers are not even required to provide correct specifications and measurement values. But, of course, it would be great if more studies could be carried out that deepened the knowledge about headphones and about who prefers what.