ZMF Caldera Headphone Review

[zach915m]

I am pretty sure the real story is that you noticed that we value the Harman target, realized you were not compliant with it and that you would not get a good review so decided to ghost me.

This is untrue. Why would I reach out to you to state I found the work you were doing biased? It's best to leave sleeping dogs lie.

I just showed you two examples of ways to be closer to harman via two different ZMFs. Had you reached out I would have pointed you to the ultra perf pads and told you there will be a shortfall to your target in the bass, as in pointed out in my first post in this thread.

I can prefer the Caldera in a way that's not Harman, and prefer the Bokeh in a way that's closer to Harman and be the same person. The research is not done. This is not "settled science." Sorry.

 

[amirm]

In an open system the ways to tune to Harman that I know of add distortion and can be seen in other open planars measured by this site.

They can? Here is such a headphone with Harman compliance:

Notice how 94 and 104 dBSPLs hug the zero line. This is yours:

The only advantage you show is in 114 dBSPL in bass. Otherwise the previous headphone is superior in all cases and delivers on Harman compliance:

 

[dfuller]

I thought at this point we were past that and folks were fully educated on what my headphone measurements mean. I hate to have disclaimers and only put that in there early because of food fights.

There are new members, beginners, and people who aren't members of this forum reading reviews. I think it's quite important to have the disclaimer that headphone measurements are somewhat variable based on fitment, seal, artificial pinna shape, yadda yadda.

 

[Benesyed]

I understand the wariness in this regard but I don't believe any Harman brand was making a headphone that was designed to follow the Harman curve before the research was conducted. Therefore I don't think you can argue that they had a standard they were already following and set out to prove it was better. Besides, if I wanted to do that I would have cherry-picked my samples so that I could bump the overall percentage of predicted preference to way more than 64%.

This is in pretty stark contrast to medicine where a pharmaceutical company synthesizes a drug at huge cost so will fully intend their trial to demonstrate a significant effect. Moreover, they will invent explanations to validate the chemistry behind the effect even when they have no idea what is going on, such as the "chemical imbalance in the brain" theory of depression to push SSRIs.* With the Harman target curve, however, it was proved first that frequency response was the most important indicator of listener preference.

Interestingly, producing a headphone then arguing for the superiority of its frequency response after the fact, using unverifiable explanations, is precisely what is going on for this headphone.

I wouldn't say no to someone going through the Harman research to look for p-hacking and base rate fallacies and the like, however.

* worst case scenario, not saying this is always what happens.

Even if they didn't have headphones that matched the research. once it's done they can and they can say here you go here's our evidence that you will like our tuning. Moreover just having your name attached to data adds greater weight to your audio equipment even if it doesn't match the measurement due to prestige.

The order doesn't actually have the relevance you think you do. There's plenty of pre clinical research before drugs are even developed that can have bias.

The second bias is that if you are being paid to research and find a better headphone tuning that is reliable, you have incentive and bias to find a strong result. that big is undeniable.

 

[amirm]

This is untrue. Why would I reach out to you to state I found the work you were doing biased? It's best to leave sleeping dogs lie.

You would do that in the interest of professionalism and high ethics. Remember, you reached out to me to test your headphones. Leaving me in the lurch when I gave up the opportunity to test the Verite was in very poor form.

BTW I have had plenty of companies tell me their products are not Harman optimized when I ask for samples and we leave it at that. Same happens in other product categories. Companies have no obligation to loan me anything and I respect them not volunteering to get negative press.

 

[Benesyed]

I am also a MD and I know how medical research is done, asking what you did to Amir/ASR members shows (sorry to put it in this way), a lack of understanding of the situation. Doing proper research to corroborate Harman (which is probably not perfect but the best we have), needs a whole team of experts, lots and lots of working hours and not to mention quite expensive. I am surprised you asking that.

Ofcourse it's expensive! But that's the standard to actually prove something. Typically you require multiple corroborating studies by different groups or with very strict parameters.

What I do agree with is that its much better than nothing and a reasonable but not a absolute way to analyze things.

 

[zach915m]

You would do that in the interest of professionalism and high ethics.

I'm sorry but starting a conflict based on different points of view and perceived bias is not what I would consider useful for either of us.

However reviewing a headphone in the Caldera, without understanding and doing a minimum or work to see there are various pads and a tuning system that will make it closer comply with the target (with caldera ultra perf pads) you use is not responsible, and it should be your goal to accurately present information regardless of the source of gear.

 

[amirm]

However reviewing a headphone in the Caldera, without understanding and doing a minimum or work to see there are various pads and a tuning system that will make it closer comply with the target (with caldera ultra perf pads) you use is not responsible, and it should be your goal to accurately present information regardless of the source of gear.

The headphone was tested as bought by the customer. If these pads are not performant, you should not sell them. If they are, then you should accept them being tested as such. It is not my job to go and hunt for other pads to test with it. Nor do I accept that the other pads change the story here.

 

[amirm]

What I do agree with is that its much better than nothing and a reasonable but not a absolute way to analyze things.

There will never, ever be an absolute statement here. As I have said repeatedly, music is operating with no standard. Content tonality is a variable that cannot ever be taken out of the equation. Headphones also don't always offer the same seal and positioning. All of these conspire from us ever having a anything close to 100%. Current research is about as good as it gets and we are lucky to have gotten this far with them.

 

[DenverW]

Most of the votes are just evaluating the measurements, I’m sure.

I've listened to the Calderas a few times. I think they are solid headphones if you can borrow them from a buddy. But unfortunately since I’m not made of money, I have to factor in the value proposition, so these are a nonstarter for me.

I appreciate yours and the other responses. To answer @amirm my question was based on audio, not visual, so hearing the headphones.

I'll clarify a bit regarding my question. I've been poking around this forum for years, and have never registered. There is another forum that's the antithesis of this one where I have done the same. Honestly, I find them intimidating. As I don't live anywhere that I can really demo any gear I would rely heavily on reviews and forums to decide what to try, and what to spend on. That led to discovering some things I enjoyed, and some things I really did NOT (I'm looking at you, ATH M50!).

However, through this journey, I would run into some subjective vs objective differences, as I'm sure many people have. I believe my first real deviance from objective preference would be the Bottlehead Crack. Tube amp, I'm sure it measures terribly, and has lots of distortion. However, for me, I preferred it so much more (I had hd600, and later hd800) than the drop THX amp.

So this lead me to feel like I couldn't contribute here. Some of the more harmon-esque tuned headphones such as the Aeon didn't ring my bell. It honestly made me wonder if something was wrong with my preferences or how I listened. How can I like something where the majority of other people here DON'T like it, or view it as badly tuned, or distorted? I believe one reply expressed that this is "bs." It's not easy to be a little fish that doesn't swim the same way as the rest of the pond, here or on other forums, so I generally keep quiet and form my own opinion.

So my question was based on that. I was curious of the different ways that people evaluated and gave their opinion; if it was based on the review and measurements, or if it was based on subjective listening experiences. Or both, such as Amir. Cheers, and I hope this is an acceptable explanation.

 

[Benesyed]

There will never, ever be an absolute statement here. As I have said repeatedly, music is operating with no standard. Content tonality is a variable that cannot ever be taken out of the equation. Headphones also don't always offer the same seal and positioning. All of these conspire from us ever having a anything close to 100%. Current research is about as good as it gets and we are lucky to have gotten this far with them.

yes it's very helpful resource for sure and much needed. In a world where cables are being tauted for tonality and dacs/amps being over priced with terrible distortion (and I don't even mean tubes) it's a great resource

 

[Benesyed]

I'm sorry but starting a conflict based on different points of view and perceived bias is not what I would consider useful for either of us.

However reviewing a headphone in the Caldera, without understanding and doing a minimum or work to see there are various pads and a tuning system that will make it closer comply with the target (with caldera ultra perf pads) you use is not responsible, and it should be your goal to accurately present information regardless of the source of gear.

I don't think this is a realistic expectation since this review was done by a reader donated pair.

Also since there had been initial talks about Amir doing a review after which he was sadly "ghosted" as the kids say.

 

[Resolve]

However reviewing a headphone in the Caldera, without understanding and doing a minimum or work to see there are various pads and a tuning system that will make it closer comply with the target (with caldera ultra perf pads) you use is not responsible

We should all just review them padless. Padless headphone reviews are the next meta.

(this is a joke)

 

[solrage]

Did you click on the link and read the whole post? It does, in fact, fully explain the difference at 4k and he even shows a the difference between his calculated compensation and the provided compensation (which I'm guessing is used in your graph above). Surprise surprise, big difference at 4k.

"How am I supposed to tell which is more accurate except with my ears?" Go by the more accurate measurement (Amir's) rather than hoping a less accurate measurement will better support your opinion.

I would recommend absolutely trying out the EQ (with the same pads) as you speak about at the bottom and seeing which you prefer.

"I really feel like I'm EQ'ing the recording as much as the phones" This hits the nail on the head and is precisely why lots of us like headphones tuned close to Harman, because different music recorded in different studios/venues (some of them well, some terribly) will sound different and you might want to bump up the treble a bit on one, or the bass down on another, etc. This is how I use my subwoofer with my speakers: I don't have it at the same level for all music, some songs I find mixed with more bass than I prefer already so want to turn it down a little. With headphones, this is easier if you have a headphone which gets you close enough to some semblance of "neutral" to begin with.

I read the whole post. The post would explain the difference at 4k if we were considering only the measurement of the Caldera by itself. What I'm saying is that it doesn't explain why the Caldera and Stealth measure identically in that region on one rig and wildly differently on Amir's.

I don't see the justification for assuming Amir's measurement is more accurate when we (seem to?) agree the quality of the seal can be affecting the measurement. How are we to determine which rig/measurement got the better seal?

I don't actually actually think "tuning close to Harman" helps with EQ'ing for different recordings. Let's take the easiest example of the bass region. Harman implements a ~5db bass boost with a particular slope. So which do you think is easier to EQ: a FLAT bass response to tilt it up, or the Harman boost to tilt it down? With the latter you have to deal with the particular slope they use. With a flat bass response you can decide on what slope sounds best to you. So if we're recommending headphones based on the ability to easily EQ the bass to taste, it would seem a headphone with a flat bass response and low distortion (like the Caldera according to Amir's measurement) would be much easier to tune than one with a bass boost already there.

The rest of the frequency range I might agree with you, though even for things like the 1k-3k dip I think you can make a practical argument for deviating from Harman based on the fact that without that dip the music that's mixed hot in that region will sound really bad (I have/have heard a lot of such recordings), while those mixed well won't sound terrible with it. That dip, if done smoothly (the Caldera is) is also really easy to EQ to taste, at least as easy as it would be to EQ the Harman curve down. That just leaves the 4k dip, which I'm still skeptical is audible, or at least as audible as Amir's measurement suggests it should be.

 

[TK750]

I'm sorry but starting a conflict based on different points of view and perceived bias is not what I would consider useful for either of us.

However reviewing a headphone in the Caldera, without understanding and doing a minimum or work to see there are various pads and a tuning system that will make it closer comply with the target (with caldera ultra perf pads) you use is not responsible, and it should be your goal to accurately present information regardless of the source of gear.

Haven't read the rest of the thread and not sure I dare to go into it, so sorry if it has already been mentioned, but I don't agree with this at all. If you expect all the pads (6 pairs) to be tested, don't you think you should include them with the $3.5k headphones at least? From looking at the site there is a pair included and one additional pair of the purchaser's choice thrown in. In order to try the full range you mention, a customer needs to spend an additional 4 * 80 = $320 on pads. I also noticed you mentioned the mesh, which whilst it is much more reasonable at $20, again should it not be thrown in if you are wanting/expecting people to experiment with these options and find what suits? Perhaps I'm cynical but it seems more a case of, if they are willing to pay $3.5k for headphones, a few more hundred dollars for pads will be nothing to them...

 

[zach915m]

Haven't read the rest of the thread and not sure I dare to go into it, so sorry if it has already been mentioned, but I don't agree with this at all. If you expect all the pads (6 pairs) to be tested, don't you think you should include them with the $3.5k headphones at least? From looking at the site there is a pair included and one additional pair of the purchaser's choice thrown in. In order to try the full range you mention, a customer needs to spend an additional 4 * 80 = $320 on pads. I also noticed you mentioned the mesh, which whilst it is much more reasonable at $20, again should it not be thrown in if you are wanting/expecting people to experiment with these options and find what suits? Perhaps I'm cynical but it seems more a case of, if they are willing to pay $3.5k for headphones, a few more hundred dollars for pads will be nothing to them...

We talk with the customer about their use and preferences before purchase to make sure they get the configuration that will work best for them, in most cases if there's a preference that they don't get with the headphone, it would usually be one additional set of pads and/or mesh at a discounted rate (given through email). In this case if we were told the preference was Harman, we would've immediately suggested the ultra perf and to EQ the bass to taste. This is the typical communication that usually happens between us and our users/owners/reviewers. In the case of a reviewer it would be sent out regardless of the source of headphone for no cost.

 

[TK750]

We talk with the customer about their use and preferences before purchase to make sure they get the configuration that will work best for them, in most cases if there's a preference that they don't get with the headphone, it would usually be one additional set of pads and/or mesh at a discounted rate (given through email). In this case if we were told the preference was Harman, we would've immediately suggested the ultra perf and to EQ the bass to taste. This is the typical communication that usually happens between us and our users/owners/reviewers. In the case of a reviewer it would be sent out regardless of the source of headphone for no cost.

I can understand that, do you have a rough estimate of how many people that buy headphones from you actually get in contact with regards to this at all? I suppose I'd also have to counter with a lot of customers are likely (and understandably) unsure about what their actual preferences are or have been misled, I still think they should be included really. I appreciate a business needs to make money however and I am not the intended market for ZMF, appreciate the response.

 

[Benesyed]

I think we can all agree too the below assessment:

If you want a default Harman tuned headphone then it is not for you, especially at a sub 500 price.

If you don't mind deviations and or use EQ, then it might be worthwhile to look into it if you like the non FR regard aspects (build, comfort etc). Especially if cost isn't an issue.

In a cost conscious decision making it's really hard to justify though it is anywhere from 400-700 dollars cheaper than the DCA expanse so you have to ask how much EQ freedom means to you.

Full disclosure I ordered it because I liked it when I heard it and seeing that it responds well to EQ helped seal the deal but I don't need to be cost conscious and it serves a purpose as a nice piece of woodwork/sculpture in my home office. Also on my head it's more comfortable (from when I auditioned both) than the expanse which I use for shorter listening sessions.

 

[zach915m]

I'd also have to counter with a lot of customers are likely (and understandably) unsure about what their actual preferences are or have been misled

There's quite a lot of info subjective and objective on the web now about the Caldera. Most users spending that much on a headphone either know their preferences or reach out, or post on a forum somewhere and get help from other owners. Since we go to Canjams and shows/meets a lot many owners hear it ahead of time before buying.

In the rare case they don't have info ahead of time and aren't satisfied they reach out afterwards and we help out. Here's another recent informative review by techpowerup with objective measurements and subjective impressions that is more thorough: https://www.techpowerup.com/review/zmf-caldera-open-back-planar-magnetic-headphones/

 

[L0rdGwyn]

Caldera sounds very good without EQ to me here is my pair in Kingwood (rosewood), driven by a tube amp of my design, along with a NOS DAC of my design with a tube output stage. The horror! Shield your eyes lest you go blind