ZMF Caldera Headphone Review

[amirm]

We as headphone audiophiles are a very particular group, and what I've found is that many headphone audiophiles are not part of that 64 percent, and ZMF certainly wouldn't exist of those 64 percent were such a majority.

That is just an assumption. You don't know it factually. Only a controlled experiment where people don't know what they are listening with statistical rigor tell you that. There are all kinds of assumptions like this that are proven wrong. Harman automotive group (Becker) for example was bidding for the audio system of a Japanese car manufacturer. They were competing against Japanese audio companies claiming that Asian listeners had different taste than what Harman assumed so they should get the business. Dr. Olive set up a controlled test and demonstrated there were no such distinct preferences. And with it, won that large contract. FYI they binaurally captured the sound of automotive systems and played that back using headphones in front of both western and Asian listeners, showing that result.

Such arguments were also made regarding tonality of speakers and again, proven wrong:

As you see above, a number of different categories of listeners were recruited from audio reviewers to students and Harman trained listeners. While the absolute scores were very different (trained listeners are much more critical of tonality errors), the relative ranking of the speakers did not change. Speaker M was the worst of the bunch across all listener groups for example (green).

The current high-end headphone market is heavily influenced by what people read and watch on youtube. People can be convinced anything is good and a business made around it. We live this everyday as we lift the curtain and show the true nature of audio products. Owner after owner who has sent products to me is surprised and admit that they just followed the crowd and wouldn't make the same decision knowing what they know now.

So please don't rely on such assumptions. It is not hard to verify them. Get one of your headphones, properly calibrate it to Harman's (using the right test fixture), and do controlled, blind listening tests. I think you will be surprised at the results.

 

[Benesyed]

That is just an assumption. You don't know it factually. Only a controlled experiment where people don't know what they are listening with statistical rigor tell you that. There are all kinds of assumptions like this that are proven wrong. Harman automotive group (Becker) for example was bidding for the audio system of a Japanese car manufacturer. They were competing against Japanese audio companies claiming that Asian listeners had different taste than what Harman assumed so they should get the business. Dr. Olive set up a controlled test and demonstrated there were no such distinct preferences. And with it, won that large contract. FYI they binaurally captured the sound of automotive systems and played that back using headphones in front of both western and Asian listeners, showing that result.

Such arguments were also made regarding tonality of speakers and again, proven wrong:

As you see above, a number of different categories of listeners were recruited from audio reviewers to students and Harman trained listeners. While the absolute scores were very different (trained listeners are much more critical of tonality errors), the relative ranking of the speakers did not change. Speaker M was the worst of the bunch across all listener groups for example (green).

The current high-end headphone market is heavily influenced by what people read and watch on youtube. People can be convinced anything is good and a business made around it. We live this everyday as we lift the curtain and show the true nature of audio products. Owner after own who has sent products to me is surprised and admit that they just followed the crowd and wouldn't make the same decision knowing what they know now.

So please don't rely on such assumptions. It is not hard to verify them. Get one of your headphones, properly calibrate it to Harman's (using the right test fixture), and do controlled, blind listening tests. I think you will be surprised at the results.

or even simply have two separate Caldera stations one with equalization applied and one without and see which one most listeners preferred without telling them which one is which.

although ultimately perhaps none of this is truly all that necessary because the overall excellent design of the caldera allows it to be EQd to Harmon. this is clearly not a headphone that you use on the go so anyone can apply equalization to it easily in the use case and get the full benefits of the Harman tuned headphone and enjoy the craftsmanship etc should they desire.

 

[Benesyed]

That is just an assumption. You don't know it factually. Only a controlled experiment where people don't know what they are listening with statistical rigor tell you that. There are all kinds of assumptions like this that are proven wrong. Harman automotive group (Becker) for example was bidding for the audio system of a Japanese car manufacturer. They were competing against Japanese audio companies claiming that Asian listeners had different taste than what Harman assumed so they should get the business. Dr. Olive set up a controlled test and demonstrated there were no such distinct preferences. And with it, won that large contract. FYI they binaurally captured the sound of automotive systems and played that back using headphones in front of both western and Asian listeners, showing that result.

Such arguments were also made regarding tonality of speakers and again, proven wrong:

As you see above, a number of different categories of listeners were recruited from audio reviewers to students and Harman trained listeners. While the absolute scores were very different (trained listeners are much more critical of tonality errors), the relative ranking of the speakers did not change. Speaker M was the worst of the bunch across all listener groups for example (green).

The current high-end headphone market is heavily influenced by what people read and watch on youtube. People can be convinced anything is good and a business made around it. We live this everyday as we lift the curtain and show the true nature of audio products. Owner after owner who has sent products to me is surprised and admit that they just followed the crowd and wouldn't make the same decision knowing what they know now.

So please don't rely on such assumptions. It is not hard to verify them. Get one of your headphones, properly calibrate it to Harman's (using the right test fixture), and do controlled, blind listening tests. I think you will be surprised at the results.

Could you clarify the groupings what is Aced1 vs Aced2?

 

[IAtaman]

I keep an open mind when tuning our headphones to find what works best for each system.

I am sorry, maybe others know who you are and what is your role in the development and tuning of this headphone but I am not aware of that, so would be glad if you could clarify.

One thing that does change the peaks and valleys especially in the 1khz+ area is spinning the Caldera pads during measurement (circular around the baffle to adjust the z axis of the pad and where it lands) which can also be done in use on your head. The Caldera pads are Asymmetrical purposely to adjust peaks and valleys via fit to each person's HRTF/anatomy and it is worthwhile when doing during measurement as well.

Oh, interesting. Does this mean if the pads were spun around to another position, the measurement results would have been different, or do you mean that the measurements published here do not reflect the tuning you/ZMF were going for?

 

[amirm]

Could you clarify the groupings what is Aced1 vs Aced2?

I believe they are all university students. This is the make up:

"The second group (S) consisted of 14 university students
from two California universities. One group (CALP)
consisted of undergraduate electrical/mechanical engineering
students with an interest in audio engineering. The
other student group (UC) was enrolled in programs
preparing them for careers in music and recording industries.
Based on personal observations it would be safe to
say that the student group was the youngest group in this
study and had the least amount of experience judging the
sound quality of loudspeakers."

 

[solderdude]

I am sorry, maybe others know who you are and what is your role in the development and tuning of this headphone but I am not aware of that, so would be glad if you could clarify.


Oh, interesting. Does this mean if the pads were spun around to another position, the measurement results would have been different, or do you mean that the measurements published here do not reflect the tuning you/ZMF were going for?

Well... when one has played with drivers, tuning headphones, pads (I believe he made his own pads) and has a few test fixtures and a keen interest to learn about this and has many years of experience it is quite likely one can get good results.

Also there are quite a few people (amongst them his buyers) that do not want/desire Harman bass or want a 'mellower' tuned headphone to be used without EQ.
Chances are most of their buyers do not use EQ anyway but prefer the headphone to color it the way they like it.

I reckon if Zach really wanted to (and now he may have to) could probably design a headphone that could follow the Harman curve. Dan can so he is the man.
Could be a nice challenge and maybe he could even get a few potential customers this way.

That said.... EQing a cheaper headphone is cheaper but may not be as comfy nor look the way the ZMF headphones do.

 

[all24bits]

The current high-end headphone market is heavily influenced by what people read and watch on youtube. People can be convinced anything is good and a business made around it. We live this everyday as we lift the curtain and show the true nature of audio products. Owner after owner who has sent products to me is surprised and admit that they just followed the crowd and wouldn't make the same decision knowing what they know now.

Are you literally saying people enjoy their headphones until you measure them and then they decide based on what you say and show about them that they don't enjoy the sound of them even tho they did before ?

They follow one crowd and then jump into following another crowd?

Because I've never done this. I heard the 1266 and enjoyed it even before I read here that the headphone measured poorly and no one should buy it. Then after I read all the negative views here, I still enjoyed the headphone.

Are you saying the readers don't make up their own minds and just edit their tastes to whatever ASR says they should enjoy or dislike?

 

[amirm]

Are you literally saying people enjoy their headphones until you measure them and then they decide based on what you say and show about them that they don't enjoy the sound of them even tho they did before ?

Once they equalize them to Harman response, their response is that the sound is much more pleasing now. You don't get to invert that and say the old sound was not enjoyable. Many of us enjoy music at varying levels of fidelity. I like music in my car even though it doesn't hold a candle to my system at home. So your statement is non-squitter. It is not binary.

 

[amirm]

They follow one crowd and then jump into following another crowd?

Only if you get your logic out of fortune cookies..... We are evidence, science and engineering based. The other is not.

 

[solderdude]

Are you literally saying people enjoy their headphones until you measure them and then they decide based on what you say and show about them that they don't enjoy the sound of them even tho they did before ?

They follow one crowd and then jump into following another crowd?

Because I've never done this. I heard the 1266 and enjoyed it even before I read here that the headphone measured poorly and no one should buy it. Then after I read all the negative views here, I still enjoyed the headphone.

Are you saying the readers don't make up their own minds and just edit their tastes to whatever ASR says they should enjoy or dislike?

More often than not a proper EQ can improve the sound quality.
Of course the wrong EQ can make it a little better or even worse in some aspects.
Then there is the difference in ear-gain between people and preference for a certain coloration.

 

[Benesyed]

Are you literally saying people enjoy their headphones until you measure them and then they decide based on what you say and show about them that they don't enjoy the sound of them even tho they did before ?

They follow one crowd and then jump into following another crowd?

Because I've never done this. I heard the 1266 and enjoyed it even before I read here that the headphone measured poorly and no one should buy it. Then after I read all the negative views here, I still enjoyed the headphone.

Are you saying the readers don't make up their own minds and just edit their tastes to whatever ASR says they should enjoy or dislike?

I think the point he is making that in general most people if blinded would like the Harman EQd version more and so is a "better" default standard. Even many who say they don't in a sighted setting.

If you have tried the EQ and you don't like it then that is fine and that is great. Whether it is from bias or not ultimately doesn't matter since you will not be listening blinded anyway.

 

[all24bits]

Once they equalize them to Harman response, their response is that the sound is much more pleasing now. You don't get to invert that and say the old sound was not enjoyable. Many of us enjoy music at varying levels of fidelity. I like music in my car even though it doesn't hold a candle to my system at home. So your statement is non-squitter. It is not binary.

What "decision" would the owner of the headphones make differently then? Would they have purchased a different headphone or would they have EQ'd differently?

 

[Thomas_A]

What "decision" would the owner of the headphones make differently then? Would they have purchased a different headphone or would they have EQ'd differently?

Many prefer to get a headphone that is close to the target as possible, other bother with EQuing headphones. The Harman target is based on preferred average response that mimics (good) speaker response in a small room. Situation could be different for large hall live recordings, binaural recordings or for the studio technician. But not for the majority of listener preference. I an satisfied with my Bose QC25 and DT150 with DT100 pads, except for binaural recordings. Then I think bass is too much.

DT150_DT100 pads with in-ear microphones, my ears, with correction curve for my speaker-room response. Not accurate >5 kHz though:

The same using Bose QC25. More low end bass due to active correction.

 

[Mulder]

None of that matters if you get tonality wrong, putting aside some of what you say being nonsensical. The typical reviewer gushing about soundstage width and such is totally imaginary, or made up. And at any rate, is listener dependent. You are also wrong in saying no one commented on it. I did in the review:


Spatial effects refers to separation of instruments outside of left and right ear. By the way, if you care about those effects, then the response of this headphone works against you. Per my review comment, those effects only manifested themselves once I filled the holes in lower treble. That is where a lot of channel separation is in your content and where your ability to locate things exists.

Texture of bass and other things like that are just audiophile nonsense. They have no place in this forum.

That is just an assumption. You don't know it factually. Only a controlled experiment where people don't know what they are listening with statistical rigor tell you that. There are all kinds of assumptions like this that are proven wrong. Harman automotive group (Becker) for example was bidding for the audio system of a Japanese car manufacturer. They were competing against Japanese audio companies claiming that Asian listeners had different taste than what Harman assumed so they should get the business. Dr. Olive set up a controlled test and demonstrated there were no such distinct preferences. And with it, won that large contract. FYI they binaurally captured the sound of automotive systems and played that back using headphones in front of both western and Asian listeners, showing that result.

Such arguments were also made regarding tonality of speakers and again, proven wrong:

As you see above, a number of different categories of listeners were recruited from audio reviewers to students and Harman trained listeners. While the absolute scores were very different (trained listeners are much more critical of tonality errors), the relative ranking of the speakers did not change. Speaker M was the worst of the bunch across all listener groups for example (green).

The current high-end headphone market is heavily influenced by what people read and watch on youtube. People can be convinced anything is good and a business made around it. We live this everyday as we lift the curtain and show the true nature of audio products. Owner after owner who has sent products to me is surprised and admit that they just followed the crowd and wouldn't make the same decision knowing what they know now.

So please don't rely on such assumptions. It is not hard to verify them. Get one of your headphones, properly calibrate it to Harman's (using the right test fixture), and do controlled, blind listening tests. I think you will be surprised at the results.

How many subjects participated in the survey in total? What level of statistical significance does the result of the survey have, i.e. what is the range?

 

[Thomas_A]

How many subjects participated in the survey in total? What level of statistical significance does the result of the survey have, i.e. what is the range?

There are error bars on the plot, but if it includes errors for multiple testing is not clear to me (e.g. selected point to point difference may or may not be significant). Nevertheless, the trend is very clear from the graph.

 

[Mulder]

There are error bars on the plot, but if it includes errors for multiple testing is not clear to me (e.g. selected point to point difference may or may not be significant). Nevertheless, the trend is very clear from the graph.

I just realised the plot is for speakers an not for headphones. I rephrase the question. What is the statistical probability (and in what range) that a randomly selected individual prefers a headphone with a sound profile according to the harman curve?

 

[Thomas_A]

I just realised the plot is for speakers an not for headphones. I rephrase the question. What is the statistical probability (and in what range) that a randomly selected individual prefers a headphone with a sound profile according to the harman curve?

There is a summary here:

https://acousticstoday.org/wp-content/uploads/2022/03/The-Perception-and-Measurement-of-Headphone-Sound-Quality-What-Do-Listeners-Prefer-Sean-E.-Olive.pdf

 

[Mulder]

There is a summary here:

https://acousticstoday.org/wp-content/uploads/2022/03/The-Perception-and-Measurement-of-Headphone-Sound-Quality-What-Do-Listeners-Prefer-Sean-E.-Olive.pdf

Thanks. After reading the paper I can draw some conclusions.

It can be stated that, according to the study, a randomly selected individual with a 64 percent probability prefers a headphone that matches the Harman-curve. But it is determined with equal objectivity that a minority of 36 percent prefer a different profile (some prefer less bass, some prefer more)

So it is not so simple that you can say that there is a right and a wrong based on the Harman curve. As Olive himself states there is a strong need to be able to adapt, above all, the lower frequencies to individual preferences, but also to different program material - precisely because there is no uniformity when it comes to recorded material. Some are older material that we would like to continue to listen to.

The sample consisted of 130 listeners, approximately half were trained and the other half were untrained. This means that subgroups such as age, gender (women) are probably quite small. (It's not really clear what I could read how small) Olive's research is solid, but in an ideal world it would need to be supplemented. But as Olive points out, conducting these types of studies is cumbersome and expensive. And probably not highly prioritized by the industry (hello Apple!!)

This is how Olive writes himself: "As expected, there are also critics whose headphone tastes in sound may not agree with
the research. The Harman target is intended as a guideline and is not the last word on what makes a headphone sound good.
One legitimate criticism is the limited number of headphones, programs, female listeners tested, and questions raised about the
confluence of variables like hearing loss and its effect on headphone preference. Future studies will hopefully
address this. Finally, I hope that this article encourages others to continue the research and improve our knowledge
of the perception and measurement of headphones sound quality. Although listeners largely agree on what
makes a headphone sound good, there are still many unanswered questions and more to learn."

 

[Madlop26]

Thanks. After reading the paper I can draw some conclusions...

Well, thank you for the review, Harman study is certainly the best we have for factual reality in a sea of subjectivism, but let's face it, the sample was not too big, at least not enough to know if the troublesome subgroup of people which I myself belong to, males over 50s.; differs any from the Harman curve. Let's hope someday, somebody does additional research.

 

[IAtaman]

Well, thank you for the review, Harman study is certainly the best we have for factual reality in a sea of subjectivism, but let's face it, the sample was not too big, at least not enough to know if the troublesome subgroup of people which I myself belong to, males over 50s.; differs any from the Harman curve. Let's hope someday, somebody does additional research.

What was the average age of listeners who rated headphones in the Harman research?