Resolve's B&K 5128 Headphone Target - you can try the EQ's.....

[Robbo99999]

View attachment 274739
Well it says Harman over ear 2018 in Auto EQ in Peace.


Yes, indeed. I need to check what I have saved vs the PDFs on dropbox to make sure I was comparing vs Harman tuning. I have to say, I am a bit disappointed AutoEQ is not reliable as it was very convenient.

Just be safer to use Oratory's latest EQ's from his reddit site, avoid all pitfalls that way.....and he also does update his pdf's from time to time, so best to check his actual site rather than downloading one of his pdf's and assuming it will just stay the same over the course of months or a year.

It's somewhat related, although I can see why you'd say that. A change in positioning showing a slightly different result isn't exhaustively indicative of the degree of variability on different heads (given the ears and coupling would be different too), however you can still get a sense of how that would change given the differences in default wearing positions across users. As an example, for some headphones, like the HD 560 S, or the Focal closed backs, extremely subtle seating changes can yield distinctly different results. I think we can expect those effects to vary across users as well. Of course, that doesn't include all variables, but it's at minimum an indication that we should probably be less confident about those fine-grained minutia on those headphones versus say... and HD 800 S where the same subtle change doesn't yield any meaningful difference, since they're less predictive as a result.

I think it's a reasonable suggestion from you to think there could be some relation between positional variability on a measurement fixture vs variability between users, although I don't think you can be certain until the measurements are done on different people. And I agree that if a headphone shows more positional and also more interpersonal variation then as you say you can't be as confident in the fine grained minutia re those particular headphones.

Re variability of various headphones between reseatings on measurement rigs, I thought I'd quickly drop some pictures showing reseat variability of a number of my headphones, to me the HD560s had the lowest variation between reseats vs my other headphones, and that was including the K702 which has been proven to be very reliable in terms of variation between different people's heads (well the K701 which is the same design was measured in the published study in on head deviation between different heads), so here some quick thumbnails of my headphones measured on my miniDSP EARS (showing all measurements & of both channels on each graph) showing the HD560s to have the least deviation (by eyeball) between reseats and even compared to the K702 which is supposed to be very stable with positioning and between different heads:

These were all measured in a roughly central position to avoid pinna deformation in a bid to replicate how a user would intelligently place a headphone on their own heads, which they'd do with some accuracy as they can feel how their ears sit within the earcup. 20 measurements in all (10 for each channel), and 10 reseats for each headphone. In terms of reseat variation to me it looks like closed back NAD HP50 is clearly the worst with the highest variation between reseats, then open back planar HE4XX, then open back K702, then the best with least reseat variation the open back HD560s. So this does seem to conflict with what you've been saying re HD560s reseat variability vs other headphones.
(I did all these measurements almost exactly 1 year ago)

 

[solderdude]

. So this does seem to conflict with what you've been saying re HD560s reseat variability vs other headphones.

Consider Resolve meant that the HD560S driver being angled and not big in diameter it will have a bit more varying pinna-effects which will differ between actual humans vs a fixture.

 

[Robbo99999]

Consider Resolve meant that the HD560S driver being angled and not big in diameter it will have a bit more varying pinna-effects which will differ between actual humans vs a fixture.

That could well be true (especially if closed mic on head measurements of various headphones are forthcoming that support that), but Resolve has flip flopped between emphasising either the variation of HD560s between humans or positional variation seen whilst measured on a fixture. I was just showing that I get the least variation on a fixture using an HD560s vs my other headphones.

 

[usern]

here some quick thumbnails of my headphones measured on my miniDSP EARS showing the HD560s to have the least deviation (by eyeball)

Why eyeball? Is there not an option to plot +-standard deviation and mean of multiple measurements in REW?

 

[IAtaman]

Why eyeball? Is there not an option to plot +-standard deviation and mean of multiple measurements in REW?

You are going to need quite a bit of samples to be able to calculate any meaningful standard deviation no?

 

[IAtaman]

So I did check the the EQs that were supposed to be Oratory's 2018 Harman OE target curves which I have used to compare Resolve. Edition XS is corect, looks like I have entered it manually. HD600 is not. LCD-X is not either. Duh.

 

[Robbo99999]

Why eyeball? Is there not an option to plot +-standard deviation and mean of multiple measurements in REW?

I haven't got REW installed on this PC at the moment, you couldn't have a quick look in the menus could you & post up a screenshot of where that is located in the menus? (if it does exist). I think eyeballing is pretty valid to see overall trends in variation, and comparative variation in terms of how much variation is going on vs different samples.

You are going to need quite a bit of samples to be able to calculate any meaningful standard deviation no?

Well, it would be valid actually, because we're talking about variation between reseats of the same headphone, so it's not like you have to measure more than one unit of headphone. As long as you've done enough reseat measurements then it's valid (I did 20 measurements over 2 channels, 10 reseats, so that should be valid).

So I did check the the EQs that were supposed to be Oratory's 2018 Harman OE target curves which I have used to compare Resolve. Edition XS is corect, looks like I have entered it manually. HD600 is not. LCD-X is not either. Duh.

Ha, well if you like redo your testing on the affected headphones & report back, if you want to.

 

[solderdude]

That could well be true (especially if closed mic on head measurements of various headphones are forthcoming that support that), but Resolve has flip flopped between emphasising either the variation of HD560s between humans or positional variation seen whilst measured on a fixture. I was just showing that I get the least variation on a fixture using an HD560s vs my other headphones.

You have a flatbed fixture with non compliant coupler and positioning a headphone repeatedly in closely the same position is not the same as positioning it willfully in a different position and also not using different heads and different test fixture.
So Resolve might have seen a lot more variance with his fixture/experiments on different heads than you found on your head and fixture.

I think Resolve is definitely moving in the right direction and if I had the funds, sponsoring and time he seems to have I would have done some things in a very similar way as he is doing now. He might not have time, access to labs and possibilities Harman research has/had access to but given what he has to work with I like it.

 

[Robbo99999]

You have a flatbed fixture with non compliant coupler and positioning a headphone repeatedly in closely the same position is not the same as positioning it willfully in a different position and also not using different heads and different test fixture.
So Resolve might have seen a lot more variance with his fixture/experiments on different heads than you found on your head and fixture.

I think Resolve is definitely moving in the right direction and if I had the funds, sponsoring and time he seems to have I would have done some things in a very similar way as he is doing now. He might not have time, access to labs and possibilities Harman research has/had access to but given what he has to work with I like it.

You've conflated the arguments that were being made somewhat, but it doesn't matter, I'm not gonna pursue it with you. I don't agree with wilfully placing the headphone in extreme unnatural positions when measuring headphones as you wouldn't wear them like that, but if he did that then of course he would have seen more variation. Yes, I was placing mine roughly centrally, not pinpoint precisely, but roughly centrally - regardless of that it was the same process for all of my headphones and the HD560s saw the least reseat variation.

I too think that Resolve's work is commendable, I think it's interesting and possibly useful. As I said I've been enjoying his HD560s EQ with just a small 1.5dB bass shelf added at 70Hz.

 

[solderdude]

I don't agree with wilfully placing the headphone in extreme unnatural positions when measuring headphones as you wouldn't wear them like that

Some people do wear headphones in different positions for comfort or even sound quality reasons. Some headphones sound sharp in the optimal position or less sharp in the optimal condition or prefer some leakage or perfect seal. You and I may not do that or wear headphones that way but some do/might.
In the interest of research I can understand that. Even Tyll did that in his plots (the grey lines) so in the interest of finding target and perception the findings of Resolve make sense to me. So do yours, mine and every one else's for that matter. But I am sure Resolve has been experimenting more and different things than you, me and others. I would not dismiss or disagree with reported findings but being critical is a good thing, so is making someone else think or redo some testing.

 

[IAtaman]

I do wear the QC35s upside down (i.e. headband under my chin) during long flights when the top of the head starts to get too hot

 

[IAtaman]

I have been listening to HD600 with the correct Oratory EQ and comparing it to the Resolve EQ, again, for a while now. Luckily for me, the outcome is more or less the same; Resolve's EQ sounds a but darker mostly due to the pull down at 5.8K. Once that band is zeroed, tonality overall sounds good to me.

 

[isostasy]

You've conflated the argument somewhat but it doesn't matter. I don't agree with wilfully placing the headphone in extreme unnatural positions when measuring headphones as you wouldn't wear them like that, but if he did that then of course he would have seen more variation. Yes, I was placing mine roughly centrally, not pinpoint precisely, but roughly centrally - regardless of that it was the same process for all of my headphones and the HD560s saw the least reseat variation.

I too think that Resolve's work is commendable, I think it's interesting and possibly useful. As I said I've been enjoying his HD560s EQ with just a small 1.5dB bass shelf added at 70Hz.

I don't want to ask too much of resolve given I agree with solderdude, but ideally I'd want to see pictures of different seating positions of the headphones if they're obvious enough to warrant it. You've said 'extreme unnatural positions', I've been assuming so far this is not the case and Resolve has even said it's 'extremely subtle seating changes'. However, if there are some easily reproducible scenarios it would be really interesting to see pictures of this and the accompanying frequency response. e.g. with my HD6XX, there are two obviously different ways of putting them on my head: headband towards the front of the head, with the cups exactly vertical; or headband towards the back of the head so the the cups are slightly angled relative to vertical.

I admit I'm still not entirely satisfied with some of the explanations given by resolve, particularly: 'What folks need to realize with headphone measurements is that it’s really not worth caring about the fine-grained minutia, since it’s unlikely to be a predictive element'. But 'fine-grained minutia' is the only thing being addressed by the EQ profile in the treble response of the HD600 and HD650.

I haven't done stats for ages but don't you need to work out what the resolution of your equipment is first and then make sure you're not trying to do anything beyond that? In this case, you could say your resolution is limited by positional changes. Your HATS may be able to detect 0.1dB changes in response but if unit variation and positional changes are in the region of +-1dB you can't make any adjustments to a higher resolution than 1dB because they won't mean anything.

What I'm trying to say is that shouldn't the target response be more of a region than a line? And ideally this would depend on the headphone so you'd have a different target 'region' for each headphone depending on the resolution you can reasonably attain given unit variation and positional changes. What this would mean in practice is that if your target response for the HD650 ends up being around +-2dB above 500Hz you would look at the measurements you've got and conclude you're not going to EQ at all above 500Hz because any adjustments you make are less significant than unit variation and positional variation, and therefore you have no idea the real effect they will have, whether 'good' or 'bad', and more to the point, because you don't know what you're really doing at this resolution, the EQ may end up taking some units further away from the target curve.

Someone on reddit has collected different sources of 5128 measurements so there's now four HD650s (including a HD6XX) each of which, I think, support what I'm saying: there may be some for which the EQ brings the response slightly closer to the target, but others which it takes further away, whilst overall they were all close enough to begin with that no such narrow-band adjustment was necessarily 'needed', subjectively, to begin with.

Obviously if you have a B&K 5128 and can measure your own set and perfect the target response you can aim for a high resolution, but this isn't what we're doing: we're measuring on a 5128 and then asking for feedback from people putting their different headphones on in different positions.

The reason I'm still harping on about this is that it depends on what the purpose of the Community Input Thread is. I'm worried it will lead to some incorrect conclusions. For example, especially with the HD600 and HD650, you might conclude the treble response of your target curve is 'better' or more correct because most people who tried the EQ said they liked it. But the filters used are so small, to the effect of potentially being smaller than positional or unit variations, that you would be remiss to conclude this because all that's really being shown is that the filters didn't make anything worse.

Sorry this ended up being a bit of a mini essay. I appreciate the hard work but as solderdude said, trying to be constructively critical whilst accepting I don't know much so might be barking up the wrong tree anyway.

 

[usern]

There are scientific tools to investigate the variance between measurements. Mean (average) and standard deviation and graph it like here. Why not use them?

EDIT: This example is only meant to show how it would look like graphically (the middle graph), not referencing the content of the study.

This is more of a case of isolating positional differences than it is a case of significant unit variation.

Instead of isolating, you could show the variance by measuring multiple reseats and presenting mean and +-stdev.

Latest review by @amirm has the same issue.

I should note that there was fair amount of variability with the slightest change in the alignment.

If there is variability, then it could be shown instead of presenting just one measurement sample.

And the same question to @oratory1990 - do you present just one measurement in the SPL Frequency Response graphs or is it an average over multiple measurements? What about plotting standard deviation in the same graph?

I haven't got REW installed on this PC at the moment, you couldn't have a quick look in the menus could you & post up a screenshot of where that is located in the menus? (if it does exist).

I took a quick glance at REW GUI and I cannot see that it is possible to calculate stdev between different measurements.

 

[IAtaman]

There are scientific tools to investigate the variance between measurements. Mean (average) and standard deviation and graph it like here. Why not use them?

I read the post (thanks for the link by the way). It is not offering a standard deviation for each headphone, it is looking at different published measurements for a variety of headphones and calibrating them against oratory's measurements to calculate how much variation is there in the measurement done by different reviewers. The variants are probably caused by a mix of unit variation, different placements and seals, different measurement rigs with different ear canals and pinnas and face plates etc. I am not an expert but I think it would be very hard to untangle how much variation is coming from where. Everybody who knows the topic has been saying headphone measurements are not exact for a while now, I think it is about the time we accept that.

 

[IAtaman]

What I'm trying to say is that shouldn't the target response be more of a region than a line?

To your point, Oratory at least shows the target as a band rather than a line. That might cater for the "lack of resolution" as you called it on the vertical axis. There there is also the lack of frequency depended resolution of our hearing, i.e. how wide a peak/dip should be at different frequencies to have an audible signature. I think maybe smoothing of the target curve takes care of that, but given how our hearing works, shouldn't the smoothing need to be frequency dependent as well?

 

[usern]

I read the post (thanks for the link by the way). It is not offering a standard deviation for each headphone, it is looking at different published measurements for a variety of headphones and calibrating them against oratory's measurements to calculate how much variation is there in the measurement done by different reviewers.

I edited my post to clarify that it is just refernece how it can look like graphically. In this thread the question has been about positional variance and that should be simple enough to address with multiple measurements while reseating the headphons between each one.

 

[IAtaman]

I edited my post to clarify that it is just refernece how it can look like graphically. In this thread the question has been about positional variance and that is simple enough to do with multiple measurements while reseating the headphons between each one.

Yes, I think that is a fair point. How much a headphone is susceptible to positioning is an important piece of information, I would definitely like to know that. I suspect it might not be as straight forward however. Amir says in his headphone review preface for example, that he does not do averaging. I am not sure what is his logic. Maybe because you need a detailed protocol that will be hard to consistently follow, and probably will be incomplete no matter how pedactic you get anyway, given the wide variety of headphone sizes, shapes and material available? How to position, what counts as a valid measurement, if a headphone loses seal easily does it count as variation or not; should it be with new new pads or worn out pads; or maybe both to simulate how a headphone ages over time; what do you do with just released headphones, what do you do with on-ears; what about clamp force - do you measure different seatings for different head sizes; do you have to develop a separate protocol for different size and shape of headphones; if you need 8 data points to calculate std dev does this mean you need 8 measurements for each parameter to calculate a std dev for its impact on FR etc - which can be tricky to implement consistently I suppose even for the same person measuring it. Maybe we need to wait for fully robotized headphone measurement to arrive for all that

 

[Robbo99999]

I have been listening to HD600 with the correct Oratory EQ and comparing it to the Resolve EQ, again, for a while now. Luckily for me, the outcome is more or less the same; Resolve's EQ sounds a but darker mostly due to the pull down at 5.8K. Once that band is zeroed, tonality overall sounds good to me.

That's cool, at least you know you're using the right settings now......once you've tested all your headphones you could let him know over on Resolve's website (and here too).

I don't want to ask too much of resolve given I agree with solderdude, but ideally I'd want to see pictures of different seating positions of the headphones if they're obvious enough to warrant it. You've said 'extreme unnatural positions', I've been assuming so far this is not the case and Resolve has even said it's 'extremely subtle seating changes'. However, if there are some easily reproducible scenarios it would be really interesting to see pictures of this and the accompanying frequency response. e.g. with my HD6XX, there are two obviously different ways of putting them on my head: headband towards the front of the head, with the cups exactly vertical; or headband towards the back of the head so the the cups are slightly angled relative to vertical.

I admit I'm still not entirely satisfied with some of the explanations given by resolve, particularly: 'What folks need to realize with headphone measurements is that it’s really not worth caring about the fine-grained minutia, since it’s unlikely to be a predictive element'. But 'fine-grained minutia' is the only thing being addressed by the EQ profile in the treble response of the HD600 and HD650.

I haven't done stats for ages but don't you need to work out what the resolution of your equipment is first and then make sure you're not trying to do anything beyond that? In this case, you could say your resolution is limited by positional changes. Your HATS may be able to detect 0.1dB changes in response but if unit variation and positional changes are in the region of +-1dB you can't make any adjustments to a higher resolution than 1dB because they won't mean anything.

What I'm trying to say is that shouldn't the target response be more of a region than a line? And ideally this would depend on the headphone so you'd have a different target 'region' for each headphone depending on the resolution you can reasonably attain given unit variation and positional changes. What this would mean in practice is that if your target response for the HD650 ends up being around +-2dB above 500Hz you would look at the measurements you've got and conclude you're not going to EQ at all above 500Hz because any adjustments you make are less significant than unit variation and positional variation, and therefore you have no idea the real effect they will have, whether 'good' or 'bad', and more to the point, because you don't know what you're really doing at this resolution, the EQ may end up taking some units further away from the target curve.

Someone on reddit has collected different sources of 5128 measurements so there's now four HD650s (including a HD6XX) each of which, I think, support what I'm saying: there may be some for which the EQ brings the response slightly closer to the target, but others which it takes further away, whilst overall they were all close enough to begin with that no such narrow-band adjustment was necessarily 'needed', subjectively, to begin with.

Obviously if you have a B&K 5128 and can measure your own set and perfect the target response you can aim for a high resolution, but this isn't what we're doing: we're measuring on a 5128 and then asking for feedback from people putting their different headphones on in different positions.

The reason I'm still harping on about this is that it depends on what the purpose of the Community Input Thread is. I'm worried it will lead to some incorrect conclusions. For example, especially with the HD600 and HD650, you might conclude the treble response of your target curve is 'better' or more correct because most people who tried the EQ said they liked it. But the filters used are so small, to the effect of potentially being smaller than positional or unit variations, that you would be remiss to conclude this because all that's really being shown is that the filters didn't make anything worse.

Sorry this ended up being a bit of a mini essay. I appreciate the hard work but as solderdude said, trying to be constructively critical whilst accepting I don't know much so might be barking up the wrong tree anyway.

I can let you know roughly the tolerance in mm for my headphone measurements on my miniDSP EARS. There's some visible screws in the flat cheek section of the miniDSP EARS so they can act as useful "reference dots" to see where your headphone is lying in relation to them - based on me trying to put all my headphones roughly centrally on my rig I'd estimate I place my headphones +/-0.5cm, meaning there's probably a total range of 1cm max difference for any given point. I always looked at my headphone in fair detail after I'd reseated it on the rig out of interest to see how it was positioned in relation to the visible screws (sometimes they weren't visible though depending on placement or headphone, but it was pretty easy to guage what kind of +/- deviation I was getting from that. That doesn't seem like much but I know when I place my headphones on my head that I get it well within that zone, better than that. A lot of headphones don't have much room in the earcups so pretty much everyone is gonna be placing their headphones on their head so they have minimal ear to headphone contact, which doesn't leave much margin for deviation - I think 1cm range of deviation overall in that scheme of things is quite a lot. Certainly you raise a very useful & interesting point when you call for transparency on just how much variation in terms of placement that measurers like Resolve/Oratory/Crin, etc (anyone) is putting into their measurement protocol, it's a very valid point.

With your concerns on how different people position headphones on their heads, I've always thought of it as minimal ear contact. So I know from my own experience that I will adjust them in the ways you mention - headband rotated more forward or more back to change the angle of the "ear oval", lifting up the earcups & shifting them front/back/up/down - people just gonna swivel them round till they're most comfortable, ie least ear contact. The exception to this rule if they have some funky jaw bones going on or something that they know is preventing them getting a seal, in which case they move them into more extreme positions, but I think that's gonna be the minority for two reasons: number one because anatomically that's more likely to be an outlier and number two because that person has to be aware enough about seal to even do that in the first place, so I think we're talking minorities falling into that category.

For your headphone concerns re narrow band adjustments and their relevance - I reckon if you're concerned about this (re unit variation & person variation & positional variation) then don't use the fine-grained EQ in the treble - say for instance your headphone is averaging the Resolve target curve and instead the EQ is calling for fine band adjustments either side of that area, then try EQ in that area vs no EQ in that area and report on your findings. This is only relevant if your headphone is fluctuating in fine band conditions around the target curve on average, but if the average is deviating from target then you'd have to do something about that because then you're not anywhere close to the target - so in that situation you'd have to use the EQ or instead create your own wide band low Q EQ that will bring the average of your fine band fluctuations down to the target curve.

 

[Robbo99999]

I took a quick glance at REW GUI and I cannot see that it is possible to calculate stdev between different measurements.

Cheers, when I next install REW I'll remember to take a look to see if I can find something re stddev - there's no reason why such a statistical line on the graph couldn't be calculated from the data, so I'm sure it's within the capability of the software, as long as it's been programmed for it. It would be quite a useful tool.