why don't we graph DACs ? ( i did !)

[JeffS7444]

Measuring IEMs can be very tricky, as inserting them a little bit differently into the measurement instrument can affect the measured results. When using MiniDSP's EARS device, once I get a fit which looks pretty good, as much as possible, I do measurements without touching anything, particularly not the IEMs.

 

[Matias]

There is no room involved though

I know. The point is that averaging a random signal is a valid measurement.

 

[MaxwellsEq]

yes i did put the eartip on the mic hole on the laptop (seal comes from the silicon eartip elasticity), redid the sampling 3 times to make sure it was not changing when i reapply the eartip on the mic hole

Some time ago I had to design a headphone level limiting approach (to prevent hearing damage) which meant I had to measure hundreds of different headphones.

I rapidly discovered that repeatability is almost impossible. Even in a properly equipped lab with clamps, professional microphone mechanical isolation, acoustic isolation, accurate sound sources and precision measuring gear, I would get significant variations run-to-run and over time.

While I admire your idea, based on my experience, run-to-run variations will swamp 10x or 100x the difference between two properly working DACs with optimal filtering.

 

[solderdude]

From my experience doing MMM measurements in room, after a few seconds and some 30 samples the curve is quite static and not changing anymore. I doubt that adding hundreds of samples during 2 minutes would change the curve much.

For speakers yes as that is one pile of mess anyway. For headphones you get results much closer to a sweep. I have played with this extensively because some folks claimed noise would be more telling for headphones as that is closer to music. Turns out that was not the case when you average the (random) noise long enough.

Why would that be?

It would depend on whether you use averaging (which will show the average droop) or you use peak level in which case the droop is hardly visible.
The reason for that is the sampling frequency, when doing a long measurement, will always have some high frequency amplitudes that make it all the way up to '0dB'.
Averaging and sweeps will not.

 

[amanieux]

Some time ago I had to design a headphone level limiting approach (to prevent hearing damage) which meant I had to measure hundreds of different headphones.

I rapidly discovered that repeatability is almost impossible. Even in a properly equipped lab with clamps, professional microphone mechanical isolation, acoustic isolation, accurate sound sources and precision measuring gear, I would get significant variations run-to-run and over time.

While I admire your idea, based on my experience, run-to-run variations will swamp 10x or 100x the difference between two properly working DACs with optimal filtering.

you are targetting scientific perfection, i just wanted rough data, noticing a +6db@50hz confirms my impression of hip2 sounding warmer than dawn 4.4, the feeling of harshness on dawn 4.4 maybe more complex to identify looking at the graphs

 

[solderdude]

I don't see the difference in 'warmth' between the 2 plots. The Moondrop appears to have less treble.
Bass extension difference could just as well be a less perfect seal against the mic of the laptop in the first plots you posted.

It is not possible that there is a +6dB @ 50Hz unless it is hum or some bass boost is accidentally switched on.
Even a highish output R can not do this. And both will be low output R anyway.

 

[voodooless]

you are targetting scientific perfection, i just wanted rough data, noticing a +6db@50hz confirms my impression of hip2 sounding warmer than dawn 4.4, the feeling of harshness on dawn 4.4 maybe more complex to identify looking at the graphs

That’s just confirmation bias. Without proper measurement you cannot with certainty say that the + 6 dB is real.

… garbage in, garbage out…

And even if confirmed, there is still no proof that that is truly what you hear. For that you’ll need further investigation, for instance by EQ’ing them to the same leven and then comparing again. And don’t forget level matching!

 

[MaxwellsEq]

you are targetting scientific perfection, i just wanted rough data, noticing a +6db@50hz confirms my impression of hip2 sounding warmer than dawn 4.4, the feeling of harshness on dawn 4.4 maybe more complex to identify looking at the graphs

I was not "targetting (sic) scientific perfection". I was doing a job as a qualified professional electronic engineer working in a studios complex dealing with a safety issue.

I would have been happy with a reliable +/- 3dB, but it was almost impossible to get that. I take my hat off to anybody who does a good job of headphone measurements. If you read Amir's headphones reviews, you will see how difficult he finds it to get consistent results with some devices!

Your idea may be valid and possibly useful, but the variances you are posting are evidence that your methodology is sufficiently flawed that you should not draw any conclusions.

If I was you and wanted to make this a useful approach. I would:

• Find a way to reliable couple the two devices in a way that they cannot be disturbed even by 1/10th of a millimetre (since that is the tolerances you will need to work with) e.g. by gluing them together;
• Make sure the coupled devices are mechanically isolated from their surroundings (holding them together with your hands is absolutely certain to mess up any measure);
• Ensure the coupled devices are a long way from any electrical interference sources;
• Fix the cables so they cannot be moved and don't pass mechanical perturbances to the coupled devices;
• Measure the signals using 400Hz and 1kHz and ensure both devices are giving the same output +/- 0.1V
• If you are using a noise source, pick a good one which has excellent randomness and run each test for several minutes
• Alternatively use REW and do frequency sweeps
• Repeat each test 20 times and create an averaged plot
• Look for any outliers, they can be interesting

 

[wwenze]

One additional thought... Noise is randomness. (Pink noise is filtered randomness.) That makes it hard to get precise measurements, or to get the same-exact results twice.

You are getting obvious differences with your setup so that's not the main concern here but a frequency-sweep can give much more accurate-precise frequency response.

It's randomness with expected behavior... here is a pink noise generated and measured by Audacity, everything in the data domain

Frequency sweep vs noise is an issue that would be way out of the scope of this thread. It's for certain that with noise, due to the way averaging works, unless you select a long duration to FFT right from the start (above is 2.5 minutes) the graph will be very squiggly, but still within a few dB of the expected trend.

 

[wwenze]

yes i did put the eartip on the mic hole on the laptop (seal comes from the silicon eartip elasticity), redid the sampling 3 times to make sure it was not changing when i reapply the eartip on the mic hole

That's not good enough. You need more hands to do this properly

The IEM needs to be at exactly the same spot for both DAC measurements. Removing it to switch the DAC and then placing it back is not precise enough.

What ^ said.

This sort of situation is exactly why when you "redo the sampling", you should be removing it to switch the DAC, every time you sample. Then it counts as a sample. Basically whatever you did to the second DAC each time, you need to do to the first DAC each time. That way you would have observed and concluded if a variation occurred from physically moving things around if the measurements of DAC A don't match.

Of course this is usually only relevant if you observed a difference in the first place. Example, technically shifting a cable around results in measurable difference. But if the difference doesn't show up or is in frequency or amplitude ranges of no consequence, then it's whatever.

 

[pablolie]

Does anyone really care about DAC measurements anymore? There is litle difference between perfection and even more perfection. It amounts to an obsession with measurements in an area where it is no longer relevant. Speaker measurements? Oh yes please (but also please start including meaurements with matched subs). Amps? Absolutely. DACs? It's like trying to measure perfect sun-sets in Kauai, or aurora borealis in Iceland. Spectacular stuff, even a bad one is pretty darn good.

 

[wwenze]

Does anyone really care about DAC measurements anymore? There is litle difference between perfection and even more perfection. It amounts to an obsession with measurements in an area where it is no longer relevant. Speaker measurements? Oh yes please (but also please start including meaurements with matched subs). Amps? Absolutely. DACs? It's like trying to measure perfect sun-sets in Kauai, or aurora borealis in Iceland. Spectacular stuff, even a bad one is pretty darn good.

Because people hear "hip2 sounding warmer than dawn 4.4, the feeling of harshness on dawn 4.4" and want to show their proof.

 

[antcollinet]

macbook white internal mic

wait what?

How are you coupling the IEM to the microphone?

 

[antcollinet]

you are right i was looking for a way to get objective data to confirm my subjective impression when switching dacs on my iem, when things sounds different we must always try to get rid of the placebo effect that is always present.

The only way to do that is to test blind.

The problem with hearing something then measuring for a difference, is you have no idea if the difference you measure is the cause of the difference you hear.

 

[kemmler3D]

noticing a +6db@50hz confirms my impression of hip2 sounding warmer than dawn 4.4, the feeling of harshness on dawn 4.4 maybe more complex to identify looking at the graphs

How many runs did you do? I have found that fit affects measured bass a lot with headphones, often much more than +/- 6dB.

Can you get the same measurement from the same DAC on two different insertions of the IEM? Not sure if we saw that ITT yet, apologies if I missed it.

 

[thulle]

the chain is simple Pc->DAC->iem(through bal cable)

No, the chain doesn't stop at the IEM, next it goes from IEM to mic, and I asked specifically that step, as other have filled in

How do you rule out different angle of the IEM? Different position relative to the mic? Different interaction with the environment? You don't even mention it and then point straight to the DAC as the culprit in the whole chain.

you are targetting scientific perfection, i just wanted rough data, noticing a +6db@50hz confirms my impression of hip2 sounding warmer than dawn 4.4, the feeling of harshness on dawn 4.4 maybe more complex to identify looking at the graphs

Why this reluctance to challenging yourself and your test, instead challenging and dismissing the critique? I'm just stunned at this reply. MaxwellsEg's saying that they worked their ass off to rule out things interfering with the measurements, and any DAC differences just drowns in the other stuff anyway. In response you complain it's a lot of work. You're basically going "nope, my single coinflip confirms my impression that coins always land heads up and that's good enough for me".

 

[Matias]

What is the point of IEM insertion into the coupler if it stays unchanged, as only the DAC dongle is changed at the other end of the cable?

 

[solderdude]

The issue in this test is that there is no coupler.
OP just presses the IEM to the mic of his laptop.
No controls for mechanically induced noises (pressing), no similar seal, no similar positioning which are nuisance variables you don't want.
The 'detected' +6dB at 50Hz for instance probably is not there at all and highly unlikely. The difference in treble is probably real and due to the used reconstruction filter.
OP heard a difference and part of it could be explained but not by the first measurements and now we know why.

When he would be using a coupler that stays stable (mic in tube) and only cares about differences and averages long enough the test is valid.
You would not be comparing DACs but DACs + internal amplifiers.

 

[antcollinet]

You would not be comparing DACs but DACs + internal amplifiers

Plus distortion of the IEM, likely swamping any difference's between dacs. Not to mention environmental noise picked up by the microphone.

This method of comparison is meaningless.

 

[amanieux]

I don't see the difference in 'warmth' between the 2 plots. The Moondrop appears to have less treble.
Bass extension difference could just as well be a less perfect seal against the mic of the laptop in the first plots you posted.

It is not possible that there is a +6dB @ 50Hz unless it is hum or some bass boost is accidentally switched on.
Even a highish output R can not do this. And both will be low output R anyway.

Sorry i was not clear, i mean the bass starts 6bd higher on the hip2@50hz to get to the same level@130hz