why don't we graph DACs ? ( i did !)

[amanieux]

Does anyone really care about DAC measurements anymore? There is litle difference between perfection and even more perfection. It amounts to an obsession with measurements in an area where it is no longer relevant. Speaker measurements? Oh yes please (but also please start including meaurements with matched subs). Amps? Absolutely. DACs? It's like trying to measure perfect sun-sets in Kauai, or aurora borealis in Iceland. Spectacular stuff, even a bad one is pretty darn good.

Sorry about my loose terminology by dac i mean these products i am using (dongle dac, portable dac) that are in reality dac+amp, the difference must certainly come from the amp

 

[solderdude]

Sorry i was not clear, i mean the bass starts 6bd higher on the hip2@50hz to get to the same level@130hz

The problem is that I suspect you had a poor seal in the case of the first Moondrop measurements and this made you draw the wrong conclusion because it seemed to verify what you believe you heard.
This is evident in the following plots you made which did not show the early bass roll-off and in fact matched the ifi in the lows.
This clearly shows there is no tonal difference between the 2 devices.

 

[amanieux]

I was not "targetting (sic) scientific perfection". I was doing a job as a qualified professional electronic engineer working in a studios complex dealing with a safety issue.

I would have been happy with a reliable +/- 3dB, but it was almost impossible to get that. I take my hat off to anybody who does a good job of headphone measurements. If you read Amir's headphones reviews, you will see how difficult he finds it to get consistent results with some devices!

Your idea may be valid and possibly useful, but the variances you are posting are evidence that your methodology is sufficiently flawed that you should not draw any conclusions.

If I was you and wanted to make this a useful approach. I would:

• Find a way to reliable couple the two devices in a way that they cannot be disturbed even by 1/10th of a millimetre (since that is the tolerances you will need to work with) e.g. by gluing them together;
• Make sure the coupled devices are mechanically isolated from their surroundings (holding them together with your hands is absolutely certain to mess up any measure);
• Ensure the coupled devices are a long way from any electrical interference sources;
• Fix the cables so they cannot be moved and don't pass mechanical perturbances to the coupled devices;
• Measure the signals using 400Hz and 1kHz and ensure both devices are giving the same output +/- 0.1V
• If you are using a noise source, pick a good one which has excellent randomness and run each test for several minutes
• Alternatively use REW and do frequency sweeps
• Repeat each test 20 times and create an averaged plot
• Look for any outliers, they can be interesting

you are right +-3db variation upon samples seem to be the order of magnitude i see when comparing my few samples (variations in eartip placements on the mic it seems) i need a bigger pool of samples to have conclusive results

 

[MaxwellsEq]

you are right +-3db variation upon samples seem to be the order of magnitude i see when comparing my few samples (variations in eartip placements on the mic it seems) i need a bigger pool of samples to have conclusive results

I think you do.

Also, a good engineer or scientist should be sceptical about their results, until they are satisfied they are irrefutable. You should really have challenged yourself over some of your earlier statements. There is ample evidence that two well designed DACs using similar filter strategies and level matched should measure within 1dB of eachother. The fact you were seeing such wild differences should really have made you double check your methodology.

 

[amanieux]

I think you do.

Also, a good engineer or scientist should be sceptical about their results, until they are satisfied they are irrefutable. You should really have challenged yourself over some of your earlier statements. There is ample evidence that two well designed DACs using similar filter strategies and level matched should measure within 1dB of eachother. The fact you were seeing such wild differences should really have made you double check your methodology.

i think i had a confirmation bias i was looking for a warmer graph on hip2

here are 10 samples of each (only looking at 47hz value)

dawn 4.4 :
-19,-21,-19,-22,-19,-18,-17,-17,-19,-18 - average=-18.9

hip2 :
-19,-18,-18,-19,-19,-18,-21,-22,-17,-17 - average=-18.8

so it looks very identical, sorry guys my initial data is not reliable, but i still hear a difference so it must be 100% placebo after reading that hip2 was supposed to be warmer .

conclusion : always be afraid of the big bad ugly placebo effect

 

[voodooless]

Thanks for taking the effort! This is exactly what we’ve been trying to explain

If there is indeed something to hear, it will be in the high frequency bit. Try doing the same at say 10 kHz.

 

[kemmler3D]

The difference in treble is probably real

Dunno, this is also highly sensitive to positioning in some cases. If OP moved the IEM 1mm relative to the mic hole, it could throw off the treble that much. Maybe not, but we don't know.

 

[wwenze]

Dunno, this is also highly sensitive to positioning in some cases. If OP moved the IEM 1mm relative to the mic hole, it could throw off the treble that much. Maybe not, but we don't know.

What next, spinorama of headphones?

 

[kemmler3D]

What next, spinorama of headphones?

Not as silly as it sounds... I believe headphone measurement is difficult in large part because you have a lot of cancellations / comb filtering between the headphone driver and the microphone inside the ersatz ear canal on the measurement jig.

By the same token, headphone preference is very idiosyncratic because everyone has a differently-shaped ear, and so every headphone actually does sound different to every listener.

However, if we had 3D scans of our ears handy (big "if", but not that big given today's tech) and if there were some kind of headphone measurement tool akin to the NFS that would derive the sound field coming from the headphone in high res, eliminating the influence of reflections, we could combine the measurement and the ear scan and come up with the actual response at the eardrum for every listener, as well as the perfect EQ curve to correct a given headphone to our preference.

This is the impossible dream today, but "spinorama for headphones" would perhaps be even more useful than normal spinoramas.

 

[solderdude]

Dunno, this is also highly sensitive to positioning in some cases. If OP moved the IEM 1mm relative to the mic hole, it could throw off the treble that much. Maybe not, but we don't know.

OP mentioned

in Non-oversampling filter mode

So NOS filter mode which is no filter, even when it would be simulated the effect will be the same.
Later he measured it in 'slow' roll-off (which has less roll-off) mode which showed higher treble levels.

 

[kemmler3D]

OP mentioned

So NOS filter mode which is no filter, even when it would be simulated the effect will be the same.
Later he measured it in 'slow' roll-off (which has less roll-off) mode which showed higher treble levels.

Good point! Pretty interesting. We don't often see the effects of DAC filters like that.

 

[Colonel7]

i sample the sound from my iem through the dac and not the voltage out of the dac, the graph must be the combination of response of dac+iem and not dac alone i guess

The reason no one measures a DAC/amp with an earphone using a mic in a tube as there are too many variables here 'measured' at the same time.
Of course from earphone to plot will be the same path but what you call a DAC is actually a DAC+headphone amp.
There could be a different output resistance in the amp which can change the response of the earphone but is not part of the DAC.

So the: why don't we graph DACs ? ( i did !) should say:
Why aren't combinations of DAC/amps and headphones measured on acoustical output (I attempted this)

The reason is there are too many variables. There are literally hundreds of head/earphones and thousands of possible combinations between DAC/amp/headphones possible so picking just one combination is pointless other than to the owner.
On top of that sticking an earphone in a tube and some mic on the other side is not very scientific and explains why your result is what it is and does not look like pink noise.

Conn, sonar! DAC ahead, 1600 meters, err 1600 Hz off the port bow!