• WANTED: Happy members who like to discuss audio and other topics related to our interest. Desire to learn and share knowledge of science required. There are many reviews of audio hardware and expert members to help answer your questions. Click here to have your audio equipment measured for free!

Controlled Listening - Explained

Joined
Nov 14, 2022
Messages
30
Likes
18
When I say 'explained', can you explain it to me :D

I really appreciate the work that goes into all these measurements and reviews, it's really impressive. It is certainly not something I would ever want to do myself, but the 'controlled listening' is something that I could subscribe to, in fact I kinda need to as I just took delivery of a RME ADI-2 DAC, and want to compare to an SU-9N before I return it.

When you say controlled, what should I be looking for other than the following.

I will pick 5 tracks, all streamed in the same manner from Amazon Music, through my Bluesound Node at a 'fixed volume / output' (cant remember what its called but it locks the volume at max so it can only be controlled by the next in line, out via the same Coax cable, into each DAC, and from there out to my CXA81 amp via the same XLR cables.

In terms of levels / volume etc, I will have both DAC's in DAC only mode, neither in the preamp modes, and will leave the amp at the same volume.

Am I on the right track?

I cannot match voltage or watts etc, wouldn't know where to start and even if I did that may give me a different volume.

Cheers guys
 

DVDdoug

Major Contributor
Joined
May 27, 2021
Messages
2,920
Likes
3,835
The odds are you won't hear a difference. The only time I've heard a "defect" from a DAC is from a noisy soundcard.

It's not always easy to do with hardware but volume matching is pretty-much "required"... Unless your trying to listen-for a volume difference. ;)

If there is a volume difference it can be perceived as a volume difference plus "something else". And higher gain will amplify any background noise so you could hear hum or hiss when there is no difference in the signal-to-noise ratio.

With a Blind ABX Test you eliminate any known variables and then you can switch freely between A (known) B (known) and X (unknown) and you try to identify X. The test is repeated multiple times (usually more than 10) to get a statistical result. ...Obviously, there is a 50/50 chance of "guessing" correctly if you only do it once.

An ABX test doesn't tell you which one is better, it just confirms that you can (statistically) hear a difference.

If there is a difference it will be noise, frequency response, or distortion, or something really gross like dropouts which can cause clicks (noise) or gaps in the audio. Don't get trapped into the audiophile-nonsense terminology-descriptions of the sound quality. (See Audiophoolery.)
 

solderdude

Grand Contributor
Joined
Jul 21, 2018
Messages
15,891
Likes
35,912
Location
The Neitherlands
Controlled means certain controls have to be in place.

A: The outputs should be level matched so play a 400Hz tone into DAC A and measure the output, do the same for DAC B and then, if at all possible lower the output level of the DAC with the highest output voltage to the exact same value (within 1%)

B: Ensure there are no tell-tale signs (a relay clicking, indicators, other sounds or visual clues etc) that might give a hint what is playing or the helper being too forthcoming with hints.

C: Let someone else (randomly) switch DACs in a way that you don't know which is which. Maybe even press buttons trying to fool you.

D: At least 10 but preferably 20 'attempts' to determine which is which or you like best.
 
OP
S
Joined
Nov 14, 2022
Messages
30
Likes
18
Controlled means certain controls have to be in place.

A: The outputs should be level matched so play a 400Hz tone into DAC A and measure the output, do the same for DAC B and then, if at all possible lower the output level of the DAC with the highest output voltage to the exact same value (within 1%)

B: Ensure there are no tell-tale signs (a relay clicking, indicators, other sounds or visual clues etc) that might give a hint what is playing or the helper being too forthcoming with hints.

C: Let someone else (randomly) switch DACs in a way that you don't know which is which. Maybe even press buttons trying to fool you.

D: At least 10 but preferably 20 'attempts' to determine which is which or you like best.
Can you explain what you mean by measuring the levels? How would I go about that bit :)

Thanks Very much
 
OP
S
Joined
Nov 14, 2022
Messages
30
Likes
18
PS - Also, the measurements that are conducted on the site by Amir and a few others, do these explain the differences in how DAC's can sound? I.e. a warmer sound with rolled off smoothed high notes etc, just as an example? Or emphasis on a certain frequency that maybe out of kilter with the other frequencies?

Just learning and trying to understand this all a bit more. I understand the terminology and desire for a DAC to be 'transparent' but not all DAC's are of course, even some extremely expensive stuff well above my level and desire choses a rolled off, supposedly more 'musical' way... rather than clinical. Are we really saying that this is nonsense, and instead we'd prefer ideally a perfect Dig-Analogue conversion, and any musicality or mere preferences are handled by EQ, Speakers etc.

I am a photographer. So I am likening this to the comments around digital photography and 'clinical' lenses, vs those with 'character'. The 'character' are basically just imperfections.
 

MaxwellsEq

Major Contributor
Joined
Aug 18, 2020
Messages
1,630
Likes
2,429
If you have an AC voltmeter able to measure high enough frequencies, that would be OK. Alternately (and it allows you to measure other things), an oscilloscope.
 

solderdude

Grand Contributor
Joined
Jul 21, 2018
Messages
15,891
Likes
35,912
Location
The Neitherlands
You can also measure using a USB soundcard. This would be the easiest way and you can directly see dB's.
Or use a multimeter, just not the $ 10.- cheapies with a lowest setting of 200V AC.

Often the DAC output voltage, digitally, can not be set in 0.1dB increments in which case a good matching (within 0.1dB = about 1%) is not possible.
A way around that would be to do this as close as possible and allow some time to pass between switching between DACs. You cannot do a real time A-B switching.
 

bluefuzz

Major Contributor
Joined
Jan 17, 2020
Messages
1,043
Likes
1,774
I understand the terminology and desire for a DAC to be 'transparent' but not all DAC's are of course
A DAC that isn't 100% 'transparent' is simply broken. End of story.
even some extremely expensive stuff well above my level and desire choses a rolled off, supposedly more 'musical' way
'Supposedly' being the operative word. If you want a 'warmer' presentation or other subjective adjustments – and there is nothing wrong with that I hasten to add – then that should be done with DSP/room correction before it reaches the DAC. The DAC itself should simply pass through what it receives as cleanly as possible. The RME ADI-2 you are looking at has a transparent DAC that is as good as it gets with the added bonus of optional DSP to tune the sound to your needs.
 
OP
S
Joined
Nov 14, 2022
Messages
30
Likes
18
Thanks all, my likening to photograph and lenses is pretty accurate then in that case.

Blue fuzz - Saying something that isn’t totally transparent is ‘broken’ or I guess inaccurate is another term is pretty much what I mean with a non perfect or ‘character’ lens.

I did the best job I could without reading these messages. Spent two hours doing A B test.

I cut out a lot dead time in between sources by plugging two usb c to usb b cables into my laptop, then one into each DAC. Than each Dac connected to the amp. And swapped sources that way. Rather than changing coax from the node and then two Xlr cables.

I then downloaded a Decibel meter, played a 1khz frequency and adjusted volume until both were the same. I didn’t quite know how to get DAC only mode on the RME but the smsl was in that mode already.

I played between 10 and 30 seconds of each song and swapped back and forth. I did this a lot. I couldn’t tell them apart in the slightest. At one stage I thought the smsl had an edge (before downloading decibel meter) but that soon was quashed.

Both basically as goood as I ever need.
 

antcollinet

Master Contributor
Joined
Sep 4, 2021
Messages
7,414
Likes
12,301
Location
UK/Cheshire
Thanks all, my likening to photograph and lenses is pretty accurate then in that case.

Blue fuzz - Saying something that isn’t totally transparent is ‘broken’ or I guess inaccurate is another term is pretty much what I mean with a non perfect or ‘character’ lens.

I did the best job I could without reading these messages. Spent two hours doing A B test.

I cut out a lot dead time in between sources by plugging two usb c to usb b cables into my laptop, then one into each DAC. Than each Dac connected to the amp. And swapped sources that way. Rather than changing coax from the node and then two Xlr cables.

I then downloaded a Decibel meter, played a 1khz frequency and adjusted volume until both were the same. I didn’t quite know how to get DAC only mode on the RME but the smsl was in that mode already.

I played between 10 and 30 seconds of each song and swapped back and forth. I did this a lot. I couldn’t tell them apart in the slightest. At one stage I thought the smsl had an edge (before downloading decibel meter) but that soon was quashed.

Both basically as goood as I ever need.
Well done on the test. Decibel meters are generally not consided accurate enough for proper level matching, but in this case it doesn’t matter because you still couldn’t detect a difference.

this is the result we would expect based on the measurements.
 

kemmler3D

Major Contributor
Forum Donor
Joined
Aug 25, 2022
Messages
3,022
Likes
5,643
Location
San Francisco
+1, a better attempt than most people ever make. The level matching is so key because louder sounds better, not just louder.

At this point I am sure you are convinced that DACs are so good nowadays that your time and money is definitely better spent looking into speakers / headphones and EQ / acoustics. There is plenty of rabbit hole to go down even if you ignore dacs, amps, and sources altogether. :)
 

BDWoody

Chief Cat Herder
Moderator
Forum Donor
Joined
Jan 9, 2019
Messages
6,949
Likes
22,627
Location
Mid-Atlantic, USA. (Maryland)
I played between 10 and 30 seconds of each song and swapped back and forth. I did this a lot. I couldn’t tell them apart in the slightest.

Pretty eye opening, isn't it? Big Kudos to you for actually hassling with this. So many make claims for years without ever taking the couple of hours you took to actually find out for yourself. 'Blinding' yourself to which one was playing would be the next step if you want to add a bit more rigor to the test.

I had a similar experience after stumbling across this place some years back, and back went the expensive DAC, and in came the new speakers...
 

Speedskater

Major Contributor
Joined
Mar 5, 2016
Messages
1,610
Likes
1,333
Location
Cleveland, Ohio USA
For level matching:
a] Electronic components. any DMM (Digital Multi-Meter) with a 2 Volt AC scale will do. It doesn't need to be the least bit accurate, just repeatable.
use a 400 Hz or 1 kHz tone.
b] Loudspeakers. Use a SPL meter, or 'cal' microphone or some cell phones. Again, it doesn't need to be the least bit accurate, just repeatable.
This time use a 'pink' noise track. The tricky part is repeating the speaker to mic distance and mic positioning.
 
OP
S
Joined
Nov 14, 2022
Messages
30
Likes
18
So what was most enlightening was this (long winded version):

I compared the DAC in my Node and the CXA81 AMP (a good dac apparently - ES9016) compared to the SMSL SU-9n - maybe less scientifically than I did with the RME but still listening (to my ear) at the same volume, from same position etc. There was a huge difference between the Node / Amp and the SMSL, with the SMSL winning by a long way, well a long way compared to how I thought they may sound.

I then assumed that the more I spent, the better and better my audio would sound, so I started looking at better stuff... After reading much here I saw that the CXA81 DAC and the Node 2i (one before mine) actually scored pretty poorly. Hence my findings of better sound. And without realising the SMSL was actually at saturation point in terms of performance, any from there I wouldn't hear a difference regardless of what I spent.

Very interesting to me
 
Top Bottom