KTB vs Topping E30 listening tests

[Pdxwayne]

about 0.21dB difference.

Just a tad too much when comparing directly but you won't be switching between outputs directly.

Next attempt... try to figure out which DAC is which in a blind comparison (have someone randomly plug in 1 of the 2 DACs)
The helper can also try to bait you by unplugging DAC A and plugging it in back again, even a few times.

Make 20 'attempts' with random DAC and figure out by ear only which is which. The 0.2dB is no issue when time has passed between attempts.
You can spread this test over several days. It is fatiguing after some time.

Good luck telling them apart.

I can't tell them apart using headphones even when doing sighted quick switch.
; )

 

[solderdude]

That's not what you said earlier:

"my subjective impression is that KTB is sharper and E30 is warmer."

 

[Pdxwayne]

That's not what you said earlier:

"my subjective impression is that KTB is sharper and E30 is warmer."

Please read post #72. I must admit my initial impressions were not apple to apple.

KTB via speakers setup most of the time. KTB via headphones was mostly before I updated firmware, when I complained about sharpness. That is before I got E30.

Only since I started this thread, I did apple to apple comparison with headphones.

Next is speakers setup comparison.

 

[Pdxwayne]

about 0.21dB difference.

Just a tad too much when comparing directly but you won't be switching between outputs directly.

Next attempt... try to figure out which DAC is which in a blind comparison (have someone randomly plug in 1 of the 2 DACs)
The helper can also try to bait you by unplugging DAC A and plugging it in back again, even a few times.

Make 20 'attempts' with random DAC and figure out by ear only which is which. The 0.2dB is no issue when time has passed between attempts.
You can spread this test over several days. It is fatiguing after some time.

Good luck telling them apart.

As noted earlier, with USB, 400 Hz with REW at -3 dB:
KTB: 6.23 V
E30: 6.07 V.

With node2i, playing 120 Hz tone, which is supposed to be at 0 dB, I got from L30 preout:
KTB: 5.63
E30 5.49

60Hz at 0dB:
KTB: 5.6
E30: 5.46

30hz at 0dB:
KTB: 5.6
E30: 5.47

I wonder where is the difference between USB via computer (6.23V) vs coaxial via node2i (5.6V) comes from.

Direct measure from DAC RCA out:
KTB:1.956
E30: 1.909

I will assume they are both close enough and will not volume match when testing with speakers at first. Only when I can hear a difference will attempt to volume match.

 

[solderdude]

It's most likely the meter. Cheap meters may well be inaccurate at frequencies above 100Hz.

 

[Pdxwayne]

It's most likely the meter. Cheap meters may well be inaccurate at frequencies above 100Hz.

Curious, my 20+ years old integrated amp has a rating of 100 watts for 1V input. The amps are only rates for >125 watts per channel. Would that mean it won't handle 2V input well? Possibility of clippings?

 

[solderdude]

It just means that when the volume control is turned all the way up at 1.12V the amp will reach clipping point. For short peaks it may be a bit higher.
As soon as you turn the volume knob down a few notches you will need 2V to reach clipping levels and even further down you won't reach clipping levels at all.
Amplifiers have a limit for the max. input voltage but this is rarely specified.

 

[Iglo]

about 0.21dB difference.

Just a tad too much when comparing directly but you won't be switching between outputs directly.

Next attempt... try to figure out which DAC is which in a blind comparison (have someone randomly plug in 1 of the 2 DACs)
The helper can also try to bait you by unplugging DAC A and plugging it in back again, even a few times.

Make 20 'attempts' with random DAC and figure out by ear only which is which. The 0.2dB is no issue when time has passed between attempts.
You can spread this test over several days. It is fatiguing after some time.

Good luck telling them apart.

First off I enjoyed reading this topic. It's very cool to see the effort the topic starter is making to validate his hypothesis.
I would recommend (in the name of science) to really randomize which DAC is plugged in. Your helper making decisions will influence the result. Unconsciously you could start guessing what the helper is plugging in ("he wouldn't pick A 7 times in a row..."). The helper also will (unconsciously) try to make it harder to guess for you (let's do A and then A again) which in result can make it easier to guess (humans are really bad at randomness).

You want to have a random allocation with no relationship to the prior assignments. A real double-blind test. This is not difficult if you add a few steps and use a coin and dice.

First of all, these steps need to be done without you seeing any of it.

With that out of the way, your helper will start marking one DAC A and one B.

After the first step, we need to select a sequence to split the 1-6 range of numbers later on in the process. Your helper can split range (1-3 / 3-6) or even/uneven. Just let a coin flip settle on which sequence will be used. Write down the split (range vs even/uneven).

After this step your helper will throw the dice 20 times (without you seeing it), writing down each result.
Now he ends up with a list of 20 numbers ranging 1-6.

Now we need to do one last coin flip. Your helper will use this coin flip to find how the 1-6 numbers correspond with A and B. If the initial coin flip decided on the range split let the coin decide which range (1-3 or 3-6) is A and which is B, the same goes for even/uneven. Use the split-sequence to transform the 20 numbers on the list to A or B.

Now you end up with a truly random sequence and you can start testing!

p.s. keep in mind that you have to score more than 14 out of 20 right to come to a statistically valid conclusion. p-value would be .010547 so the result is significant at p < .05.


* as mentioned below, this is not double-blind but still a well-controlled blind test.

 

[solderdude]

It still isn't double blind. Your suggestion is merely letting a dice or coin make the decision to determine the used DAC.
It is a good suggestion none the less.

When it is double blind both the helper and tester have no way of knowing which one is used.
So it remains a blind test where it is important there is no hinting and one must only use ears and there should be no accidental clues (a small tick for instance)

 

[Iglo]

Yes, correct. I should have removed the double-blind reference. I was writing the extra steps to get true double-blind but decided it is hardly worth the effort.

 

[Sgt. Ear Ache]

All of that is excellent advice in general. But in this specific case, probably not necessary. It's a pretty safe bet that Pdxwayne will quickly realize under even simple controlled conditions that he can't identify any difference and any further, more-stringent testing will be discontinued. lol...

 

[BDWoody]

All of that is excellent advice in general. But in this specific case, probably not necessary. It's a pretty safe bet that Pdxwayne will quickly realize under even simple controlled conditions that he can't identify any difference and further more-stringent testing will be discontinued. lol...

So true.

Tighten up as you need to, but you probably won't need to get to the point of a 1% level match and DB/X protocol before you've learned what you need to...

 

[Pdxwayne]

Thanks all for comments.

Yeah, using headphones, even only doing the sighted quick switch, without doing volume matching, I can't hear a difference that can truly identify which is which when go blind.

Now, that is with a $150 headphones.

Next is with a pair of bookshelf speakers that is current top of the line from Paradigm. Supposed to be very reveling. If I can't even hear a different using sighted quick switch, no point going further doing blind test.
; )

That part might take some time to set up....

 

[M00ndancer]

That part might take some time to set up....

Good luck and I look forward to your posts!

 

[Pdxwayne]

Finally had a little free time to quickly setup in my bedroom and did a quick speakers listening test.

Integrated Amp: Yamaha DSP A1
Speakers:. Paradigm Persona B.

Listeners: My son and my daughter (end of one song)

Songs:
Diana Krall's A Case of You.
Esperanza Spalding's Inutil Paisagem
Front Line Assembly's Everything that was before.

My son identified item A (KTB) as the shaper sounding one for Diana Krall and Frontline Assembly. He said Esperanza was a coin toss and opted for B (E30). My theory for not able to identify which is sharper for Esperanza: The song only have vocals and double bass.

My daughter walked into the end of listening session and caught the last quick switches for Frontline Assembly's song. She agreed with my son that A (KTB) is the sharper sounding one.

Note that none of them care about my music or DAC. The whole thing took maybe a few minutes. Front Line Assembly song is electronica with lots of high notes. Does it proof anything when both of them identify A (KTB) as the sharper sounding one when playing that song?

 

[Sgt. Ear Ache]

Not a whole lot really. I mean was A the one you played last for them? Maybe they heard the most recent as sharpest. Or, if you knew which was which maybe you gave some sort of "hint" that you aren't aware of. Were they blind-folded? Your daughter "agreeing" with your son certainly sounds like she is being given a clue. I mean for this sort of test to mean much it has to be done in a way that makes random luck unlikely and you can't have the subjects discussing their thoughts with each other before making their choices. Picking the KTB as sharpest after a couple of songs could just be luck. It's a 50/50 guess right?

And also, it's you who claimed to hear differences before. Not your kids. That was what started this whole process...

 

[solderdude]

you can't have the subjects discussing their thoughts with each other before making their choices.

Yep.

And in the end.. were you able to identify the DAC's and who did the switching (or pretend to switch) ?
Were you or others aware which DAC was playing ?

 

[BDWoody]

Finally had a little free time to quickly setup in my bedroom and did a quick speakers listening test.

Good for you for continuing to mess around with this.

How did you match the volume levels? Was any of this unsighted?

 

[Pdxwayne]

Not a whole lot really. I mean was A the one you played last for them? Maybe they heard the most recent as sharpest. Or, if you knew which was which maybe you gave some sort of "hint" that you aren't aware of. Were they blind-folded? Your daughter "agreeing" with your son certainly sounds like she is being given a clue. I mean for this sort of test to mean much it has to be done in a way that makes random luck unlikely and you can't have the subjects discussing their thoughts with each other before making their choices. Picking the KTB as sharpest after a couple of songs could just be luck. It's a 50/50 guess right?

And also, it's you who claimed to hear differences before. Not your kids. That was what started this whole process...

First of all, there were 3 songs. I gave my son no clue as to which is which. He listened and asked for multiple switches before deciding. Afterward, my son said 1 and 3 songs were easier to identify. I already said why I think 2nd song is harder.

What validated my original observations more? A person (me) who you claim to have a bias doing the test, or two people who care not and know not about my setup?

 

[Pdxwayne]

Yep.

And in the end.. were you able to identify the DAC's and who did the switching (or pretend to switch) ?
Were you or others aware which DAC was playing ?

There were no discussion.

My daughter didn't even want to help me until she walked in to ask for my son to help her with something else.

I asked my daughter to wait for just a little to let my son identify the sharper one for last song. After a few switches, my son said the one before this is sharper. My daughter said yup.

I asked my son during dinner if he is sure what sharper meant. He said less muddy. My daughter on the other hand said something surprised me. She said less annoying......