frequency-, phase-, distortion (and noise) response enough to determine sound quality?

[Jazigo]

Hi everyone!

So, a chistmas present for me to me was a Topping D50 III and a pair of B100s. This replaces a set of Topping E50 and a PA5. I expected the B100s to be a complete waste of money, so I was prepared to return them after a day of two.

However, there seems to be a clear difference between the two. I known that i can be biased and sort of hear what I want hear, so I decided to do some measurements with what I have on hand.
From my understanding, frequency-, phase-, distortion (and noise) response should be all that matter for audible changes. So I measured with a UMIK-1 and I made sure that the settings on my signal chain had the same voltage output. I was not able to get the voltage perfect, but I got so close that it would amount to less than a 0.1 dBspl difference on the speaker.

The measurements:

(edit): They both use the same DAC, the topping D50 III in the measurements above.

As one can see, no meaningful differences in the output of the speakers. Does that mean that I am just biased? I am happy to admit it if the science is clear about it.
The main differences that I can see in my signal chain is that I know use 0db gain on B100s compared to the 19.1db gain on the PA5. This means that I have to set the DAC to -21.5db with the PA5 and -2.5 on the DAC with the B100s to get the same volume.

Could be that the gain staging is doing the differences, but that does not show up in the response? I was also thinking about maybe checking some SMPTE/DIN distortion. Maybe that is affected by having the DAC at a lower volume.

Subjective perspective (feel free to skip):
I mainly use this system for playing games, which often can have very complex sound stages that have a lot of effects going on at once and two things stands out admittedly when listening:

1. Small sound effects are not so muted (while a lot is going on), like they were Motown mixed (every instruments have their on frequency range and they are not allowed to bleed into each other range)
2. It sounds effortless for the amplifier to produce all of these sounds as once.

 

[staticV3]

record the same track, played at close to the same volume, through both Amps.

load the recordings into Pkane Deltawave.

click Match.

wait for DeltaW to do its thing, then go to Listen->ABX Comparator

now you can compare their sound free of bias.

if you can consistently tell them apart, then there is a difference and DeltaW's various graphs will tell you where that difference lies.

 

[kemmler3D]

Agree with @staticV3 as usual.

If there is a clear audible difference you can probably find it pretty easily with Deltawave and then hunt the cause that way.

You've got an excellent start here, but it might be easier just to do a null test. There are types of distortion that aren't very easy to measure with REW.

I would also trot out the suggestion to do a proper blind test, but it's probably easier just to do a few close mic recordings and compare those.

 

[Keith_W]

You might be doing the wrong test. You are measuring at one volume, but music isn't a steady state signal at the same volume - there is loud and soft. Try doing a loudspeaker compression measurement - place your mic 1m from the speaker, then play pink noise. Using an SPL meter, adjust the volume until it reads 76dB at 1m. Then sweep. Repeat the sweeps by increasing the volume in REW by 10dB each time - you will have measurements at 76dB, 86dB, 96dB, and 106dB. In the "All SPL" tab, choose "Align SPL" and align all the curves at 1kHz over 2 octaves. Using Trace Arithmetic A/B, divide all the curves by the 76dB curve. You will see something like this:

(I stole this image from Erin). The purpose of this test is to look for deviations from the frequency response at different volumes.

Now, we know that amps operating within their linear range will sound extremely similar, if not the same. But who says that amps are always operating in their linear range? That's the purpose of this test - to stress your amp a bit and look for deviations.

You may also be interested in examining distortion products between the two amps.

Bottom line: if you can hear it, you can measure it. But you have to take the correct measurement.

 

[solderdude]

It sounds effortless for the amplifier to produce all of these sounds as once.

An amplifier does not amplify different sounds.
It amplifies the sum of all signals that were mixed in the studio.
2 voltages (stereo) that simply vary in amplitude over time. An amplifier simply amplifies that, it does not 'process' it in any way or 'favor' one or more instruments.
An amplifier does not care nor know nor have any preference for the signal and as long as it can reproduce 20kHz cleanly at full power into the connected load with low distortion it can handle every 'dynamic' signal you can throw at it.

One should compare amplifier output signals (with the speakers connected) and not through speakers.
You will introduce differences that are bigger than what the amplifiers do.
Even when you leave the mic in place and speakers in place you can still get measurable differences by standing at a slightly different spot in the room for instance.

 

[solderdude]

Now, we know that amps operating within their linear range will sound extremely similar, if not the same. But who says that amps are always operating in their linear range? That's the purpose of this test - to stress your amp a bit and look for deviations.

The purpose of this test is not how the amplifiers behave (nor get stressed) but how the speakers/drivers behave at larger excursions.
And... for active speakers with tweeter protection how that protection behaves.

That test will say nothing about the amplifiers... well until it clips of course but we assume not clipping signals.

 

[Keith_W]

The purpose of this test is not how the amplifiers behave (nor get stressed) but how the speakers/drivers behave at larger excursions.
And... for active speakers with tweeter protection how that protection behaves.

That test will say nothing about the amplifiers... well until it clips of course but we assume not clipping signals.

That is true. But if his amplifiers are running out of steam, this test will show it. Of course it's better to do electrical measurements, but I assume he's not equipped for it. And actually, neither are most of us, including me

 

[Sokel]

As you already use rew and your interest is about your amps it might be interesting to do a FSAF measurement comparison.

It won't be perfect as UMIC1 is not suitable for it but as long as you don't move your mic and speakers FSAF can be fun and reveling with the same signal (Pink noise usualy but you can even use music) .

 

[Jazigo]

As you already use rew and your interest is about your amps it might be interesting to do a FSAF measurement comparison.

It won't be perfect as UMIC1 is not suitable for it but as long as you don't move your mic and speakers FSAF can be fun and reveling with the same signal (Pink noise usualy but you can even use music) .

That sounds interesting! I will give it a try to see if something comes up.

 

[Panelhead]

I like your tests. Testing an amplifier with a resistor is a long way from testing with an actual speaker. I do not believe if two amplifiers measure the same loaded with a resistor it means they measure the same driving a loudspeaker.
I would not test the your amp-speaker at 76, 86, 96, & 106dB. The loudspeakers will be crying uncle at 106dB. Maybe double digit distortion.
Try 56, 66, 76, & 86dB.
I purchased a PA5 II to drive three way horns. Cheap enough. Loved it. Best bass they had ever delivered. Turned off sub to see if they had enough bass with PA5.
Purchased a pair of B100’s and expected to hear no difference. Used mid gain on B100’s.
Easy to hear difference. BUT, this is listening with a speaker that is 105dB efficient and 16 ohms. Listening to music at low levels, classical, mid 80’s peaks.
I now use low gain, the E70 Velvet set to 5 volt max. So amps are capable of a slamming 1.6 watts with a 0dB digital signal input. Dac is normally set toaround -20 to -10dB. Compressed music around -20dB. Good classical with wide dynamics set around -10dB to hear the lower levels better. The streamer shows the tracks are -40 to -20dB, not close the limit of 0dB.
Amplifier is outputting around 100mW on the peaks. The low levels on many classical tracks maybe 10mW.
These low levels are where the B100’s shine. On the Topping website the linearity plots for amplifiers are shown down to 20mW output level.
Except the B100’s. They are plotted down to 0.2mW. At a 40dB lower level.
Topping recommended the B100’s for the three way horns. Good enough.

 

[Holmz]

Hi everyone!

So, a chistmas present for me to me was a Topping D50 III and a pair of B100s. This replaces a set of Topping E50 and a PA5. I expected the B100s to be a complete waste of money, so I was prepared to return them after a day of two.

However, there seems to be a clear difference between the two. I known that i can be biased and sort of hear what I want hear, so I decided to do some measurements with what I have on hand.
From my understanding, frequency-, phase-, distortion (and noise) response should be all that matter for audible changes. So I measured with a UMIK-1 and I made sure that the settings on my signal chain had the same voltage output. I was not able to get the voltage perfect, but I got so close that it would amount to less than a 0.1 dBspl difference on the speaker.

The measurements:
View attachment 497909View attachment 497910View attachment 497912
(edit): They both use the same DAC, the topping D50 III in the measurements above.

As one can see, no meaningful differences in the output of the speakers. Does that mean that I am just biased? I am happy to admit it if the science is clear about it.
The main differences that I can see in my signal chain is that I know use 0db gain on B100s compared to the 19.1db gain on the PA5. This means that I have to set the DAC to -21.5db with the PA5 and -2.5 on the DAC with the B100s to get the same volume.

Could be that the gain staging is doing the differences, but that does not show up in the response? I was also thinking about maybe checking some SMPTE/DIN distortion. Maybe that is affected by having the DAC at a lower volume.

Subjective perspective (feel free to skip):
I mainly use this system for playing games, which often can have very complex sound stages that have a lot of effects going on at once and two things stands out admittedly when listening:

1. Small sound effects are not so muted (while a lot is going on), like they were Motown mixed (every instruments have their on frequency range and they are not allowed to bleed into each other range)
2. It sounds effortless for the amplifier to produce all of these sounds as once.

That 150-300 Hz phase range looks wild.

 

[solderdude]

I like your tests. Testing an amplifier with a resistor is a long way from testing with an actual speaker. I do not believe if two amplifiers measure the same loaded with a resistor it means they measure the same driving a loudspeaker.
I would not test the your amp-speaker at 76, 86, 96, & 106dB. The loudspeakers will be crying uncle at 106dB. Maybe double digit distortion.
Try 56, 66, 76, & 86dB.
I purchased a PA5 II to drive three way horns. Cheap enough. Loved it. Best bass they had ever delivered. Turned off sub to see if they had enough bass with PA5.
Purchased a pair of B100’s and expected to hear no difference. Used mid gain on B100’s.
Easy to hear difference. BUT, this is listening with a speaker that is 105dB efficient and 16 ohms. Listening to music at low levels, classical, mid 80’s peaks.
I now use low gain, the E70 Velvet set to 5 volt max. So amps are capable of a slamming 1.6 watts with a 0dB digital signal input. Dac is normally set toaround -20 to -10dB. Compressed music around -20dB. Good classical with wide dynamics set around -10dB to hear the lower levels better. The streamer shows the tracks are -40 to -20dB, not close the limit of 0dB.
Amplifier is outputting around 100mW on the peaks. The low levels on many classical tracks maybe 10mW.
These low levels are where the B100’s shine. On the Topping website the linearity plots for amplifiers are shown down to 20mW output level.
Except the B100’s. They are plotted down to 0.2mW. At a 40dB lower level.
Topping recommended the B100’s for the three way horns. Good enough.

30dB (factor 1000) lower distortion/noise at 20mW and 200mW... very likely to be an audible difference (suspect only noise level) ?

 

[Jazigo]

I have done some samples now. I have tried my best to keep everything as similar as possible with the setup I have. I sampled two runs on the B100s and two runs on the PA5, so that one can compare run-to-run variance against amp differences. I haven't been able to analyze them myself yet, but here is the sources at least:
B100-vs-PA5

 

[Jazigo]

Did a new sample where I was more careful with the settings: https://drive.google.com/file/d/1qtwrhEBCuwYW_EeacA6F23XQEF54nbNE/view?usp=sharing

I have also analyzed the files with DeltaWave and here we can see the difference:

B100 vs B100:

B100 vs PA5:

PA5 vs PA5:

It may look a bit worse on the B100 vs PA5 graph due to the graph scaling, but it seems things start to differ from 15 kHz and upwards.

 

[staticV3]

Did a new sample where I was more careful with the settings: https://drive.google.com/file/d/1qtwrhEBCuwYW_EeacA6F23XQEF54nbNE/view?usp=sharing

I have also analyzed the files with DeltaWave and here we can see the difference:

B100 vs B100:
View attachment 500612
B100 vs PA5:
View attachment 500613
PA5 vs PA5:
View attachment 500614
It may look a bit worse on the B100 vs PA5 graph due to the graph scaling, but it seems things start to differ from 15 kHz and upwards.

Interesting pattern, but still just ±0.4dB and, as you said, only in the high treble.

The important question is:
Once Deltawave is done matching the files, can you still tell them apart by ear using the Play->Comparator function? Now that loudness is precisely matched, you're listening blind, and can switch instantly?

 

[Panelhead]

The difference signal is interesting.
The B100 - B100 comparison has more noise than the PA5 - PA5 comparison. Unexpected finding.

 

[Jazigo]

So I have done some A/B testing now, with speaker, headphones and IEMs. I am never able to get a clean run, best have been 8/10, but in the end, I am never consistent with it. In other words, it isn't a clear difference as I thought initially.