• WANTED: Happy members who like to discuss audio and other topics related to our interest. Desire to learn and share knowledge of science required. There are many reviews of audio hardware and expert members to help answer your questions. Click here to have your audio equipment measured for free!

Benchmark...first watt....ABX...facepalm!

OP
Blumlein 88

Blumlein 88

Grand Contributor
Forum Donor
Joined
Feb 23, 2016
Messages
20,523
Likes
37,054
I have a question, from a scientific perspective. The best testing would be using test equipment, but lacking such tests or equipment, why wouldn't it be "proper" to use a test frequency (or frequencies) as a listening test to evaluate an amplifier (or system)?

Scientific testing in many cases require a system to be broken into individual smaller systems to provide usable information. In this case a specific frequency test appears to be valid to evaluate the response of the amplifier over a specific (narrow) frequency range. It would seem the argument should be that many additional frequencies should be tested. When music is used, it would seem that evaluating individual frequencies....


Well evaluating the capabilities of something you want to have a margin or a safety factor. If I am stressing a beam I want to stress it with a little bit more than I think I will ever apply to it to see if it breaks. I want to flesh out the envelope of performance. In audio there may not be the safety issue, but you do want some margin beyond your requirements.

Despite what most people imagine, test tones can lay bare the limits of performance in ways music usually doesn't. One of the very toughest test tones is the twin tone IMD test. Two high frequency signals combined to maximum level. One can imagine having 20 tones instead of 2 would be more stressful, but that is not normally the case with amplifiers. One may wish to use a sweep or a series of spot tones to flesh out flatness of frequency response.

So if music is a lesser stressor test tones can give you a margin of performance. Do well on the test tones, and you don't even need to test it with music most of the time. Nor is this unusual in designing products. Many things are designed and tested in sensible ways that don't require use of the product as it will be used to confirm. Sometimes there are gotchas, but more often than not there are none.

As for the idea never have blind tests been done with test tones, well of course they have. People can detect marginally audible distortion at lower levels with tones than they can music. They can more readily detect and detect at smaller levels of difference frequency response differences in gear with pink or white noise. So if listening tests of tones are clean, and wideband noise are clean, you'll be able to use the gear for music knowing all is well. Wideband noise has all the frequencies all the time.
 

Wayne

Active Member
Forum Donor
Joined
May 26, 2017
Messages
172
Likes
46
Location
Los Angeles, CA
As for the idea never have blind tests been done with test tones,

@Blumlein: Thank you, I think we are in agreement.

Just a comment concerning "blind tests." from a "scientific" perspective, a sighted (non-blind) test would not be objective, but subjective to individual biases. So IMHO, all listening tests must be double blind to be of maximum value and in addition, the higher the number of participants (n) the better.
 

RayDunzl

Grand Contributor
Central Scrutinizer
Joined
Mar 9, 2016
Messages
13,200
Likes
16,982
Location
Riverview FL
Wideband noise has all the frequencies all the time.

Wideband noise contains all frequencies within a range of bandwidth, with randomly changing amplitude for individual frequencies.

Here's about 50ms of some white noise, with an attempt to observe only the 5khz component within it:


upload_2017-9-18_18-7-20.png

This has high and low pass filters at 5khz applied multiple times, so the amplitude shown is meaningless, but the range of amplitude could be of interest.
 
Last edited:

amirm

Founder/Admin
Staff Member
CFO (Chief Fun Officer)
Joined
Feb 13, 2016
Messages
44,368
Likes
234,386
Location
Seattle Area
Why don't we see peer reviewed DBTs of people listening to test tones? I've never heard of such a thing.
They are done pretty routinely in research. Here is a Dolby paper for example on impact of jitter:

DolbyJitterStats.PNG


dolbyjitter.png


When they switched to testing music, they found that jitter was far, far less audible than in tones:

upload_2017-9-18_19-19-3.png



So test tones did provide a worst case scenario compared to music (as expected due to effects of masking).
 

Nightlord

Active Member
Joined
Mar 15, 2017
Messages
207
Likes
102
Location
southern Sweden
Amps are used primarily for playback of music/drama signals, not test tones. The test conditions here are pathological. A real world test using music would have been more representative of how amps are actually used. They decided against this and for good reason.

So who cares if the amp can be distinguished by listening to a test tone? Do you listen to test tones?

If the music I listen to happen to be a tone only, then yes. But it's beside the point... if there's a test tougher than the usecase to use in designing, then the issues will be so much less in the actual use case.... I prefer having something 10x unhearable than 1x unhearable if I can choose... as it's not the only gear used and distortions may combine until hearable if the happen to match.

Any time it's distinguished, there's room for improvement.
 

Dimitrov

Active Member
Joined
Apr 29, 2017
Messages
140
Likes
43
They are done pretty routinely in research. Here is a Dolby paper for example on impact of jitter:

View attachment 8762

View attachment 8763

When they switched to testing music, they found that jitter was far, far less audible than in tones:

View attachment 8764


So test tones did provide a worst case scenario compared to music (as expected due to effects of masking).

Sorry, lack of communication on my part. I didn't mean to say that people don't listen to test tones in DBTs, but it came across that way. What I meant to say and failed to articulate is that people rarely perform DBT's of amplifiers and use test tones as a source. They use music as the source.

Virtually every amp or DAC or cable DBT that I've read about is done using music. If you fail to distinguish a difference using music (ie the intended usage), but succeed with a 1 kHz test tone, what useful information can be taken away from that exercise?
 
Last edited:

RayDunzl

Grand Contributor
Central Scrutinizer
Joined
Mar 9, 2016
Messages
13,200
Likes
16,982
Location
Riverview FL
If you fail to distinguish a difference using music (ie the intended usage), but succeed with a 1 kHz test tone, what useful information can be taken away from that exercise?

That your ears are rather forgiving little devices when presented with complex information?

---

I know I like my Krell/MartinLogan setup at higher volume levels, where my little JBL LSR 308s have higher distortion (as measured with tones).

See https://audiosciencereview.com/forum/index.php?threads/jbl-lsr-308-in-the-house.1066/#post-28025

At more casual levels, where the harmonic distortion readings are lower, and I'm not staring blindly at the sweet spot, I can't say I prefer one over the other, and use the JBLs as my daily econo drivers now.
 

Pdxwayne

Major Contributor
Joined
Sep 15, 2020
Messages
3,219
Likes
1,172
A new thread about Benchmark interview mentioned the first watt article. When searching for more info, I found this old thread.

This a spreadsheet that tracks the sinad of each amp.


I wonder if I can use the spreadsheet to roughly estimate which amp would likely have audible distortion at 0.01 watt.....
 

egellings

Major Contributor
Joined
Feb 6, 2020
Messages
4,005
Likes
3,245
Wouldn't you just about need to have your ear on the cone to hear a 10mW level?
 

NTK

Major Contributor
Forum Donor
Joined
Aug 11, 2019
Messages
2,656
Likes
5,819
Location
US East
Wouldn't you just about need to have your ear on the cone to hear a 10mW level?
Not really. 10 mW is -20 dB from 1 W. If your speaker's sensitivity is 86 dB SPL @ 1 W 1 m, then it will give 66 dB SPL at 1 m with 10 mW.
 

pjug

Major Contributor
Forum Donor
Joined
Feb 2, 2019
Messages
1,775
Likes
1,561
I found a Class AB amplifier that seems to be about as bad as what Benchmark describes as the "traditional" Class AB. This is my Kentiger MA-200 with Class AB TDA7388 chip amp. Looking at the amplifier voltage, I am getting about -40dB 2nd harmonic when playing at 100mW (distortion is buried in the noise when I try 10mW). Even so, when I play a tone at 0.28V on a 86dB speaker, it sounds pure to me.

It would be interesting to make a tone with -50dB (or maybe worse?) distortion using @pkane 's Distort, and then do ABX against a pure tone playing at 67dB SPL same as the Benchmark test description, see if it is as easy to hear this as Benchmark claims. Does anyone know how to make the same distortion profile as shown in the Benchmark scope shot?
 

Pdxwayne

Major Contributor
Joined
Sep 15, 2020
Messages
3,219
Likes
1,172
I found a Class AB amplifier that seems to be about as bad as what Benchmark describes as the "traditional" Class AB. This is my Kentiger MA-200 with Class AB TDA7388 chip amp. Looking at the amplifier voltage, I am getting about -40dB 2nd harmonic when playing at 100mW (distortion is buried in the noise when I try 10mW). Even so, when I play a tone at 0.28V on a 86dB speaker, it sounds pure to me.

It would be interesting to make a tone with -50dB (or maybe worse?) distortion using @pkane 's Distort, and then do ABX against a pure tone playing at 67dB SPL same as the Benchmark test description, see if it is as easy to hear this as Benchmark claims. Does anyone know how to make the same distortion profile as shown in the Benchmark scope shot?

I started check hearing sensitivity to distortions thread at

If you have not check, it maybe good for you to check how sensitive you are using Klippel web site first.

Edit:
Also, unless you have another clean amp to compare, it is likely that you won't sense the harmonic that easy. It could be just a slight tone change that you won't notice until you can also play the same test song using a clean amp like Benchmark amp.
 
Last edited:

pjug

Major Contributor
Forum Donor
Joined
Feb 2, 2019
Messages
1,775
Likes
1,561
I started check hearing sensitivity to distortions thread at

If you have not check, it maybe good for you to check how sensitive you are using Klippel web site first.
I did the Klippel with the two tone test and I got about -54dB, so above average. A long time ago I did the music test and did much much worse on that, I think I was at or below average on that one. I assume I would do worse with a single tone than the two tone?

I did the Klippel test on headphones and louder than the 67dB of the Benchmark test; I know I would do much worse at lower volume.

Anyway, when playing the 1000Hz at 10mW I had nothing to compare it just sounded pure on its own. Of course that isn't a good test.
 

Pdxwayne

Major Contributor
Joined
Sep 15, 2020
Messages
3,219
Likes
1,172
I did the Klippel with the two tone test and I got about -54dB, so above average. A long time ago I did the music test and did much much worse on that, I think I was at or below average on that one. I assume I would do worse with a single tone than the two tone?

Anyway, when playing the 1000Hz at 10mW I had nothing to compare it just sounded pure on its own. Of course that isn't a good test.
Yeah, I just edited my prior post to add more comment. Maybe you missed it.

Here is it again:
Also, unless you have another clean amp to compare, it is likely that you won't sense the harmonic that easy. It could be just a slight tone change that you won't notice until you can also play the same test song using a clean amp like Benchmark amp.
 

pjug

Major Contributor
Forum Donor
Joined
Feb 2, 2019
Messages
1,775
Likes
1,561
Yeah, I just edited my prior post to add more comment. Maybe you missed it.

Here is it again:
Also, unless you have another clean amp to compare, it is likely that you won't sense the harmonic that easy. It could be just a slight tone change that you won't notice until you can also play the same test song using a clean amp like Benchmark amp.
Yes, there needs to be a comparison. Seems the easiest way is to use software to make the distortion rather than try to do ABX with amps. Also then a group can do the same ABX at 67dB SPL to get an idea of typical sensitivity to this distortion.
 

pkane

Master Contributor
Forum Donor
Joined
Aug 18, 2017
Messages
5,632
Likes
10,205
Location
North-East
It would be interesting to make a tone with -50dB (or maybe worse?) distortion using @pkane 's Distort, and then do ABX against a pure tone playing at 67dB SPL same as the Benchmark test description, see if it is as easy to hear this as Benchmark claims. Does anyone know how to make the same distortion profile as shown in the Benchmark scope shot?

Can you point me to the Benchmark scope shot?
 

pkane

Master Contributor
Forum Donor
Joined
Aug 18, 2017
Messages
5,632
Likes
10,205
Location
North-East

pjug

Major Contributor
Forum Donor
Joined
Feb 2, 2019
Messages
1,775
Likes
1,561
Seems like a combination of crossover and harmonic distortion. DISTORT can simulate both, but there's no way to reproduce this without additional measurements and at least the amplitude scale.
Thanks. Is it possible to come close to reproducing it if you know that the top waveform is 2.83 Vrms, and the bottom waveform relative scale has been blown up by a factor of 1024? And then, if you can reproduce it, would the distortion at 10mW be expected have the same components as the 1W distortion that they are showing?
 

pkane

Master Contributor
Forum Donor
Joined
Aug 18, 2017
Messages
5,632
Likes
10,205
Location
North-East
Thanks. Is it possible to come close to reproducing it if you know that the top waveform is 2.83 Vrms, and the bottom waveform relative scale has been blown up by a factor of 1024? And then, if you can reproduce it, would the distortion at 10mW be expected have the same components as the 1W distortion that they are showing?

Possibly, but it'll take some guessing and a lot of trial and error.
 
Top Bottom