• WANTED: Happy members who like to discuss audio and other topics related to our interest. Desire to learn and share knowledge of science required. There are many reviews of audio hardware and expert members to help answer your questions. Click here to have your audio equipment measured for free!

Is Audio Science Review going about it all wrong? Or partly wrong? Or all right?

daftcombo

Major Contributor
Forum Donor
Joined
Feb 5, 2019
Messages
3,687
Likes
4,068
Even if a test shows that 99.999% of the people tested cannot distinguish between two audio samples it is still possible there are perceptible differences that can be heard if our focus is directed appropriately. So that would render double blind test inconclusive as well, no?

It would still be difficult to impress friends with such a barely different-sounding setup then.
 

Bliman

Senior Member
Joined
Nov 9, 2018
Messages
419
Likes
150
Location
Belgium
So that means I cannot do my blind tests. No way I can switch the setup under 6 seconds. I would need to switch of the amplifier, switch the cables and set the music back to the same point. There goes my test.
 

andreasmaaan

Master Contributor
Forum Donor
Joined
Jun 19, 2018
Messages
6,652
Likes
9,399
So that means I cannot do my blind tests. No way I can switch the setup under 6 seconds. I would need to switch of the amplifier, switch the cables and set the music back to the same point. There goes my test.

You would need to use a switch box to give yourself the best chance of discerning a difference. You could do the test with slower manual switching, but you would then be relying on non-echoic memory, which is less reliable.
 

Cosmik

Major Contributor
Joined
Apr 24, 2016
Messages
3,075
Likes
2,180
Location
UK
Can someone explain the main motivation for testing for hearing 'a difference'? I can only see the point of it if the two setups are vastly different in cost and the test is to see what can be 'got away with'. But as far as I can tell, cost in audio is a red herring: audio equipment can be built from components that cost $0.002; diamond tweeters and so on are not necessary if the speaker is designed not to need diamond tweeters.

If one of two setups measures better than the other, then use the design of the one that measures better. Don't try to get away with the DAC chip that costs $0.50 if the one that costs the princely sum of $2.00 is that much better.
 

MrGoodbits

Member
Forum Donor
Joined
Sep 6, 2018
Messages
63
Likes
110
Location
Knoxville, Tennessee
What would even confuse the subject even more is to present another photo similar to the first but different slightly and to ask what is different between the two photos. Most would remember wrong or only remember one or two things that stood out to them that were biased to their previous memories and experiences.
Making fair analogies between audibility tests and visibility tests is tricky, but we can make the photograph comparison scenario a little closer to the spirit of audio DBTs: images A and B would be displayed on a computer monitor one at a time; pressing a keyboard key would instantly switch between A and B.
 

SIY

Grand Contributor
Technical Expert
Joined
Apr 6, 2018
Messages
10,383
Likes
24,749
Location
Alfred, NY
Making fair analogies between audibility tests and visibility tests are tricky, but we can make the photograph comparison scenario a little closer to the spirit of audio DBTs: images A and B would be displayed on a computer monitor one at a time; pressing a keyboard key would instantly switch between A and B.

That happens anyway- your actual undistorted visual field is shockingly small, so you have to move your vision from one object to the other.

Of course, there's the blink method of detecting motion of objects in astronomy...
 

Cosmik

Major Contributor
Joined
Apr 24, 2016
Messages
3,075
Likes
2,180
Location
UK
To determine audibility thresholds for various forms of distortion, for example.
But in order to do the experiment you need to have the equipment already that is capable of meeting or exceeding the lowest distortion in your experiment. So don't bother to do the test; simply use that design.

As I said above, if the real cost of the best-measuring equipment is $100,000 then maybe there's a point to seeing what you can get away with, but otherwise I can't see the point.
 

andreasmaaan

Master Contributor
Forum Donor
Joined
Jun 19, 2018
Messages
6,652
Likes
9,399
But in order to do the experiment you need to have the equipment already that is capable of meeting or exceeding the lowest distortion in your experiment. So don't bother to do the test; simply use that design.

As I said above, if the real cost of the best-measuring equipment is $100,000 then maybe there's a point to seeing what you can get away with, but otherwise I can't see the point.

Well in the case of DACs, amps, etc., a lot of the best-measuring equipment does cost thousands or tens of thousands of dollars. Knowing that the SOTA is unnecessary is quite valuable, wouldn't you agree?
 

Cosmik

Major Contributor
Joined
Apr 24, 2016
Messages
3,075
Likes
2,180
Location
UK
Take the Quad 405 amplifier. Back in 1975 or whenever it was, Quad produced an amplifier with very, very low distortion and no adjustments to make. It may have had a flaw in its overly-sensitive overload circuitry but this was, I believe, later corrected with a MkII version.

Question: given the stellar measurements, why did anyone bother to design another amplifier after that?

If you believed in measurements, why would you do listening tests on it? What would your hypothesis be? That there was something in the sound that couldn't be measured?

True science requires you to have a hypothesis to try to falsify. For the Quad 405, what would that be?
 

Cosmik

Major Contributor
Joined
Apr 24, 2016
Messages
3,075
Likes
2,180
Location
UK
Well in the case of DACs, amps, etc., a lot of the best-measuring equipment does cost thousands or tens of thousands of dollars.
But in my earlier post I said I didn't believe that: I know about electronic components and how much they cost. Sure, the final products may be dressed up and sold for thousands of dollars to audiophiles, but I see that as separate from the universal, timeless aspect of science.
 

Cosmik

Major Contributor
Joined
Apr 24, 2016
Messages
3,075
Likes
2,180
Location
UK
That the level of distortion produced by the device was inaudible.
But if you don't have a better amplifier to test it against, how are you going to do that? And if you do have a better amplifier, why not just use that all the time?

And if you don't know which is the better amplifier (because they measure differently in multiple dimensions as any two amplifiers will), what's the hypothesis you're testing using difference-only listening tests?
 
Last edited:

andreasmaaan

Master Contributor
Forum Donor
Joined
Jun 19, 2018
Messages
6,652
Likes
9,399
But in my earlier post I said I didn't believe that: I know about electronic components and how much they cost. Sure, the final products may be dressed up and sold for thousands of dollars to audiophiles, but I see that as separate from the universal, timeless aspect of science.

The best-measuring DAC I know of is a DCS unit costing tens of thousands. It really does measure slightly - but definitely - better than the best-performing $100-$1000 DACs we all discuss frequently on this site.

But if you don't have a better amplifier to test it against, how are you going to do that? And if you do have a better amplifier, why not just use that all the time?

Because it may be more expensive. See my comments on DACs above.
 

SIY

Grand Contributor
Technical Expert
Joined
Apr 6, 2018
Messages
10,383
Likes
24,749
Location
Alfred, NY
Question: given the stellar measurements, why did anyone bother to design another amplifier after that?

"You don't necessarily need a choice of 23 underarm spray deodorants or of 18 different pairs of sneakers when children are hungry in this country."
 

MrGoodbits

Member
Forum Donor
Joined
Sep 6, 2018
Messages
63
Likes
110
Location
Knoxville, Tennessee
That happens anyway- your actual undistorted visual field is shockingly small, so you have to move your vision from one object to the other.

Of course, there's the blink method of detecting motion of objects in astronomy...
Exactly, a blink comparison is the direction I was going. The A and B images in the DBT would be displayed on the monitor at the same position and size, and the person taking the test could instantly swap them.

I did that sort of test in my job with images scanned from (historic) documents, trying to figure out the optimum JPEG 2000 compression settings to balance small file size against fidelity losses from compression. How small could the compressed file be with the resulting image visually indistinguishable from the uncompressed original? Eventually I started digitally subtracting the original and compressed images to see and quantify the more subtle differences. Hmmm, a little like an audio null test...
 

LTig

Master Contributor
Forum Donor
Joined
Feb 27, 2019
Messages
5,760
Likes
9,442
Location
Europe
Take the Quad 405 amplifier. Back in 1975 or whenever it was, Quad produced an amplifier with very, very low distortion and no adjustments to make. It may have had a flaw in its overly-sensitive overload circuitry but this was, I believe, later corrected with a MkII version.

Question: given the stellar measurements, why did anyone bother to design another amplifier after that?
Because the Quad has not enough power for insensitive speakers? The last presentation I witnessed used 4 mono blocks of 1200 W each to feed a pair of line array highend speakers in biamping mode.
 

Cosmik

Major Contributor
Joined
Apr 24, 2016
Messages
3,075
Likes
2,180
Location
UK
Because the Quad has not enough power for insensitive speakers?
As far as I'm aware, the Quad concept is scalable to whatever power you like - flea-powered it is not. At the time, the 405 was notable for its muscularity in comparison to most other amps.
 
Top Bottom