• WANTED: Happy members who like to discuss audio and other topics related to our interest. Desire to learn and share knowledge of science required. There are many reviews of audio hardware and expert members to help answer your questions. Click here to have your audio equipment measured for free!

Consumer evaluation strategies personal evaluation or relying on external evidence

fas42

Major Contributor
Joined
Mar 21, 2016
Messages
2,818
Likes
191
Location
Australia
Just talking about "audio transparency" for a moment - this one is pretty easy: going back to the TV comparison I introduced earlier this would be comparing walking into a room where there is a very large TV built into the end wall - imagine very well calibrated, using premium quality video recordings, lighting conditions just right - you might say, "That's damn good TV!" .. with walking into that room and seeing the end wall gone, and the room placed into some bizarre alternate university where what you see is completely at odds with what you logically know is outside that part of the house.

The first scenario matches typical decent sound reproduction; the second what is possible when "audio transparency" is fully operational - some may need lots of DBTs to pick the difference, but personally I don't require such ... ;)
 

Cosmik

Major Contributor
Joined
Apr 24, 2016
Messages
3,075
Likes
2,180
Location
UK
Just talking about "audio transparency" for a moment - this one is pretty easy: going back to the TV comparison I introduced earlier this would be comparing walking into a room where there is a very large TV built into the end wall - imagine very well calibrated, using premium quality video recordings, lighting conditions just right - you might say, "That's damn good TV!" .. with walking into that room and seeing the end wall gone, and the room placed into some bizarre alternate university where what you see is completely at odds with what you logically know is outside that part of the house.

The first scenario matches typical decent sound reproduction; the second what is possible when "audio transparency" is fully operational - some may need lots of DBTs to pick the difference, but personally I don't require such ... ;)
But in the case of the 'holodeck' we know what is required, and it *cannot* be generated by a 2D screen, no matter how bright and well-calibrated. We would need some sort of super-Oculus Rift, using science and engineering to ensure that a 3D model of a world is translated into the correct photons hitting the correct parts of the observer's retinas at the right time.

It is possible to do something similar in audio e.g. based on something like the Smyth Realiser. Are you saying that this is not necessary? That two stereo speakers in an arbitrary room and a static recording (no head tracking and dynamic modelling) will generate you an 'audio holodeck' that you can walk around, experiencing a realistic acoustic scene - even if the scene was acquired with arbitrarily-placed microphones set up by a 17 year old tea-maker's apprentice? Good luck with that!
 
OP
J

Jakob1863

Addicted to Fun and Learning
Joined
Jul 21, 2016
Messages
573
Likes
155
Location
Germany
If the effect of a digital cable is the item in question, but we don't understand how it could possibly introduce a difference, surely that means that we don't understand how the rest of the experimental apparatus works, either..?

Might be so, but, why should that be a problem?
It´s usually the foundation of model construction, that it is a restricted approximation to reality, that works sufficiently in some parts but surely can (and must) be modified if new evidence is gathered.

It is no good pointing to the switching mechanism and saying that it is truly random, etc. because it may also contain these mysterious effects that shouldn't be there.

Therefore anyone planning experiments has to be very cautious before adding new circuitry and claiming that it wouldn´t harm.

At some point, you have to draw a line and say "We understand how a wire works in the context of where we are using it". Otherwise, we don't understand anything.

If you stop questioning the models you´re working with, you most likely stop working scientifically.
There are a lot of aphorism reflecting that like:
"Todays scientific knowlegde is the scientific error from tomorrows point of view" or
"scientific process is to err in a creative way"
(to give the gist of it)

Drawing the line means in that context that you use the models where they obviously work, in most cases a "digital connection" works for audio in a sufficiently way, and to accept that there is still room for model evolution.
 
OP
J

Jakob1863

Addicted to Fun and Learning
Joined
Jul 21, 2016
Messages
573
Likes
155
Location
Germany
Just talking about "audio transparency" for a moment - this one is pretty easy: going back to the TV comparison I introduced earlier this would be comparing walking into a room where there is a very large TV built into the end wall - imagine very well calibrated, using premium quality video recordings, lighting conditions just right - you might say, "That's damn good TV!" .. with walking into that room and seeing the end wall gone, and the room placed into some bizarre alternate university where what you see is completely at odds with what you logically know is outside that part of the house.

The first scenario matches typical decent sound reproduction; the second what is possible when "audio transparency" is fully operational - some may need lots of DBTs to pick the difference, but personally I don't require such ... ;)

Audio transparency in our context just means, that a certain component does not introduce any perceivable difference.
As Cosmik said, that listeners really confuse a two channel stereophonic loudspeaker reproduction with a real acoustical event is imo quite rare.

Binaural recordings should be able to be near match to reality as the real ear channel soundfields are approximated.
 

fas42

Major Contributor
Joined
Mar 21, 2016
Messages
2,818
Likes
191
Location
Australia
It is possible to do something similar in audio e.g. based on something like the Smyth Realiser. Are you saying that this is not necessary? That two stereo speakers in an arbitrary room and a static recording (no head tracking and dynamic modelling) will generate you an 'audio holodeck' that you can walk around, experiencing a realistic acoustic scene - even if the scene was acquired with arbitrarily-placed microphones set up by a 17 year old tea-maker's apprentice? Good luck with that!
I am saying exactly that! You can't walk through the scene, it's always in front of you, but no matter where you walk it maintains a robust solidity - precisely this happened to me 3 decades ago, completely unexpectedly - WTF!! , blah, blah, blah ...

It happens because our hearing wants to make sense of what the ears pick up - but normally the sound is too corrupted by subtle flaws and any illusion easily fails. But, attain a sufficient competence, low enough distortion in key aspects - and then the "holodeck" holds strong.

The competence of the recording is immaterial. The auditory clues are always picked up, irrespective of what the person playing with the mics is trying to do - and our minds do the rest.
 

fas42

Major Contributor
Joined
Mar 21, 2016
Messages
2,818
Likes
191
Location
Australia
As Cosmik said, that listeners really confuse a two channel stereophonic loudspeaker reproduction with a real acoustical event is imo quite rare.
Yes, quite rare - but it doesn't have to be that way ... ;)
 

RayDunzl

Grand Contributor
Central Scrutinizer
Joined
Mar 9, 2016
Messages
13,198
Likes
16,981
Location
Riverview FL
but no matter where you walk it maintains a robust solidity - precisely this happened to me 3 decades ago, completely unexpectedly

What setup and environment prompts that perception?
 
Last edited:

fas42

Major Contributor
Joined
Mar 21, 2016
Messages
2,818
Likes
191
Location
Australia
What setup and environment prompts that perception?
The setup in a conventional sense is not important - what matters more than anything else is the quality of the audio being broadcast from the speaker driver surfaces, however they may be constructed. One key requirement, for me at least, is to make sure the speaker cabinet is firmly locked into position with respect to the room - jokes about Blu Tack now allowed!! - conceptually, the frame of the speaker driver which the diaphragm moves in relation to should be as stable, as unmoving as possible; this is particularly important for the bass content to work subjectively, I find.

The environment also has no requirements - I'm sure some conditions will be more conducive than others, but I have never investigated this.
 

Sal1950

Grand Contributor
The Chicago Crusher
Forum Donor
Joined
Mar 1, 2016
Messages
14,068
Likes
16,598
Location
Central Fl
As Cosmik said, that listeners really confuse a two channel stereophonic loudspeaker reproduction with a real acoustical event is imo quite rare.
Always brings me back to an observation made by J Gordon Holt (or maybe it was Paul Klispch?),
You walk out your door and someone down the street is playing an instrument live, immediately your ear perks up and you JUST KNOW it's the real thing.
I'm not so sure even the best of todays systems can fool people in that way?
YMMV
 

fas42

Major Contributor
Joined
Mar 21, 2016
Messages
2,818
Likes
191
Location
Australia
Always brings me back to an observation made by J Gordon Holt (or maybe it was Paul Klispch?),
You walk out your door and someone down the street is playing an instrument live, immediately your ear perks up and you JUST KNOW it's the real thing.
I'm not so sure even the best of todays systems can fool people in that way?
YMMV
Yep, that's the goal ...

And the news is good - that quality is very much achievable ... but it's either pretty expensive, or one has to go to a lot of effort and fussing about to get there - at the moment ...
 

Cosmik

Major Contributor
Joined
Apr 24, 2016
Messages
3,075
Likes
2,180
Location
UK
Might be so, but, why should that be a problem?
It´s usually the foundation of model construction, that it is a restricted approximation to reality, that works sufficiently in some parts but surely can (and must) be modified if new evidence is gathered.



Therefore anyone planning experiments has to be very cautious before adding new circuitry and claiming that it wouldn´t harm.



If you stop questioning the models you´re working with, you most likely stop working scientifically.
There are a lot of aphorism reflecting that like:
"Todays scientific knowlegde is the scientific error from tomorrows point of view" or
"scientific process is to err in a creative way"
(to give the gist of it)

Drawing the line means in that context that you use the models where they obviously work, in most cases a "digital connection" works for audio in a sufficiently way, and to accept that there is still room for model evolution.
Where I am going with this, I think, is that without objective evidence, you have no science. The audiophile listening test is just a preamble to where the real science starts - it is not science in itself.

Science demands a hypothesis, and for good reason it demands that you test/demonstrate the hypothesis with objective measurements. Even if you show a clear, repeatable result with a listening test, you have not demonstrated or tested a hypothesis: all you have done is to show that two sets of circumstances led to an apparent audible difference - and without objective measurements, *you cannot say what those two sets of circumstances are*. Saying "All we changed was the colour of the cable, and the cable was hidden from view, therefore cable colour causes audible difference" is not good enough. Only objective measurements can show why you really got a difference and how to make the results relevant outside your experiment. Only objective measurements can show that your experiment isn't flawed in some way you haven't thought of.

A would-be scientist who states implicitly that they don't believe in objective measurements (e.g. digital cables used in situations where they cannot make any difference are making a difference, but the effect cannot be measured objectively) has written themselves out of the scientific realm. Their listening test is not science in itself. Only by coming up with a hypothesis about why the cable might be having a physical effect, and then investigating it with objective measurements can they regain their scientific credentials. If they cannot find a measurable effect, they *must* conclude that their listening test is most likely faulty. Without that measurable difference, there is nothing to publish.
 

Fitzcaraldo215

Major Contributor
Joined
Mar 4, 2016
Messages
1,440
Likes
632
Spoken like a true hard, physical scientist. The implication is that social, behavioral, perceptual or even much medical science focused on the responses of human test subjects is invalid and not"true" science. Educationally, I have studied hard science as well as social and behavioral science, plus statistics. The notion that objective studies in those fields do not first pose a hypothesis does not square at all with what I have learned.

Of course, the physical sciences are different in many ways. It is nice to mix chemical A with chemical B in a test tube and replicate the exact observable and measurable results science has told us to expect. (Yes, most physical science is not that simple or clear cut.). But, anything involving humans is far noisier and less predictable as it pertains to any one individual. Different ball game, but still incredibly useful science in my book, albeit with many, many differences in the techniques and disciplines vs. the physical sciences.
 
OP
J

Jakob1863

Addicted to Fun and Learning
Joined
Jul 21, 2016
Messages
573
Likes
155
Location
Germany
Always brings me back to an observation made by J Gordon Holt (or maybe it was Paul Klispch?),
You walk out your door and someone down the street is playing an instrument live, immediately your ear perks up and you JUST KNOW it's the real thing.
I'm not so sure even the best of todays systems can fool people in that way?
YMMV

It might depend on the recording quality. Generally the directivity of loudspeakers and real acoustical instruments is vastly different, so if the listener is in the same room (typical small room) that will be a factor.
But usually we notice the difference between a life event and a recording even if it happens in the next room and we have only the sound coming through the door opening for evaluation.

But, as said above, if you listen to live recorded acoustical events without any postprocessing, just the raw (high quality) microphone signal on tape or digitized, that´s usually a sursprising experience. As said above if binaural recorded and reproduced via a high quality headphone/inear, it is even more stunning.
 
OP
J

Jakob1863

Addicted to Fun and Learning
Joined
Jul 21, 2016
Messages
573
Likes
155
Location
Germany
Where I am going with this, I think, is that without objective evidence, you have no science. The audiophile listening test is just a preamble to where the real science starts - it is not science in itself.

Science demands a hypothesis, and for good reason it demands that you test/demonstrate the hypothesis with objective measurements.

There seems to be a misunderstanding.
The listening test is part of the scientific process and is already working on the basis of a hypothesis, it is the null-hypothesis expressing the point of view that no perceptable difference _can_ exist. The experiment called listening test tries to find evidence against this hypothesis.

<snip>Only objective measurements can show why you really got a difference and how to make the results relevant outside your experiment. Only objective measurements can show that your experiment isn't flawed in some way you haven't thought of.

As said several times before, our problem is mainly that "objective" measurements exist that show differences, but the hypothesis nevertheless claim that they can´t be perceived.
Sometimes there aren´t measurements of "something" but were later developed or just reinvented.

A would-be scientist who states implicitly that they don't believe in objective measurements (e.g. digital cables used in situations where they cannot make any difference are making a difference, but the effect cannot be measured objectively) has written themselves out of the scientific realm. Their listening test is not science in itself. Only by coming up with a hypothesis about why the cable might be having a physical effect, and then investigating it with objective measurements can they regain their scientific credentials. If they cannot find a measurable effect, they *must* conclude that their listening test is most likely faulty. Without that measurable difference, there is nothing to publish.

Fitzcarraldo215 already pointed out a few things about your hypothesis and i´d add that the replication crisis in science exists because our understanding and therefore the models we´ve developed still often aren´t good enough. Humans are complex (and often nonlinear) systems.

If you state that "..where they cannot make a difference.." that is just a conclusion from you model, but it might not reflect reality in a sufficiently way. The audiophile habit of generalization is often not helpful but the "nonaudiophiles" urge to dismiss everything isn´t either.
 

fas42

Major Contributor
Joined
Mar 21, 2016
Messages
2,818
Likes
191
Location
Australia
Everything makes a difference, that's how the universe works. Next step, is it measurable, using conventional techniques? Many times, it's very difficult, or extremely time costly and even money costly to do so - so often it's not worth the effort - at that point in time.

What's relevant, in the audio enthusiasts' world, is whether that inevitable difference is audible ... this is the tricky bit, where people start hurling tin cans at each other - but IME it usually comes down to how 'fussy' people are in their hearing.
 
Top Bottom