Master Thread: Are measurements Everything or Nothing?

[IAtaman]

If we don't measure everything we can't explain everything.
Tell me, do we measure everything?

Everything is an abstraction, like infinity, not a real world object. So we are not measuring everything for sure.

If the question is are we measuring all that can be relevant to the sound quality or transparency of an equipment, I am not qualified to answer that question as an expert, but my personal opinion (like yours I suspect) is no, we don't, depending on the equipment we are talking about. I.e. for cables, yes we do. For headphones, no we don't. How significant is what we don't measure to the sound quality and transparency, I don't know. I am trying to walk the thin line of keeping an open mind and BS some pure subjectivists are spewing out is making it really hard to do that some times

If someone's view is that there are qualities to audio that can never be measured but only be perceived by humans however, then I think this view should be discussed at astrology forums and not on ASR.

 

[Blumlein 88]

Transducers are what let us down. Electronics are mostly a solved issue. Headphones, speakers and microphones are not.

 

[solderdude]

If someone's view is that there are qualities to audio that can never be measured but only be perceived by humans however, then I think this view should be discussed at astrology forums and not on ASR.

I am off to astrology then....

The point Blumlein 88 makes is very true.
For electronics all aspects of signal fidelity can be measured in great detail under various loads.
The problem is the words 'all aspects'. Usually only some aspects are measured/published that are the most telling and or easy to understand/explain and usually also with simple loads under laboratory conditions.

Measurements can tell everything here if everything is measured/tested for. Which often is not the case.

Speakers can be measured but interact with rooms and positioning. A spinorama can provide a lot of data but extrapolating that data on how that speaker will sound in your room/conditions is no easy task if at all possible. It would have to be measured in situ. And this is also no simple thing. A microphone in the listening position can surely improve things so can room conditioning.
So speakers can be measured but only some aspects and should be measured in the actual circumstances.
Speaker measurements can show certain issues of that speaker which is valuable info but not all telling.

Headphones can be measured acc. to a standard. The problem here is that the measurement fixture used may not be the same as your hearing. Then there are seal issues and other aspects (pad wear for instance) and fit issues that could be measured but often are not.
So here too measurements can show problems and expected tonal balance in 'ideal' conditions. Don't expect the measurements to be accurate to your ears and a headphone to perform exactly the same on your head.
Measurements are good and valuable indicators though but not all aspects are measured and accuracy leaves to be desired.

 

[IAtaman]

Transducers are what let us down. Electronics are mostly a solved issue. Headphones, speakers and microphones are not.

The culprit is psychoacoustics isn't it - the complex physics and physcology of hearing? From what I have seen, as all discussions about transparency and sound quality boils down to a discussion about audibility after a while.

For electronics, on their own maybe but when they are interacting with transducers, doesn't that bring some level of ambiguity into the equation? I was discussing this on another thread actually, but a headphone amp that is rated 120db SINAD for example, can be as low as 80db at normal listening levels, and that is assuming resistive load. Do we know enough to conlude that the output linearity will stay linear and transparent despite reactive components introduced by transducers?

 

[IAtaman]

I am off to astrology then....

Typical Scorpio!

Speakers can be measured but interact with rooms and positioning.

Headphones can be measured acc. to a standard. The problem here is that the measurement fixture used may not be the same as your hearing.

Right, it is the transducers interacting with complex and unknown real world in chaotic ways including rooms, our heads and bodies, our ears and most confusingly our brains.

 

[solderdude]

Typical Scorpio!

Astrology didn't work here... I am a Capricorn (not a unicorn )
Using correct data (date of birth) would have given the correct answer.

The brain, ego, circumstances and ears are definitely responsible for the disconnect between measurements and perception.
Fortunately, in the end, it is all about enjoying music.
Measurements can show 'technical performance' under standardized (or not) conditions.
They can be valuable for that when done properly and one understands all the measurement results and circumstances under which those are obtained.

 

[Deleted member 48726]

Everything is an abstraction, like infinity, not a real world object. So we are not measuring everything for sure.

If the question is are we measuring all that can be relevant to the sound quality or transparency of an equipment, I am not qualified to answer that question as an expert, but my personal opinion (like yours I suspect) is no, we don't, depending on the equipment we are talking about. I.e. for cables, yes we do. For headphones, no we don't. How significant is what we don't measure to the sound quality and transparency, I don't know. I am trying to walk the thin line of keeping an open mind and BS some pure subjectivists are spewing out is making it really hard to do that some times

If someone's view is that there are qualities to audio that can never be measured but only be perceived by humans however, then I think this view should be discussed at astrology forums and not on ASR.

I agree.

Your question as to how significant the data we don't get because of what we don't measure, I think it could explain and validate the claim that I (and many other) have; that amplifiers can sound different with the same set of speakers because of the load/amplifier combination. And I'm not proposing measuring with microphone in these scenarios but some well chosen complex loads to represent real speaker loads.

I think it's utterly stubborn and naive to dismiss things that haven't been thoroughly scrutinized. It's not in the spirit of science as I know it.

 

[bodhi]

I agree.

Your question as to how significant the data we don't get because of what we don't measure, I think it could explain and validate the claim that I (and many other) have; that amplifiers can sound different with the same set of speakers because of the load/amplifier combination. And I'm not proposing measuring with microphone in these scenarios but some well chosen complex loads to represent real speaker loads.

I think it's utterly stubborn and naive to dismiss things that haven't been thoroughly scrutinized. It's not in the spirit of science as I know it.

You and many others can validate your claims much, much more easily with a properly controlled test. When this phenomenon has been observed it's time to find explanation for it.

 

[Zinda]

@amirm I'd like to know your position on the question. I imagine you are somewhere in the middle or else why bother with subjective listening impressions in your reviews? In a number of these listening reviews you have said how you couldn't really hear a fault that you measured and that you weren't always sure whether EQ improved a dip/peak or not.

Sometimes you was surprised by how much you enjoyed a speaker, I think the JBL 4349 was one of these, even though the measurements were far from the best you had seen. It seems the speakers provided something in reproduction that the measurements themselves didn't make obvious, which would suggest that measurements (at least as they currently stand) are not the be all and end all, at least when it comes to enjoying music.

Your thoughts?

Ever think that the actual measurements could be flawed from the get go? Just because there's a program written for testing it doesn't mean that it's going to provide accurate measurements in every single test. If all tests were done "the old fashioned" way (analog) I wouldn't ever question any results. But after dabling with digital equipment and test programs I finding there's quite a large difference in what I see compared to what I hear. Digital will never sound as good as analog except for in 1 aspect - background noise or silence. That's the one thing that digital has a purpose. Or you only listen to pure electronic music that has no strings or voices or horns or pianos or etc...

I have my own suspicions that programs are made with certain ideas in mind that aren't necessarily based on perfection but okay a role in a larger idea that could be based purely on $. Even a free program can be a huge mony generating product by changing the way people think. Controlling a single aspect within a program can make a huge shift in how people perseve audio and music reproduction. The purists know that analog is the only way to go and it's impossible to get that to change unless digital can suddenly become infinite.

By removing one control aspect there's a chance to prove a new way of thought. The problem comes in when you can't get to if these programs to show identical results!
What's more is how a non digital based test will show huge changes in sound when an analog EQ is used to correct response and then a digital program is used just to find out that the digital program is showing a very small literally no change at all.

This is showing me that their main objective is based on the idea that you can't correct sound using an EQ but yet I can test it with any number of other tests that all show a dramatic change in sound and balance can be achieved using an EQ?

Which one is correct? The one that has been used since the invention of the ocsilliscope or the programs that have been constantly changing over the last 15 years? When a program is so difficult to get set up correctly unless you use specific hardware shows me that there is an agenda and is not flexible which reduces chances of finding comparable results with any other equipment. This way there is a reduced amount of possible results that don't agree with other results generated using the same program.

There's also quirks within the operating systems and sound cards that can mislead or alter results. I don't think I've been able to get 2 exact results yet while using a computer based test programs yet every time I test using a dedicated test set up I can see the exact same results as the last test I did.

Using a powered mic and a USB interface should allow a few different ways to set up for testing yet I can only get it to accept 1 way to set it up and get any results at all. Using anything other than the mic as a USB mic results in no sound output but the sound output still works through the operating system thereby changing the mic from an analog item to a digital. That's where everything goes wrong. If I can't use an analog mic what good is the program? It's already rounding up and down as it pleases before the signal is even read it's been changed and could be flawed right from the get go!

Ive tried using the analog output from the interface to a mic input and as a line input but can't get it to work. Using the interface for both USB mic and speaker out just created feedback. So that rules out any other possible way to set up even though the program suggests it's possible to use different set ups I've fou d that to be untrue.

 

[fpitas]

Ever think that the actual measurements could be flawed from the get go? Just because there's a program written for testing it doesn't mean that it's going to provide accurate measurements in every single test. If all tests were done "the old fashioned" way (analog) I wouldn't ever question any results. But after dabling with digital equipment and test programs I finding there's quite a large difference in what I see compared to what I hear. Digital will never sound as good as analog except for in 1 aspect - background noise or silence. That's the one thing that digital has a purpose. Or you only listen to pure electronic music that has no strings or voices or horns or pianos or etc...

I have my own suspicions that programs are made with certain ideas in mind that aren't necessarily based on perfection but okay a role in a larger idea that could be based purely on $. Even a free program can be a huge mony generating product by changing the way people think. Controlling a single aspect within a program can make a huge shift in how people perseve audio and music reproduction. The purists know that analog is the only way to go and it's impossible to get that to change unless digital can suddenly become infinite.

By removing one control aspect there's a chance to prove a new way of thought. The problem comes in when you can't get to if these programs to show identical results!
What's more is how a non digital based test will show huge changes in sound when an analog EQ is used to correct response and then a digital program is used just to find out that the digital program is showing a very small literally no change at all.

This is showing me that their main objective is based on the idea that you can't correct sound using an EQ but yet I can test it with any number of other tests that all show a dramatic change in sound and balance can be achieved using an EQ?

Which one is correct? The one that has been used since the invention of the ocsilliscope or the programs that have been constantly changing over the last 15 years? When a program is so difficult to get set up correctly unless you use specific hardware shows me that there is an agenda and is not flexible which reduces chances of finding comparable results with any other equipment. This way there is a reduced amount of possible results that don't agree with other results generated using the same program.

There's also quirks within the operating systems and sound cards that can mislead or alter results. I don't think I've been able to get 2 exact results yet while using a computer based test programs yet every time I test using a dedicated test set up I can see the exact same results as the last test I did.

Using a powered mic and a USB interface should allow a few different ways to set up for testing yet I can only get it to accept 1 way to set it up and get any results at all. Using anything other than the mic as a USB mic results in no sound output but the sound output still works through the operating system thereby changing the mic from an analog item to a digital. That's where everything goes wrong. If I can't use an analog mic what good is the program? It's already rounding up and down as it pleases before the signal is even read it's been changed and could be flawed right from the get go!

Ive tried using the analog output from the interface to a mic input and as a line input but can't get it to work. Using the interface for both USB mic and speaker out just created feedback. So that rules out any other possible way to set up even though the program suggests it's possible to use different set ups I've fou d that to be untrue.

No offense intended, but I think you need a lot more technical understanding before you can reliably judge.

 

[bodhi]

No offense intended, but I think you need a lot more technical understanding before you can reliably judge.

Some common sense is enough. Any time you start thinking that some kind of consipiracy is to blame for results you don't like you should take a deep breath and think things through one more time.

 

[MdeVelde]

Came across this thread a bit late. But here's my 2c..

Measurements + knowledge of what these measurements actually say / mean is everything.

Let's take speaker and room measurements for instance.
- THD by itself doesn't say much. In general higher order harmonics are much more audible than lower harmonics. But even this is a generalization. One has to know about masking and the sensitivity of our ear at various frequencies etc.
- IMD is perhaps more relevant / audible. Though with knowledge we can make an educated guess based on harmonic distortion.
- Compression at higher SPL is a very audible thing. Often not measured or mentioned in specs.
- Why are speakers always measured with a linear amplitude sine sweeps? Actual music is in general almost like pink noise (with about a 3dB/oct dropoff above 40-50Hz or so).
- Many people stare too blindly at on-axis freq response of course. But also a smooth dropping power response isn't saying everything. The big elephant in the room here is that speakers sound "dead" at the back of the speaker. This is a very audible thing and is what makes speakers sound like "speakers" instead of "real" in my opinion. It's in the measurements but still the knowledge of what it does to sound / audibility of this in an actual room is very limited in almost all people (personal opinion, not intended to offend)
- Many seem to think that an omni microphone in a room reflects objectively the sound of the speakers in that room. It does not! Our ears are nothing like an omni microphone! To understand how we hear you have to take the head related transfer function into account. Which means that we hear very differently depending on the direction the sound comes from. A reflection coming from the side or from behind may measure exactly the same but sound totally completely different to an actual human.

And this gets to the point I think. The measurements often used are not wrong themselves. But the way they are normally presented can be improved a great deal in my opinion (to match actual human perception a lot better than what is currently standard).

 

[antcollinet]

Ever think that the actual measurements could be flawed from the get go? Just because there's a program written for testing it doesn't mean that it's going to provide accurate measurements in every single test. If all tests were done "the old fashioned" way (analog) I wouldn't ever question any results. But after dabling with digital equipment and test programs I finding there's quite a large difference in what I see compared to what I hear. Digital will never sound as good as analog except for in 1 aspect - background noise or silence. That's the one thing that digital has a purpose. Or you only listen to pure electronic music that has no strings or voices or horns or pianos or etc...

I have my own suspicions that programs are made with certain ideas in mind that aren't necessarily based on perfection but okay a role in a larger idea that could be based purely on $. Even a free program can be a huge mony generating product by changing the way people think. Controlling a single aspect within a program can make a huge shift in how people perseve audio and music reproduction. The purists know that analog is the only way to go and it's impossible to get that to change unless digital can suddenly become infinite.

By removing one control aspect there's a chance to prove a new way of thought. The problem comes in when you can't get to if these programs to show identical results!
What's more is how a non digital based test will show huge changes in sound when an analog EQ is used to correct response and then a digital program is used just to find out that the digital program is showing a very small literally no change at all.

This is showing me that their main objective is based on the idea that you can't correct sound using an EQ but yet I can test it with any number of other tests that all show a dramatic change in sound and balance can be achieved using an EQ?

Which one is correct? The one that has been used since the invention of the ocsilliscope or the programs that have been constantly changing over the last 15 years? When a program is so difficult to get set up correctly unless you use specific hardware shows me that there is an agenda and is not flexible which reduces chances of finding comparable results with any other equipment. This way there is a reduced amount of possible results that don't agree with other results generated using the same program.

There's also quirks within the operating systems and sound cards that can mislead or alter results. I don't think I've been able to get 2 exact results yet while using a computer based test programs yet every time I test using a dedicated test set up I can see the exact same results as the last test I did.

Using a powered mic and a USB interface should allow a few different ways to set up for testing yet I can only get it to accept 1 way to set it up and get any results at all. Using anything other than the mic as a USB mic results in no sound output but the sound output still works through the operating system thereby changing the mic from an analog item to a digital. That's where everything goes wrong. If I can't use an analog mic what good is the program? It's already rounding up and down as it pleases before the signal is even read it's been changed and could be flawed right from the get go!

Ive tried using the analog output from the interface to a mic input and as a line input but can't get it to work. Using the interface for both USB mic and speaker out just created feedback. So that rules out any other possible way to set up even though the program suggests it's possible to use different set ups I've fou d that to be untrue.

Suspicions are utterly pointless. How about coming up with some actual science or engineering based justification for your thoughts.

Or even just some actual facts. For example Digital measurements are quite capable of showing the effects of EQ. Everone here using DSP room correction uses digital measurments to implement the filtering and then to check the results. So what is this misunderstanding you have that somehow digital measurements are in some way inferior to analogue ones?

Don't mistake your own inability to set up your system properly as a fundamental problem with digital.

 

[fpitas]

Some common sense is enough. Any time you start thinking that some kind of consipiracy is to blame for results you don't like you should take a deep breath and think things through one more time.

Well, yes. That and guessing that everybody is all wrong, for reasons you can't elucidate or understand. And no one but you has noticed.

 

[Killingbeans]

Just because there's a program written for testing it doesn't mean that it's going to provide accurate measurements in every single test. If all tests were done "the old fashioned" way (analog) I wouldn't ever question any results.

Sounds like you live in some strange alternate reality.

As if Klippel or AP made some really good hardware and then just left the software development to the cleaning lady

If you don't like these "sterile" and "impersonal" ways of doing measurements, just say so. But don't spread nonsense about tolerance and precision.

 

[Mulder]

If someone's view is that there are qualities to audio that can never be measured but only be perceived by humans however, then I think this view should be discussed at astrology forums and not on ASR.

I have a friend who collects 78 rpm shellac records and old 78 record turntables. Some of them mechanical and not even electric. And she enjoys listening to these recordings on the old equipment from time to time. For her, it has a quality of it´s own. Much the same as is the situation with vinyl I think. She is also a piano player so she´s really into music. What this example says is that perceived audio quality in the end is very subjective and something different than measured audio quality. The former can´t be measured in any very meaningful sense. What we can measure is transparency, ie the absence of distortion and other electrical or other physical properties. We can then note that we can in some cases hear certain forms of distortion or noise. But that in itself is only one aspect of audio quality. In my opinion, the greatest value of ASR is that it is a source of enlightenment in a HiFi-world full of myths and false information.

 

[antcollinet]

I have a friend who collects 78 rpm shellac records and old 78 record turntables. Some of them mechanical and not even electric. And she enjoys listening to these recordings on the old equipment from time to time. For her, it has a quality of it´s own. Much the same as is the situation with vinyl I think. She is also a piano player so she´s really into music. What this example says is that perceived audio quality in the end is very subjective and something different than measured audio quality. The former can´t be measured in any very meaningful sense. What we can measure is transparency, ie the absence of distortion and other electrical or other physical properties. We can then note that we can in some cases hear certain forms of distortion or noise. But that in itself is only one aspect of audio quality. In my opinion, the greatest value of ASR is that it is a source of enlightenment in a HiFi-world full of myths and false information.

Don't mistake percieved quality with preference or enjoyment.

It is quite possible to enjoy lower quality music for reasons other than the quality or even because of the low quality - even when knowing it is lower quality.

 

[fpitas]

Don't mistake percieved quality with preference or enjoyment.

It is quite possible to enjoy lower quality music for reasons other than the quality - even when knowing it is lower quality.

Or music altered by special effects like a SET amp, for example. At that point the argument is never, "Do you enjoy it?", it's instead, "Is that actually hi-fi anymore?"

 

[Mulder]

Don't mistake percieved quality with preference or enjoyment.

So what is quality then?

 

[fpitas]

So what is quality then?

Whatever it is, he did qualify it as pereceived quality. So, there isn't much objective information left; i.e., "Someone likes it".