Master Thread: Are Measurements Everything or Nothing?

[sonitus mirus]

Yes, they generally do, though unlike delta-sigma converters it's not inherent to the approach.

It's an expensive boutique device so it's not hard to guess what the "love" is for, and it seems like a well-engineered converter, but so are ESS/AKM ICs that cost less than the sandwich I'm having for lunch.

Is that a DACwood sandwich?

 

[Newman]

You‘ve got it wrong. I‘m on your ignore list, maybe.

No, I read you. All good.

We know that measurements as taken today are not all descriptive in regard to sound quality. HD was shown, for example by Geddes, to have little to no correlation to sound quality.

That doesn't make anything I said 'wrong'. Neither I nor others ever said that every measurement is critical to sound quality, and at every level. Are you erecting a straw man?

BTW Geddes' paper on HD actually described which harmonic distortions we are most acutely sensitive to and dislike most. Rather the opposite of your description of it. Perhaps you meant to write 'THD'.

 

[MAB]

I fully appreciated that some other people might not be used to being challenged and will perceive it as a threat to their beliefs but that ironically is like people claiming they can hear differences that don’t exist, some people will choose to subjectively interpret something that doesn’t actually exist.

Funny you missed the first post. How come I’m not surprised?

Seems like so many review threads get challenged with:

1. Measurements are not everything.

2. You all never listen.

3. I trust my ears, not graphs.

4. I don't listen to graphs. I listen to music.

5. You all must not listen to music at all.

6. Why don't you all buy the best SINAD gear?

7. I have heard your best SINAD gear and they sound terrible. I don't like any of this Chinese stuff.

8. You don't trust your ears. I/we do.

9. All these reviewers/youtubers/audophiles say these amps, DACs, etc. sound different and you say they don't. They can't all be wrong.

10. Surely designers have created certain house sound for each equipment which your measurements don't show.

11. Your measurements are only at one frequency. You need to also measure X, Y and Z like impulse response, slew rate, etc., etc.

12. You guys run a cult here where you only go by measurements and no one is allowed to disagree.

On and on...

I have had to answer these so many times that I thought it is time to stop having them go into every review as they are not product specific. From here on, any such questions should be posted here. Answers will be given in this thread and simply referenced in future challenges in other threads.

Thanks. You all are free to discuss this topic, provide answers, argue, whatever, in this thread.

 

[solderdude]

I wish there was a lot more blind testing but that is difficult in practice.

Most certainly is....

Well conducted blind testing with the goal to provide irrefutable proof in a scientific way that can convince others and is verified by others repeating the experiment is indeed quite a task and whatever one does at home will never be able to convince others ... when differences are found/perceived.

• When one cannot discern differences in blind tests but could when one tested subjectively they are very likely to believed as it matches what is often found in proper research.
• When one cannot discern differences or isn't bothered by insignificantly small differences... be glad it could save you quite a lot of money.
• When one is of the opinion that one can hear differences with everything ... good for you. Don't bother trying to convince others... just be happy YOU can.
  And ... don't demand explanations from the 'measurebators' as proving proof is on the claimant which of course they can't/won't.

Using blind testing at home for your own education is another matter and can be very enlightening.
Chances are that those tests may well be flawed but in general are a better way of testing than just subjective observations/evaluations.

What is easy to test 'blind' but always requires a helper:

interlink cables.
Mains cables.
Music files.
Certain 'tweaks' like ground boxes and obvious snake-oil.

Blind testing DACs, amplifiers etc already requires test equipment for checking the setup + helper.

Speakers in a room is an entirely different animal as for instance 2 different speakers next to each other might influence the sound of the other speaker.
You can easily test for preference though but one should at least perform some kind of level matching (very difficult to do and would require narrow noise bands for instance or warble tones).

Headphones ... Yeah... one can tell them apart just by how they feel on the head alone which is a tell and that's not the only issue.
Listen for preference instead when you want 'enjoyment'. They all sound (and measure) different anyway.
Use known good monitors as reference (preferably measured in situ) when the goal is to match the monitors and you need to really know how to EQ by ear (starting from presets obtained with measurements) when the goal is accuracy to 'speakers in your room'.

• Subjective Audiophiles should use their ears to decide what they 'look for' (enjoyment). This can become a very expensive hobby.
• Sound engineers should aim for a reference (accuracy). This can turn out to become expensive depending on the required level of accuracy at various SPL.
• Objective audiophiles should buy gear that satisfies their preferred signal fidelity demands and acoustically measure the crap out of their room at the listening position and then trust the results. When music sounds crap ... it is the recording. When music sounds great ... it is the recording.
  This has the potential becoming a bit less expensive as the other 2 use cases, but ... transducers should probably not be skimped on.

In all cases the final arbiter will be perception. It is a matter of how to use perception and what the goal is.

 

[Heinrich]

No, I read you. All good

Me too.

BTW Geddes' paper on HD actually described which harmonic distortions we are most acutely sensitive to and dislike most. Rather the opposite of your description of it. Perhaps you meant to write 'THD'.

My bad. The summary was too short, leaving out that detail. Restart?

I found myself to be perfectly happy with a—by SINAD, medium/high rated set of digi amps and pretty high ranked speakers plus ‚perfect‘ subs, driven by a highly qualified USB interface. I would like to generalize this finding as a recommendation to the common man, who may have like 2.5k left to spend for a once for all purchase. To get it done, what to do?

No in situ evaluation needed, measurements tell it all. All components meet spec/ of „better than good/needed“. That might work out, after equalization it‘s just perfect.

Next recommendation, it‘s about 1k of money, which for many is still a lot. Not even AsciLab with their sensational quality for price could deliver the goods. Compromise is needed, but which? Evaluate based on individual, subjective feelings. Or is there a formula like (SINAD x directivity / smoothness + powercord length) x (size - power^2)?

Decision making with limitless budget (money, real estate, reputation, …) is easy. Most perfect, taken against the competition is a no-brainer. That‘s a less common situation, though. Thing is, all those measurements are neither comparable to a non-exsisting good enough, nor do they share a common scale.

 

[krabapple]

Science and technology moves on. Newton gets replaced by Einstein.

That's a century ago now. And it concerned questions vastly more difficult than measuring the 'sound' of a DAC or amp. Or loudspeakers for that matter

Materials improve, storage improves, processing improves , knowledge improves etc There is invariably few absolutes especially when it comes to human interaction. None of that is to diminish the research and measurements that we do have at all. That is just the way it goes. The ASR survey shows that hard core measurement absolutists are in the minority tail of the normal like curve.

The scientists you credit for improving technology and science are also a 'minority tail' in the population.

Please point me to the "ASR survey" you cite.

 

[MattHooper]

Heads up:

There is a very nice interview with Floyd Toole on the current Audio Unleashed podcast.

All of it was great, but I especially enjoyed when Toole was asked to address
“ subjectivists,” including the misconception that “objectivists” like Toole only look at graphs and don’t listen to or care about music (and also Toole’s elaboration on what he sees as the goal of his work versus choosing to colour the sound at the consumer end).

 

[Heinrich]

The scientists you credit for improving technology …

Measurement is not science, but engineering.

The problem with Newton‘s theory of gravitation was about which strings attached would transceive the immense forces e/g between sun and our mothership, earth.

Audio isn‘t a model driven search for truth. Measurement is done for the sake of (commercial) competition. Then after using it for that purpose it’s meaning is as important as the demanding listener‘s pleasure.

I wonder why audio as a hobby refers to science that often. Do you want to pull science in? Well, it doesn‘t care that much. There are so many open questions, why not start to find reliable answers? Don‘t expect them from anybody else, get started.

 

[Jim Taylor]

Measurement is not science, but engineering.

Really? Then how was Neptune discovered? How did we learn the value for gravity on Mars? Who "engineered" the speed of light?
And why is the journal of metrology (called "Measurement") known as a scientific journal rather than an engineering journal?

Measurement (journal) - Wikipedia

en.wikipedia.org

Perhaps science creates measurements and engineering uses them ... would that be better?

Audio isn‘t a model driven search for truth.

Had someone said that it was? I must have missed that.

. Measurement is done for the sake of (commercial) competition.

Measurement is done to ascertain what something is and what it does. As for audio ... does this item do what is advertised? If nothing is advertised, then is there not a need to measure it to find out what it does and does not do? After all, it could be a total fake, couldn't it?

I wonder why audio as a hobby refers to science that often.

Audio enthusiasts are in love with music, but try making a recording of that music without using science. Science sets the rules for the process as well as the function of the equipment.
Try playing back that recording without using equipment built using scientific principles.


As for the improvement of all processes ... including audio ... can we quantitatively improve something without using science? Personally, I don't think so.

 

[RexrothPigeon]

I presume read Toole is a fair response but is there a summary somewhere.

"Dudes would rather argue online for weeks on end than read a book"

 

[RickS]

Let’s keep a more civil tone here please. The opinion trolling was previously reported and a thread ban was issued. I empathize with some of the frustration but is better addressed by reporting than posting.

 

[Heinrich]

Really? Then how was Neptune discovered?

I‘m afraid you read me wrong. Newton‘s theory of gravity failed. It could calculate trajectories of massive object pretty well, except for Poincare‘s chaos, but never revealed the mechanism of how the forces involved would ‚travel‘ through space. What is the medium that ‚connects‘ stuff, angels’ strings? Newton made gravity accessible to calculation, but the mental model was just wrong, not only incomplete. Same with Goethe‘s theory of color, just plain wrong. (It is still propagated in arts class, though, while biology class tells the contrary.)

As I said before, measurement in audio as done today allows to select a best, and a fine ranking of devices, that some are eagerly after. What it doesn‘t help is rational decision making beyond that ranking.

How would one ‚trade‘ a smooth frequency response for less distortion? we neither have a good enough (basically all
amps and dacs pp are good enough today) that is commonly accepted (see SINAD competition), nor could we compare measurable properties of one kind to the other in regard to ‚overall quality, listener satisfaction‘.

It is the audio enthusiast‘s obligation to do some homework. A statement that measurement was science is a bit off, from a rational, objectivist‘s perspective—mine.

 

[Jim Taylor]

Newton‘s theory of gravity failed.

I am not an astrophysicist, but it has been agreed for many years (by others who are) that the discovery of Neptune was a confirmation of Newton's Theory of Gravitation.

As I said before, measurement in audio as done today allows to select a best, and a fine ranking of devices, that some are eagerly after. What it doesn‘t help is rational decision making beyond that ranking.

Many people use measurements in audio to "select a best". Personally, I don't look at sets of measurements that way. I use them to avoid both generally-under-performing equipment as well as equipment that has an "Achilles heel" ... a glaring deficiency in an otherwise acceptable set of characteristics. But ... to each their own.

How would one ‚trade‘ a smooth frequency response for less distortion? we neither have a good enough (basically all
amps and dacs pp are good enough today) that is commonly accepted (see SINAD competition), nor could we compare measurable properties of one kind to the other in regard to ‚overall quality, listener satisfaction‘.

As for evaluating different characteristics, I say today, as I have always said, that once you have winnowed the chaff, the remainder is a personal choice.

It is the audio enthusiast‘s obligation to do some homework.

Oh, I most heartily agree with this! (Every day that I read the posts here, I learn something new. Learning is not easy for an old fart like me, but I try.)

A statement that measurement was science is a bit off, from a rational, objectivist‘s perspective—mine.

I think this is something about which we must agree to disagree. I can accord you your viewpoint if you can accord me mine.

 

[solderdude]

Science is a broad field of study that involves using rigorous methods to investigate the natural and physical world.
Common duties of a scientist include creating hypotheses, reading past scientific research, and designing experiments.

Engineering involves using mathematical equations and scientific knowledge to solve problems.
It's a type of applied science, a field of study that also includes electronics and acoustic sciences.
Engineers perform research, design devices and test the effectiveness of their prototypes.

 

[Heinrich]

I A am not an astrophysicist, but it has been agreed for many years (by others who are) that the discovery of Neptune was a confirmation of Newton's Theory

From that statement alone I could, but never would, call you a science denier. You ignore the recent take on gravitas. But more so, you didn‘t grasp the differential between calculus, measurement and modelling reality. That‘s Fine, science is for specialists.

I think this is something about which we must agree to disagree. I can accord you your viewpoint if you can accord me mine.

The most reasonable Statement in this discussion so far.

I came to the conclusion that I secretly remain an objectivist, but when asked for advice I would tell „take what you like, sighted“. We as objectivists don‘t have proper arguments … (happy face)

 

[rdenney]

Oh, fer cryin' out loud. Engineering is the application of scientific principles to everyday problems so that humans can address those problems. The boundary between science and engineering is not in any way distinct, and the use of the word "engineering" (lower-case E) implies scientific understanding. Newton's model of gravitation is like all models--ultimately false but often useful.

Measurements exist for a single purpose: To validate. They either validate a theory for scientists, or for engineers they validate that an objective has been attained or requirements fulfilled. Engineers that don't know the science and scientists that have no relationship to engineering both undermine outcomes.

Robert Herman and Ralph Alpher estimated the background radiation from the Big Bang in 1948, using only physics and mathematics, while at the Johns Hopkins Applied Physics Laboratory. In 1964, Penzias and Wilson "discovered" that radiation while at Bell Labs using purely engineering techniques by the definition I'm seeing here--they systematically measured and rejected all competing causes for what they were observing without having formulated a theory. So, who were the engineers and who were the scientists? They were all considered to be scientists, but all of them were application-driven. Robert Herman would have rejected any dichotomy utterly. During WWII, he did theoretical work developing a proximity fuse for naval ordinance, but then shifted to testing and perfecting that work in the field, for which he was decorated by the Navy. He spent his later years at the University of Texas at Austin teaching traffic science specifically to engineers (myself included), and to be sure those engineers are still applying that science to real problems as the basis for developing engineering methodology. Engineers should be able to do that, even though many cannot, just as scientists should be conversant in application and ultimately driven by application requirements, but often are not.

Don't define people into boxes they should transcend as a matter of course.

Rick "an engineer who has developed a number of ways of measuring things" Denney

 

[Jim Taylor]

From that statement alone I could, but never would, call you a science denier.

Why, thank you! I am so very appreciative of your restraint!

 

[MAB]

Newton‘s theory of gravity failed.

Newton gets replaced by Einstein.

Grandiose statements, factually and philosophically wrong though. Slightly funny to read, like watching YouTube videos of people saying embarrassing things in public while people are taking videos with cellphones.

Einstein superseded Newton, it's an important distinction that you get wrong in your rush to score points here in the audio forum.

Neither does Chandrashekhar make Boltzmann wrong for not being able to describe the statistics that describe a White Dwarf star.

Ohm's Law still works for consumer electronics, despite many advanced extensions, modifications, and understandings on how transport actually occurs.

Phenomena like Integer and Fractional Quantum Hall Effects doesn't make the Hall Effect we observe in day to day life wrong or failed in any way whatsoever, and are of zero use in describing transport in a magnetic field like a speaker driver... Not one iota.

Both Newton and Einstein fail to actually describe gravity, yet both provide correct results in particular frames of reference. And for the purpose of audio, we don't need Einstein to describe how a turntable tonearm balances when used on our planet's surface. Or how a speaker cone sags in Earth's gravitational field.

You two are desperate to argue at all cost with no actual point, no barrier in your use of rhetorical nonsense and fake-gotchas in your debates. Both of you go onto ignore.

 

[dfuller]

I am not an astrophysicist, but it has been agreed for many years (by others who are) that the discovery of Neptune was a confirmation of Newton's Theory of Gravitation.

While we're on this tangent, it was a similar thing that led to Newtonian gravity no longer being the most accurate model. Mercury's orbit could not be explained without adding an extra planet that was not possible to find. However, general relativity does explain it well, along with everything else observed - though it isn't the final law of nature either, as GR itself breaks down at extremely high densities (see: black holes, extremely early universe) and starts spitting out infinities.

Anyway none of this is relevant to audio gear, we can pretty damn well characterize sonic performance of equipment, rooms, etc. - it's matching the objective onto subjective that is harder.

 

[Jim Taylor]

Anyway none of this is relevant to audio gear,

Correct. I should not have used analogies that were off the subject of audio.

Now back to our previously scheduled programming ...