Master Thread: Are measurements Everything or Nothing?

[Newman]

Dragging something here from Hoffman forum as evidence of something real is the epitome of irony.

 

[MattHooper]

Thanks Blumlein88. I was hoping you would take a look.

Really you should avoid Hoffman forums. It is like drinking from a polluted lake.

I'm a big boy, I can navigate the silliness there ;-)

I enjoy engaging in some of the music discussions, as well as some of the audio discussions. I don't use it as a place to rigorously vet claims about gear.

My guess would be if nothing else speed variations from run to run. I bet they could blind detect instance A, B and C even if no changes were made. In fact in this case, including one of the cables recorded two different times as a choice would have been a good idea.

Interesting. Good point.

So are they detecting this blind? I don't doubt you could. Is it because the cables sound different? Probably not. Along with maybe the different cable construction's capacitance effects the cartridge frequency response.

So I'm clear: does that mean it's possible they were hearing differences between the cables? (In how the different cables interacted with cartridge...)

I didn't delve into the details much, as it is a waste of time with those forums. In full disclosure I loath Steve, his forum and his moderators and with good reasons.

Ok, fair enough, then while you have raised some interesting issues, I shouldn't take your comment as exhaustive in terms of possible explanations.

 

[MattHooper]

Dragging something here from Hoffman forum as evidence of something real is the epitome of irony.

I'm a skeptic regarding cable differences. I noticed that some SH members seemed to have detected differences between the cables using and ABX test. Which made me think "hmmm, that's interesting, I wonder what might explain those results." The point of posting it here was to have people more technically knowledgeable to take a look and suggest what might be going on.

If this were just more sighted listening it wouldn't be of any interest I agree. But ASR members are always demanding "Ok, now do it without peeking." And since some apparently detected differences with ABX, then mere dismissal starts to look like the "ASR is only about mocking and will dismiss anything that doesn't match with their pre-conceived beliefs" stuff.

So this seems to me a good opportunity for ASR to provide some informed commentary.

Do you have anything to contribute?

 

[Blumlein 88]

Looking at the first page of comments, the OP showed Foobar results, everyone else except one was just saying I listened I heard a difference. As to why I won't bother, if the first 15 posters had said, we listened and heard nothing, Steve or his minions would have disappeared the thread. It would vanish. SOP there, and why it is not worth bothering with. If the fellow wanted to post his results here, then maybe someone would look. Otherwise, if you want to spend time there enjoy yourself. I am sure I'm not alone in taking a dim view of Steve Hoffman, and his forum.

 

[MattHooper]

Looking at the first page of comments, the OP showed Foobar results, everyone else except one was just saying I listened I heard a difference. As to why I won't bother, if the first 15 posters had said, we listened and heard nothing, Steve or his minions would have disappeared the thread. It would vanish. SOP there, and why it is not worth bothering with. If the fellow wanted to post his results here, then maybe someone would look. Otherwise, if you want to spend time there enjoy yourself. I am sure I'm not alone in taking a dim view of Steve Hoffman, and his forum.

I've been locked out of several threads, myself, on the SH forum (usually when I dared challenge the cables stuff or dared extol the usefulness of blind testing). I know how touchy they are. But there are conversations that happen there that I still enjoy, that tend not to happen here.

 

[MattHooper]

Do you have anything to contribute?

I guess not....

 

[Ghostofmerlin]

I guess not....

Existential crisis?

 

[Blumlein 88]

I guess not....

He probably feels like I do. Don't mistake not wishing to contribute for not having anything to contribute.

 

[MattHooper]

He probably feels like I do. Don't mistake not wishing to contribute for not having anything to contribute.

Once again, I appreciate that you gave a response.

Nobody has to contribute of course.

But in terms of reasons for not contributing...

For someone like Newman who continually talks about the necessity of blind testing, the breezy dismissal of apparently positive results for detecting cable differences, without engaging in the data, seems odd.

Scientifically, if there are valid positive results for a blind test it shouldn't matter where results come from.

Just dipping in to insult the place where the results came from..and that's it...seems something of a dodge in that respect.

It smacks of "I won't give anyone from the Steve Hoffman forum any credit ever, no matter what they do, since it's the Steve Hoffman forum."

And that frankly plays in to the feeling many outside have of ASR of a place so close minded to some things, like cable differences, that they won't even accept evidence when it's the type of evidence they are always demanding.

 

[Blumlein 88]

It smacks of "I won't give anyone from the Steve Hoffman forum any credit ever, no matter what they do, since it's the Steve Hoffman forum."

And that frankly plays in to the feeling many outside have of ASR of a place so close minded to some things, like cable differences, that they won't even accept evidence when it's the type of evidence they are always demanding.

Yes, you understand me at least. If someone wants to take part elsewhere then they'll get full consideration. Steve Hoffman's forum is a rigged forum, and I'll have no part of it period.

Now the last part about close mindedness here is a whole different matter and one that is not correct. If regular Hoffman participants get that wrong idea then I really could not care less. The recent thread on digital filter differences is the counterpoint. It has gotten pretty fair treatment and it appears the fellow could hear an improbable thing. I took some part in it. Anything from Hoffman or his forum gets no such treatment at least from me. Steve earned it and then some. If someone takes part and presents stuff there, they may be innocent, but too bad. Speaking only for myself which I hope is obvious.

 

[JP]

I'm a skeptic regarding cable differences. I noticed that some SH members seemed to have detected differences between the cables using and ABX test. Which made me think "hmmm, that's interesting, I wonder what might explain those results." The point of posting it here was to have people more technically knowledgeable to take a look and suggest what might be going on.

If this were just more sighted listening it wouldn't be of any interest I agree. But ASR members are always demanding "Ok, now do it without peeking." And since some apparently detected differences with ABX, then mere dismissal starts to look like the "ASR is only about mocking and will dismiss anything that doesn't match with their pre-conceived beliefs" stuff.

So this seems to me a good opportunity for ASR to provide some informed commentary.

Do you have anything to contribute?

For samples A and B the lead-in is a dead giveaway, and the ~0.2dB level difference and ~0.0544s alignment error don’t help. Overall speed fluctuation is around 22 samples max which is quite good.

My 20/20 score was done by listening to the first two seconds of the files. Let’s see if the
OP takes another shot with the corrected files I posted.

 

[Newman]

For someone like Newman who continually talks about the necessity of blind testing, the breezy dismissal of apparently positive results for detecting cable differences, without engaging in the data, seems odd.

For someone like Matt who continually disputes the need for blind testing, the breezy lack of criticality of apparently positive results for detecting cable differences, without engaging in the data, doesn't seem odd at all.

What other possible explanations for detecting the difference have you put forth? None?

How about Exhibit 1A: nobody says cables must sound indistinguishable, since they can differ in LCR (hence frequency response) and shielding (hence injected noise).

Do you see published LCR, frequency response, and noise shielding measurements for the cables used in the test, hooked up to the gear used in the test? No??? Then what is even slightly interesting? Nothing!

Which is exactly like I said in my 'breezy dismissal'. Most people who understand the importance of controlled listening tests would know that without having to be told. But, for those who are still waiting for the measurements to 'measure the right things' and hence confirm their personal sighted/uncontrolled listening observations - and I wonder who that could be! - then amateurish test results of surprising detectable differences have to be explained away, one by one, and every explanation disputed point by point, because they want the day to come when their sighted listening observations are validated.

Back to standard programming. Thank you and good night!

 

[Mnyb]

Yes experimental error is to be expected if unlikely or unreasonable things gets trough ABX test ?

properly designed cables does not change the signal ( within reason for audio applications ), trivial measurements can show that .
ABX this is more of test of the humans involved and theor ability to not botch the test.

you can show that the electrical signals are practically identical . Engaging in hard to do complex ABX test are an invitation for experimental errors .

member @JP seems to have found just that error .

lead in did they realy use vinyl the repeatability of you dropping the cart in the track is nonexistent. You must remove tells .

 

[Mnyb]

For someone like Matt who continually disputes the need for blind testing, the breezy lack of criticality of apparently positive results for detecting cable differences, without engaging in the data, doesn't seem odd at all.

What other possible explanations for detecting the difference have you put forth? None?

How about Exhibit 1A: nobody says cables must sound indistinguishable, since they can differ in LCR (hence frequency response) and shielding (hence injected noise).

Do you see published LCR, frequency response, and noise shielding measurements for the cables used in the test, hooked up to the gear used in the test? No??? Then what is even slightly interesting? Nothing!

Which is exactly like I said in my 'breezy dismissal'. Most people who understand the importance of controlled listening tests would know that without having to be told. But, for those who are still waiting for the measurements to 'measure the right things' and hence confirm their personal sighted/uncontrolled listening observations - and I wonder who that could be! - then amateurish test results of surprising detectable differences have to be explained away, one by one, and every explanation disputed point by point, because they want the day to come when their sighted listening observations are validated.

Back to standard programming. Thank you and good night!

That is also a thing the pickup cable application is one where Capacitance is a factor .

so by not checking LCR parameters and fr response before the test . They might end up as captain obvious proving the high C cables alter the response of your cartridge or something else trivial.

But the problem as you say is really the mysticism they approach the subject with that ends up producing meaningless flawed and unecessary experiments .

Beforehand you could easily deduce that there is no real point in making another cable ABX test like this.

There can ofcourse be valid reasons for doing this , if you have a better hypothesis and perspective.
This could have been a test on how audiable the cable load are on certain MM cartridges . With measurements of the electrical differences as a companion . And in cases where no electrical differences are found those are in there to improve statistics.

 

[MattHooper]

Yes, you understand me at least. If someone wants to take part elsewhere then they'll get full consideration. Steve Hoffman's forum is a rigged forum, and I'll have no part of it period.

Now the last part about close mindedness here is a whole different matter and one that is not correct. If regular Hoffman participants get that wrong idea then I really could not care less. The recent thread on digital filter differences is the counterpoint. It has gotten pretty fair treatment and it appears the fellow could hear an improbable thing. I took some part in it. Anything from Hoffman or his forum gets no such treatment at least from me. Steve earned it and then some. If someone takes part and presents stuff there, they may be innocent, but too bad. Speaking only for myself which I hope is obvious.

Ok thanks for explaining your position.

 

[MattHooper]

For someone like Matt who continually disputes the need for blind testing,

You'll never get it. Will you.

the breezy lack of criticality of apparently positive results for detecting cable differences, without engaging in the data, doesn't seem odd at all.

Nobody who has seen the amount of skepticism I've poured on cables, on this forum, could be intellectually honest in implying I'd suddenly become credulous about cables. If I were displaying a "breezy lack of criticality" about the SH results I wouldn't have posted the results here. Rather, I saw that people *apparently* were passing some ABX tests in identifying the turntable cables, and BEING SKEPTICAL of the results is why I posted it here, to be analyzed. My view was "I do not expect the cables to sound different, so my hunch was there is likely an explanation for people detecting differences that doesn't have to do with the actual change in cables. Though since I'm not an electronics expert, perhaps there is a reason why the cables could sound different I'm unaware of." These are the issues I felt the ASR members could cast some light on. Which is why I brought it here instead of just "uncritically accepting the results." .

So the situation is..as usual Newman...precisely the opposite of what you claim about me. (Would it just kill you to ever act in good faith?)

What other possible explanations for detecting the difference have you put forth? None?

How about Exhibit 1A: nobody says cables must sound indistinguishable, since they can differ in LCR (hence frequency response) and shielding (hence injected noise).

Ok. So then it's plausible the cables could sound different in that application. I didn't know whether that's the case.

Do you see published LCR, frequency response, and noise shielding measurements for the cables used in the test, hooked up to the gear used in the test? No??? Then what is even slightly interesting? Nothing!

Ok, so you are positing, or at least implying some reasons, or variables, that could explain the test results. Good. That's the POINT of my asking the questions.

If there was something wrong with the test, I was looking for explanations. That's why I was looking for more than "it's just the steve hoffman forum."

Which is exactly like I said in my 'breezy dismissal'. Most people who understand the importance of controlled listening tests would know that without having to be told.

One can understand that controlled listening tests are important, without having the electronics knowledge to know precisely what needs to be controlled for. This was an opportunity to elucidate what was wrong in these tests, for me and anyone else reading the posts who might not be able to put their finger on exactly what could have explained the test results.

 

[MattHooper]

For samples A and B the lead-in is a dead giveaway, and the ~0.2dB level difference and ~0.0544s alignment error don’t help. Overall speed fluctuation is around 22 samples max which is quite good.

My 20/20 score was done by listening to the first two seconds of the files. Let’s see if the
OP takes another shot with the corrected files I posted.

Fantastic! Just the type of reply I was looking for! Thanks JP!

That would also help explain the results. Cheers.

 

[rdenney]

Here's my curated review of the last 15 or 20 pages for those who are trying to catch up:

I understand your point, but in my case I dont hear differences between the Hidisz dongle and the headphone output of my Marantz receiver.

So it looks to me that I am the opposite of a subjectivist audiophile: they think they hear differences where there aren't, while I don't hear differences where there should be.

If you don't hear a difference, there's nothing you have to prove to anybody. Controlled subjective testing such as blind ABX testing demonstrates that the difference one perceives can be demonstrated with repeatability. If one doesn't perceive a difference, proving that is a waste of time. Everyone has different thresholds of perception, though most are not near as sensitive as they think they are.

Distortion is almost surely fine at -60 db or less especially with music. Noise needs to be lower. SINAD combines noise and distortion. So -60 db SINAD being good enough depends upon how much of that is noise. OTOH, if you get SINAD down to -100 db or lower then both noise and distortion are low enough to be of no concern in nearly all circumstances.

I don't agree with all this. I think basic standards should be for young people of good hearing. We don't need specs for 60+ years old. I also don't believe you can hear -80 db distortions. The tests done on that are with a test tone at the frequencies where we are most sensitive to hearing which show -60 db. You'll not get that with music no matter any training you have. If there are any people who can do it with a test tone it will be a tiny portion of people in the world. I actually doubt there are such people until shown otherwise. I'm speaking of distortion here not SINAD. Noise will need to be -80 or -90 db or more depending upon particulars. Also there is a sweet spot for hearing distortion in humans so no younger people won't hear better because they can play louder. Our ears themselves begin to distort somewhere around 75 or 80 db or louder. Eventually around 110 or 120 db and above the air distorts at a level above minimum air SPL. Distortion unless really horrible isn't much of an issue. Noise can be, but increasingly that is not either.

Now yes for any particular thing you don't believe you can test yourself to prove it to yourself or to other people.

Just a note: Phillips originally planned on digital being 14 bit as after much study they found no reason for more in music distribution. Sony once partnered on the CD introduction insisted on 16 bit partly because 8 bit increments made more sense.

Yes. But we should distinguish between source components at final amplifiers. The signals from final amps are not further amplified, so as long as the distortion products (which only occur when the music is playing) are lower than what we can detect, we've achieve all we can with respect to distortion. For me, that's probably no more demanding than -40 dB for harmonic distortion. Intermodulation distortion is probably a bit more demanding, maybe -50 dB.

Noise happens when the music isn't playing, and that's why it is more noticeable. But for final amps, if my listening environment has an ambient noise level of, say, 38 dB SPL (which is what I have measured on a quiet evening), and I'm listening to music loudly with the average signal around 90 dB SPL (which is LOUD, especially for dynamic music), the -60 dB noise in the electronics will be sitting around 30 dB SPL in the room and will be masked by all those little computer fans, air handlers, wind against the windows, and so on. And if the clothes dryer is running or the Redhead is taking a shower or washing dishes, forget hearing noise from just about any amp.

(And let's not forget that the amp's clean signal is going through speakers that we fantasize might be as good as 1% distortion, -40 dB, in the most hearing-sensitive range; worse in the lowest two octaves.)

But source components will get amplified by as much as 40 dB, depending on the component. Lots of old source components were designed for peak line-level output of 1V RMS or less. A line-stage preamp might increase that by maybe 10-15 dB, and the speaker amp another 25 dB, if both are being played at full capability (meaning: LOUD). My old TEAC open-reel tape deck has a nominal peak output of 400 mV, not nearly the signal put out by a standard CD player at 2V. But even 2V gets amplified significantly on its way to speakers. So, a noise level in a source component of -80 dB may be -40 dB by the time it gets to the speakers, and in loud playing, might be noticeable in the quiet bits. So, we expect more from source components, which is fine because it's a lot easier for source components to be really clean.

There are threads that discuss audibility thresholds. Those who are higher than my numbers above have received special training, probably, and are using special listening techniques to find the distortion artifacts, or they are holding their ear to the tweeter and cranking up the gain to hear noise.

Listening to music is a personal and subjective adventure.

Yes--music is emotional and subjective. Designing electronics is not emotional and subjective, but follows well-known principles and objectives, whatever the rock-star designers or their advertising people may say. And measuring equipment faults is easy and will find effects far smaller than we can hear; those are the effects that if large enough will prevent that subjectively emotional art from reaching our ears.

...
I agree that sighted listening is much less reliable than blind, but I've never found much correlation between expectation and result.

Bias doesn't always work in the way we expect, and that's what makes it impossible to filter out without careful controls. If bias was solely the result of conscious expectation, we could probably overcome that consciously in our listening. But our "heart of hearts" knows things our conscious brains haven't articulated, and often has the opposite expectation that we consciously believe we are bringing to the scenario. You expect your biases to be reasonable (even stating there's "no reason" to have a certain bias), but your heart of hearts eschews reason whether you want it to or not.

in my opinion, and it is just that, science can only go so far. There needs to be room for subjective evaluation as well. Science cannot explain everything.

There's nothing wrong with subjective evaluation. But without controls, it is not instructive for others. Just because you like something doesn't mean I or anyone else will or should like it.

Of course, the notion that we can hear something we can't measure has been rebutted effectively a thousand times in this long thread alone. But it is true that science cannot explain why you or I might like something, though it can certainly measure empirically and statistically whether our preferences fit within some larger population of preferences.

Most of the time, though, this statement is used to justify preferring something over something else when any real science (e.g. controlled subjective testing) would reveal that the person asserting that preference can't actually hear the difference reliably depending only on their ears.

What if it measures well, but you don't like the sound Keith?

Maybe you've trained yourself to prefer certain colorations and distortions. The victim of that training isn't you--you like what you like--but the artist whose work you have altered without their permission and others who believe reviews and advice based on similar training. The artist may not care, of course. But the authoritative transference of one person's preference for colorations and distortions to others causes a drift away from high fidelity as it moves away from ground truth. Even the most innocent preference in opposition to measured fidelity, if not rigorously disclosed as such, eventually leads to ridiculous tweaks, because the principles that make them ridiculous had to be rejected even to justify that first innocent preference.

The thing that always puzzles me about ‘measurements are not everything’ is, if measurements don’t tell you everything, then how the heck do you know which part of your design is doing what?

Do designers just throw everything in a box and hope and pray it ‘sounds’ good? Surely there are electrical principles that need following? How were these electrical principles founded? By listening? Or measuring?

For instance, you always hear some bollocks about Naim amps having PRaT. If you can’t measure PrAT (which they seem to say you can’t), how the hell does the designer know which component or design to utilise to insert said PrAT?

Makes no sense to me

Yes. This is a point I and others have made repeatedly. Either rock-star designers know what measurements they want or their products are happy though unrepeatable accidents. That the rock-star designer listens to each component to select just the right one based on their hearing is both unrealistic and unscalable, but it sure makes their product sound like a craft product instead of a manufactured product. That allows them to add a digit or two to the price. Of course, they became rock stars in the first place on the basis of one ground-breaking product, and their subsequent work identifies clearly enough whether they really knew design or were just lucky. But by that time they are rock stars, and for some that credential alone will be enough to dominate their decisions. High fidelity then moves away from fidelity towards personality cults, and I say that in full recognition of my own high regard for the greats of our industry, even when their best stuff was just a happy accident.

Rick "bored today" Denney