Master Thread: Are Measurements Everything or Nothing?

[solderdude]

that there is no difference in anything if SINAD has a good enough value

Where is it stated that SINAD is a sound determining/describing metric ?

There is not a single ASR member that thinks that and on the off chance that some do then they don't understand the value of measurements at all.

 

[solderdude]

there is blind testing and then there is blind testing.
That seems to be where this discussion comes down to.
Your vision of blind testing differs radically from blind testing where undeniable proof is required.

 

[Miguelón]

Seriously, you don't think I know what you're talking about, you don't answer my criticism of blind testing, you don't know anything about electronics according to your own words, you are incredibly active but why do you think I should take you seriously? Yes why should anyone take you seriously at all.

Blind tests are the only valid and your criticism is no based in logic. Many variables will invalidate any single test, weather blinded or not.

If you perform any test, with for example 95% of confidence against null hypothesis for each variable measured and you use 10 variables then just for probability you will obtain biased results in at least one of them or more

This is called the “too much variables bias” and is important no mater the topic,

I was mathematician and actually I’m physical therapist: I conducted research in both areas. I answered to the need to show your type of test and describe it (it was requested to you in other occasions), you can search on the historical.

Being an amateur in audio don’t disqualify the basis of knowledge I have, and of course don’t disqualify the basis on physics I know very well.

Your electronic experience don’t have more validity that any other unless you prove what you’re stating.

In my case I agree with results I researched on signal conversion and transmission that were published, and think is only psychological influence the mater of cables or DAC, as was demonstrated to exhaustion.

If you disagree with science it belongs to you to defend your position and to be taken seriously by transparent and solid arguments not based in your personal experience or sensorial beliefs

 

[Miguelón]

I think you are right, I experience systems where I can clearly hear differences and systems where I can't, I hear differences with some music and no difference with other music etc.

You don’t understand what he’s saying: nothing to do with your experiences results but with your methodology. Undeniable proof means consistent, reproducible and unbiased

 

[BDWoody]

And if the test is to be fair, it must be the best hifi system available and if that's the case, I can promise you that everyone can hear the difference between e.g. cables.

We don't need promises, we need evidence. Of that, you have none.

That's enough in this thread.

 

[Purité Audio]

Unsighted comparisons are used purely to determine whether there is an audible difference between two components.
The two stipulations are that the subject listening must not know which of the. two components is in circuit and that if necessary the level must be matched to .1 dB.
Thats it.
Keith

 

[Miguelón]

Interestingly you are a doctor so you must know that the blind test is tied to one variable and one constant, how many variables do you think there are in hifi, I can help you there are many and that makes the blind test pretty much useless with one exception if it is positive.
But the vast majority experience the opposite, because the system is not good enough, because the acoustics are bad, because the music selection is wrong, etc. And therefore the test is misleading and outright wrong.

I answered to that before.

The more variables measured on a test, the less trustable will be the result.

As an example: if you conduct a study to measure differences on listening two DACs and you measured 10 different aspects in you can perceive a difference, by probability one or more can be positive even if the DACs are exactly the same.

This is called the “so many variables error”

To perform an adequate test you should measure if possible one single outcome: “can I perceive a difference between them?” will be a possible one.

Blind the DACs correctly (no voltage differences in output below a defined threshold, occult all possible ways to identify them other than the value you test, even more you should take a third person or a random switch to change and memorize the changes of device in order to not influence your subjects with language or non verbal communication), choose an enough number of listeners (in a binary “yes or no” variable this can be as low as 6 to 95% confidence interval, but don’t trust me: better consult an expert).

And other aspects I don’t know about concrete acoustic tests: these are basic to all blind randomized trials

 

[Miguelón]

Is there a standard for blind testing that includes stereo system, music and acoustics? After following Sean Oliver's tests for several years, I don't get that impression.
As long as there is no standard, blind testing cannot be used.
And if the test is to be fair, it must be the best hifi system available and if that's the case, I can promise you that everyone can hear the difference between e.g. cables.

Don’t need a standard to blind test: blind your cables an just ask if the subject can identify A from B, or take a third cable as a control. If answers are consistent then you can hear the cables or perhaps any interference on the setup exists.

Then publish all your setup and let others repeat and criticize the test and wait for confirmation. A single test has little validity, except on unusual high quality and quantity ones

 

[Jim Taylor]

Is there a standard for blind testing that includes stereo, music and acoustics?

No standard includes stereo. This was explained to you. You seem to think that "test" is the same as "comparison". A controlled test is much more rigorous.

As long as there is no standard, blind testing cannot be used.

There are controls and protocols, as was discussed here. Using the word "standard" as having exceptional meaning is just semantics. It's a ruse used by trolls.

And if the test is to be fair ...

"Fair" is not a word that applies to scientific methodology. It's an arbitrary and subjective evaluation based on emotion. Tests are not "fair"; beauty pageants and elections are "fair". Tests, in audio, radiology or chemistry, are "controlled". If they are not, they are worthless. Remember the cold fusion debacle?

it must be the best hifi system available and if that's the case, I can promise you that everyone can hear the difference between e.g. cables.

The quality of the system means nothing. The capability of the system may mean something; it depends on what (exactly) you are trying to isolate. And the operative word here, by the way, is "isolate". That is the purpose of a test, whether a drug test, an accuracy test in artillery, a test for annealing or a test in audiology.
A test is useful is if isolates one variable at a time. If it tries to isolate two or more variables at one time, it goes by another term:

MESS.

Jim

 

[Newman]

A new approach. We know who the returning previously banned member is and you have read about us giving him a seventh chance. To sink or swim, climb or fall. Do we let him stay and is there any point. Is he here to constructively exchange ideas and concepts or is he here on some other strange and disruptive purpose?

Type-1 For he has earned extra time.

Type-2 For his time is up.

This way the outcome is determined by his peers and not the guy on Team ASR.

Look. If someone has been banned, if they have an ounce of good faith, they need to come back via an open conversation with mods about what they need to be doing differently, and agree to do that, then demonstrate in their posts that they are doing that. In fact, the fact they didn’t have that conversation before their initial ban as they were getting into strife with their posts, is not a good sign at all.

If they just come waltzing back in with a fresh pseudonym and doing *exactly* what they were doing before, especially attitude-wise, then they are automatically a bad-faith actor and deserve the whack-a-mole. If they do it time after time after time, it’s beyond a joke and I feel for the mods and this person can’t be treated like they might have been misunderstood. Everything is perfectly clear. Just do it.

Please don’t turn the membership into a jury. What a mess that would be. You guys just need to be clear in your own minds, and act on that clarity. And I thank you for that.

cheers

 

[solderdude]

Is there a standard for blind testing that includes stereo system, music and acoustics? After following Sean Oliver's tests for several years, I don't get that impression.
As long as there is no standard, blind testing cannot be used.
And if the test is to be fair, it must be the best hifi system available and if that's the case, I can promise you that everyone can hear the difference between e.g. cables.

You can blind test a lot of things easiest (1) to hardest (7) to do properly.

1. digital files (super easy) no help needed
2. interlink cables. (not very difficult) help needed or special ABX box.
3. speaker cables. (more difficult) help needed or special ABX box
4. amplifiers (difficult but possible, requires accurate metering) help needed or special ABX box
5. DACs (difficult but possible) help needed or special ABX box
6. speakers (very hard to do) help needed
7. headphones (nearly impossible) forget about this.

In all cases special care must be taken to avoid tells, these can even be ones that seem harmless.

There is no standard blind testing (aside from digital files).
Level matching for both channels is essential within 0.1dB
Statistical relevant attempts needed (15 to 20 attempts minimum) can be spread over a few days as AB testing is really exhausting.

 

[Mulder]

Correct blind testing can for sure be valuable if you want to objectively determine preferences of one kind or another, but for the individual HiFi enthusiast the value, in my experience, lies more on the educational level. Many years ago I read that uncompressed wave always sounds better than FLAC. (It was the owner of a prestigious DAC manufacturer who claimed this) So I compared, and oh so much better uncompressed wave sounded. No doubt about it. Fuller, warmer.. all those qualities were in place that are usually emphasized in the mainstream HiFi press. But later I read about expectation bias. So I thought that maybe after all I ought to peform a blind test. Said and done. I conducted a blind test. Guess what. I could not register a single difference. It was pure guesswork. The differences I heard so clearly before were now gone. I immediately re-encoded all the uncompressed wave files to FLAC and after that have never trusted differences in what I think I hear. Conclusion! Correctly performed measurements are king.

 

[bodhi]

Correct blind testing can for sure be valuable if you want to objectively determine preferences of one kind or another, but for the individual HiFi enthusiast the value, in my experience, lies more on the educational level.

Personal preference testing is completely different thing. The issue here is audibility.

The floor for evidence is low. We don't need to know if something is audible for majority of listeners. We want to know if any human can perceive the difference. If that is proven , just for a single genetic oddity, then the discussion would completely change.

 

[Mulder]

Personal preference testing is completely different thing. The issue here is audibility.

The floor for evidence is low. We don't need to know if something is audible for majority of listeners. We want to know if any human can perceive the difference. If that is proven , just for a single genetic oddity, then the discussion would completely change.

Audibility or preferences. Sure. I could have chosen my words better, but my point was about something completly different. For the common HiFi-enthusiast it will hardly be about prove anything.

 

[AdamG]

Look. If someone has been banned, if they have an ounce of good faith, they need to come back via an open conversation with mods about what they need to be doing differently, and agree to do that, then demonstrate in their posts that they are doing that. In fact, the fact they didn’t have that conversation before their initial ban as they were getting into strife with their posts, is not a good sign at all.

If they just come waltzing back in with a fresh pseudonym and doing *exactly* what they were doing before, especially attitude-wise, then they are automatically a bad-faith actor and deserve the whack-a-mole. If they do it time after time after time, it’s beyond a joke and I feel for the mods and this person can’t be treated like they might have been misunderstood. Everything is perfectly clear. Just do it.

Please don’t turn the membership into a jury. What a mess that would be. You guys just need to be clear in your own minds, and act on that clarity. And I thank you for that.

cheers

It was an experiment and maybe not my best moment.

We have spam closed and deleted all his posts. Until his next reincarnation. Enjoy your Weekend and remember that we are humans too and sometimes we try something different to see what happens. Otherwise we get stuck in a locked in mindset and that’s not good either. Trying to do this job with compassion as I constantly ask from the members. Thank you for your input.

 

[Jim Taylor]

It was an experiment and maybe not my best moment.

We have spam closed and deleted all his posts. Until his next reincarnation. Enjoy your Weekend and remember that we are humans too and sometimes we try something different to see what happens. Otherwise we get stuck in a locked in mindset and that’s not good either. Trying to do this job with compassion as I constantly ask from the members. Thank you for your input.

DON WURE ... U B OK.

Jim

 

[Miguelón]

I’m gonna miss Knud82: his or her assessments were so stimulating. Somehow like George Orwell’s Animal Farm: “All cables are equal but some cables are more equal than others”

 

[Jim Taylor]

I’m gonna miss Knud82: his or her assessments were so stimulating.

So is an enema. I considered the results to be the same.

Jim

 

[Miguelón]

So is an enema. I considered the results to be the same.

Jim

I can’t agree: contrary to Knud’s claims, the results of an enema are well proven.

 

[ahofer]

He’s a return sock-puppet? I’m not shocked. My comments weren’t ill-mannered, they were prophetic!