Master Thread: Are measurements Everything or Nothing?

[Miska]

So one or more of these formats resulted in the graph you posted, and you won't tell us which one? And all anyone has to do is run a simple test and one of the formats will act strange like you showed?

A guess before doing any testing would be it is one of the DSD formats.

If you go back, you can see, I posted graphs for 44.1k alt setting 1, 352.8k alt setting 1 and DSD256 alt setting 3 (PCM rate 352800). Mentioned in the graph headers.

Of those, 44.1k PCM was worst performer, 352.8k PCM was a bit better but had picketed noise floor, and DSD256 was cleanest but still abnormal noise floor modulation.

In addition I think I posted graph showing how 19+20 IMD test tone at 705.6k PCM trips up the output.

PCM input is 24-bit TPDF dithered data.

 

[Sgt. Ear Ache]

Oh yeah, I can imagine I could easily spend few million euros on subjective listening tests and still would be faced with the same skepticism about the results. At least by one or two persons, who would demand spending a few million more. And then that wouldn't be enough either. So why bother?

Yeah, you bet. If all we have is you saying "I spent millions of dollars conducting subjective tests and they proved noise at -120dbs is audible" then you'd definitely still face skepticism. Just like if somebody on the internet said "I spent millions of dollars conducting tests and have now proven humans can levitate." Saying stuff on the internet proves nothing. Now of course, it's not reasonable to expect individuals to conduct large scale public testing to prove this stuff...the reality is there isn't much you could do privately that would really prove anything. However, it is reasonable to ask the question "why hasn't any large scale testing been done that does prove any of this?" I mean wouldn't it be in the interests of some of the big manufacturers of expensive audio gear to actually prove the benefit of their products over other less-expensive gear (if they could)?

 

[Miska]

Be careful of accusing others of what you are yourself demonstrating.

Rick “write a paper for the AES, then” Denney

I'm not AES member. And I don't see a reason why I would bother with that?

 

[Miska]

However, it is reasonable to ask the question "why hasn't any large scale testing been done that does prove any of this?"

Because it is expensive. And especially difficult because you need to make sure that the test subject is not aware of the test. Usually someone being aware that something is being tested already spoils the results because they don't behave in normal way anymore.

 

[Sgt. Ear Ache]

Because it is expensive. And especially difficult because you need to make sure that the test subject is not aware of the test. Usually someone being aware that something is being tested already spoils the results because they don't behave in normal way anymore.

lol...sorry, but that's nonsense.

 

[Miska]

Since you claim to have done the tests, it wouldn't cost you anything to provide more details about them and how you got the results you did, whatever they were.

The graphs and charts are nice, but if you are claiming something is audible to multiple people that is in the neighborhood of -120dBfs, without gain riding, using real music, it's going to need more than just 'because I say so.'

Unless you realize your tests won't stand up to scrutiny, I'm not sure where the reluctance comes from.

OK, let's put it this way. How much you see ESS, AKM or TI documenting and publishing stuff about their subjective audio testing?

Or for example my ex employer Nokia?

None.

So probably I shouldn't say anything either. I'm very sorry about saying anything on the subject. I was refusing to state anything about it for a while, but people insist about "do you think this is audible". And I say yes, I say that -120 dB stuff is audible based on my experience. And I'm not going to open up that any further. Period.

 

[Miska]

lol...sorry, but that's nonsense.

Based on what? On what scientific evidence it is nonsense? Please provide more data and evidence about testing psychology.

Already at tests at school I get so freaking nervous that I forget half of the things. And failed my first driving license test for the same reason.

 

[BDWoody]

And I say yes, I say that -120 dB stuff is audible based on my experience. And I'm not going to open up that any further. Period.

Then...

Uh huh.

 

[pkane]

Asking for more and more evidence costs nothing but beating mouth or keyboard. Providing more and more evidence is much more expensive.

Sure it is like typical war tactics where you make enemy consume all it's resources.

I get it. Let me summarize:

1. Everyone here has an opinion that can't be changed... except you're the only one expressing opinions about others, and making assumptions as to their motives and inability to have an intelligent discussion
2. Everyone here is an enemy, because they don't automatically believe you when you make unsubstantiated claims and they dare to ask questions
3. Typing a description of how you conducted a test is too difficult and expensive because you'd rather argue about nonsense and deflect questions

Did I get this about right?

 

[SIY]

If you go back, you can see, I posted graphs for 44.1k alt setting 1, 352.8k alt setting 1 and DSD256 alt setting 3 (PCM rate 352800). Mentioned in the graph headers.
View attachment 181245

That graph right there shows a problem with the test setup. That's exactly what you see when a scan is interrupted, a connection is intermittent, or an amplitude changes during acquisition. It could also be something in the software chain. Regardless, you need to get your measurements right.

 

[SIY]

OK, let's put it this way. How much you see ESS, AKM or TI documenting and publishing stuff about their subjective audio testing?

If by "subjective" you actually mean "uncontrolled," I hope it's zero, since their stocks are doubtless part of my 401k portfolio, and investing in companies that practice voodoo is a bad idea.

 

[Miska]

1. Everyone here has an opinion that can't be changed... except you're the only one expressing opinions about others, and making assumptions as to their motives and inability to have an intelligent discussion

No, every time I say something here I get shit load of posts making assumptions about my ability and motives.

2. Everyone here is an enemy, because they don't automatically believe you when you make unsubstantiated claims and they dare to ask questions

Yes, every time I say something here, or even just post measurement results without making any further statements about those, I get under hostile attack wave.

3. Typing a description of how you conducted a test is too difficult and expensive because you'd rather argue about nonsense and deflect questions

Yes, at most I will say that based on my experience -120 dB things are audible. I already said it is totally blind AB test. More details I won't divulge.

Did I get this about right?

Not quite.

 

[Geert]

I was refusing to state anything about it for a while, but people insist about "do you think this is audible". And I say yes, I say that -120 dB stuff is audible based on my experience.

If you have evidence for a groundbreaking observation, than why keep it for yourself? Are you running a business and don't want your competitors to know? Unfortunately Chord Electronics is already working on the -300dB level (without evidence of course ).

 

[Miska]

That graph right there shows a problem with the test setup. That's exactly what you see when a scan is interrupted, a connection is intermittent, or an amplitude changes during acquisition. It could also be something in the software chain. Regardless, you need to get your measurements right.

Ahh, always good excuse when the results don't please you. Somehow, on the exact same setup, tens of other DACs don't have these issues.

 

[Miska]

If by "subjective" you actually mean "uncontrolled," I hope it's zero, since their stocks are doubtless part of my 401k portfolio, and investing in companies that practice voodoo is a bad idea.

I guess you didn't watch the ESS CTO's presentation?

 

[Sgt. Ear Ache]

Yes, at most I will say that based on my experience -120 dB things are audible. I already said it is totally blind AB test. More details I won't divulge.

Then it is completely and utterly meaningless and proves absolutely nothing.

 

[Sgt. Ear Ache]

Based on what? On what scientific evidence it is nonsense? Please provide more data and evidence about testing psychology.

Already at tests at school I get so freaking nervous that I forget half of the things. And failed my first driving license test for the same reason.

hahaha, oh my. Yes, being asked "do you hear anything" is totally a stressful event like taking an exam or driving test. Although I suppose if a person has some sort of weird investment in hearing something that they know they actually can't hear and they've spent years cultivating the belief that they can do so even though they know they can't, and now they know that belief is about to be blown up it might be stressful.

 

[SIY]

I guess you didn't watch the ESS CTO's presentation?

Not interested in marketing. Or sitting through long videos.

 

[voodooless]

More details I won't divulge..

That’s the whole problem.

 

[SIY]

Ahh, always good excuse when the results don't please you. Somehow, on the exact same setup, tens of other DACs don't have these issues.

Because you fucked up that particular measurement and won't go figure out how you managed that and repeat it properly.