Why you can't trust audio measurements by GoldenSound

[faheem]

Experienced Engineer vs Excited Hobbyist.

One understands the process to a solution, the other is pressing buttons to get to a result.


See this all the time, in all fields of engineering. Grinds my gears.

 

[sonitus mirus]

Isn't this just more click-bait? I fell for this before, I'm am not even going to watch this video as the title of it and the producer is all I need to know to understand it has little to nothing of value to offer except to stir the pot.

 

[pointprime]

Search this forum on Goldenone and find out..

I searched, but didn't find anything about GoldenSound opting out of a blind test.

Does anyone have a link?

 

[Vini darko]

Sigh. Another youtube video designed to generate forum threads to farm clicks for money.

 

[dc655321]

I searched, but didn't find anything about GoldenSound opting out of a blind test.

Does anyone have a link?

Enjoy.

 

[Ambient384]

That video is just pure hardcore subjectivist cringe. I've got a few mild Subjectivist views like me enjoying the ER4XR despite It being 1% at 1KHz. Isn't this the dude who claims DAC's affect sound then imploded when challenged?.

 

[JSmith]

Isn't this the dude who claims DAC's affect sound then imploded when challenged?.

Correct... I have no time for Goldfinger nor his repeated defamatory accusations directed @amirm in the past.

It is rather amusing to see the conflict between a nice new APx555 and certain mindsets... now we get some finality to see this video about not being able to trust measurements, which says it all really.


JSmith

 

[Blockader]

I think this is the most controversial part in his video.

Does he really think distortion products below -120db make an audible difference?

Or does he think Fourier principles do not apply to human hearing? I think it's the latter. He posted this in his discord server:

Fourier is cancelled

This is the article he posted. I think this is a new wave of subjectivism where Fourier principles are challenged and denied. Interesting.

Honestly I do not care how much difference amplifiers make, to my ears, frequency response is the most important factor in sound quality. And from my experience, especially low order distortion is not very audible unless the signal level is very low. Subjectivists and Objectivists can fight all day long, I do not take sides, I will be enjoying my Neumann speakers meanwhile. However, I am curious about incentives of this new subjectivist movement. (Or if there is any truth to what they claim)

 

[Lambda]

Exactly. That is precisely what I use in the form of templates (so you can't overwrite them by accident). You load up the template and 99% of the settings are the same.

Are they public?
And then why not publish the full test report PDF?

Some of the other issues he is talking about is brought onto himself. He has cooked up his own way of testing products that in some cases are the same as mine, and in others, they are not. So naturally you can't compare those measurements to mine. If he really wanted these problems to not exist, he should not be making the changes.

He should not be making the changes to your way of doing things?!
There is more then one valid way to do things.

for example i never understand why you would only measure DACs at 2/4V this is not how most would use them.
The user usually controls the volume behind the DAC.

Finally, finishing the video with saying two headphone amps with the same SINAD "sound vastly different" is so absurd as to undo everything else he said up to that point. There is really no reason for him to test anything if he can't point to any test that identifies that "vast" difference.

this is bullshit but it isn't undoing everything else he said.
He has a valid points and mixes them with bullshit to please a specific audience.

Only if you are sitting there randomly messing with the dials to measure things.

Well true random has no chance of being influenced from conformation bias...

Doing so will quickly get you in trouble the first time a manufacture challenges your measurements. Then the truth comes out that you were gaming the measurements and you will be done with anyone trusting you.

Or you hand wave it away...
"Normal variances"
So what is Measurement uncertainty and Measurement error at -120dB?

What if some measurement was taken with an output "clipping" or closes to saturating.
Go on and test at 2/4V as always or see this limitation and test at lower voltage.
I remember some controversy about this with some integrated amplifier dac thing.

In other words, there is tremendous pressure to do things right and the same across full range of products tested.

It is ok to make mistakes.
The question is how you react to them.

Perfections can't be achieved but coming extremely close is all you need to ultimately determine one thing: "is the product well engineered?" That is what you want to know, isn't it? That for all the money you spend, someone cared to build a very high performance product.

Sure this is what we want to know.
But SINAD at 1K 4V 44.1khz with 20khz bandwidth is not the only or way to determine this.

Maye some extra care was taken to isolate the USB gnd from the audio gnd.
This can makes way more real world performance difference then 3dB SINAD at 12xdB
Or headroom for inter sample over.

 

[JSmith]

Are they public?

One just has to ask...

Emotiva XPA-DR2 Review (Stereo Power Amplifier)

@amirm is no news, bad news? They asked for my project file which I provided a week ago. I also explained the test. That was that unfortunately. I just shipped the unit back to its owner.

www.audiosciencereview.com

JSmith

 

[theREALdotnet]

There are are measurements that

1. Have enough reproduction info to be successfully reproduced, and are reproducible
2. Have enough reproduction info to be successfully reproduced, and are not reproducible
3. Do not have enough reproduction info to be reproduced, and therefore indicate you're being presented info by someone who is sloppy or dishonest.

All three of those provide useful info.

No, #1 is useful information, #2 is a fluke (measurement error) and #3 is an anecdote.

 

[theREALdotnet]

Exactly. That is precisely what I use in the form of templates (so you can't overwrite them by accident). You load up the template and 99% of the settings are the same. Yes, there is some room for mistake like measuring unbalanced vs balanced (if both connected), etc. But vast majority of what he talks about is eliminated.

The vast majority of operator errors, yes, but not of operator choices.

Some of the other issues he is talking about is brought onto himself. He has cooked up his own way of testing products that in some cases are the same as mine, and in others, they are not. So naturally you can't compare those measurements to mine. If he really wanted these problems to not exist, he should not be making the changes.

So he should be using your settings? How about, both him and you (and everybody else who posts measurements) publish details of the test setup, DUT configuration and analyser settings, so that the results can be replicated, compared and trusted?

Finally, finishing the video with saying two headphone amps with the same SINAD "sound vastly different" is so absurd as to undo everything else he said up to that point. There is really no reason for him to test anything if he can't point to any test that identifies that "vast" difference.

It’s not absurd at all. There is a lot more an amp can do wrong beside noise and harmonic distortion. All measurable, or course, but not reflected in the SINAD score.

 

[Deleted member 46664]

Once again ... if you base yourself on thresholds of hearing ... "Will this be audible"... small differences in measurements (and even a few big ones) do not matter. Amir is a great asset to us all. But, who really cares if Amir says -100db and the others say -105 ... none of it is audible and there you have your answer... no matter your source.

Now if Amir says 10% distortion and the others are saying 1% ... that could be audible... then it's time to look for 3rd or 4th sources and to believe the majority.

There's no need to demonize anyone here ... the variations shown in the video and thus the argument about them are trivial.

 

[Newman]

And then why not publish the full test report PDF?

Amir has responded to this question numerous times. Example. IIRC it will bury the cogent stuff in stuff that makes no difference. The purpose of a test report is to extract meaning from raw test data, efficiently.

 

[theREALdotnet]

Amir has responded to this question numerous times. Example.

That example doesn’t apply. It’s about people asking for additional measurements, not details about the test setup.

 

[JSmith]

But, who really cares if Amir says -100db and the others say -105 ... none of it is audible

It's not about audibility when getting into those figures as you say... it's about exemplary design and engineering at this point. If a designed can't be bothered to address noise and distortion in their products, then I don't want to buy them which is where Amir's data becomes invaluable.


JSmith

 

[DudleyDuoflush]

Seems to me what he is describing is the inherent issues with all forms of test and calibration. You always have all sorts of variation that need to be accounted for and reported. That's why, if you want testing or measurements done to the highest standard, you use an ISO17025 accredited laboratory (or a national lab). You'll then know the processes being used have been validated, the measurement uncertainty has been calculated and reported and comparisons have been made to other laboratories using the same sample or instrument. All this takes a huge amount of time and costs a lot of money and would, in my opinion, be massive overkill for what is essentially a hobby. Without ISO17025 accreditation you need to trust the testing laboratory - I'm very happy with the measurements on ASR.

 

[amirm]

Are they public?
And then why not publish the full test report PDF?

I have fully documented and published the DAC ones: https://www.audiosciencereview.com/...derstanding-digital-audio-measurements.10523/

I have also done one for speakers but keep forgetting to post it.

My plan is to do a video on each category, walking through all the parameters.

I am hugely against dumping a whole confusing PDF auto generated by AP software. Go ahead and download the ones Schiit produces and tell me if you can figure out what is what. Tons of noise is in there that has nothing to do with performance.

 

[amirm]

The vast majority of operator errors, yes, but not of operator choices.

??? What choices? I use the exact same template for each device of that type so operator choice does not enter the equation. Anything that is needed above that is indicated right on the graph. Headphone amp gain for example, or balanced versus unbalanced. All documented on the graphs which is more than I can say for others doing these measurements and just dumping graphs.

Since these graphs get hotlinked on other sites, it is critical that they be self documenting as I do them. A dump of parameters in another place or text in the blog does not work.

 

[theREALdotnet]

I have fully documented and published the DAC ones: https://www.audiosciencereview.com/...derstanding-digital-audio-measurements.10523/

Thanks, bookmarked. It might be worthwhile linking to this at the bottom of DAC reviews.