Why you can't trust audio measurements by GoldenSound

[hardisj]

For someone to know what was done correctly or not actually requires substantial knowledge of measurement (protocols), interpretation of results and a complete description of all test parameters and setup.

 

[RandomEar]

I support the general idea of making the test setup and measurement settings/boundary conditions clear and easy to find. I don't get why some members advocate against providing more info - if people can't "digest" the stuff in a PDF report, that's not a problem. They can either learn how to read and interpret it correctly, or they can ignore it. But then, at least people with the required knowledge can look into the details, cross-check results or more easily compare it to their measurements. If you like you can add the AP project files, but these are really only useful for people with AP analyzers and that's not what this little tumoil is about - it's about transparency towards the regular members and readers of this forum. Or maybe more broadly towards "the public".

I would prefer a single post or thread per measuerement type (DAC, amp, HP amp, HP, speaker) with a clearly structured overview of all relevant settings and the test setup. No unnecessary info or explanation. And then link to the respective post/thread in each new review. Documentation done - many questions answered. Easy. Posts like this are a start, but for me, there's just too much clutter in there.

I also support the proposal of @amirm to do one video per measurement type to present that same info - it will certainly be easier to follow for interest amateurs (like me). But I think a video alone is not enough and is also harder to skim through when looking for specific data o settings. Please: Create posts, write down the settings, make it short and clear.

 

[BDWoody]

Just inserting a point

Correct... I have no time for Goldfinger nor his repeated defamatory accusations directed @amirm in the past.

It is rather amusing to see the conflict between a nice new APx555 and certain mindsets... now we get some finality to see this video about not being able to trust measurements, which says it all really.


JSmith

He was banned after a clear display of bad faith in his interactions with the forum and its admins. He isn't someone who is going to be posting here. He has tried posting under alternate names claiming he has been 'shadowbanned,' and since I'm not even sure what that means, I want to make sure our members know this.

Carry on.

 

[MaxBuck]

Why should I trust Darko, Guttemberg and the rest of the gang that is not even aware that Black Metal and Grindcore exist?

All my test tracks sound "noisy", "cranky", "boomy", "nauseatingly distorted", "grinding", "opaque", "abhorrent", "nameless", and all the other Lovecraftian-based audiophobe terms you can imagine.

The problem is that if I retorted to Diana Krall for test samples, I may die, become depressed or fall asleep, so not much of a testing method.

If you fall asleep when Diana Krall is playing, I fear for your immortal soul.

 

[Vacceo]

If you fall asleep when Diana Krall is playing, I fear for your immortal soul.

I find most typical 'audiophile music', boring to no end.

 

[MaxBuck]

I find most typical 'audiophile music', boring to no end.

I don't characterize Krall as "audiophile music." I characterize her as "beautifully rendered vocal/piano jazz."

YMMV.

If you're a sociopath.

 

[Vacceo]

Lullaby music. But I may fall asleep with some old Mayhem records, so...

 

[DonR]

I find most typical 'audiophile music', boring to no end.

You could always play some of her husband's music instead.

 

[Vacceo]

Elvis Costello? Why? If I want something to sound slow, I'd rather go for King Crimson.

 

[SIY]

This is what i mean by proprietary. it only works with one vendors specific software.


But PDF is some what "open" standard that can be oped by anyone.


Sure it would need to be set up in the right way to capture al the relevant information.


This this just an assumptions:
If you review for example a DAC you would first test it in bench mode to see if it behaves as expected.
You would maybe change The analyzer input range the DAC volume and maybe the dBFS output level.
But i assum for the final "screenshot" you keep bandwidth, fft size, window and everything the same.
(unless you have fund a specif reason not to an mention it explicitly)

So my assumption is you can then load some sort of proprietary "automation file" that contains all your normal setting for a DAC test. and it generates a PDF report with all the graphs you want and all the measurements you usually do in Bench Mode?

I’m sure you’ll write a post with measurements showing everyone the correct way to measure and review DACs and enlighten us. You can’t use REW, because it’s “proprietary.” You can’t use ARTA or Virtins because their data files are “proprietary.” I’m baffled by this notion, but clearly you have a better idea than any of the rest of us who do serious measurement. Remove our veils of ignorance.

 

[SIY]

Elvis Costello? Why?

I ask that question frequently.

 

[Katji]

Why should I trust Darko, Guttemberg and the rest of the gang that is not even aware that Black Metal and Grindcore exist?

All my test tracks sound "noisy", "cranky", "boomy", [...]

Maybe they are too busy listening to sound of their new microphone cables. Do they even have time to listen to music, they must be so busy making new videos, and checking their Youtube statistics.

boomy... My room needs some acoustic treatment, I need a UMIK microphone, but I've been thinking that most of the booming is the music / the production - soul/R&B/hip-hop edits with emphasized bass. ...I can see myself asking, when I do get a mic, is this the room or does this music make my speakers look sound fat?

 

[Vacceo]

Maybe they are too busy listening to sound of their new microphone cables. Do they even have time to listen to music, they must be so busy making new videos, and checking their Youtube statistics.

boomy... My room needs some acoustic treatment, I need a UMIK microphone, but I've been thinking that most of the booming is the music / the production - soul/R&B/hip-hop edits with emphasized bass. ...I can see myself asking, when I do get a mic, is this the room or does this music make my speakers look sound fat?

Let's say you play Streetcleaner from Godflesh and it does not sound boomy and saturated. Your gear is garbage.

 

[sq225917]

This thread title really need a comma

 

[Katji]

^^Yes...then it got worse when i found that the intention was something else.

 

[Vacceo]

I ask that question frequently.

I have found your comments about music in line with mine often and for completely different reasons, that you´re making me re-value Heraclitus.

 

[luft262]

I agree with Amir that some of the things GoldenSound is saying, like "two amps with the same SINAD levels can sound very different", is basically wrong and shouldn't be the case under typical circumstances. However, I also like that GoldenSound is telling people to take measurements as just one more factor in a purchase decision that should also include features, aesthetics, etc. and to never completely trust only a single source of information.

I'll link an ASR article below that summarizes what is most likely to be an audible difference in measurements. Chasing the last 1 or 2 dB of performance probably isn't worth it for anyone other than a rich, trained, audio enthusiast.

Audibility thresholds of amp and DAC measurements

Introduction A recurring theme on ASR is whether or not the various measured qualities of the devices are audible. In this post, I'll present some clear and visual thresholds for when those imperfections can be considered a potentially audible concern. I will not explain the basics of amp/DAC...

www.audiosciencereview.com

On a side note, even if some measurements aren't perfect they are still far and away better than subjective, non-reproducible opinions.

 

[Lambda]

. You can’t use REW, because it’s “proprietary.” You can’t use ARTA or Virtins because their data files are “proprietary.” I’m baffled by this notion, but clearly you have a better idea than any of the rest of us who do serious measurement. Remove our veils of ignorance.

You have any idea what Proprietary means? because you said it is not Proprietary.
https://en.wikipedia.org/wiki/Proprietary_software
Sure you can uses Proprietary_software and export to non Proprietary data.

 

[danadam]

He has tried posting under alternate names claiming he has been 'shadowbanned,' and since I'm not even sure what that means, I want to make sure our members know this.

Usually "shadowbanning" means, that the user is banned in such a way, that he is not aware of that. So he can still post, but his posts don't appear to anyone but him.

What I think GoldenOne means, is that the status under his avatar (also after hovering over it) still shows "Active Member", so in this case it is the other members that are not aware he was banned and may assume that he doesn't respond for one reason or other. Frankly, I agree with him on that. If you banned him, then change his status.

 

[amirm]

This thread title really need a comma

Actually, it is fine as is. See this sample of his measurement:

You see how the bottom row is cut off so you can't tell any of the analyzer settings? See how there is no text at all on it to tell you about conditions of the device tested either? These are all the things he complained about yet he didn't see fit to follow his own advice. Here is my dashboard view:

Now you can tell the measurement sample rate (44.1 kHz), that ASIO emulation was used (ASIO4ALL), that measurement bandwidth was from 10 Hz to 22.4 kHz. Also, what volume level was used. What device was tested with which inputs, etc. I have done these things from day one. He attempted to copy me but not realizing all these important steps that I had taken.

As SIY explained, AP has two modes of operation: bench and batch mode. Bench more is an interactive instrument and generates the above result dashboard. There is no pdf there to post. Batch mode generates pdf but it is mostly useless information and presents a more error prone interface due to its automation. With bench mode, as the operator you get to examine the results, tune and optimize setup. No such thing exists in batch mode. Sadly in a few cases the measurement is only available in batch mode so I have to resort to that but for almost all of my measurements, I use bench mode which has no pdf as I mentioned.

I suggest folks go and complain to him about state of his presentation of data, not mine.

So bottom line, this is a clickbait title meant to advance an improper point of view. By creating FUD around measurements, he is hoping people won't take any of them seriously so that he can continue to say stuff sounds all different. To that end, the title is correct.