• WANTED: Happy members who like to discuss audio and other topics related to our interest. Desire to learn and share knowledge of science required. There are many reviews of audio hardware and expert members to help answer your questions. Click here to have your audio equipment measured for free!

Audio foolery 2.0 - The rise and fall of objectivism

liu

Active Member
Joined
Aug 24, 2020
Messages
142
Likes
345
The top ranking Topping L30 Headphone Amp fried a few users headphones leaving them permanently damaged. That is a huge flaw in my book and it underscores that SINAD performance is but one metric of engineering. It was an eye opening moment for me and I own Topping gear.
Well, at least Topping recalled their L30 and sent every user a fixed device so they already did much better.

If you stay in this forum long enough, you would know Topping DX3 Pro wined the highest recommendation from amirm (which I believe is super double standard, especially in its linearity test) at that time. There was a design issue which would make the circuit become unstable and fail. A lot of user complained about their device burned out their headphones and so on, and more users reported they have a dead unit after a few weeks of usage. Topping never apologized for this, and never did a recall like they did for L30 this time. A lot of users requested amir to put a warning in his review, but he refused to do so.

Even without such disaster, Topping DX3 Pro can never be regarded as a good product --- it has relay clicks whenever you pause your music or switch tracks, and the firmware is very unreliable in that it would go silent in the middle of a track.

I previously owned the O2+ODAC combo, which works very, very reliably. I sold my O2+ODAC and bought Topping DX3 Pro when I saw the good Topping DX3 Pro reviews on this site and--- It just beat my old combo in every way! Turned out that is the worst decision I've made. Had this site not existed, I'd still enjoy my O2+ODAC combo today.
 
  • Like
Reactions: nhs
OP
nimar

nimar

Active Member
Forum Donor
Joined
Jan 25, 2021
Messages
213
Likes
216
Location
Ontario, Canada
Had this site not existed, I'd still enjoy my O2+ODAC combo today.

The takeaway for me is not a problem with Topping (would be nice if this thread could avoid becoming the Topping bashing thread. Many companies make mistakes), but how could the reviews, or how we as users think about reviews be improved to avoid this type of situation.

What facts / metrics could be included to help provide a more holistic perspective that would help users make a better informed decision.
 

tmtomh

Major Contributor
Forum Donor
Joined
Aug 14, 2018
Messages
2,636
Likes
7,487
The takeaway for me is not a problem with Topping (would be nice if this thread could avoid becoming the Topping bashing thread. Many companies make mistakes), but how could the reviews, or how we as users think about reviews be improved to avoid this type of situation.

What facts / metrics could be included to help provide a more holistic perspective that would help users make a better informed decision.

Despite my irritation at the thread subject title, I do think your point has merit: one can certainly argue that a truly complete review would cover every major usability issue in addition to pure measured performance, since usability becomes more important than measurements once measurements reach a certain threshold of high quality.

But that said, I think some important caveats are in order:
  • First, @amirm already notes many build-quality and usability aspects in most of his reviews. If you are concerned about brightness or dimness of LEDs, readability of displays, tactile feel of volume and input-selection knobs, ease of understanding user interfaces and accessing settings, heat generated by a unit, whether a small unit is likely to be moved around by the weight of connected cables - Amir covers all of that already - and more.
  • Second, plenty of "usability issues" noted in this this thread are not actually usability issues - they are personal preferences and there is nothing approaching an objective or consensus view about them. Amir can't possibly anticipate all the individual preferences or requests folks might have about feature details, and it's silly (not to mention ungrateful) for anyone to characterize a lack of info about those types of things as a deficiency in his reviews (let alone "the fall of objectivism").
  • Third, there are some things that no reviewer - whether it's Amir or the most subjectivist reviewer at The Absolute Sound or anyone in between - can reasonably be expected to find out in the brief review period. Unit-to-unit variation; obscure electrical problems that result in recalls (or should result in recalls) months after the unit is designed and 100s or 1000s of samples are out in the wild; and so on. Again, that's not a problem with objectivism or with the reviews here.
One way I can think of that would help with this situation, and would be in keeping with the ethos of this place, would be for you or someone else concerned about this to compile a list of key usability questions that you would like to see covered in reviews of various kinds of equipment. Amir could then cover those if he wanted to and felt it was feasible - or, alternatively, you and/or others could chime in with a comment shortly after each review is published, with info about those aspects, just as there are folks who post preference scores for tested speakers and who regularly provide more info in the review thread comments.
 

liu

Active Member
Joined
Aug 24, 2020
Messages
142
Likes
345
The takeaway for me is not a problem with Topping (would be nice if this thread could avoid becoming the Topping bashing thread. Many companies make mistakes), but how could the reviews, or how we as users think about reviews be improved to avoid this type of situation.

No I'm not bashing Topping at all. Their competitors have the same problem as well (a recent one that I encountered last week).

The issue here is once standard test is given,
companies would just produce products that optimized for the test only,
and ignore everything else.
The above SMSL problem I listed is just because Amir never tests single channel performance at all.

Chinese companies are especially good at this.
I was raised in China before teenage so I know how test driven the country is.
Chinese people do super super well in all kinds of standard tests and it has been known for a long time.
So it's not rare to see Chinese companies occupy most of the top positions in SINAD chart.
While fail on essentially everything else.
 
Last edited:

RayDunzl

Grand Contributor
Central Scrutinizer
Joined
Mar 9, 2016
Messages
13,198
Likes
16,981
Location
Riverview FL
I figure the tests here are roughly equivalent to a dyno run for a car/bike.

Raw performance.

Not much more, nor much less.
 
OP
nimar

nimar

Active Member
Forum Donor
Joined
Jan 25, 2021
Messages
213
Likes
216
Location
Ontario, Canada
would be for you or someone else concerned about this to compile a list of key usability questions

My thinking as well. I proposed a couple to get things started, thought others might also have some ideas https://www.audiosciencereview.com/...se-and-fall-of-objectivism.22794/#post-759189

I agree with all three points, certain things (like having a good remote) bleed into subjective qualities, but the remote only working within a 1-2 degree range is just bad design. The reviews do contain a smattering of comments on the appearance of build quality, the brightness of the screen etc. Some even comment on the poor UX, as in the crazy interface of the Khadas Tone board.

So what simple measurable features can be used as proxy for quality / UX? In a perfect world price has some correlation but we know that is not at all the case in HiFi. (bad car analogy, you expect your BWM to go fast and have nice seats / a good entertainment system).

Raw performance.

Agreed, though I'd argue we can expand what we quantify as raw performance? And even be so bold as to develop a balanced score that took into account these additional metrics to get a really meaningful number.
 

RayDunzl

Grand Contributor
Central Scrutinizer
Joined
Mar 9, 2016
Messages
13,198
Likes
16,981
Location
Riverview FL
I'd argue we can (do stuff)

"We" who?

"Work consists of whatever a body is obliged to do. Play consists of whatever a body is not obliged to do."
- Samuel Clemens, The Adventures of Tom Sawyer
 
Last edited:
OP
nimar

nimar

Active Member
Forum Donor
Joined
Jan 25, 2021
Messages
213
Likes
216
Location
Ontario, Canada
Raw performance.
"We" who?

"Work consists of whatever a body is obliged to do. Play consists of whatever a body is not obliged to do."
- Samuel Clemens, The Adventures of Tom Sawyer

The royal we, the plural we? Anyone that wants to take part in the thought process.

I'd assume there are two plausible outcomes from this process or some mix of the two.

A. Come up with X number of new metrics which are considered to provide high value to help determine quality of a product / brand. And kindly ask reviewers (here / elsewhere) to consider including them.

B. Interested parties dig up these metrics and append them in comments etc to reviews.

C. (Guess I can't count). B. happens but is then added to the official review. The point is that the "official review" carries a lot more weight than any number of comments.

As Amir et al have noted, the reviews done here have the power to push manufactures to improve their products. Now that even very cheap products are audibly transparent it seems it might be time to push on other fronts.
 

tmtomh

Major Contributor
Forum Donor
Joined
Aug 14, 2018
Messages
2,636
Likes
7,487
I agree with all three points, certain things (like having a good remote) bleed into subjective qualities, but the remote only working within a 1-2 degree range is just bad design.

I agree - a frustrating remote with a narrow "beam" range of functionality is an issue. I am not sure, but I think the absence of that kind of info might be partly due to the way this site has grown over time. If I'm not mistaken, the earliest reviews were focused mostly on a desktop/near-field listening scenario. At the very least, component-sized amplifiers (as far as I know - maybe I'm wrong) and far-field speakers were not among the dominant types of equipment reviewed initially. So I don't know that there is a tradition of, for example, Amir sitting 8-10 feet from reviewed equipment and trying out the remote. He certainly is not shy about commenting on aesthetic and UX issues that bother (or please) him, which I find very helpful in the reviews. So it might just be that his use-case during review does not necessarily emphasize things like remote angle range, which might perhaps become clear only with more extended far-field use.

Just speculating, but again I think it speaks to the importance, at least IMHO, of maintaining the semi-DIY nature of this place and chipping in with crowdsourced knowledge.
 

Vict0r

Addicted to Fun and Learning
Joined
Nov 6, 2020
Messages
647
Likes
1,580
Location
The Netherlands
I think the solution to this is rather simple; never buy anything after reading just one single review. If you'd have combined Amir's technical measurements with some other reviews or impressions, or asked around in the forums about the usability of the product, I think you could have avoided the disappointments. :)
 
OP
nimar

nimar

Active Member
Forum Donor
Joined
Jan 25, 2021
Messages
213
Likes
216
Location
Ontario, Canada
I think the solution to this is rather simple; never buy anything after reading just one single review. If you'd have combined Amir's technical measurements with some other reviews or impressions, or asked around in the forums about the usability of the product, I think you could have avoided the disappointments. :)

Sounds a little too simple.

Would like to avoid talking about a specific product but where are these honest critiques of the Topping D90/A90's short comings? Every single review, from across the internet gushes over these products, rather like the big echo chamber many complain the internet has become. My favourite line used more often than not, that I've begun to steer well clear of is "amazing value for money". Having a complete unit, A/D90 cost 25% more than the RME I replaced it with. The only place I found anyone say a single bad thing about Topping was super hifi friends or whatever its called and it was overtly clear that they had an axe to grind with ASR so kept clear of it. Granted there are the odd user post with people having specific issues, ground faults, bad power supplies etc (with products in general). The same is true for almost any company / product. Look at the Apple forums and see the number of people with issues. Generally those issues are a fraction of a % of happy customers and not systematic flaws in the design themselves. Before someone is tempted to go down a rabbit hole of some Apple product with a flaw, it is just an example and going there is a distraction.

Not to mention user comments are of varying quality, mostly great and others somewhat less so. There are about half a dozen posts in different threads with people stating that they wouldn't consider the RME DAC unless it upgraded to the AKM 4499 because Topping is using the AKM 4499.
 
Last edited:

charleski

Major Contributor
Joined
Dec 15, 2019
Messages
1,098
Likes
2,239
Location
Manchester UK
What happened? How is it possible that the highest regarded products on ASR could be so disappointing. Let me be absolutely clear, they don’t sound disappointing, they sound great, but so does my old DAC. They don’t sound particularly better than it.
You've just discovered the Great Truth of Digital Audio: almost all this stuff sounds the same. Maybe this should be pinned to the top of the reviews forum. Digital audio became a Solved Problem back in the '90s. All that's happened since then is that engineers have found ways to get the price down.

The whole point of testing this gear is that some of it doesn't sound the same, it sounds worse because it's badly designed. Here's an example from yesterday of a $2k amp whose DAC can't even handle 16bits properly. The point of looking at measurement reviews is to weed out the garbage, then choose from what's left based on other criteria, like price, features, support, aesthetics, etc. As Amir himself says, "I don’t suggest readers look too closely at the SINAD numbers. Instead, go by what “bucket” they fall in based on color-coding." Most people won't be able to tell the difference between DACs in the green and blue buckets. On a good day, some people with excellent hearing who've trained themselves to detect subtle distortion might be able to tell the difference between the right-hand of the green bucket and the left-hand of the blue, but I wouldn't lose sleep over it. DACs in the orange band will probably be OK for uncritical purposes, but anything in the red is just junk. Obviously, the boundaries here are fuzzy, there aren't any magic thresholds.

So yeah, anyone who thinks the Objective approach is to choose the DAC with the best numbers is seriously failing to understand what being an Objective Audiophile actually entails. it isn't just a case of More = Better.
 

Tim Link

Addicted to Fun and Learning
Forum Donor
Joined
Apr 10, 2020
Messages
722
Likes
638
Location
Eugene, OR
I've been thinking about upgrading my electronics for years now, looking for something with stability, durability, ease of use and the desired functionality without breaking the bank. I like to see the raw specs so I love this site but I also look for user reviews about negative issues and keep finding that stability issues are still common with digital equipment almost 20 years newer than mine. Amir does comment on usability and has given borderline measuring products the thumbs up because of excellent feature sets. He can't really test for durability and stability because that would take more time than he has. I'd definitely trade some SINAD for a strong feature set.

What I want is an all in one integrated 6 channel amp with powerful active crossover, equalization, time alignment DSP built in so I can run my 3 way active speakers from a single box. It could be separate boxes for the amps and DACs but I want them to coordinate with the HDMI as if they were a single unit. I'd rather not have to individually turn them on and off. It needs to have HDMI inputs and HDMI out to the TV, and support all that cool power synching stuff so if I turn the Apple TV or Nintendo or DVD player on or off or hook up my computer, everything else coordinates automatically. It occurred to me recently that everything I listen to now is using HDMI output.
 
Last edited:
OP
nimar

nimar

Active Member
Forum Donor
Joined
Jan 25, 2021
Messages
213
Likes
216
Location
Ontario, Canada
Maybe this should be pinned to the top of the reviews forum

I was also thinking something along those lines. Before entering ASR you should read "The Truth about HiFi / Transparent electronics". And in self defence, I was generally aware that the electronics was more or less a solved problem. I didn't buy a Topping D90 entirely based on any single number. The logic was slightly more complicated. Read "->" as "Should equal".

Good engineering -> Good design
Flagship product -> Better design / better implementation
Rave reviews -> Good product
"Amazing value for money" -> Value

I've since learned that "Amazing value for money" is one of the biggest conn's in Hifi, you can say it about any product about any price and it means just as little. If the item is $5 or less, that's amazing value for money. $1100 CAD for a stand alone DAC with bad UX is less good value.

ps. Thanks for the link to Amir's post, don't recall if I read it already. You are right that people are encouraged to look beyond the SINAD, maybe it just needs to be done a little louder as there is still very much (in parts, not everyone) an air or culture of better number == better product.
 

devopsprodude

Senior Member
Joined
Mar 9, 2021
Messages
332
Likes
324
Location
Beaverton, OR
Engineering and Design/UX are COMPLETELY separate fields and skill sets. That a company can do well in one doesn't mean they will do well in the other.

You had specific requirements that products needed to meet before purchasing anything. @amirm reviews aren't about product features, they are largely only looking at whether gear is engineered competently and that they do what they claim to do. So you needed to look elsewhere or ask questions in the owner's forums to see if these things had those features.
 
OP
nimar

nimar

Active Member
Forum Donor
Joined
Jan 25, 2021
Messages
213
Likes
216
Location
Ontario, Canada
Engineering and Design/UX are COMPLETELY separate fields and skill sets. That a company can do well in one doesn't mean they will do well in the other.

You had specific requirements that products needed to meet before purchasing anything. @amirm reviews aren't about product features, they are largely only looking at whether gear is engineered competently and that they do what they claim to do. So you needed to look elsewhere or ask questions in the owner's forums to see if these things had those features.

I agree they are separate, yet they are very much related.

Really I didn't have any such requirements before purchasing. I naively assumed that in the last nine years since my original DAC was built, basic features were a given because they were solved problem. It was only after purchasing that I discovered that a flagship product from a respected company had regressed in every domain apart from transparent sound reproduction.

And I am not attempting to single out @amirm or his reviews. He's doing a stellar job, but rather have a conversation as to how things could be improved.
 

devopsprodude

Senior Member
Joined
Mar 9, 2021
Messages
332
Likes
324
Location
Beaverton, OR
I agree they are separate, yet they are very much related.

Really I didn't have any such requirements before purchasing. I naively assumed that in the last nine years since my original DAC was built, basic features were a given because they were solved problem. It was only after purchasing that I discovered that a flagship product from a respected company had regressed in every domain apart from transparent sound reproduction.

And I am not attempting to single out @amirm or his reviews. He's doing a stellar job, but rather have a conversation as to how things could be improved.
None of the features you described as requirements are basic.
 

Kegemusha

Senior Member
Joined
Jul 16, 2020
Messages
488
Likes
462
Like someone above wrote, I too wanted to have a kind of block approach, dedicated AMP, DAC and my old CD player and streamer.
To me 16/44.1 FLACS (with mutiroom), CD and spotify. And that was my requirement for my system.

I love music and I very interested in music since I am 12 years old.

After 10-15 years of static gear and moved to another house I started to change my speakers, then for 3 weeks ago my old pre amp/dac to new dac and new amp (a Ncore one). (DAC and amp thanks to ASR measurements and more reading from another sites)
Oh boy what a joy, all sounds so good! Spotify, CDs, FLACS, AAC, MP3, etc. I listen like 10-14hs per day music while working or taking a break.

My Yamaha CD player (I think older than 10 years) as transport and an 2nd hand sonos streamer (just because we have multiroom system, I bougth all Sonos stuff 2nd hand) and that it is.
I will be happy now until something breaks I guess.

I think is up to you at the end to decide what to buy, ASR present the numbers.
 
OP
nimar

nimar

Active Member
Forum Donor
Joined
Jan 25, 2021
Messages
213
Likes
216
Location
Ontario, Canada
By the way, glad you ultimately found something that meets your requirements. Hope you at least had fun trying the Topping stuff out.
Well, I was never in it to just "try" stuff so not exactly "fun", and ended up close to a grand (CAD) worse off after selling the stuff. The silver lining is that because the RME DAC is cheaper than the Topping stack I ended up roughly even. Seems like an ironic statement but I got RME quality at Topping prices (more than I should have paid).

Though I did find the process a somewhat enlightening experience and an idea for a little project is forming in my head as a result.

Re. your above comment about not basic features. What would you say are the most important features of a DAC that would lead you to choose product A over B?
 
Top Bottom