• WANTED: Happy members who like to discuss audio and other topics related to our interest. Desire to learn and share knowledge of science required. There are many reviews of audio hardware and expert members to help answer your questions. Click here to have your audio equipment measured for free!

Feedback on "quick review" process

Soniclife

Major Contributor
Forum Donor
Joined
Apr 13, 2017
Messages
4,500
Likes
5,417
Location
UK
I'm in favour of keeping the full reviews for everything, and am happy if this leads to less reviews. Quality over quantity.

I don't like the idea of something being missed with a quick review, and once standardised the possibility of manufactures gaming the system becomes possible. I also fear that everyone in favour of mini reviews has their own cherished measurement, and will demand it if skipped.

Bailing out early on anything that has already earned the headless status makes sense, but you do that already.

I don't know how much time you spend reading the review threads, some of which are massive, but having those threads stick to just discussing the measurements would possibly save time.
 

Jimbob54

Grand Contributor
Forum Donor
Joined
Oct 25, 2019
Messages
11,066
Likes
14,699
Just a thought. What do the photos add (other than the panther)? Is it copyright that stops you just lifting the front and back pics from manufs sites and materials?

If you cant crib from stock, forget the photos, or at least the booth ones, a quick and dirty panther in situ would suffice.

Just take the centrefolds down from the walls in the lab first ;-)
 

marsman2020

New Member
Joined
Aug 13, 2020
Messages
1
Likes
3
I'm new here - I've actually just been reading and this is my first post. The wealth of information is amazing, and as an engineer I appreciate seeing actual data instead of vague terms that belong at a wine tasting, not in a review of a technological device.

I've so far purchased one of the Apple USB-C headphone dongles and am probably going to pick up a set of ELAC DBR-62s based on the reviews here.

Some thoughts from my new person perspective:
  • For any review if something just measures terribly that seems like a good point to consider going with a reduced review.
  • For DACs and Headphone amps, as long as the current products with a great ratio of price/performance (the E30, L30, the Apple dongle) are on the market, it seems like reduced content reviews in the areas where there are already excellent options would to be acceptable. Maybe with exceptions for devices that offer unique features.
  • The full speaker reviews are very interesting. I'm not sure I'd cut content here. If there is a subset of the graphs that community members can create from the measured data that comes with each review, it might make sense to allow the community to do some of the chart-generating to free up some time.
  • For retro/vintage/discontinued stuff, personally I'd consider availability of the item (how many are available from used sources?) when deciding to review or not review.
All that said - generally quality over quantity seems like a good approach to me. This may be the only time a lot of this gear gets measured to this level of detail.
 

tecnogadget

Addicted to Fun and Learning
Joined
May 21, 2018
Messages
558
Likes
1,011
Location
Madrid, Spain
The full reviews could continue with the actual pace or even higher, without burning out Amir if they were a team work instead a 1 man job.

Limitation are that only Amir owns the measurement gear so that part is on him.

But then the extra data mining, graph plot and publishing could be distributed between the “ASR Panther Team”.

Don’t know if there is any local member around his area, nor if Amir wants any real world help. But moving around stuff, reshipping gear, etc, those are all time and physical consuming task that could be delegated.

Maybe to “hire” for free a young student for an ASR internship where in return he or she will learn the deepest secrets of hifi :p
 

Chrise36

Major Contributor
Joined
Dec 23, 2018
Messages
1,043
Likes
359
Hello everyone. My workload of reviews continues to be very, very heavy. I provide the same set of tests regardless of desirability of the unit. Often I skip a test or two only to be asked where they are.

Question is, can we agree on a "quick review" format where stuff gets tested quickly allowing higher throughput and owner getting his gear back sooner? Or do we continue as is?

As an example, personally the moment I look at the dashboard on a DAC, I learn 90% of what is in the rest of the test, if not 100%. The dashboard takes five minutes to setup and capture. The rest of the tests multiply the effort proportionally making the job much more resource intensive.

In the case of speakers, the spinorama alone could be the main measurement for say, a DIY speaker that is not of great interest. Since I include spinorama export in the review, others have been posting directivity and such anyway. Graphs like CSD/waterfall, heatmap directivity, etc. are all time consuming to capture.

Every review right now blows an entire day. Measurements usually take a couple of hours and photographing and annotating them takes another two hours. Then there is the write-up in the post.

Anyway, the default path is to continue as is but thought I bring up the topic and see what members think.
Amir i believe you hold the Guinness record for reviewing so many different devices in such a short time!!!
 

Rick Sykora

Major Contributor
Forum Donor
Joined
Jan 14, 2020
Messages
3,518
Likes
7,030
Location
Stow, Ohio USA
As much as I like to see the detailed testing, a few years ago, I realized I was spending to much time measuring my system (rather than listening to my music). So, am in line with you on reducing your test time for each review. As we have been, would be selective about DIY but like that we have been able to try things with DIY that is not practical otherwise. :cool:

Somehow missed the headphone rig effort. I understand the interest (notably by younger generations and/or apartment dwellers), but I find them too isolating. I'd rather do something else when I cannot listen to music via speakers. When I get really desperate, I listen on my Amazon Echo speakers. Since you are asking, would rather see these and other smart speakers tested than any vintage or budget passives (including DIY ones). :)
 

raistlin65

Major Contributor
Forum Donor
Joined
Nov 13, 2019
Messages
2,279
Likes
3,421
Location
Grand Rapids, MI
I'd suggest give more extensive measurements on the segments of the market that need your attention.

For example, I think the DAC and headphone amp market is now putting in much more effort at paying attention to how their products measure. Supply the dashboard and move on unless something in particular catches your attention you want to include.

AVRs? Seems like to me this market segment could benefit from having measurements published to challenge what they are producing.

So I guess what I'm saying is choose based on where you feel there is the most need for advocacy, as I always understood that's why you got into this in the first place: to initiate change.
 
Last edited:

Thomas_A

Major Contributor
Forum Donor
Joined
Jun 20, 2019
Messages
3,422
Likes
2,407
Location
Sweden
A "minimum" speaker review could just be "listening impression notes" + spinorama.
From those "data" there is a choice to test further and I would focus on those with best performance. The issue is how to handle the different price ranges.
 

Jimbob54

Grand Contributor
Forum Donor
Joined
Oct 25, 2019
Messages
11,066
Likes
14,699
Go For It! You take only Friday nights off. You need more time to yourself!
No!
1597354004340.png
 

minimalist

Member
Joined
Feb 6, 2020
Messages
20
Likes
14
I'd prefer quality over quantity.
With DACs I'm always happy to see multitone and Coax/Toslink jitter tests. I also see some good reasons to continue testing of discontinued devices. It's interesting to see in which areas audio technology really advanced and how yesterdays hyped devices perform on an independent test bench. Also, some folks like to buy used gear from time to time, so it's very helpful to know which brands and/or devices to better avoid.
 

nrabbit

Member
Joined
Jun 5, 2020
Messages
20
Likes
19
If anything I think reviews should be longer. People are still debating all over the place whether DACs have sound signatures or about double blind testing of amps. It would be great if we could get a set of measurements which comprehensively describe how something sounds. By making mini reviews that gives less data points to look at and leaves more to the imagination. Can just a single SINAD number or dashboard or spinorama comprehensively describe how something sounds? It's just going to become a numbers pissing contest.
 

restorer-john

Grand Contributor
Joined
Mar 1, 2018
Messages
12,581
Likes
38,283
Location
Gold Coast, Queensland, Australia
My suggestion a while back was to present the objective review measurements in a locked thread with a link to an accompanying discussion thread on the same. That locked thread review could be the mini review.

That way the reviews can be short, concise and to the point. They also won't get polluted with infighting and ultimately derailed or locked (as was a recent review). @amirm , you can then choose to participate and at what level. You could add content if you wanted to this thread.

The reviews are then clean and easy to search. People can choose "just the facts" or go down the rabbithole of discussion.
 

Jimbob54

Grand Contributor
Forum Donor
Joined
Oct 25, 2019
Messages
11,066
Likes
14,699
If anything I think reviews should be longer. People are still debating all over the place whether DACs have sound signatures or about double blind testing of amps. It would be great if we could get a set of measurements which comprehensively describe how something sounds. By making mini reviews that gives less data points to look at and leaves more to the imagination. Can just a single SINAD number or dashboard or spinorama comprehensively describe how something sounds? It's just going to become a numbers pissing contest.

The measurements of electronics don't really describe how something will sound. They describe how faithful or not it is to source and/or what impact some characteristics could have on downstream kit. You can infer what those deviations might do to the sound coming out of the transducers if they are large enough.
 

bigguyca

Senior Member
Joined
Jul 6, 2019
Messages
477
Likes
617
Hello everyone. My workload of reviews continues to be very, very heavy. I provide the same set of tests regardless of desirability of the unit. Often I skip a test or two only to be asked where they are.

Question is, can we agree on a "quick review" format where stuff gets tested quickly allowing higher throughput and owner getting his gear back sooner? Or do we continue as is?

As an example, personally the moment I look at the dashboard on a DAC, I learn 90% of what is in the rest of the test, if not 100%. The dashboard takes five minutes to setup and capture. The rest of the tests multiply the effort proportionally making the job much more resource intensive.

In the case of speakers, the spinorama alone could be the main measurement for say, a DIY speaker that is not of great interest. Since I include spinorama export in the review, others have been posting directivity and such anyway. Graphs like CSD/waterfall, heatmap directivity, etc. are all time consuming to capture.

Every review right now blows an entire day. Measurements usually take a couple of hours and photographing and annotating them takes another two hours. Then there is the write-up in the post.

Anyway, the default path is to continue as is but thought I bring up the topic and see what members think.


The reviews are as short as they can be and still include the important information. Actually some of the reviews could be longer. A thermal picture of every AVR and amplifier would be useful. The dashboard can be very misleading without added measurements.

You could reduce the number of reviews.

Reviews of non-current equipment aren't always interesting. Especially if the equipment wasn't noteworthy when it was available.

Mod edit , Removed off topic .
 
Last edited by a moderator:

restorer-john

Grand Contributor
Joined
Mar 1, 2018
Messages
12,581
Likes
38,283
Location
Gold Coast, Queensland, Australia
There seem to be a lot of reviews of products made in The People's Public of China from little known manufacturers. If these reviews are included it would be nice to have at least some sort of certification that the products, components or any other supplies or materials weren't made by the 1,000,000 Uyghur's who are in concentration camps or used as forced labor in China. Within the past year there was an article about children's clothes made for Costco in these camps, for example. Most American's don't seem to care about this sequel to WWII, but it rubs me the wrong way.

I think this strays into the political/humanitarian discussions we try to keep out of reviews. I understand the concern, but where do you draw the line? Lots of components in HiFi, some are sure to be made in poor working conditions...
 

Robin L

Master Contributor
Joined
Sep 2, 2019
Messages
5,213
Likes
7,591
Location
1 mile east of Sleater Kinney Rd
That SINAD chart on the Review splash [ish] page tells me about the bar that should be set for the reviews: Anything below green, there will be a weekly page listing DACS and Headphone amps with the product names, prices, contact information and SINAD score. No more information is needed when a SOTA product can be obtained for around $100.

Seems like speaker measuring protocol is still being ironed out, those reviews still need to be more or less complete. As regards power amps and AVRs, I'd expect pretty much the same scale of measurement as Headphone Amps and DACs should apply, but with the scale adjusted to accommodate generally lowered expectations.
 

Wes

Major Contributor
Forum Donor
Joined
Dec 5, 2019
Messages
3,843
Likes
3,788
Quick Review for amps, DACs makes sense if it will speed up speaker reviews.

That would put the effort where the issues are.
 

bobbooo

Major Contributor
Joined
Aug 30, 2019
Messages
1,479
Likes
2,079
There seem to be a lot of reviews of products made in The People's Public of China from little known manufacturers. If these reviews are included it would be nice to have at least some sort of certification that the products, components or any other supplies or materials weren't made by the 1,000,000 Uyghur's who are in concentration camps or used as forced labor in China. Within the past year there was an article about children's clothes made for Costco in these camps, for example.

Most American's don't seem to care about this sequel to WWII, but it rubs me the wrong way.

That would also rule out, among others, Apple (who make our beloved dongle), Samsung (who own Harman, who make our beloved Revel speakers), and Sony (who make our b...argain Sony SS-CS5).
 

HooStat

Addicted to Fun and Learning
Joined
May 11, 2020
Messages
856
Likes
933
Location
Calabasas, CA
Apologies if already suggested above, but you could provide a list of upcoming reviews with your suggested review level, and allow people to vote on whether it should be adjusted up or down. That way you could get a sense of what people are interested in, and how much effort you should expend on each.
 

jhaider

Major Contributor
Forum Donor
Joined
Jun 5, 2016
Messages
2,822
Likes
4,514
Proposed quick formats:

Speakers
  1. Spinorama
  2. Distortion Spectrum (86dB vs. 96dB)
  3. Heatmap directivity

For speakers IMO, it would save time to only measure the things that can be done in a Klippel scan. In addition to 1 and 3 above I would keep the beamwidth graphs, because those are so cool.

Getting rid of polar maps would be a big mistake for any speaker review. I think it's clear Spinorama ERDI and DI hide the ball a little bit regarding directivity.

I know distortion is a pet thing, but I'm not convinced it's worth a damn and it's the also least replicable data. CSD/waterfall are total wastes of time. You can see resonances by looking at polar maps, or (though it's a little harder) seeing where on axis/listening window have bumps but the DI does not.

No comment on electronics, because that's not an area of much interest to me.
 
Top Bottom