Feedback on "quick review" process

[Cahudson42]

Shouldn't leave out all vintage - if so, we would have missed the RX-596...

 

[bluefuzz]

It's your time. You are under no obligation to anyone to test a device per day or more. If you only feel like doing one a week or one a month, then that's fine. There's no hurry.

However, I do think the stuff you do test is tested as fully as you do now. I prefer quality over quantity. In fact for complex multifunction devices like dac/streamer/preamps or integrated amps/receivers the real world useabilty/functionality is every bit as important as raw SINAD performance. I would like to see more focus on that side of things.

That said, I think there is little point testing vintage, obsolete or otherwise not currently available products. And prioritise new on the market gear over stuff that has been on the market for years even though still available. It's a waste of everyone's time testing obvious snake-oil/audiophool products or megabucks 'audio-jewellery'.

Personally, I am mostly interested in DIY gear – not just speakers – and would be sad to see that disappear or lose priority. Also I would much rather see interesting, original, boutique, small shop etc. products tested over big name mainstream gear.

I could easily live without ever seeing another AVR review. And don't even think about testing headphones ...

 

[bogart]

Few thoughts:

• Vintage: I think we can say that all vintage *digital* measurements are unnecessary. We know that SOTA has moved on, so retrospective reviews (Amir's Levinson DAC aside) are at best confirming that we've come a long way. Amp quality and grunt is still interesting, but I suspect these aren't taking much time or bandwidth.
• Speakers: Subjective impressions have been useful in some cases to say "measurements and experience don't line up!" in ways that are educational. I could live with no EQ experimentation and abbreviated listening notes if it sounds like it measures. I'm OK to reduce distortion measurements to minimal for speakers whose base performance doesn't merit it. Understanding "how bad they get" is limited value. I'd be interested in a sampling approach for speakers of a 'good enough' threshold to get more scrutiny - first articles from a supplier, unique price point items, etc.
• AVRs: I had been frustrated with these since they were an endless stream of bad news and took so much time, but the recent Denon reviews identified a brand I'd be willing to buy. I think that this category needs the pressure of detailed reviews.
• DACs/HP amps: Solved problems. Abbreviated reviews unless exceptional performance that deserves a spotlight.
• Oddities: For folks on the site already, we know the degree to which nearly all gizmos don't work. If we won't convince anyone through objective measurements, these can be abbreviated or eliminated.

I think it's also fair to look for us in the community to help with the relative rankings and extended visualizations. Except vs. reference performance, the item review can usefully refer to thresholds ("top tier") but not incur the work of visuals. Particularly as they've gotten busier, they're less and less useful in static form.

 

[murraycamp]

@amirm - I say do whatever you need to do in order provide a reasonable level of relevant information and keep your sanity.

Just my .02

 

[blueone]

I appreciate the full reviews. If you're getting burnt out, pace yourself better and adjust your work-life balance.

 

[ezra_s]

without any scientific base on my part I would do quick reviews or "stop right here" the moment you see the typical measure of a specific device looks to be low quality unless such device has grown much interesting from many.

 

[blueone]

without any scientific base on my part I would do quick reviews or "stop right here" the moment you see the typical measure of a specific device looks to be low quality unless such device has grown much interesting from many.

My take: if an individual member goes through the expense and inconvenience to ship Amir equipment, he deserves to have the stuff reviewed, bad or good.

 

[RayDunzl]

@amirm

Do whatever keeps it fun for you.

 

[garbulky]

I prefer you stick with the full reviews. I don't think anybody could say you are providing inadequate numbers of reviews! You're one of the most prolific review sites I've seen coupled that with a reasonable amount of detail, I don't think your review process should get shorter or quicker imo.

 

[Jimbob54]

Forget the numbers and s**t. Just use longer, flowery words and spellcheck

 

[Chromatischism]

without any scientific base on my part I would do quick reviews or "stop right here" the moment you see the typical measure of a specific device looks to be low quality unless such device has grown much interesting from many.

However, in some cases, deeper dives revealed the cause of the problem. Example: Denon HDMI multichannel input/low SINAD.

 

[Frank Dernie]

I think you are stuck with full reviews simply because anything you leave out will be requested by a member sooner or later.

On a personal note though:-
I don't even bother reading the DAC reviews any more since transparent ones are readily available, the features, fit, finish and styling would make more difference to me in choosing but I don't imagine I will ever buy another.
Nothing headphone related interests me either, since I only use them in noisy environments and SQ isn't at issue.
I find the speaker measurements the most interesting, spin and distortion in particular but also impedance tells a lot.

 

[Racheski]

My guess is that the majority of the ASR membership is more than happy with the pace of your reviews, but you feel like you are behind because the back-log of devices is starting to pile-up.

You may already have something like this in place, but create an inventory system that democratizes the work among ASR members. When members send in a device for review, we can fill out a brief template that includes the date we mailed the device, along with other pertinent data points for the Tableau dashboards. The ASR "data team" (for lack of a better term) can upload this template to a Google Doc and boom, you have an inventory of devices to review that you don't have to maintain. We could also indicate whether we are OK with a "quick review" on the template.

The inventory allows you to better manage the backlog of devices to review, and provides information to create polls to see what ASR membership wants reviewed next (have someone else create the polls). It also helps you make more accurate estimates of how long it will take to review a device after a member sends one in.

 

[newbie-]

I really like having a big database of devices to explore. So a quick review format would be fantastic to get my bearings. I agree with Amirm that the dashboard can paint a pretty clear picture.

But... I vote to continue the detailed reviews on the current schedule. Here's why - Amirm only has each device on a temporary basis. There's no opportunity to return to a device to answer any questions or get additional details. Once it's gone, that's it. It's better to have a more complete review for each device.

Maybe there are opportunities to streamline the detailed measurements.

 

[Thomas_A]

I've mentioned it before but e.g. for speaker reviews, how interesting are poor performing speakers? I don't know the best process for selection, but being more selective towards potentially good sounding speakers would be one way.

 

[MarsianC#]

Outsource as much as you can and focus on getting data, not preparing it for us. I'd love to see full tests as we are used to, including your listening experiance.
Take a break, you've earned it!

 

[VMAT4]

Go For It! You take only Friday nights off. You need more time to yourself!

 

[ezra_s]

My take: if an individual member goes through the expense and inconvenience to ship Amir equipment, he deserves to have the stuff reviewed, bad or good.

Yes, yes, but I see lots of reviews of low quality devices which also do not cost much while amirm reveals he usually spends a whole day on a single review. I wouldn't go a deep dive in such devices.

 

[Thomas savage]

Mini reviews are a good idea imo they should serve to make the full feature reviews seem more spectacular and eventful.

You will have to decide what items get what treatment, some might moan .

Maybe reducing your participation in the aftermath of the mini reviews will help you too , no point in doing this then spending the equivalent time dealing with all the back and forth in thread.

 

[pierre]

I,would,do only quick,reviews. If the;results stands out (10% maybe less) then do a full one.
would a script that takes the measurement and prefill a post would be useful? It could generate half the post from a template?