Feedback on "quick review" process

[Jimbob54]

A few comments:

1) Always measure from an analog source and a digital source when possible to cover use cases
2) Sweep AVRs pre outs to 4V
3) My preference is that you spread yourself to thin by adding headphones
4) When you find a best in class product spend a little more time with it

Definitely 1) for dac /amp combos

 

[kaka89]

I think DACs can use a quick review format, especially for brands that had couple of reviews in ASR already, like SMSL and Topping. DAC is a very mature technology, most brand use the same chip and performance is similar.

On the other hand, I think power amplifier and speaker should have comprehensive reviews. Speaker performance is still largely a mystery to me as objective and subjective result some times not match. And there are many different power amp implementation out there that worth to have a proper review.

 

[Chromatischism]

I would say drop DACs altogether, but I guess most people won't feel the same. So quick review for DACs sound good. I feel speakers are the most important thing right now, so make a shorter format review only if it's an unimportant one or the spinorama reveals very poor performance so there is no point measuring other aspects.

If this is coming up because of the headphone reviews idea, I think you should drop the headphone reviews. There are more than enough headphone reviewers out there, and it looks like diving into it will just create even more confusion, with the whole compensation deal.

Concur. Let's focus where the attention is needed most and cut time that would be less fruitful.

 

[Josq]

Newbie here... I like the horizontal/vertical heatmaps a lot (contour maps are perhaps even better): 1) I think they give a pretty good indication what can be EQ'd and what not. 2) They show how you should position the speakers (interesting example: https://www.audioholics.com/bookshelf-speaker-reviews/polk-t15/conclusion )

 

[Chromatischism]

Just to chime in, the reflection graphs are SO USEFUL to know about proper placement. They are one of my favorite features of the reviews. Your last point is, well, on point. No need for Amir to use his money to make his life hell (apart from ALL OF THIS, lol).

No kidding. All of the things he is proposing we skip are the most important things we need to know about the speaker. The reflections, directivity - that is where speaker design is focusing right now.

 

[MZKM]

No kidding. All of the things he is proposing we skip are the most important things we need to know about the speaker. The reflections, directivity - that is where speaker design is focusing right now.

The two Directivity Index graphs in the Spinorama tell you the info pretty much, the heat maps are mainly for a deep-dive.

I agree the Early Reflections graph is useful.

 

[patate91]

Small and underground manufacturers get a good visibility from ASR, which I think it's very positive. We discovered some good gems this way. So I don't think it's a good thing to cut on DIY or less known products.

I don't think cutting on objective data is a good thing too. There's a lot of experts here that can help to explain those graphs.

I would drop the subjective part. If people skips the objective data for the subjective impression I guess the ASR goal is missed.

 

[GXAlan]

I have an original Denon ALPHA DAC with Amir for testing.

1) Dashboard to give basic spec
2) Since this is the only way to test Alpha on/off, extra testing to see if it affects IMD or specialized test tones is helpful because of the AVR line and determining if Marantz vs Denon or Denon 3700 vs 4700 makes sense.

reproducing Denon marketing -60, -80, -90dB pictures vs comparing it to 2020 generation studf

is there Harm from Alpha?
Is there Benefit from Alpha?
does it change the digital filter roll off?
(Supposed to be dynamic)

Maybe a voting system to see what people want extra testing on makes sense? Unless you have sent in stuff, I don’t think the readership understands your backlog. Personally, I would have you keep the DAC longer to do more tests on it when you have time

 

[Rja4000]

Why not to stop at dashboard when performance is crap ?
Or just dashboard + 31 tones.
You get an idea of a lot of things with those 2.

Then, if you think there is a chance you could recommend the device, you'll have to go through the whole set before you do...

Given the % of headless devices, you will most probably save quite some time...

PS: 'Crap' means below average for average price or higher.

 

[sweetchaos]

IMO, drop or significantly reduce vintage and discontinued products. Since they can’t be easily bought, reading these reviews is interesting, but doesn’t lead to any purchases.

I would even say, that if the item is not sold currently, there should be extra $ incentive for Amir to measure, in order for the review to make it to the front page.

Also, I’m in favour of shorter reviews if it means more reviews overall.

 

[Cahudson42]

Well, I for one look forward to Amir's Subjective impressions, especially for Speakers (and I expect for headphones, should we get to that).

Keep them for Full speaker reviews at least.

 

[q3cpma]

If it were myself doing it, I'd have a hard time sleeping once the work charge dwindles and I know I didn't take my time doing it "properly" when I could. I'm sure we all prefer quality over quantity, no?

If I were to make a detailed criticism about the speaker review, I find the current CSD useless compared to a spectrogram; so maybe don't include it in this form. Also, either bin the subjective review (but it might be the most interesting thing for you) or shorten it (no need to name songs, for example); we'll get way less useless drama too.

tl;dr slowing down to get a little rest would be better than botching some reviews.

 

[LTig]

As an example, personally the moment I look at the dashboard on a DAC, I learn 90% of what is in the rest of the test, if not 100%. The dashboard takes five minutes to setup and capture. The rest of the tests multiply the effort proportionally making the job much more resource intensive.

In the case of speakers, the spinorama alone could be the main measurement for say, a DIY speaker that is not of great interest. Since I include spinorama export in the review, others have been posting directivity and such anyway. Graphs like CSD/waterfall, heatmap directivity, etc. are all time consuming to capture.

Dashboard and Multitone tell many things. You need the dashboard for the numbers, and multitone can replace IMD and frequency response - it actually tells us THD, noise and IMD over frequency.

 

[tecnogadget]

Quick reviews is a great idea since the volume of gear is overwhelming.

All ASR needs is just a consensus on what deserves the full review or quick one.

Just my personal vote and opinion:

-Full Review: SOTA performance independent of price. Gear that performs great and its also afordable, aka budget champions. Special gear that caught many user’s attention/interest.

-Quick Review: Average performers, duplicated (similar performance and price as previously measured gear), broken gear, headless panther.

My only concern is about how accurately one can tell if the gear is a real SOTA or Broken design by means of fewer metrics or Quick Review ??

 

[Martin]

I’m all for mini-reviews. Dashboard, SINAD, IMD and jitter for DACs. Dashboard, SINAD, SNR and output impedance for headphone amps. I think speakers and AVR’s need more complete measurements. I like the speaker reviews but am not looking to replace what I have. I am most interested in DAC, amp (both headphone and speaker) and AVR measurements. Anything to help lighten the load. Perhaps requesting a nominal fee for more detailed measurements?

Edit: I also agree that there is no reason to provide free reviews of older discontinued equipment.

Martin

 

[pjug]

I like the complete reviews. Maybe it is time to be more selective on what to review? Or have owners wait to send stuff until they get a message from you that you are ready to receive it.

 

[Vasr]

IMO, drop or significantly reduce vintage and discontinued products. Since they can’t be easily bought, reading these reviews is interesting, but doesn’t lead to any purchases.

Vintage perhaps not. But a lot of discontinued items are available for sale often at very good prices.

So, I would just say avoid one-of or not available for sale regardless of whether they are discontinued or not.

I would also recommend prioritizing newly purchased (brand new or refurbished) in the last 6 months or so over a device that has been sitting somewhere for much longer (including pre-owned sales that aren't factory refurbished). I don't think the latter is a good sample necessarily to base a judgment on when you have to do with just one sample.

 

[Chromatischism]

The two Directivity Index graphs in the Spinorama tell you the info pretty much, the heat maps are mainly for a deep-dive.

I agree the Early Reflections graph is useful.

That may be, but the heat map and white version are a lot easier to see the data (beamwidth).

 

[Chromatischism]

Vintage perhaps not. But a lot of discontinued items are available for sale often at very good prices.

So, I would just say avoid one-of or not available for sale regardless of whether they are discontinued or not.

Yep, plenty of things can be found on eBay, etc. A quick look to see if it's been covered at ASR is helpful.

 

[DuxServit]

@amirm Just do fewer selective reviews. There’s just too many devices being produced, most are forgotten after a year or so.

(e.g. Virtually every AVR system reviewed has been disappointing).