• WANTED: Happy members who like to discuss audio and other topics related to our interest. Desire to learn and share knowledge of science required as is 20 years of participation in forums (not all true). Come here to have fun, be ready to be teased and not take online life too seriously. We now measure and review equipment for free! Click here for details.

A Scientific Review of AudioScienceReview

  • Thread starter Deleted member 30699
  • Start date
Status
Not open for further replies.
D

Deleted member 30699

Guest
The following is a review of AudioScienceReview (ASR) as if it were a concept submitted to a journal.
In science, the peer-review is the evaluation of work by one or more people with similar competencies as the producers of the work. It functions as a form of self-regulation by qualified members of a profession within the relevant field. Peer review methods are used to maintain quality standards, improve performance, and provide credibility.
This review is single-blind (reviewer is anonymous, author/work is not), which is the current trend in science.


Process:
Peer reviews typically have a rating section, and feedback to author/editor section.
This review is provided s.t. a potentially revised ASR can be even better.

Review rating:
Rating 4/5 for electronic components
Rating 2/5 for speakers
Rating 3/5 for other equipment


Review feedback:
ASR has the potential to bring objective information to audio end-users. This is information is needed because audio equipment is increasingly sold online, i.e. potential customers cannot audition, and home-trails are more cumbersome. ASR provides two types of results: objective measurements and subjective feedback (listening tests or preference score).
This review holds ASR to a high standard than any other review outlet because of its claim of objectivity.

Electronic components:
The author provides key metrics and a comprehensive objective evaluation of electronic components. Analysis is provided predominantly in the frequency domain. This is state-of-the-art in audio and appropriate for this purpose. However, modern systems and control theory has moved to time domain analysis since it has advantages in the description of nonlinear, multi-input-multi-output, and constraint systems. Lack of understanding thereof may have contributed to some unfortunate statements, e.g. that phase may not matter in (
). There are several issues with the video, where ASR discards that phase shift is relevant, the main ones are
- Low/no phase shifts result in a output that is measurably better, hence higher fidelity
- Phase shift is a frequency-dependent time delay (or advance). This is audible especially at low frequencies. Also, random phase shifts (without changing the magnitude) can model certain distortions.
- The author himself acknowledges that (differential) phase shift is audible. Hence the manufacturers claim of audibility are satisfied and the burden of proof is on the author.
- Science is about finding the "truth" and models are always in question. This is especially true when the main references are 40-50 years old and human listening preferences change over time (e.g. increased bass in music).

Speakers:
ASR provides speaker measurements based on a nearfield technique. This is a good approximation to anecoic measurements and can extrapolate a in-room response based on assumptions of a room. Furthermore, ASR provides listening tests. The objective information is overall very good. However, there are some issues
(1) The author owns Revel Saloon speakers, possibly for his preference of these speakers over others. Hence, listening tests may have an unintentional bias toward this sound signature. However, listening tests are also always differential (humans are adaptive and we tend to bias towards what we are used to). Hence, speakers may also be unintentionally benchmarked against this sound. Overall, this may result in a preference for Reven speakers (or Harman products in general).
(2) In science, independence is critical and COI is a serious topic. For example, it is unacceptable to take part in a drug study and have any financial interests in the drug sales or company producing the drug. A scientist would need to excuse her/himself from such a study or at the very least refrain from making subjective claims.
(3) Not every speaker concept adheres to the underlying theory including panel speakers (Magnepan), dipole speakers, or omnidirectional speakers (Ohm, Linkwitz). These speakers, usually, intend to leverage the room and can achieve different things (including being more problematic in some rooms). However, such speakers should not be judged using the same concepts.

Other equipment:
ASR also reviews cables and power conditioners. Overall, ASR carries the message that these things don't matter, which is usually correct. However, ASR fails to educate how these things can matter.
Cables for example can matter if terminals (or wire connecting to terminals) oxidize and become less conductive, which is quite common.
Power conditioners can matter if the local grid has issues, power cables are close to mic/phono cables (typically before preamp), or in presence of old equipment (typically designed before the existence of switched power supplies). Usually, this should not be a relevant for most modern systems

Summary:
ASR is a very interesting concept with some rough edges to address.
 
Last edited by a moderator:

blueone

Addicted to Fun and Learning
Forum Donor
Joined
May 11, 2019
Messages
556
Likes
742
Location
USA
The following is a review of AudioScienceReview (ASR) as if it were a concept submitted to a journal.
In science, the peer-review is the evaluation of work by one or more people with similar competencies as the producers of the work. It functions as a form of self-regulation by qualified members of a profession within the relevant field. Peer review methods are used to maintain quality standards, improve performance, and provide credibility.
This review is single-blind (reviewer is anonymous, author/work is not), which is the current trend in science.


Process:
Peer reviews typically have a rating section, and feedback to author/editor section.
This review is provided s.t. a potentially revised ASR can be even better.

Review rating:
Rating 4/5 for electronic components
Rating 2/5 for speakers
Rating 3/5 for other equipment


Review feedback:
ASR has the potential to bring objective information to audio end-users. This is information is needed because audio equipment is increasingly sold online, i.e. potential customers cannot audition, and home-trails are more cumbersome. ASR provides two types of results: objective measurements and subjective feedback (listening tests or preference score).
This review holds ASR to a high standard than any other review outlet because of its claim of objectivity.

Electronic components:
The author provides key metrics and a comprehensive objective evaluation of electronic components. Analysis is provided predominantly in the frequency domain. This is state-of-the-art in audio and appropriate for this purpose. However, modern systems and control theory has moved to time domain analysis since it has advantages in the description of nonlinear, multi-input-multi-output, and constraint systems. Lack of understanding thereof may have contributed to some unfortunate statements, e.g. that phase may not matter in (
). There are several issues with the video, where ASR discards that phase shift is relevant, the main ones are
- Low/no phase shifts result in a output that is measurably better, hence higher fidelity
- Phase shift is a frequency-dependent time delay (or advance). This is audible especially at low frequencies. Also, random phase shifts (without changing the magnitude) can model certain distortions.
- The author himself acknowledges that (differential) phase shift is audible. Hence the manufacturers claim of audibility are satisfied and the burden of proof is on the author.
- Science is about finding the "truth" and models are always in question. This is especially true when the main references are 40-50 years old and human listening preferences change over time (e.g. increased bass in music).

Speakers:
ASR provides speaker measurements based on a nearfield technique. This is a good approximation to anecoic measurements and can extrapolate a in-room response based on assumptions of a room. Furthermore, ASR provides listening tests. The objective information is overall very good. However, there are some issues
(1) The author owns Revel Saloon speakers, possibly for his preference of these speakers over others. Hence, listening tests may have an unintentional bias toward this sound signature. However, listening tests are also always differential (humans are adaptive and we tend to bias towards what we are used to). Hence, speakers may also be unintentionally benchmarked against this sound. Overall, this may result in a preference for Reven speakers (or Harman products in general).
(2) In science, independence is critical and COI is a serious topic. For example, it is unacceptable to take part in a drug study and have any financial interests in the drug sales or company producing the drug. A scientist would need to excuse her/himself from such a study or at the very least refrain from making subjective claims.
(3) Not every speaker concept adheres to the underlying theory including panel speakers (Magnepan), dipole speakers, or omnidirectional speakers (Ohm, Linkwitz). These speakers, usually, intend to leverage the room and can achieve different things (including being more problematic in some rooms). However, such speakers should not be judged using the same concepts.

Other equipment:
ASR also reviews cables and power conditioners. Overall, ASR carries the message that these things don't matter, which is usually correct. However, ASR fails to educate how these things can matter.
Cables for example can matter if terminals (or wire connecting to terminals) oxidize and become less conductive, which is quite common.
Power conditioners can matter if the local grid has issues, power cables are close to mic/phono cables (typically before preamp), or in presence of old equipment (typically designed before the existence of switched power supplies). Usually, this should not be a relevant for most modern systems

Summary:
ASR is a very interesting concept with some rough edges to address.

The original post:

1/5 for technical relevance
1/5 for accuracy
1/5 article structure

:facepalm:

Amir has said repeatedly his subjective impressions of speakers are his opinion (however well trained) and nothing more. His ownership of Revel Salon2s does not affect his measurements.

Your comments on phase shift audibility are confusing. Are you saying Paul McGowan's view has merit?

Your comment on cables having audible differences has nothing to do with cable construction.

If you're going to be so critical, I suggest you start with your own post.
 

Yuhasz01

Member
Joined
Aug 8, 2020
Messages
38
Likes
23
Hire a proof reader. Or use a spell checker.

That aside, seriously, why?
Proof reading comment irrelevant here.

Duh? Audio “Science “ Review is name of this web site. The review methods are fair game as well as reviewer bias. A relevant topic.
 

steve59

Senior Member
Joined
Dec 18, 2019
Messages
367
Likes
223
The 'what the science shows' continuation thread from avr, lol
 

Berwhale

Major Contributor
Forum Donor
Joined
Aug 29, 2019
Messages
1,892
Likes
2,189
Location
UK
In science, the peer-review is the evaluation of work by one or more people with similar competencies as the producers of the work

Please explain how you qualify as a peer reviewer.
 

Jimbob54

Master Contributor
Forum Donor
Joined
Oct 25, 2019
Messages
6,978
Likes
8,177
Proof reading comment irrelevant here.

Duh? Audio “Science “ Review is name of this web site. The review methods are fair game as well as reviewer bias. A relevant topic.
OK, I'll get to the rub of it then. Criticism of the various review methodologies belong in the various threads set up for just that.

This thread is a thinly veiled accusation about Amir's independence and competence. Written badly.
 

amirm

Founder/Admin
Staff Member
CFO (Chief Fun Officer)
Joined
Feb 13, 2016
Messages
34,993
Likes
130,409
Location
Seattle Area
(3) Not every speaker concept adheres to the underlying theory including panel speakers (Magnepan), dipole speakers, or omnidirectional speakers (Ohm, Linkwitz). These speakers, usually, intend to leverage the room and can achieve different things (including being more problematic in some rooms). However, such speakers should not be judged using the same concepts.
Such speakers are a tiny, tiny fraction of what exists in the market. So assuming your criticism is correct (which is not as I also "reviewed" them just like anyone else does by sticking them in a room and playing music), your feedback on this front does not matter.

Now if you are saying these speakers are not reviewable, then you are against established science which has reviewed them.
 

CtheArgie

Active Member
Forum Donor
Joined
Jan 11, 2020
Messages
237
Likes
292
Location
Agoura Hills, CA.
"(2) In science, independence is critical and COI is a serious topic. For example, it is unacceptable to take part in a drug study and have any financial interests in the drug sales or company producing the drug. A scientist would need to excuse her/himself from such a study or at the very least refrain from making subjective claims."

The doctors example is also relative. It is very common for expert physicians to be authors of studies, enroll patients, consult for the company and be remunerated for their work.

They disclose this. It is when they do not disclose that you can get a problem. You can even find online the list of physicians that receive funding from drug companies and how much. It is a federal law.
 

amirm

Founder/Admin
Staff Member
CFO (Chief Fun Officer)
Joined
Feb 13, 2016
Messages
34,993
Likes
130,409
Location
Seattle Area
Speakers:
ASR provides speaker measurements based on a nearfield technique. This is a good approximation to anecoic measurements and can extrapolate a in-room response based on assumptions of a room.
You are mistaken on the first sentence. An anechoic chamber is itself an approximation of the "free field" radiation of a speaker. The ideal situation would have no reflections anywhere around the speaker to infinity. Anechoic chambers -- even large ones -- cannot achieve this so produce incorrect low frequency response. So they are not the target as they themselves have problems.

Measurement resolution is also limited in anechoic chambers as it is time consume to make thousands of measurements to represent high angular resolution. Standard spinorama has 70 measurements compared to thousand plus that Klippel NFS produces.

Klippel NFS therefore produces higher resolution and more correct response than anechoic chamber. There are some rare exceptions to this but in general, your assessment and positioning of our measurements is incorrect. If you are going to peer view something, you need to demonstrate proper knowledge of the domain, not lay understanding of it in this manner.

As to that "extrapolation," that is again based on science and makes predictions. That prediction like a weather prediction, is not always accurate but can be highly useful. Science needs to close shop if all you wanted from it was perfection.
 

Berwhale

Major Contributor
Forum Donor
Joined
Aug 29, 2019
Messages
1,892
Likes
2,189
Location
UK
? Well, by having demonstrated a similar level of study and feedback from others in the field. Academic standards. Scrutiny of one's methods and conclusions.

Since you seem to be speaking on their behalf, please provide examples of where @ophiuchia has demonstrated any of the above.
 
Status
Not open for further replies.
Top Bottom