In response to the concerns over our subjective appraisal of speakers I've started this thread to talk about what we can do to maximise the value of this part of our reviews .
Please keep it constructive.
Please keep it constructive.
-
WANTED: Happy members who like to discuss audio and other topics related to our interest. Desire to learn and share knowledge of science required. There are many reviews of audio hardware and expert members to help answer your questions. Click here to have your audio equipment measured for free!
Practical sighted evaluation guidelines for speakers
- Thread starter Thomas savage
- Start date
- Joined
- Feb 23, 2016
- Messages
- 20,689
- Likes
- 37,411
I think listening to one speaker you've never heard before can of course be useful. Better however is to compare it to a speaker you know as a reference to determine which is better.
Level matching is needed for this. Following guidelines from Toole's book, pink noise filtered with 12 db/octave roll off below 500 hz and above 2000 hz. is suggested.
Quick switching works best so whether comparing music or wideband pink noise listening to one speaker at a time with the ability to switch between them is very helpful in gauging differences. I'd suggest symmetrical placement left and right in a room if the room is symmetrical.
I suppose even this topic could be split into two parts. Sighted evaluations of a pair of speakers comparatively. And sighted evaluation of speakers in isolation without direct reference to another speaker. I think the latter is going to be less accurate, but sometimes you have no choice.
Level matching is needed for this. Following guidelines from Toole's book, pink noise filtered with 12 db/octave roll off below 500 hz and above 2000 hz. is suggested.
Quick switching works best so whether comparing music or wideband pink noise listening to one speaker at a time with the ability to switch between them is very helpful in gauging differences. I'd suggest symmetrical placement left and right in a room if the room is symmetrical.
I suppose even this topic could be split into two parts. Sighted evaluations of a pair of speakers comparatively. And sighted evaluation of speakers in isolation without direct reference to another speaker. I think the latter is going to be less accurate, but sometimes you have no choice.
OP
- Joined
- Feb 23, 2016
- Messages
- 20,689
- Likes
- 37,411
About level matching, I've also read blind listening investigation found listeners use the 100 hz to 500 hz range as a baseline to describe whether a speaker has too much, not enough or about the right amount of bass response. And whether the frequencies above this range make a speaker bright or even or dull sounding. So this might bear some investigating for level matching purposes.
Coach_Kaarlo
Active Member
I don't know much, and despite following this topic closely do not have much to contribute specifically (caveat).
However, I work in design, and when solving a new problem I always start with first principles, so maybe we do not need to reinvent the circular object which underpins transportation - just apply some methodology and controls as per the scientific method.
From our good friends at wikipedia;
Experiments:
Once predictions are made, they can be sought by experiments. If the test results contradict the predictions, the hypotheses which entailed them are called into question and become less tenable. Sometimes the experiments are conducted incorrectly or are not very well designed when compared to a crucial experiment. If the experimental results confirm the predictions, then the hypotheses are considered more likely to be correct, but might still be wrong and continue to be subject to further testing. The experimental control is a technique for dealing with observational error. This technique uses the contrast between multiple samples (or observations) under differing conditions to see what varies or what remains the same. We vary the conditions for each measurement, to help isolate what has changed. Mill's canons can then help us figure out what the important factor is.[86] Factor analysis is one technique for discovering the important factor in an effect. Scientists assume an attitude of openness and accountability on the part of those conducting an experiment. Detailed record-keeping is essential, to aid in recording and reporting on the experimental results, and supports the effectiveness and integrity of the procedure. They will also assist in reproducing the experimental results, likely by others.
Evaluation and improvement:
The scientific method is iterative. At any stage, it is possible to refine its accuracy and precision, so that some consideration will lead the scientist to repeat an earlier part of the process. Failure to develop an interesting hypothesis may lead a scientist to re-define the subject under consideration. Failure of a hypothesis to produce interesting and testable predictions may lead to reconsideration of the hypothesis or of the definition of the subject. Failure of an experiment to produce interesting results may lead a scientist to reconsider the experimental method, the hypothesis, or the definition of the subject.
Other scientists may start their own research and enter the process at any stage. They might adopt the characterization and formulate their own hypothesis, or they might adopt the hypothesis and deduce their own predictions. Often the experiment is not done by the person who made the prediction, and the characterization is based on experiments done by someone else. Published results of experiments can also serve as a hypothesis predicting their own reproducibility.
Confirmation:
Science is a social enterprise, and scientific work tends to be accepted by the scientific community when it has been confirmed. Crucially, experimental and theoretical results must be reproduced by others within the scientific community. Researchers have given their lives for this vision; Georg Wilhelm Richmann was killed by ball lightning (1753) when attempting to replicate the 1752 kite-flying experiment of Benjamin Franklin.[93]
To protect against bad science and fraudulent data, government research-granting agencies such as the National Science Foundation, and science journals, including Nature and Science, have a policy that researchers must archive their data and methods so that other researchers can test the data and methods and build on the research that has gone before. Scientific data archiving can be done at a number of national archives in the U.S. or in the World Data Center.
So in essence, at high level, a SIGHTED EVALUATION should;
Lots of the discussion on this topic starts with "you are wrong, and let me tell you how and why" - maybe acceptance is a better goal, than seeking agreement or consensus (the term herding cats springs to mind). This new thread seems like a way to start with the basics and take everyone on the same journey, and will hopefully prove more effective than working from a niche of specialist knowledge or experience and decrying the efforts of others to share their specific knowledge or experience (when relevant of course), or arguing over semantics and misunderstandings.
If logic and reason are involved others can and will follow - if personal preference or anecdotal experience is the only claim then the debate rages on ad infinitum.
However, I work in design, and when solving a new problem I always start with first principles, so maybe we do not need to reinvent the circular object which underpins transportation - just apply some methodology and controls as per the scientific method.
From our good friends at wikipedia;
Experiments:
Once predictions are made, they can be sought by experiments. If the test results contradict the predictions, the hypotheses which entailed them are called into question and become less tenable. Sometimes the experiments are conducted incorrectly or are not very well designed when compared to a crucial experiment. If the experimental results confirm the predictions, then the hypotheses are considered more likely to be correct, but might still be wrong and continue to be subject to further testing. The experimental control is a technique for dealing with observational error. This technique uses the contrast between multiple samples (or observations) under differing conditions to see what varies or what remains the same. We vary the conditions for each measurement, to help isolate what has changed. Mill's canons can then help us figure out what the important factor is.[86] Factor analysis is one technique for discovering the important factor in an effect. Scientists assume an attitude of openness and accountability on the part of those conducting an experiment. Detailed record-keeping is essential, to aid in recording and reporting on the experimental results, and supports the effectiveness and integrity of the procedure. They will also assist in reproducing the experimental results, likely by others.
Evaluation and improvement:
The scientific method is iterative. At any stage, it is possible to refine its accuracy and precision, so that some consideration will lead the scientist to repeat an earlier part of the process. Failure to develop an interesting hypothesis may lead a scientist to re-define the subject under consideration. Failure of a hypothesis to produce interesting and testable predictions may lead to reconsideration of the hypothesis or of the definition of the subject. Failure of an experiment to produce interesting results may lead a scientist to reconsider the experimental method, the hypothesis, or the definition of the subject.
Other scientists may start their own research and enter the process at any stage. They might adopt the characterization and formulate their own hypothesis, or they might adopt the hypothesis and deduce their own predictions. Often the experiment is not done by the person who made the prediction, and the characterization is based on experiments done by someone else. Published results of experiments can also serve as a hypothesis predicting their own reproducibility.
Confirmation:
Science is a social enterprise, and scientific work tends to be accepted by the scientific community when it has been confirmed. Crucially, experimental and theoretical results must be reproduced by others within the scientific community. Researchers have given their lives for this vision; Georg Wilhelm Richmann was killed by ball lightning (1753) when attempting to replicate the 1752 kite-flying experiment of Benjamin Franklin.[93]
To protect against bad science and fraudulent data, government research-granting agencies such as the National Science Foundation, and science journals, including Nature and Science, have a policy that researchers must archive their data and methods so that other researchers can test the data and methods and build on the research that has gone before. Scientific data archiving can be done at a number of national archives in the U.S. or in the World Data Center.
So in essence, at high level, a SIGHTED EVALUATION should;
- produce a result which can be repeated, either in a blind / double blind test or in a test conducted in other places or at when tested at other times or when tested by other people
- contribute to the body of available information on speakers and the testing (evaluation) of same [not unlike the spinorama database initiative]
- use simple (where reasonable) equipment, methods, processes, so as to lower the barrier of entry and allow greater participation - which will increase the body of information available, and if procedures are followed increase the accuracy of any results (larger sample size)
- use a control or base or reference
Lots of the discussion on this topic starts with "you are wrong, and let me tell you how and why" - maybe acceptance is a better goal, than seeking agreement or consensus (the term herding cats springs to mind). This new thread seems like a way to start with the basics and take everyone on the same journey, and will hopefully prove more effective than working from a niche of specialist knowledge or experience and decrying the efforts of others to share their specific knowledge or experience (when relevant of course), or arguing over semantics and misunderstandings.
If logic and reason are involved others can and will follow - if personal preference or anecdotal experience is the only claim then the debate rages on ad infinitum.
Depends on what parts of the subjective experience you want to evaluate. For example, large vs small speaker, or high vs low directivity: which gives better sense of envelopment? In that case, level matching would not be enough, you would want to match the entire frequency response curve by EQ, I guess, to exclude the contribution of that variable.
Similar threads
- Poll