Awesome, hadn't seen that thread before. Will review tomorrow and work to give you an apples-to-apples plot then.
Although not the only training, of course (just doing dozens of ABX on file formats is another form) but I would recommend checking Sean Olive's blog posts on the Harman training software. There are a few posts here but the easiest to find is HERE.
I definitely can't speak for @amirm but yes, I would strongly disagree with that. There are really only two possibilities:
Just for fun, let's forget for now the concept of skill and training, and the concept or spotting differences. Let's say we are talking one subject, with it's set of skills, good or bad, and one single product. Would you claim that the objective evaluation of all the aspects of the performance of a product is more accurate after one test, under one controlled condition than after 1000 test under random conditions?Although not the only training, of course (just doing dozens of ABX on file formats is another form) but I would recommend checking Sean Olive's blog posts on the Harman training software. There are a few posts here but the easiest to find is HERE.
I definitely can't speak for @amirm but yes, I would strongly disagree with that. There are really only two possibilities:
1) There is an obvious audible difference between the two DUT's - in which case it should remain so regardless of repetition. In fact, since you would almost certainly be much more focused in a zero-day test than you would after 10 years... it might even become inaudible to you eventually.
2) There is no significant audible difference at all. This means the preference was caused by something other than something being heard, and thus could easily present itself at any time during the 3000 day period or not.
In either case the odds of listening as carefully at the end as one did at the beginning are incredibly small. That's why people (myself included) are easily convinced of nearly any tweak making some difference in sound... because we started listening for it - and our brains take care of supplying it. That happens regardless of any actual differences. Hence the need for measurements by machines which are impervious to such limitations.
Yes I don't mind taking this elsewhere, Note than I'm only commenting and replying tough, I didn't asked "the first question", I was participating in a discussion that our host himself was engaging, in this very thread, with statements that I disagreed on. But guide me and I'll answer there. We don't have to agree on everything, but reading what you say I don't know if we are even talking about the same thing. Listening tests. The "margin of error of the test equipment"? your ear?The "durability of the analysis equipment"? you mean testing if you are going deft? Random conditions is not about repeatability and comparison, it's about extending the scope of analysis. If you always have the same input in the same environment, with the same equipment under test, yes, obviously, you'll always get the same output. But you'l know nothing on how it performs with limit cases, under stress, etc. Anyway...@PeteL I'd recommend moving this discussion to a different thread (there are many on here already - perhaps one of the threads regarding listener training, audio memory reliability, etc). In the context of this thread there's a perfect example of the problem in your first question in the preceeding dozen or so posts:
If the conditions are random, there is almost no repeatability... and therefore the results can't reliably be compared to previous results. Presuming there is no error in either equipment, parameters, etc. it should be possible to replicate the previous results (outside the margin of error or resolution of the test equipment itself). Just like statistics, by modifying the parameters sufficiently it may be possible to achieve significantly disparate results... which is why the control of as many variables as possible in every test is of paramount importance.
Once truly controlled, any more than 2-3 passes (in a case where the validity of the results is in question) should make the source of the anomaly apparent. I can think of no case in which 1000 tests under random conditions would serve any purpose other than to test the durability of the analysis equipment, and the patience (or obsessive compulsions) of the operator.
Again, sorry, are we still taltking about LISTENING tests?
This will be short as it's been a busy week (is it only Wednesday?) We are basically done with our internal evaluations. I wrote a 2-page summary plus a set of design criteria today and will publish it shortly for the community to see. It will explain the reason for slightly elevated THD (hint: A digital volume pot was required to enable perfect channel tracking for balanced armature IEMs).
I did not say that at all. I explained why your FFT was not a good way to read the dynamic range from. If the driver had worked, I would run these tests:
This point is also incorrect. I don't mind the look of it. The thing I take issue with it is that it pretends to be a "Pro" product by copying the look of such devices but without delivering the performance. And making the device harder to use for consumer applications. Kind of reminds me of the El Camino:
This point is also incorrect. I don't mind the look of it. The thing I take issue with it is that it pretends to be a "Pro" product by copying the look of such devices but without delivering the performance. And making the device harder to use for consumer applications. Kind of reminds me of the El Camino:
Cutting the back open doesn't make a sedan a truck!
I'm not a fan of the design either... but I wouldn't kick that one out of my garage. LOL.Thank you for the nice picture. We appreciate that you can inject levity into the discussion and deviate from the "science above all" position when discussing visual preferences and customer use cases. We hear you that beauty is in the eye of the beholder. To each his own, as they sayThis point is also incorrect. I don't mind the look of it. The thing I take issue with it is that it pretends to be a "Pro" product by copying the look of such devices but without delivering the performance. And making the device harder to use for consumer applications. Kind of reminds me of the El Camino:
Cutting the back open doesn't make a sedan a truck!
All of which is readily apparent in a simple FR sweep (and maybe a waterfall if it's a resonance vs. midrange peak causing that "shout") viewed prior to auditioning in the first place, right? And yes I'm familiar with BOSE displays and yes they sold millions of them. More to the point would be that unless the speaker was horrible... DSP and/or room treatment could be used to tame all but the most egregious disasters - not to mention that sales floor "auditions" are literally the opposite of a controlled environment evaluation.
Sure, we'll leave it there, but I don't know who's moving the goal post when you keep willing to disagree with me on stuff I never disagreed with myself? And always coming back to measurments when it was never about this. I don't mind being disagreed with, and love being proved wrong, but in this argument it's about being misunderstood, and in these case I guess I'll just have to accept that it's my bad, that my communication skills are poor I guess.
LOL. 1. Agreed.
You might even hear something differently depending on how hungry you are at the time. That's the inherent problem with all subjective analysis. However, that is the problem of the listener... not the devices in the signal chain. So your point in the context of producing a review for publication by a single individual is confusing (to me at least). There is no possibility to ever test every potential... and even if there were the results would be meaningless in any other context, and certainly to any other person.
