It occurred to me what happens when I do thought experiments is likely what many others find. Thought experiment by itself isn't enough, but can lead to getting further down the road before actually experimenting.
Yes, that's what I think, too. If any of us is to ever to do something unique, or to actually make real progress in any field, we can't spend our time doing useless experiments. In another thread it was stated quite matter-of-factly the other day that the the way to prove which digital link "sounds best" or whether they are all equivalent, is to perform DBTs. I would say: what a waste of time! You could avoid several days' work and several months of arguing over the interpretation of the 'results' by simply giving it a minute's thought and concluding that they all sound exactly the same.
Or some of them are things I can never figure out how to test reasonably.
I could think of an example: suppose you wanted to build an innovative new system that compensated for variations in your drivers' mechanical characteristics by modifying the signal according to temperature, humidity and long term ageing. It's an idea that can be stated in a sentence, designed in a week, built in a week, and in this case, you would
know that it was going to give you less measured error overall. But you would have to decide whether the system should operate dynamically while playing music, or wait until in between tracks, and at what rate it can make changes, what the limits should be and so on. As the designer, you could spend the rest of your life testing all the possible options using DBTs, or you could just build it and put it out there, anyway.
Where this sort of thing goes wrong:
http://www.stereophile.com/content/schiit-audio-ragnarok-integrated-amplifier#hZUKUTzoE4zfiXVY.97
A designer thinks of an innovative new system (microprocessor-controlled amplifier biasing and protection) but implements it in a way where test tones must be regarded as a fault condition thereby sending the amp into protection mode. By the power of Thought Experiment, he thinks that this is acceptable. Wrong! (In my opinion). In fact his idea may have been perfectly reasonable, but maybe the implementation was over-simplified. Maybe just by 'thinking it through' but not testing it properly, he got too far down the production process before the issue became apparent. (But that is just conjecture, of course).