Ears (lots of them) were used by Dr. Toole (and others) to get us where we are today, and that was done in a scientific way (as much as that is possible). Now we know many things that allow us to draw some conclusions from measurements and data, that's for speakers, for electronics ears are needed only to enjoy the music, measurements can tell you most (near all) of the story.
As for trusting your ears, have a look at the wine, violins and what have you blind tests and the result there. Here is a simple video of something similar: