Any stories of fooling ones self will do, don't get distracted by the word tweak. That was just a easy example to relate to.
The forum software will overflow if I told all the stories of mine.
Here is one that I have told elsewhere.
When I was at Microsoft, I managed the audio/video signal processing/compression teams among others. I trained myself to hear artifacts well and could outdo everyone on the team in this regard, hearing artifacts that others could not. All of which were found to be problems that were fixed.
One time I am home sick and volunteer to do more testing for the team. They were playing with different psychoacoustics algorithms for the WMA encoder. They sent me a before and after and I was horrified how much worse the new encoder was. Not being in good mood physically, I dumped on the manager for the group for not hearing such obvious degradations. The manager was pretty surprised saying they had not made any changes that would make such profound differences. I listen again and it is pretty clear to my ears that they indeed had broken the thing, fidelity wise. The manager then sends me another set of files and asks me which set I like, "A" or "B." I listen and immediately identify one as being the latest encoder with the same degradation. I was full of confidence 'till I get this horrifying response back from him, "Amir, both of those sets of files were identical; I just renamed them as two different names!" I could not believe what he was saying. Sadly that was easy enough to compare: a binary comparison shows the two sets to be identical, bit for bit. Being played through the same system, they could not possibly sound different.
I go back and re-listen and I hear them being the same. But thought, I wonder if I can hear into them what was different. I do that and the difference all comes back! Indeed, I could change my frame of mind one or the way other and can make the difference vanish, or come back on demand, damn the fact that the files were identical.
Needless to say, I told them to rely on their own testing than my opinion and they did that, and indeed improved the encoder.
These events are sobering slap in the face but necessary for one to realize what the probability of their hearing system being right is. Here I was, the king of the hill in my critical listening ability, yet I fell victim to misperception like nobody's business. For people without this type of training, there is no hope. There just isn't. Vast amount of our fidelity judgement as a group are therefore wrong. They are far more wrong, than they ever are right! To walk around otherwise, would be having one's head deep in the sand.