That's certainly one way of looking at it, but again, it's not thinking like a research scientist would. That would be dismissing data because it doesn't fit your original hypothesis. Maybe that approach works in engineering? It's very frowned upon research science.If none of those food critics can identify that burnt flavor unless they know the food was cooked on stove A ahead of time, I don't care about their subjective opinion because it's false. We already have a hypothesis that explains such situations - cognitive bias.
Rather, a research scientist would say, "it's possible that the food critics tasted a burnt flavor because they knew the identity of Stove A. Let's test that by blinding the food critics to the identity of each stove, and see if we get the same results." This is called "controlling for bias."
By the way, I'll skip to the end. What actually happened in this hypothetical example is that the the thermocouple was place in a position that measured an average temperature, and the average temps of each stove were identical. However, Stove A had hotspots that made certain parts of the frying pan hotter than others, whereas Stove B had more even thermal distribution. Hence, Stove A was actually causing some parts of the food to be burned.
If we had taken the "engineering" approach, we would have taken the "measured" data at face value and completely dismissed the overwhelming "subjective" signal (in this case, 90 food critics out of 100 tasting something burnt when the food was cooked on stove A). And in doing so, we would have overlooked an important difference between Stove A and Stove B that actually existed.
Last edited: