Eyes & ears pick up radiations-light waves or sound waves, respectively. There's no physical material in them. Your tongue and schnozzolla pick up actual molecules--physical material. It will be harder to fool those senses.
While conceptually fun, that does appear to be a bit dubious. Matter and energy are interchangeable, and we convert stimuli into electrical signals (and store them in calcium structures and etcetera) in the brain. So ultimately the same thing, perception-wise? I'm open-minded though: any interesting reading to support the argument of matter stimuli being more reliable than energy stimuli in human perception?
Edit: that'll teach me to write a post then get distracted before pressing the button ... I'll be reading @JaMaSt's stuff for a bit.