A diffuser distributes the sound evenly. Scattering doesn't.
http://www.lydogakustikk.no/?p=2137
I have heard the argument many times. What is lacking is controlled testing to show the audible preference.
Also keep in mind that those are theoretical arguments. In reality diffusers are not ideal devices and at any rate, are routinely misapplied. A broadband diffuser will be deeper than many people deploy and requires a healthy distance away from it for it to work like the textbook says. Those conditions are rarely met.
Here is the reality from Dr. Toole's book:
We see that the diffusion coefficient is frequency dependent.
In addition, speakers don't have same directivity at different frequencies. Here is a random example showing the same 4000 Hz per above:
https://tonefreqs.files.wordpress.com/2013/10/e-606-polar-pattern.jpg[/iimg]
What is delivered form the speaker is very different from forward to backward direction at that frequency. So if you put that diffuser on your front wall, it is not going to see an omni directional sound power as they do in lab testing.
And the angle hitting a diffuser may very well be different for each speaker (e.g. on a side wall). Or the secondary reflections.
Bottom advice is this:
1. If you are building a dedicated listening room that starts with an empty box, by all means use dedicated acoustic products.
Here is for example the side wall in our work theater while under construction (depth is about 12 inches)
2. For everyday listening spaces, products like above would be ugly and impractical. So opt for natural furnishings which accomplish the same thing.