Our brain has a break-in period too. But I think it entirely possible that woofer surrounds and spiders might change characteristics over time with use. Midrange drivers & Tweeters? I don't see it but I would like to see studies on this topic- maybe they too "wear in." Surely speaker system or raw driver manufacturers have this data.
There could be some slight break in of woofers but mostly it's your brain getting use to the sound.
I can tell you after twenty years when the woofer surrounds start to crack and break that will definitely sound different.