Okay, so today I tried 2 things today.
First I disabled eARC on my TV because I read in the WiiM forum that that somehow sometimes helps and the WiiM doesn't support eARC, only ARC. Asly it didn't make any difference to me.
Then I had another idea yesterday that I tried now and that surprisingly...