I have an old (>10 years?) Dell Optiplex 3020 mini desktop PC that has an i5 CPU, 8 GB of RAM, and a 128 GB SSD that is setup with a headless install of ubuntu server, Jellyfin, and LMS. It has been 100% reliable for about 5 years now, requiring only occasional updates (sudo apt upgrade). There's an external box with 3 HDDs for media storage plugged into a USB port. I have an Nvidia Shield Pro connected to the TV, and use Wiim Amp Ultra, Wiim Amp Pro, and a Wiim Pro Plus for playing music.
I used to use Plex for video, but it turned into a PITA when they decided to become a streaming platform and I had to log into their servers every time I wanted to play stuff on my HDD. Jellyfin is every bit as good as Plex ever was, and there are no issues with logins. It's entirely possible I had Plex configured incorrectly, but I see no reason to ever go back.