Where is the AI audio upscaler?

[anphex]

For video, TVs have them for a while now, Nvidia and AMD have it on their GPUs,, even a streaming stick (Nvidia Shield) is capable of real time AI upscaling with pretty impressive results.
So why doesn't anyone offer this for simple music files yet? I believe it would be a godsent for the bandwith issues and produce exceptional results for files that don't even have lossless never mind high res originals.

And I can't even think of why this would be hard to make. Just pick a few ten thousand music files with high res. Compress them with lossy formats. Feed the training data and expected results to a neural network and voila - there is the AI model with 99,999% success rate in upsampling lossy music to their original state. I think you could train this in a few days on a consumer GPU even.

 

[amirm]

Look up Soectral Band Replication. It extends the spectrum of glossy codecs which have lower bandwidth than 22.4 Khz. It sounds impressive but then gets annoying to my ears.

 

[Zensō]

For video, TVs have them for a while now, Nvidia and AMD have it on their GPUs,, even a streaming stick (Nvidia Shield) is capable of real time AI upscaling with pretty impressive results.
So why doesn't anyone offer this for simple music files yet? I believe it would be a godsent for the bandwith issues and produce exceptional results for files that don't even have lossless never mind high res originals.

And I can't even think of why this would be hard to make. Just pick a few ten thousand music files with high res. Compress them with lossy formats. Feed the training data and expected results to a neural network and voila - there is the AI model with 99,999% success rate in upsampling lossy music to their original state. I think you could train this in a few days on a consumer GPU even.

320 Ogg Vorbis and 256 AAC are so good I don't see why this would be useful.

 

[ZolaIII]

Take a look at pletorija of Sony (DSEE) and Creative (can't even remember how they ware called) trough time and neither of them worked really.

 

[anphex]

320 Ogg Vorbis and 256 AAC are so good I don't see why this would be useful.

"1080p looks so realistic, I don't see why 4k would be useful".

 

[anphex]

Take a look at pletorija of Sony (DSEE) and Creative (can't even remember how they ware called) trough time and neither of them worked really.

Those were handmade algorhythms. Machine learning upscaling is a completely different approach.

 

[Zensō]

"1080p looks so realistic, I don't see why 4k would be useful".

Not even close to a realistic comparison. Numerous well-controlled blind tests have shown that almost no one can hear the difference between high bitrate AAC and Vorbis compared to Redbook and higher, particularly if not isolating very small snippets or reverb trails. 1080P compared to 4K is easy for anyone to spot.

 

[ZolaIII]

@anphex really and AI as it is now is not just extension of compute vision and dumb as hell?

Edit: by the way before last time Creative went down they developed a huge, mammoth DSP architecture for which and that task today GPU's look pitiful.
Reedit: took me some time to find it.

ZiiLABS Touts "100-core" ZMS-40 StemCell Media Processor

100 cores in a processor? Evidently that's a thing. ZiiLABS, a media processor and platforms company that's also a wholly-owned subsidiary of Creative Technology), has just unveiled a 100-core ZMS-40 StemCell Media processor optimized for Android. You read that correctly. The ZMS-40 combines 96...

amp.hothardware.com

Seams that project and ambition of it are not dead and we see them in similar form at least with new Intel video - compute GPU's.
And belowed Pi's Vcore GPU whose exactly the same thing (conceptually) in small.

 

[radix]

Denon AVR's have a "Restorer" described as below. I'm not sure that it uses AI, but sounds like it does what you're asking about.

Restorer​

Compressed audio formats such as MP3, WMA (Windows Media Audio) and MPEG-4 AAC reduce the amount of data by eliminating signal components that are hard for the human ear to hear. The “Restorer” function generates the signals eliminated upon compression, restoring the sound to conditions near those of the original sound before compression. It also restores the original bass characteristics for a rich and expanded tonal range.

 

[ThatM1key]

Take a FLAC file, compress to ogg-vorbis 320kbps and then use AI upscale to "match" the original. What's the point of that? Data storage & bandwidth is cheap. The only reason to use something like DSEE HX/AI/Extreme/Ultimate is when your stuck with a lossy format and don't mind throwing around some DSP.

 

[Trell]

"1080p looks so realistic, I don't see why 4k would be useful".

That depends on the seating distance and the size of the display how useful it is.

With UHD we got HDR and extended color gamut, which is a real improvement.

 

[ThatM1key]

That depends on the seating distance and the size of the display how useful it is.

With UHD we got HDR and extended color gamut, which is a real improvement.

The thing I never understood about HDR TV's is that they don't have proper RGB support despite having the ability to display millions of more colors. (The color profile not the cable).

When I got my Hisense H8G, I wanted to know if my TV supported RGB (Because you know it was 2020, had HDR and had quantum dot). I looked through the manual and website, nothing. I called tech support and they were more useless than a junk yard car. They told me they had no idea if it supported RGB despite them having more resources than I do. I had to use my eyes and sure enough it didn't, which was bit disappointing. I am happy that HDR is displayed well on my hisense though. Many years ago I have gotten a "Mid-range" Samsung for Christmas. SDR colors were great and it was crystal clear. The problem with it was the HDR, the TV didn't have enough nits to display it, so the colors were more washed out then SDR itself.

 

[Trell]

The thing I never understood about HDR TV's is that they don't have proper RGB support despite having the ability to display millions of more colors. (The color profile not the cable).

When I got my Hisense H8G, I wanted to know if my TV supported RGB (Because you know it was 2020, had HDR and had quantum dot). I looked through the manual and website, nothing. I called tech support and they were more useless than a junk yard car. They told me they had no idea if it supported RGB despite them having more resources than I do. I had to use my eyes and sure enough it didn't, which was bit disappointing. I am happy that HDR is displayed well on my hisense though. Many years ago I have gotten a "Mid-range" Samsung for Christmas. SDR colors were great and it was crystal clear. The problem with it was the HDR, the TV didn't have enough nits to display it, so the colors were more washed out then SDR itself.

Dolby Vision looks very nice on my LG OLED CX, and I expect that color reproduction as well as HDR will be better as years go by. It looks much better than the eighth year old Panasonic plasma display it replaced.

 

[ThatM1key]

Dolby Vision looks very nice on my LG OLED CX, and I expect that color reproduction as well as HDR will be better as years go by.

My TV also has Dolby Vision but I don't have any players that support it and my TV doesn't support eARC, so I can't just "Put it on a flash drive" like most people do (Via ripping)

 

[Trell]

My TV also has Dolby Vision but I don't have any players that support it and my TV doesn't support eARC, so I can't just "Put it on a flash drive" like most people do (Via ripping)

I don’t have an UHD player either and still thinks that 1080p Blu-ray looks good.

I mostly watch HDR content using the builtin TV apps for streaming services like Netflix, HBO Max, Disney+. Sadly, only compressed audio is available from those.

To bad Oppo exited the player business or I would have replaced my aging Oppo BDP-93 by now as I do have many SACD.

 

[ThatM1key]

I don’t have an UHD player either and still thinks that 1080p Blu-ray looks good.

I mostly watch HDR content using the builtin TV apps for streaming services like Netflix, HBO Max, Disney+.

To bad Oppo exited the player business or I would have replaced my aging Oppo BDP-93 by now as I do have many SACD.

Yeah 1080p content still looks good on our 4K TVs. The weird thing I found out with 4K TV's is that the built in apps have better color's than the HDMI inputs, even with the same color settings.

I heard OPPO's have great SACD support, in terms of SACD ripping and/or SACD-R playback.

 

[Snoopy]

Yeah 1080p content still looks good on our 4K TVs. The weird thing I found out with 4K TV's is that the built in apps have better color's than the HDMI inputs, even with the same color settings.

I heard OPPO's have great SACD support, in terms of SACD ripping and/or SACD-R playback.

I got a LG CX 65" and I tried the same apps on the TV, ps5, Amazon 4k Cube and a apple TV 4k.

The apple TV was colour calibrated for HDR content with a iPhone. Great feature by the way.

The apple TV simply looks the best. But it really blew my socks away how much better Dolby Vision content looks.

 

[ZolaIII]

The thing I never understood about HDR TV's is that they don't have proper RGB support despite having the ability to display millions of more colors. (The color profile not the cable).

When I got my Hisense H8G, I wanted to know if my TV supported RGB (Because you know it was 2020, had HDR and had quantum dot). I looked through the manual and website, nothing. I called tech support and they were more useless than a junk yard car. They told me they had no idea if it supported RGB despite them having more resources than I do. I had to use my eyes and sure enough it didn't, which was bit disappointing. I am happy that HDR is displayed well on my hisense though. Many years ago I have gotten a "Mid-range" Samsung for Christmas. SDR colors were great and it was crystal clear. The problem with it was the HDR, the TV didn't have enough nits to display it, so the colors were more washed out then SDR itself.

You have colorimetry and calibration (for both SDR and HDR) and adaptive HLG (as a part of the HDR10+) which is apply able to anything (if they didn't broke colorimetry totally in source).

 

[litemotiv]

One of the main reasons why upscaling is a lot easier on video than on audio lies in evolution, and the different ways our visual and auditory cortices work.

Our vision is optimized to track larger patterns and is very good in ignoring small artifacts. For instance a hunter in an ancient forest would want to be able to track an animal moving through the trees, but not be distracted by small details such as a beetle sitting on a tree branch. Upscaling favors this task, by increasing contours and contrast our eyes can accommodate more easily, and residual artifacts are relatively easy to ignore. For this same reason most people can watch a film with subtitles without problems, after a short while the subtitles become almost invisible and are processed mostly subconsciously.

With audio this works the opposite. Our hearing is most sensitive to small anomalies in ambient noise. For instance at night the ability to hear a small twig snap in the distance, or the rustling of a snake through the leaves, can mean the difference between life and death. Increasing resolution has an opposite effect here. Increased contours and artifacts will reduce ambient levels and demand more attention. This will quickly cause unease and fatigue, and is impossible to ignore because our brains are predisposed to focus on them.

 

[ZolaIII]

@litemotiv and after all of this time we still have it (to a less extent than slapping predator as cat but still there).
By the way that's the reason why so far all of them (attempts) failed (felt sibling and harsh in highs where they added the most).
On the other hand you can have lossy unrecognisable and not distructive better packed as it is or tighter packaged losseles if you wish (PCM).