• WANTED: Happy members who like to discuss audio and other topics related to our interest. Desire to learn and share knowledge of science required as is 20 years of participation in forums (not all true). Come here to have fun, be ready to be teased and not take online life too seriously. We now measure and review equipment for free! Click here for details.

What Does DSEE HX AI / DSEE Ultimate / DSEE Extreme Do?

Joined
Jul 18, 2020
Messages
46
Likes
16
Location
Australia
#1
Without a DSP called DSEE HX AI, audio seems blended with collapsed soundstage. Basically smoother vocals and transient attack is what DSEE HX already had, whereas ‘height’ is by AI…

You could try out these audio players with some headphones and mp3s. It should bring out details - restore musical details and nuances - that random players can't achieve, and with DSEE HX, it should sound less digitally compressed and more open. In some cases mp3s will sound no different to its Hi-Res equivalent! I am an expert in DSEE HX and it has been Sony's consumer audio jewel since 2013. Ayataka Nishio who created DSD, along with Chinen Toru and others, together made DSEE HX in 2013. At first they used separate algorithms for each segment of songs but for ease of use, lower risk and greater stability they 'averaged' it out to a single one-size-fits-all. This was then tuned for different regions/countries and for different products. But it is mostly useless and makes no audible difference despite the extra audio data. Still, larger files naturally sound better due to the possibly faulty mechanism of most DAC chips' 8 times oversampling. Now DSEE HX AI is able to analyse the song in advance and apply a multitude of different algorithms to match each segment of a song. So it is closer to the original idea Nishio-san had but not yet perfect.

Instructions:

Software for DIY 'Harmonic and Bit Extension': wpup.html.xdomain.jp/

Also for 'DSEE HX' on Windows PCs : musiccenter.sony.net/



Click Add or Import files to add mp3s etc


Use WASAPI (or ASIO if you can) as the output setting to bypass the kernel mixer and achieve bit-perfect performance. Any PC Realtek HD audio should be ok but if you have a USB DAC it would be better. For WPUP use 32 bit to avoid white noise ( Realtek has that problem with 24 bit). For Sony's MCfPC it works without problems using their old 2013-2018 DSEE HX algorithm. It is not DSEE HX AI but we're still asking them to upgrade it.

More about DSEE HX AI

Since 2018, DSEE HX AI, now with variants: DSEE Extreme (AI for 192kHz for wireless headphones) and DSEE Ultimate (AI for both 32 bit and 192kHz first used in Xperia 1 Mk2) has improved upon the old DSEE HX a lot. DSEE HX AI is like those software algorithms that can convert black and white photos/films into Color photos/films but works on audio data instead. It's really almost like a very experienced sound engineer is remastering your music collection in real time.

What Sony has done is they ran simulation models of what is lost during the conversion from their studio master DSD/DXD files and the end result consumer PCM / AAC format that we usually listen to. Then they came out with this DSEE HX algorithm that can restore the audio back to near studio master quality. It can deliver a sound closest to what you experience when you attend a concert in real life.

The old DSEE HX achieves smoothing only.

DSEE HX AI Effect has two parts:

One is the soundstage part which adds a natural sounding soundstage height feel to your songs that has recorded soundstage information to begin with. It doesn't seem to add any additional soundstage to songs that are electronically synthesized(eurodance music).

The other part is the transient attack and smoothing part. Vocals sound smoother and Transient sound e.g. Cymbals, sounds more immediate and drums sound more impactful.

And this is how I find the DSEE HX effect varies with file resolution:

Soundstage height improvement shows up in all lossy, normal 16bit resolution and high resolution music. DSEE HX will alter how the soundstage sounds for as long as the original music content has some form of soundstage info.

As for the smoother vocals and transient attack. With higher resolution files, DSEE HX doesn't seem to add much of a noticable difference as the music is already smooth and dynamic to begin with. As for normal 16bit resolution and lossy files, that's where DSEE HX effect has the most noticeable improvement to the sound.

Sony engineers are beyond others, however their products are a lot different from the real prototypes - they are more like what you would call a “watered down” versions. Only Chinen Toru has the full DSEE HX AI that uses 100% CPU power to achieve the best quality. And he won't make it public as Sony guards their code strongly.

It's not any old SBC (see https://www.audiosciencereview.com/...led-248k-transparent-to-high-resolution.1441/ further down the first page) from decades back, but it's not audio magic come true either. Sony isn't achieving DSEE's potential, and ridiculous names like 'DSEE Ultimate / DSEE Extreme' thrown around now to get customers who are attracted to upscaling audio are simply disappointing. At the minimum, the real DSEE HX AI that uses 100% CPU power must be made available to all. This sort of thing was easy back in the day of Sony VAIO Preinstalled software. Anyone who knows SonicStage Mastering Studio and its WAVES/Sonnox plugins knows that the real DSEE HX AI that uses 100% CPU power made available to all is the only way forward. Not a low-energy weakling with an overblown name: 'DSEE Ultimate' on Xperias! It's wrongly named!
 
Joined
Jul 22, 2020
Messages
12
Likes
1
#2
Will Sony add the new functionality for “AI” to the Music Center for PC app? Now Music Center just has basic DSEE HX. I leave it in does not seem to hurt. I use Sony Music Center on Windows since have a Sony Walkman NW-ZX1. Feed the PC to a PHA3 Dac via USB. Ok headphone amp, but line-out not so good for stereo. The PHA3 has the DSEE HX, but doubt upgradable. The WALKMAN never got an Android update. So no Amazon app , new YouTube among many others.
 

RayDunzl

Major Contributor
Central Scrutinizer
Joined
Mar 9, 2016
Messages
10,812
Likes
10,244
Location
Riverview FL
#3
I haven't been able to find a before/after demo track.

Is there one?
 
OP
Saidera
Joined
Jul 18, 2020
Messages
46
Likes
16
Location
Australia
Thread Starter #4
Demo? Not really.

 
OP
Saidera
Joined
Jul 18, 2020
Messages
46
Likes
16
Location
Australia
Thread Starter #5
The official verdict is that DSEE or WPUP and others claim to replicate or restore but because it doesn't need to be strictly accurate, only an approximate guess is necessary to create better sound. By using AI algorithms it may do the job better but not enough. DSEE is useful for the consumer, and even businesses seem to require this sort of technology (Bambu). Since HE-AAC is not mainstream, DSEE becomes necessary. The changes made to the sound are so subtle that changing speakers or headphones to IEMs or using various IEMs of different characteristics will make you feel DSEE is a world of difference, or absolutely no difference at all. (It can separate instruments and so on, especially if tuned specifically for an IEM but if the IEM does not reproduce too much detail or makes bad and good sound sound alike, then DSEE makes no difference in the end).

How to check whether WPUP or DSEE are actually creating ‘good sounds’ in the lost frequencies?

High pass filter a file in Audacity and listen to the high frequencies only.

https://www.audiosciencereview.com/...led-248k-transparent-to-high-resolution.1441/

https://books.google.com.au/books?i...Q#v=onepage&q=Coding Technologies aac&f=false

Swedish company Coding Technologies AB: SBR is the ancestor and used in aacPlus – now MPEG-4 HE-AAC
also see
https://en.wikipedia.org/wiki/Mp3PRO

Lars Liljeryd, Kristofer Kjörling, and Martin Dietz received the IEEE Masaru Ibuka Consumer Electronics Award in 2013.
Clearly it is because of their 2013 award that DSEE was born – it reminded Sony of it? or a coincidence!
https://web.archive.org/web/20101130235115/http://telos-systems.com/techtalk/aacplus/aacPlus_overview.pdf
be creative, innovative, don’t just listen to those who say no.
those who like music and like technology created SBR.

WMA 10 Professional LBR:
https://web.archive.org/web/20070430153905/http://www.edn.com/index.asp?layout=article&articleid=CA6391444&industryid=22043

Details of mp3: https://www.ime.usp.br/~rbc/lixo/ch02.html

I say by 96 kbps you have flat, muddy, damp but not harsh sound in complex music encoded without enough CPU usage (DSEE then makes it harsh, spiky, in the high frequencies after restoration!)

DSEE doesn't fully address these:
  • Decreased Frequency Response
  • Lower Dynamic Range and Mutated Dynamics
  • Loss of Frequencies
  • Frequency Anomalies and Distortions, Swirlies
  • Pre- and Post-Echoes
  • Audio Content Timing Errors (Roughness & Double-speak)
  • Loss of Transients and Sound Details
  • Loss of Stereo Imaging and Panned Details
  • Phase shifts
  • Noise addition

Good quality MP3 can sound identical to the original PCM or DSD?

There is a free exciter plug-in for Audacity (also free) if you want to play-around with it.

So "psychoacoustic exciters", "harmonic exciters", or "enhancers" came first, then Spectral band replication, but now DSEE is the best option available?
https://www.soundonsound.com/techniques/what-data-compression-does-your-music

Bambu Tech appears to boast FM radio quality through AM!
https://www.bambu.tech
But is that technology, like DSEE, based on a mistaken premise?
'A 2015 study by the Consumer Technology Association found that 82% of consumers describe sound quality as critical when selecting a product.'
Audio quality matters. Really?
AWSM first adds stuff, then removes noise stuff, and it only has to be done once at either end? Clearly to do so at the end is advantageous? DSEE AI uses a strong Edge AI algorithm, but what AI does AWSM use? How does this compare with WPUP?
 
OP
Saidera
Joined
Jul 18, 2020
Messages
46
Likes
16
Location
Australia
Thread Starter #6
Now some who have used Sony's WH-1000XM4 headphones say 'DSEE Extreme feels better than wired high resolution itself.' Wireless is now on par with wired headphones thanks to DSP!

I honestly didn't find any difference in sound quality with SBC connection. It is necessary to measure the difference between AAC/SBC/LDAC?

Compared to wired headphones, sound quality is almost the same on Wh-1000xm4.
So for those who value sound quality, wired headphones are sufficient.
Only convenience favours WH-1000XM4.

Clearly it is virtually impossible to measure DSEE performance from a headphone's DAC.

What changed was the sound field, or the sound spread compared to the M3.
Perhaps this is the effect of DSEE Extreme installed from M4.
When I turn off DSEE Extreme, I feel the sound field narrows.

However, I think that this DSEE Extreme installed from M4 is a little troubling.
Unlike the previous DSEE HX, DSEE Extreme seems to use a deep learning model for upsampling ...
Sound sources with features not covered by the dataset used to train this deep learning model will be significantly distorted. This is a problem that cannot be easily avoided due to the mechanism of deep learning.

To give an example that you can see at hand, when you play a piece of music played by an orchestra in the OST of a game, the percussion sounds as if they were played through cheap speakers.
Probably not a problem if it is a recorded sound source of a normal orchestral performance (because it is definitely included in the dataset), but since this example is a sound source mastered for games, such distortion occurs. I'm expecting it to be because there are few games that release OST with high resolution sound source in the first place.

On the other hand, pop music seems to be quite compatible with DSEE Extreme.
(Although these genres have almost unnoticeable artifacts even if the sound is slightly distorted by DSEE AI...)

Therefore, I think it is wise to enable DSEE Extreme only when listening to a genre where many high-resolution audio sources are distributed.
Or even just for listening to Sony Music audio, since their 'Edge AI' relies on their database.

It is interesting to note that the AI on A55 didn’t lead to revolutions happening, nor did ZX507’s which has flaws for vocal sibilances etc although clearly the ‘height’ in live albums and the vocals’ transients were always noticeably better (whereas old DSEE made NO difference) e.g. XM2’s DSEE is what’s on the PC software – good DACs cannot allow me to hear the high frequencies well, only IEMs can, when used with Apple dongle on 24/44.1 while XM3’s DSEE is stronger and works only with oldies and live tracks to create obvious improvements (it is tuned just for XM3). And XM4 is 24/96 so it’s not that great but so many people find it useful and the difference is always obvious on any track. It’s convenient in noisy environments! What is funny is how Walkmans ZX2/ZX507 win against Xperia 1 II but XM4 can surpass some headphones and tie with 1AM2 using WM1Z…on the condition that you use AAC or LDAC.

It is still very hard to determine whether the hardware or software exerts more power over the ultimate analog output sound. DSEE makes the case for software.
 
OP
Saidera
Joined
Jul 18, 2020
Messages
46
Likes
16
Location
Australia
Thread Starter #7
There are so many titles for Sony's spectral band replication: digital sound enhancement engine; dsee; dsee extreme; dsee hx; dsee hx ai; dsee ultimate.
All this fuss over just another harmonic exciter.

I propose: PORTING DSEE HX AI to the next Music Center for PC v2.4! Audio enthusiasts must show Sony Corporation V&S division that this is necessary at all costs.

Porting DSEE HX AI to the next Music Center for PC v2.4 will increase users of Music Center for PC and increase the standard of audio experienced by many people. Especially those who use lossy formats regularly. It is uncertain whether any DSP is on par with or can create the same effect as Sony's spectral band replication.

To better understand lossy formats and how to overcome their defects, let's tell Sony Japan to port DSEE HX AI to the next Music Center for PC: http://chng.it/SDjvHxcbxL

Otherwise it is near impossible to analyse DSEE HX AI. The HX stands for 'High(er) Extreme' but who knows.
 
OP
Saidera
Joined
Jul 18, 2020
Messages
46
Likes
16
Location
Australia
Thread Starter #8
Will Sony add the new functionality for “AI” to the Music Center for PC app? Now Music Center just has basic DSEE HX. I leave it in does not seem to hurt. I use Sony Music Center on Windows since have a Sony Walkman NW-ZX1. Feed the PC to a PHA3 Dac via USB. Ok headphone amp, but line-out not so good for stereo. The PHA3 has the DSEE HX, but doubt upgradable. The WALKMAN never got an Android update. So no Amazon app , new YouTube among many others.
I answered your request belatedly. Sony WILL add the new functionality for “AI” to the Music Center for PC. I explain:

What Sony prides itself in is hardware - this tradition continues today. To keep DSEE latched to or caged in hardware in an attempt to sell more hardware simply cannot be permitted by us Sony fans. After all, look at all the Sony products measured on ASR! Which one really got highly recommended? The overall feel is disappointing. Little has changed since. DSEE is locked into hardware because without DSEE, Sony hardware wouldn't be able to independently stand or deliver 'experiences'. Us Sony fans want to see that Sony hardware engineers take action to improve and learn to independently stand without the help of software.

Sony has always been extreme, no iTunes has ever put any 12 Tone Analysis or DSEE into their players. Sony did. Sony VAIO had VST plugins of WAVES and Sonnox. It could play and record DSD via ASIO. They put SBM/Direct into VAIO software as well. Now, since DSEE HX was released in 2013 it took until 2018, 5 years for it to be ported, and only because Xperia DSEE HX was ported in xda and DSEE AI was born.

History's course will be changed however. Without intervention, DSEE AI will be ported in late 2023 or beyond. No way.

No way.

If xda ports DSEE AI now from Xperia 1 II, and the next DSEE, which could be a VST plugin or a customisable AI program for professional mastering, gets released now, it can happen. But the determinative factor is whether there is overwhelming demand for DSEE AI usage on a PC. For this, a grassroots bottom up approach is warranted. http://chng.it/SDjvHxcbxL is one aspect which could show the President of the Video&Sound division that DSEE AI should go onto PC software.

Such a small inconspicious SBR, yet so important, particularly for poor listeners who can't afford expensive hires tracks! Sony hardware engineers must improve! They cannot hide behind DSEE AI and consider their work finished. They must be exposed.

I also acknowledge that most people on ASR have dismissed DSEE as irrelevant because true hires and lossy are all they need. There simply is no middle ground with 'fake hires' born out of lossy streaming. That method must be created, and I am doing so.

And think about it - PC software can't stream, so by porting AI, what harm has been done to sales?

Incidentally, although the links were taken down, NW-A45 and ZX300 models have mods which add DSEE HX AI (from DMP-Z1), DC Phase Linearizer, Vinyl Processor, Spectrum Analyzer/Analog VU Meter/Digital Peak Meter: https://thewalkmanblog.blogspot.com/2020/08/custom-sony-walkman-firmware-by.html

I wonder why the Android version of Sony Music Center has
VPT Surround
ClearAudio+
but Sony Music Center for PC only has DSEE HX.
 
Last edited:
Top Bottom