• WANTED: Happy members who like to discuss audio and other topics related to our interest. Desire to learn and share knowledge of science required. There are many reviews of audio hardware and expert members to help answer your questions. Click here to have your audio equipment measured for free!

HDCD in Practice

MRC01

Major Contributor
Joined
Feb 5, 2019
Messages
3,424
Likes
4,030
Location
Pacific Northwest
In summary, as I understand the HDCD spec, it uses the LSBs of each sample as a bitstream to encode flags telling the DAC to trigger various functions that the HDCD spec requires to be programmed into the DAC, like different AA filters or amplitude shaping curves.

I'll ignore the AA filter flags because they should be essentially inaudible and even if you did decode them, you might want to ignore them and use the standard linear phase "sharp" filter all the time anyway.

The amplitude shaping functions are more interesting. For example, consider an extended quiet section of music. If it's 36 dB below full scale, the top 6 bits are all zero so they only have 10 bits of resolution (even less in the treble, due to the spectrum of music). In post-processing, they can digitally encode this quiet section at a higher level, getting more bits per sample. If you play it back without HDCD encoding, the quiet part is simply louder so it's like dynamic range compression. As the quiet part arrives, HDCD encodes a flag that tells the DAC to reduce the amplitude. The DAC is working in 24 bit so it can shift it down without loss of resolution. Then as the music gets louder again, the HDCD encodes another flag telling the DAC to "go back to normal".

Several of the CDs in my collection happen to be HDCD encoded. I've decoded them with various PC software and found that none of them actually use this feature. If this is common, then HDCD seems like a marketing scheme to get companies building DACs to pay license fees, and light up the HDCD icon in the DAC display to make consumers feel happy. Or, is this amplitude shaping feature commonly used, and I just have a poor sample of CDs? Or maybe my decoding software is missing the flags?
 

Wes

Major Contributor
Forum Donor
Joined
Dec 5, 2019
Messages
3,843
Likes
3,788
It does several things (or so I am told)




Components of HDCD for both Pacific Microsonics converters (Charles Hansen)





a) On-the-fly selection of the low-pass anti-aliasing filter. The machine would analyze the content of the music being converted and choose one of two filters to achieve the best sound. This does NOT require any decoding.





b) Selectable dither levels and shapes. This was quite advanced for its day as most people were not aware of the importance of proper dither in capturing low-level signals. Again, there is NO decoding required or even possible for this step.





c) Perhaps the heart of HDCD was the use of two separate “compansion" (compression/expansion) methods. The idea is that the dynamic range of the music is compressed during the recording phase and then expanded back upon playback.


c1. Low-Level Extension (LLE). When the peak music level reaches -45 dBFS, the system boosts the gain by 0.5 dB. As the level drops the boost increases, to a maximum of 4.0 dB at -63 dBFS. Was rarely used.





c2. Peak Extend - it is active at very high signal levels. It really should be called "Peak Compression", because that is what it does. It takes the top 9 dB of the recording and compresses it down to fit into 3 dB on the CD. Then an HDCD decoder will expand it back to the full 9 dB of range. This is probably the only feature that might require decoding to achieve the best sound on an HDCD disc. I say "might" because unless the decoded music reaches a level of -9 dBFS, the decoder won't do anything. About half of the albums I examined that use Peak Extend (PE) don't ever reach levels that high. Examples include most of the early Joni Mitchell catalog. The other half of the albums that use PE have peaks that reach as high as -1 dBFS and will definitely benefit from decoding.



Simply putting a disc in and seeing the light is not enough. The only way to know for sure is to use Foobar with the latest HDCD plug-in. PE is very simple to interpret -- it's either on or off. LLE is a bit trickier. Watch the gain setting in between tracks for a quick indication of whether it was used.





most HDCD's are really closer to 16 bits than 20





https://www.head-fi.org/threads/hdcd-list.65414/page-35
 

Guermantes

Senior Member
Joined
Feb 19, 2018
Messages
484
Likes
561
Location
Brisbane, Australia
There is a list of analyzed HDCDs with their enabled features (peak extension, gain and filter, etc) over at Hydrogenaudio:
https://wiki.hydrogenaud.io/index.php?title=List_of_HDCD-encoded_Compact_Discs

I have a few HDCDs that use the peak extension feature but little else. I've used the foobar2000 HDCD component to convert them to 24 bit files.

I also have a Musical Fidelity X-DAC that actually has the Pacific Microsonics PM100 inside but it has some issues at the moment and is out of service. From memory it sounded fine with HDCD encoded discs.

I'm sure @amirm could provide some info from the Microsoft perspective, if he is willing ;)
 

amirm

Founder/Admin
Staff Member
CFO (Chief Fun Officer)
Joined
Feb 13, 2016
Messages
44,370
Likes
234,414
Location
Seattle Area
I'm sure @amirm could provide some info from the Microsoft perspective, if he is willing ;)
Just came back from digging trenches in our garden for sprinkles so definitely not that willing at the moment. :)

One line summary of what HDCD aimed was to encode 20 bit content in 16 bit CD format. They took the advantage of the fact that the dynamic range of music is not always 0 to max. So they used a sliding scale to shift the available 16 bit of data up and down (volume expansion). This signal which is a few Kilohertz/second is encoded in the rightmost bits. The decoder extracts these bits and applies the range shifting automatically. Non HDCD decoders just lose the low order bits to noise.
 

Trouble Maker

Addicted to Fun and Learning
Joined
Jan 6, 2020
Messages
676
Likes
709
Location
Columbus, Ohio, US
So, basically leveraging a fixed point calculation into slightly floating point by using last bit (or last few bits) to encode what the LSB is?
 

Wes

Major Contributor
Forum Donor
Joined
Dec 5, 2019
Messages
3,843
Likes
3,788
I've also heard that the main benefit was the use of the Pacific Microsonics A to D, which was said to be exemplary back in the Late Holocene
 

tmtomh

Major Contributor
Forum Donor
Joined
Aug 14, 2018
Messages
2,636
Likes
7,494
@MRC01 , I get why HDCD could seem like a "marketing scheme" because so few HDCDs actually use Peak Extend, which is the main feature that produces an audible difference. However, my understanding is that the Pacific Microsonics A-D converter had settings/switches to enable or disable all of HDCD's features - so every HDCD ever produced had the ability to use Peak Extend (and therefore encode 20 bits' worth of depth into a 16-bit signal). But the reason most HDCDs didn't do that is simply because the mastering engineers didn't enable the feature. I would speculate that the most common reason that they failed to enable Peak Extend is a reason that we can find ironic in retrospect: for most of HDCD's life as an active technology, the Loudness Wars were not really a major thing yet. So most engineers no doubt saw little need to enable Peak Extend since where was relatively little demand for peak-limiting to begin with. I think they also probably didn't see the point of cluttering up the 16th bit of the CD/redbook signal with a lot of encoding when the vast majority of people would be playing the disc on on-HDCD players.

Also, my understanding is that any CD mastered/produced using a Pacific Microsonics will have the HDCD flag set - I think it's the one flag/"feature" that is always on. So a CD mastered using Pacific Microsonics gear will technically be an HDCD even if zero actual HDCD features are used.

Finally, it's been said elsewhere, but HDCD can be seen as the antecedent of MQA, just a simpler, more basic version and with much more honest and straightforward marketing.
 
OP
MRC01

MRC01

Major Contributor
Joined
Feb 5, 2019
Messages
3,424
Likes
4,030
Location
Pacific Northwest
Ah, so HDCD had 2 amplitude shaping features that could be used in a complimentary fashion. I'll double check my HDCD discs with a different software decoder and see if it detects any of the flags. If so, my plan is to decode them into an uncompressed WAV file then re-burn that to a new CD without HDCD. Because the likelihood that the music uses 16 bits of dynamic range, even after being decoded, is practically nil. In the extremely unlikely event that it does, I can always burn them to a DVD-A at 24-bit.
 
OP
MRC01

MRC01

Major Contributor
Joined
Feb 5, 2019
Messages
3,424
Likes
4,030
Location
Pacific Northwest
So, basically leveraging a fixed point calculation into slightly floating point by using last bit (or last few bits) to encode what the LSB is?
In HDCD, I *think* the LSB is used only to encode flags that trigger functions built into the DAC, described here (amplitude shaping, filter & dither selection).

At 44.1k stereo, the LSB don't give enough data to do much more than that. They add up to about 11 kbps (1/16 of the data rate) and that's an upper bound if HDCD used all of them. And I'm not sure it could use them all, since the LSB should be randomized.
 

JoachimStrobel

Addicted to Fun and Learning
Forum Donor
Joined
Jul 27, 2019
Messages
518
Likes
303
Location
Germany
I ripped my HDCDs with ITunes lossles, and they lost the HDCD attribute when played back through my HDCD capable DA and also the Oppo103 did not recognize them as HDCD any more. How should they be ripped?
 

Wes

Major Contributor
Forum Donor
Joined
Dec 5, 2019
Messages
3,843
Likes
3,788
HDCD doesn't seem to improve Grateful Dead albums nearly as much as that tape speed correction somebody in GDO figured out...
 
OP
MRC01

MRC01

Major Contributor
Joined
Feb 5, 2019
Messages
3,424
Likes
4,030
Location
Pacific Northwest
Now that we've mentioned Oppo players, my BDP-83 has a setting whether to decode HDCD. If I turn this ON, does that also apply to its digital outputs (coax or toslink)? E.g. Does it emit a bitstream that has the amplitude decompression applied?

The closest answer I can find from Oppo suggests that it does: https://oppodigital.azurewebsites.net/KnowledgeBase.aspx?KBID=44&ProdID=BDP-95
However, the DAC still reports the input as 44-16. So if it is doing this, it's emitting the de-compressed amplitude in 16-bit.
 

amirm

Founder/Admin
Staff Member
CFO (Chief Fun Officer)
Joined
Feb 13, 2016
Messages
44,370
Likes
234,414
Location
Seattle Area
I ripped my HDCDs with ITunes lossles, and they lost the HDCD attribute when played back through my HDCD capable DA and also the Oppo103 did not recognize them as HDCD any more. How should they be ripped?
The bit is only preserved if your playback pipeline is bit-exact. No volume control. No channel mixing. No OS pipeline. If you are using itunes as is, it will do all of that and destroy the signalling bits and the light won't come on.
 
OP
MRC01

MRC01

Major Contributor
Joined
Feb 5, 2019
Messages
3,424
Likes
4,030
Location
Pacific Northwest
Speaking of these amplitude shaping effects, here's an example from one of my HDCD discs, Reference Recordings RR-136 Saint Saens Organ Symphony. From my listening notes: On track 4 (movement 2) when the strings are at mp to mf level, they have a grain-like sound similar to tape hiss. At 4:54 there's a distinct change in this; the hiss suddenly disappears, the level shifts quieter, and the treble is attenuated.. This is just one example; it fades in and out occasionally. I wonder if this is HDCD amplitude shaping or dither switching on and off.

If anyone can check what HDCD features switch on and off at different moments, I'd be curious whether what I hear at that moment is an HDCD artifact or something else. Or listen to that spot and tell me if you hear the same thing.
 

tmtomh

Major Contributor
Forum Donor
Joined
Aug 14, 2018
Messages
2,636
Likes
7,494
Best practice for ripping an HDCD that uses Peak Extend if you want to play it back properly decoded through a computer system is to take the rip and run it through an HDCD decoder app, which will decode it to 20-bit and then pad the decoded version with 4 bits' worth of zeros to make it 24-bit. You end up with a 24/44.1kHz file that plays back decoded without the need for an HDCD capable DAC (or a bit-perfect signal chain, although a bit-perfect chain is a nice thing to have in general).
 
OP
MRC01

MRC01

Major Contributor
Joined
Feb 5, 2019
Messages
3,424
Likes
4,030
Location
Pacific Northwest
That sounds right. Yet it seems likely that you'll find after decoding that it doesn't actually use more than 16 bits of dynamic range, so 20 bit is unnecessary. In that case you can convert the 20-bit output of the decoder to 16-bit without losing any information.
 

amadeuswus

Active Member
Forum Donor
Joined
Jul 8, 2019
Messages
277
Likes
265
Location
Massachusetts
Tidal has quite a few Reference Recordings titles that first came out as HDCDs. I have read that these need no HDCD decoding for playback--presumably, the label supplied Tidal with unencoded 20- or 24-bit "masters" (whatever that may mean in this context). Does anyone here know more details? Thanks.
 

Wes

Major Contributor
Forum Donor
Joined
Dec 5, 2019
Messages
3,843
Likes
3,788
Best practice for ripping an HDCD that uses Peak Extend if you want to play it back properly decoded through a computer system is to take the rip and run it through an HDCD decoder app, which will decode it to 20-bit and then pad the decoded version with 4 bits' worth of zeros to make it 24-bit. You end up with a 24/44.1kHz file that plays back decoded without the need for an HDCD capable DAC (or a bit-perfect signal chain, although a bit-perfect chain is a nice thing to have in general).

examples for MacOS?
 

tmtomh

Major Contributor
Forum Donor
Joined
Aug 14, 2018
Messages
2,636
Likes
7,494
examples for MacOS?

Alas, I use Foobar2000 and hdcd.exe with a Windows emulator on my Mac: the former to detect if an HDCD disc has Peak Extend (otherwise it's not worth decoding IMHO); and the latter to do the actual decoding and conversion-to-24-bit-file process.

I believe the Mac version of dbpoweramp is the one app that can do this natively on the Mac side.
 

Guermantes

Senior Member
Joined
Feb 19, 2018
Messages
484
Likes
561
Location
Brisbane, Australia
Top Bottom