Transcoding challenge! With prizes!

[classy cheapo]

Let's rekindle the old discussion about high resolution audio formats

I have conducted my own tests which can be considered equivalent to double blinded and I find that I can determine correct file playback 10 times out of 10 between pairs of DSF64/256 and DSF64/FLAC192 files.

Here are the 3 files used in tests shared on Dropbox:

Dropbox

www.dropbox.com

Also, there are 3 more files with md5 extension to be able to verify bit perfect downloads.

Tests were done on Topping DX3PRo+ DAC paired with AKG K612 Pro headphones. Volume on DAC was set to 0 db.
Correct file format output was ensured by checking values displayed on DAC.
More info on the setup here: https://audiosciencereview.com/foru...ne-amp-for-classical-music.34663/post-1221817

Blindness of the test was ensured by loading a playlist consisting of paired files and following steps:

• Issuing shuffle command: mpc shuffle
• Playing first file for a first few seconds with redirecting info output to file: mpc -f %file% play > test.txt
• Playing next file for a first few seconds with redirecting info output to the same file: mpc -f %file% next >> test.txt
• Making decision on the order of played files
• Checking the actual order recorded in test.txt file

Repeat the steps 10 times.

In order to determine whether perceived differences are artifacts of encoding process or the format itself I propose following challenge:
Transcode one of the two given DSF files to any rate FLAC file and/or given FLAC file to any rate DSF file in a way where blind test as described above will not be able to definitely determine correct file playing order.

Why I do not perform transcoding myself?
Because I am not well versed in this black art and default settings not always provide best results. Even more, I am new to the field and do not know any proper audiophile lingo. Only words I can use when describing difference between given files are that one sounds "flatter" or "creamier" than other

What are the prizes?
There are two prizes awarded to first who will submit transcoded file(s) where order of playback cannot be definitely determined. One for DSF -> FLAC and other for FLAC -> DSF transcodings.
Each prize consists of one sponsored download from any of winner's choice high resolution audio sites up to 50 EUR/USD value.

In order to keep this thread civilized and practical please do not post your subjective opinions about audio formats. Stick to questions about setup, testing or links to transcoded files with transcoding parameters.
Please consider participating in this challenge only if you are well versed in audio encoding and you are reasonably sure of your good result.

May the best format win!

 

[thulle]

What's the end goal here? To see if you through your senses can discriminate different formats played on your setup?

 

[JaccoW]

It will be fun to take a shot at this.
But IIRC one of the quick telltale signs of DSD is the overall lower volume? At the very least that's what I noticed when I did this test myself.

 

[Apesbrain]

May the best format win!

Do what you like, but since your experimental design is a classic ABX test I suggest you use that tool. foobar2000 with its "ABX Comparator" component is a common way this is done. You and test participants can then publish your results in the standard format that foobar2000 provides. Be sure to ReplayGain your files before testing and set the comparator tool to adjust level according to track gain.

Keep in mind that test results between DSD64, DSD256 and PCM192 may be a reflection of your DAC's differential performance with those formats. Also, we don't know the provenance of the test files you have provided, nor the effect of the conversions done by participants. Perhaps there is something about the original files or in the DSD/PCM conversion processes that distorts the result? It is not possible to control for these variables in the test you've proposed.

I understand it's all in fun, but that's why "challenges" like this do not add meaningfully to our collective knowledge. As @thulle said, our individual experience of music is unique. Each of us "hears" differently, there is no shared standard, and no guarantee that what is obvious to you will be the same for another listener. If the DSD vs. PCM "best format" question had a true answer, there would be no call for these threads.

Finally, "best" is subjective depending on the individual's needs, but in the way I think you mean as regards fidelity requires that we compare playback of the recorded files to the actual event. How else can we each decide which is most faithful to the original sound? Otherwise, we are just conforming to bias and/or selecting the colorations we prefer (which is exactly what each of us does when we assemble a playback system and choose our preferred media formats.)

 

[DVDdoug]

I not going to bother because I don't have a DSD DAC and I probably couldn't hear a difference anyway.

But IIRC one of the quick telltale signs of DSD is the overall lower volume? At the very least that's what I noticed when I did this test myself.

One of the basic requirements of an ABX Test is that the files are level matched.

Why I do not perform transcoding myself?

You need to know they are both from the same source file. Otherwise, they could be mastered differently, or one might be re-mastered, or they could be different mixes, or even different recordings. If you convert a DSS to FLAC yourself (or vice versa) you know any differences must come from the conversion or the format differences.

Even more, I am new to the field and do not know any proper audiophile lingo. Only words I can use when describing difference between given files are that one sounds "flatter" or "creamier" than other

In this situation, there is ONLY noise, distortion, and frequency response. For example, If you want to say a slight roll-off in high frequencies makes it sound "creamy", it's best to use the technical terminology along with the audiophile nonsense so people know what you're talking about.

With speakers and room acoustics we can add reverb and other sound reflection characteristics in addition to the above three things that describe audio quality. And with lossy compression there can be artifacts that don't fit neatly into noise, distortion, or frequency response, and they might not show-up in "traditional" noise, distortion, or frequency response tests.

See Audiophoolery.

 

[Veri]

Let me give this a try once I'm at home and have eaten dinner

 

[SoNic]

In my experience the discussions about hires formats are mostly an academic exercise. In futility almost.

I am listening right now Pat Benatar "Get Nervous" remastered 2022 in 24 bit 96kHz Sample Rate.
Well.. on the Spectrum Visualizer (foobar2000) I can see the 1982 recording limitations - audio top end severely limited past 17 kHz (18kHz is barely lighting up).
It still sounds OK(ish), but not better than the CD version, because there is nothing extra on the master source material to be revealed by the 24bit/96kHz.

Now I also have "In the Heat of the Night" LP album and I have transferred that to FLAC 24-96. Almost identical results, just this time I can see small "lights" on 20 and 21 kHz. Most of the "activity" stops at 18khz. Very hard to capture a screenshot, but here is an example:

Is this something sounding astoundingly better, twice as good as CD? Heck no.

Other bands are in a similar situation.
Dire Straights "Brother In Arms" 20th Anniversary Edition SACD stops at 18kHz, sometimes lighting up the 20kHz. Recorded in 1988.
Their "Dire Straights" album from 1978, re-issued on SACD, has the same 17kHz limit as Pat Benatar's.
OK, let me try something actually recorded in this century...
Foo Fighters - "Saint Cecilia" 24 bit / 192 kHz:

Hits frequently 21kHz and occasionally 25kHz? Yes. Sounds good? Yes.
But I think that's the good mastering, not the hires. Their "Wasting Light" that I have in CD format sounds as good, even if it is "clipped" at 21kHz. Yes, their 16 bit 44.1kHz sample rate lights up the 21 kHz audio band.

PS: The thin lines are hold maxes that decay in 0.2 sec.
PPS: My headphones are a pair of HIFIMAN Deva Pro.

 

[Veri]

File A: test.wav
SHA1: 357a2201ef90378150995c914ba1a931f5e01cad
Gain adjustment: -1.99 dB
File B: test.dff
SHA1: 044b6e644213c856b3e4b6ef45e2a92a34465d05
Gain adjustment: -1.60 dB

Output:
DS : Primary Sound Driver
Crossfading: NO

20:45:41 : Test started.
20:46:12 : 00/01
20:46:24 : 01/02
20:46:48 : 01/03
20:47:02 : 02/04
20:47:17 : 02/05
20:48:19 : 02/06
20:49:20 : 02/07
20:50:22 : 03/08
20:51:23 : 04/09
20:52:25 : 05/10
20:52:55 : Test finished.

----------
Total: 5/10
p-value: 0.623 (62.3%)

testfiles.rar

Zippyshare.com - Free File Hosting

www87.zippyshare.com

 

[Sokel]

Easy to cheat by the way,and has nothing to do with volume,without conversion DSD>PCM or PCM to DSD on the player the volume can be the same.
But still easy to cheat by anyone using REW,I suggest you put your money to a nice charity.

 

[mansr]

It will be fun to take a shot at this.
But IIRC one of the quick telltale signs of DSD is the overall lower volume? At the very least that's what I noticed when I did this test myself.

Level matching between PCM and DSD depends on the DAC, and it can go either way.

 

[classy cheapo]

File A: test.wav
SHA1: 357a2201ef90378150995c914ba1a931f5e01cad
Gain adjustment: -1.99 dB
File B: test.dff
SHA1: 044b6e644213c856b3e4b6ef45e2a92a34465d05
Gain adjustment: -1.60 dB

Output:
DS : Primary Sound Driver
Crossfading: NO

20:45:41 : Test started.
20:46:12 : 00/01
20:46:24 : 01/02
20:46:48 : 01/03
20:47:02 : 02/04
20:47:17 : 02/05
20:48:19 : 02/06
20:49:20 : 02/07
20:50:22 : 03/08
20:51:23 : 04/09
20:52:25 : 05/10
20:52:55 : Test finished.

----------
Total: 5/10
p-value: 0.623 (62.3%)

testfiles.rar

Zippyshare.com - Free File Hosting

www87.zippyshare.com

Congratulations!

You win DSF -> FLAC (WAV) challenge.
I cannot reliably determine file playing order between my DSF 256 and your WAV 44100. Sample rate adds to embarrassment.

But I can reliably determine playing order between my DSF 64 and your WAV. Between my FLAC and your DFF 128 as well.

I start to suspect that all the difference amounts to small changes in loudness.

Something tells me you will not be very interested to get a free download from any high resolution site. Cannot blame you. I also suddenly lost my interest in them.

Would you pls PM me your Paypal address so that can transfer you the money.

 

[classy cheapo]

Dear all,
I hope that you will allow me to cancel the second part of my challenge. I learnt my lesson and would not like to lose another 50 EUR/USD.
Thank you and I hope that everyone had a bit of fun!

 

[Blumlein 88]

Not to rub it in, but JOB #1 in any listening comparison is precise level matching.

Plus or minus .1 db is what you need. (or within +/- 1.2 % by voltage).

Job #2 is checking frequency response. I like to say 85% of HiFi is frequency response. Certainly the great majority of real genuinely audible differences are frequency response differences.

 

[Doodski]

Plus or minus .1 db is what you need. (or within +/- 1.2 % by voltage).

That's pretty tight.

 

[Blumlein 88]

That's pretty tight.

Well at one time the consensus was +/- .2 db (+/- 2.4% by voltage), but some tests under some conditions indicated that wasn't always tight enough. If that were as close as I could get for one reason or another I'd use it. You can pretty much count on loudness causing an audible difference (which isn't heard as loudness) at a .5 db change in level.

 

[Doodski]

ell at one time the consensus was +/- .2 db (+/- 2.4% by voltage)

Are you using a online calc for that calc? I have not been using a dB calc because I rarely if ever used them in my line of work (Mechatronics).

I would not do the listening comparison as it isn't reliable enough to indicate anything useful. You can pretty much count on loudness causing an audible difference (which isn't heard as loudness) at a .5 db change in level.

Hmmz okies.

 

[Blumlein 88]

Are you using a online calc for that calc? I have not been using a dB calc because I rarely if ever used them in my line of work (Mechatronics).

Hmmz okies.

Used a calculator with log functions. I am ignoring some decimal places past the first two. So not fully exact numbers.

 

[Doodski]

Used a calculator with log functions. I am ignoring some decimal places past the first two. So not fully exact numbers.

Ahh. CooL! Something I am not proficient in. I've studied that and am aware of that but never became proficient in it because of other stuff happening at the time. Was working like 72 to 74 hours/week for a couple of years repairing gear after my electronics study and not studying formal matters thus forgot some stuff. (I was totally digging electronics and stuff.) What calculator are you using?

 

[mansr]

Used a calculator with log functions. I am ignoring some decimal places past the first two. So not fully exact numbers.

Matlab has a handy "db2mag" function.

 

[AdamG]

Congratulations!

You win DSF -> FLAC (WAV) challenge.
I cannot reliably determine file playing order between my DSF 256 and your WAV 44100. Sample rate adds to embarrassment.

But I can reliably determine playing order between my DSF 64 and your WAV. Between my FLAC and your DFF 128 as well.

I start to suspect that all the difference amounts to small changes in loudness.

Something tells me you will not be very interested to get a free download from any high resolution site. Cannot blame you. I also suddenly lost my interest in them.

Would you pls PM me your Paypal address so that can transfer you the money.

Your a new member and I say keep your money in your pocket and we will just call this little game off. All bets are null and void. I’m going to close this thread as a administrative action. I suspect @Veri did not do this for the payment. If I am wrong please contact me directly Veri.

@classy cheapo, you are more than welcome to open a new thread to continue to discuss this subject further if you wish. Sans the betting thing….