Okto dac8 stereo DAC Review

[Tks]

I agree that the multitone test is the closest of the current standard tests to music, but it's still a long way off. Real music is typically composed of far more than just 32 tones (and not evenly spread as they are in the multitone signal). Why use a test tone when you can use the real thing (or at least a proven, standardized close analogue to real music in the program simulation noise) to reveal all possible degradations of the source signal, which may include unexpected or as yet unknown mechanisms of distortion?

Problem is, what do you mean by "real thing" with respect to standardization aspirations toward a relevant benchmark? I know you mean music, but what song exactly?

There's a massive difference between something like an Alva Noto experimental piece, a pop track that is clipping, and classical pieces with huge dynamic swings.

What musical piece would be of relevant merit here? With of course as I said prior, aspirations toward creation of some sort of "standard" that can be used as a benchmark?

I like the idea of a null-dif test, and that's all fine, but you lost me on why a specific song could ever be considered as some sort of benchmark.

 

[Vasr]

Judging purely on this measurement suite of test tones, it looks so. It would be great to see its measured performance using music too, via a null difference measurement. @Okto Research seeing as you have both a class-leading DAC and the class-leading ADC capabilities of the APx555B analyzer, would you mind taking such a measurement? This would then provide a benchmark metric by which other null measurements can be compared, as well as suggesting how the SINAD metric relates to total audio degradation when playing music. SoundExpert have measured null differences for a range of DA/AD loops here using recordings uploaded to Gearslutz here. The music file they're using for the null test can be downloaded here, which would just need to be recorded out of the dac8 via the APx555B. Obviously this is just one music genre (classical), but it would allow comparison with the large set of already measured gear.

SoundExpert also calculate null differences using a 'Program Simulation Noise' signal (see their full test suite here), a standardized signal representative of music and speech created as part of standard BS EN 50332-1. I see that Audio Precision have actually included this very test signal in their 'headphone test utility', downloadable here. It would be great if you could also record playback of this test file from the dac8 by the AP Analyzer, as the difference signal calculated from this has shown very good correlation with that using SoundExpert's full 2-hour test suite of varied, real music - typically the former is ~1-2 dB higher (worse) than the latter, as can be seen from these results, suggesting the program simulation noise is a good worst-case analogue to real music.

SoundExpert's methodology and open-source software used to calculate the null difference metric (Df) can be found here (more details on the project are in this thread). This software accurately adjusts for phase/time shifts between the original and recorded signal using an iterative algorithm to find the global minimum value. All this can be calculated independently though - only the recordings of the two test files as I described are needed.

I think this is a very interesting experiment to do from a science perspective.

But not to detect some additional distortion patterns necessarily (I am not yet convinced there would be something not manifested in the multi-tone test) but to validate the model measurement itself.

If our measurements are useful then there should be a positive correlation between the measurement and the deviation from "real music". For example, higher the SINAD (as in better performance), lower the measured null difference (which is the tacit assumption behind these measurements anyway, right?). If we can show that, then that would be a huge validation of the measurement metric used... proof is in the pudding so to speak. If there is no correlation, then we would need to think of the validity of the measurement for practical use.

But the objection above by @Tks is valid. We would need to validate why a specific "music test pattern". It may have to be a suite. I can even think of a test suite of various instruments and voices on their own being sent through and the null differences computed. Create a single sample with clips of these, each clip does not need to be long. Have the off-line software break it up and score it for each clip.

I have no idea how easy or difficult it is to incorporate such a test. There needs to be a fully automated process, otherwise, it just seems very tedious and impractical.

 

[Vasr]

I have this one

https://www.nativedsd.com/albums/2XHDFT1095-audiophile-speaker-setup

It is a very good mix of music for evaluating a full stack for critical listening.

I am not sure how you would represent that breadth with a single clip for null diff testing.

 

[Tks]

I have this one

https://www.nativedsd.com/albums/2XHDFT1095-audiophile-speaker-setup

It is a very good mix of music for evaluating a full stack for critical listening.

I am not sure how you would represent that breadth with a single clip for null diff testing.

You wouldn't, but you also wouldn't do 69 null diffs either. This is also, again, falling into the "audiophile" category of music, mostly jazzy, instrumental, classical, music. This isn't even really "music" if we're being serious, just something to setup speakers it seems. Reminds me more of Chesky's binaural stuff for headphone tests to see what such recordings sound like.

But nevermind any of this. What exactly do you think multi-tone tests are missing in the first place. Like what are you looking to reveal (what sort of phenomena, seeing as how you're on some search for a new field of science to open up on)?

I'm just basically having issues understanding why we would want to do this at all in the first place? Why not multi-tones, but increase the number of multi-tones. Far less inconsistency, far less work as well. You seem to understand how this would need to be automated, which makes me wonder why anyone would bother outside of a hobbyist?

I'm just very confused is all.

 

[Sunship]

Small press publisher? Oh, Oh. I'm discomforted by the dis-synchronous color variations in the bases of the speakers. My issue.

What do you mean by ‘small press publisher”? I build the speakers.
Regarding the feet, there’s nothing easier than having all done in one color Most people love the multiple colors while some worry it affects the sound or at least their perception of it.

 

[wineds]

Amir already measured it. His results were much worse because he used his noisy computer as a source. I used my $149 iPhone 7.

https://www.walmart.com/ip/Refurbished-Apple-iPhone-7-32GB-Black-AT-T/924779379

Which brought the grand total of my computer/DAC combo setup to $169.

But isn't your output in that case only single ended?

 

[Vasr]

What exactly do you think multi-tone tests are missing in the first place. Like what are you looking to reveal (what sort of phenomena, seeing as how you're on some search for a new field of science to open up on)?

I'm just very confused is all.

Yes, partly because you are mixing up what I have said with what @bobbooo has been saying I actually said I am not convinced you will find some new form of distortion if you read my note carefully.

You are looking at this in the mindset of yet another measurement to do for a metric and so it is natural for you to question what else could you get by more measuring. I agree with you there. But I am saying something totally different as a science process (not creating new science). Perhaps these nuances get lost in quick exchanges.

But just to clarify and just leave it here for people to think about than carry on rapid responses which seldom illuminate...

The process of science often includes some limited set of measurements of an object or an event to create a model of that object/event. This is what we are doing with DUTs here. But that is usually just the first step. For that model to be considered valid and useful, we create a prediction of that model that can be tested/measured. If that prediction is borne out by the new tests/measurement then we get more confidence/credibility in our model and the original set of measurements we do for that model. This is how most of astrophysics works, for example.

Here the measurements rank DUTs under some metrics. The "prediction" (albeit implicit) is that lower the "test score", more likely the DUT would make the content we listen to sound worse. But whether something sounds worse is a subjective opinion, so we need an objective measurement/tool that captures that quality. I see the null diff test with real music sample as that measurement to validate and corroborate the measurements done with SINAD. multi-tone, etc. By showing correlation with the measurements and the outcome of the null diff test, we validate our model of the DUT with the set of measurements we have done, not show some new forms of distortion (this is where I am saying something different than @bobbooo). If the null diff tests are at odds with our model measurements, then we have a potential problem with our model. This is, again, how scientific investigation works.

The above is separate and distinct from whether we should do so ... because there are some practical limitations (which doesn't invalidate the need for the process) or whether we need to find that perfect representation of every possible music. Statistical theories suggest only a statistically significant sample (in variety) is sufficient to test our prediction hypothesis) So looking for covering every possible music is not necessary and not a valid objection to the process itself.

That is all I have to say on this.

 

[wineds]

No doubt a compromise of some sort, given the current enclosure design and position of the Pi. Perhaps @Okto Research can chime in on the motivations behind this decision?

Personally, I'm not too bothered and it is not in the way of anything. As long as there is no performance degradation, it's all good

The alternative would be to plumb three pi USB ports and one pi Ethernet jack which would be messy IMHO. I think the way into have implemented the pi provides maximum flexibility and is quite tidy.

 

[TimF]

What do you mean by ‘small press publisher”? I build the speakers.
Regarding the feet, there’s nothing easier than having all done in one color Most people love the multiple colors while some worry it affects the sound or at least their perception of it.

Small press publisher....the books and the presentation in the book case are similar to the presentation of books in small book stores of book publishers of small editions. The speakers are works of art, I must say.

 

[bobbooo]

Problem is, what do you mean by "real thing" with respect to standardization aspirations toward a relevant benchmark? I know you mean music, but what song exactly?

There's a massive difference between something like an Alva Noto experimental piece, a pop track that is clipping, and classical pieces with huge dynamic swings.

What musical piece would be of relevant merit here? With of course as I said prior, aspirations toward creation of some sort of "standard" that can be used as a benchmark?

I like the idea of a null-dif test, and that's all fine, but you lost me on why a specific song could ever be considered as some sort of benchmark.

I addressed these issues in my original post. As I said, SoundExpert calculates the difference signal using a test suite of among other things 2-hours of music - 35 tracks of a wide variety of genres, dynamic range, some with clipping, listed here (bottom of the page). On that page is also described the BS EN 50332-1 standard 'Program Simulation Noise' test signal, which was originally devised to have spectral content representative of music and speech for testing audio devices. SoundExpert also calculate the difference signal for this test file.

The justification to use the latter is in the previous results for devices already tested (here and here), which consistently reveal a difference signal for the program simulation noise ~1-2 dB higher (worse) for every device than the difference signal for the 2-hour suite of varied music tracks on that device (and that's even despite the two sets of results above using two different ADCs of differing precision). This is strong evidence that this program simulation noise is a good worst-case analogue of real music, and so would make an ideal, standardized test signal for null measurements.

 

[Sunship]

Small press publisher....the books and the presentation in the book case are similar to the presentation of books in small book stores of book publishers of small editions. The speakers are works of art, I must say.

Ah! ok The listening space was set-up by a friend who is indeed a non-fiction book collector and occasionally distributor/publisher.
I'll stop here to not pollute this DAC8 thread, exit: https://www.sunshipaudio.com/p/listening-room.html

 

[JediMa]

I'm always so impressed, such detailed and accurate reviews and comments, I really would like to improve my tech knowledge, do you have some link to share about it?

 

[bobbooo]

But nevermind any of this. What exactly do you think multi-tone tests are missing in the first place. Like what are you looking to reveal (what sort of phenomena, seeing as how you're on some search for a new field of science to open up on)?

That's the point, we don't know. But it's perfectly plausible that the complex waveform of music could introduce or exacerbate yet unknown or poorly understood mechanisms of distortion that standard test tones don't. Null measurements are guaranteed to reveal all such degradations of the source when playing real music (or a close analogue of this in the form of the program simulation noise test signal).

Subjectively, Dr. Sean Olive and Steve Temme for example have shown in this AES paper that non-coherent distortion produced using real music has a much better correlation to sound quality than multitone distortion, which along with IMD is poor in this regard, as well as beating THD (results are at 31:24 in the video):

I'm just basically having issues understanding why we would want to do this at all in the first place? Why not multi-tones, but increase the number of multi-tones. Far less inconsistency, far less work as well. You seem to understand how this would need to be automated, which makes me wonder why anyone would bother outside of a hobbyist?

I'm just very confused is all.

I'd argue it's actually much less work, and can in fact be performed by anyone with an ADC of equal or higher performance to the DUT. All that needs to be done is a recording of the test file played through the DUT using said ADC. SoundExpert's free software can then be used to calculate the 'Df' difference signal metric (or DeltaWave by @pkane which now includes an option to do the same).

And then there's the very useful benefit of seeing how SINAD correlates with the total sound degradation revealed by this null difference metric, which I alluded to in my in original post and @Vasr has expanded on in detail.

 

[JohnYang1997]

Just measured the output noise of APx555B when output is 4.22V. This is not the definitive test.
Ch1 is the channel for it. 2.3uV noise. Ch2 is loopback without output signal.

It's unbalanced connection. So balanced may have even higher noise.
This means it's very possible to have a external dac measuring better than the loopback. So yeah, may be thinking of getting or designing something else for THD+N for amplifiers now.....
The limitation though is still the inputs of AP. It will reach to 4.3uV. Still better to measure THD no N and noise independently. Then you have better than -140dB THD+N potential or higher than 140dB SINAD.

For people don't know how lower noise can affect in a system which have higher noise. You can get your head around it by searching for noise figure. If you think noise figure is difficult to understand.
Independent noise sources are essentially two perpendicular vector. You add them in vector way. Which is also known as Pythagorean theorem. As noise is getting lower and lower it's still contributing to the total noise just less and less and it's not a direct sum.

The way I measured it is a bit complex. I set the output to 100khz 4.2V. It goes through a notch filter at 100khz. It has about 65db of attenuation. This assures that the AP's input stays at 310mV range. I was going to use another noise amplifier but when I measure it it's high enough to not be essential. Then AP is set to 20khz or 22khz bandwidth. The number is CH1 shows the noise.

 

[Veri]

For people don't know how lower noise can affect in a system which have higher noise. You can get your head around it by searching for noise figure. If you think noise figure is difficult to understand.
Independent noise sources are essentially two perpendicular vector. You add them in vector way. Which is also known as Pythagorean theorem. As noise is getting lower and lower it's still contributing to the total noise just less and less and it's not a direct sum.

Interesting =) thanks!

 

[JohnYang1997]

Well now coming back to the original question. Half of the question is answered. The DAC's noise performance is easily better than the AP's output.
The other half is that dynamic range is low at 127dB instead of 131dB from spec. I would like to know if anyone can help to reproduce the spec.

 

[Veri]

Well now coming back to the original question. Half of the question is answered. The DAC's noise performance is easily better than the AP's output.
The other half is that dynamic range is low at 127dB instead of 131dB from spec. I would like to know if anyone can help to reproduce the spec.

May DAC reached 130dB DR... but Wolf uses the APx555-B. Maybe it(APx555) is the bottleneck? I mean, overall dashboard is less clean on May.

 

[JohnYang1997]

May DAC reached 130dB DR... but Wolf uses the APx555-B. Maybe it(APx555) is the bottleneck? I mean, overall dashboard is less clean on May.

APx555 and APx555B are essentially the same just a different software/firmware/security/version revision. It comes with newer version of AP software and doesn't require external hardware to run that version which is required for APx555.

 

[Tks]

I addressed these issues in my original post. As I said, SoundExpert calculates the difference signal using a test suite of among other things 2-hours of music - 35 tracks of a wide variety of genres, dynamic range, some with clipping, listed here (bottom of the page). On that page is also described the BS EN 50332-1 standard 'Program Simulation Noise' test signal, which was originally devised to have spectral content representative of music and speech for testing audio devices. SoundExpert also calculate the difference signal for this test file.

The justification to use the latter is in the previous results for devices already tested (here and here), which consistently reveal a difference signal for the program simulation noise ~1-2 dB higher (worse) for every device than the difference signal for the 2-hour suite of varied music tracks on that device (and that's even despite the two sets of results above using two different ADCs of differing precision). This is strong evidence that this program simulation noise is a good worst-case analogue of real music, and so would make an ideal, standardized test signal for null measurements.

Two hour suite though? That doesn’t sound too practical. Like imagine that on top of something like a 30 minute thermal test.

There’s also the issue of needing to get liscence free music, on top of opening yourself up to critique about music selection.

Again, the test itself doesn’t strike me as something unsound, but as you can see, noise of 1-2db of difference is hardly something I think is worth exploring beyond recognition of its existence for said device. I imagine a difference of something like 2db is seemingly only an issue with respect to final output through headphones or speakers that distort far worse than any modern DAC. A 2db diff between something like 120db SINAD and 122db SINAD seems uneventful in the least even if such diff manifests in the first place using non coherence as a metric.

Also, at these levels with the AP pushed to its limits, I imagine breathing (metaphorically speaking) over the devices as they run - would have an impact.

You’d also have to be careful on measuring noise separately (unless I misunderstand non coherent distortions).

So again, I’m just not seeing the value. Great for theoretics, and for someone that really wants to dig deeper. And possibly a test suitable if he never transitioned to testing speakers (though I’d say possibly better for speaker testing), but in reviews where I have to bug him to run Toslink measurements, SINAD over Power level, or SINAD over time, or just filter performance. I doubt this is something Amir would run. The work itself isn’t daunting as you can just create a single file of the entire suite, the time required is the problem.

 

[bennetng]

Reminded me of this... -129dB THD+N