AES Paper Digest: Do Audio Op-amps Sound Different?

[Krunok]

Just for the record, the Swedish Audio Society were able to identify a dac as recent as the Oppo Sonica Dac in a DBT about a year ago. But not with music signals though, AFAIR, only with clicking/ticking sounds. So the claim that no dacs have been spotted in DBTs is not correct, methinks. But they were not able to identify the rather cheap Yamaha WXC-50 in a similar test.

Google doesn't seem to be able to find any references to the Sonica's alleged clicking issue.

Here is an independent measurement of Sonica's frequency response (and other parameters) which doesn't show any issue with bass response, so I wonder how they were able to hear any difference with Sonica versus other DACs..

https://hometheaterhifi.com/reviews/dac/oppo-digital-sonica-dac-review/

 

[Krunok]

You well may differentiate in a true blind test. You may be validated. Then again, may not be. If you are happy with your choices don't take the test. If you wish to convince others ............................ .

I'm not trying to convince anybody, I was merely asking if there is some established level of THD/IM distortion which is generally acceptable as "hearable".

 

[Wombat]

I'm not trying to convince anybody, I was merely asking if there is some established level of THD/IM distortion which is generally acceptable as "hearable".

I wasn't suggesting you were. There has been much research done over decades re this matter and I would definitely say, yes. Others can possibly provide a brief reference to typical levels. If not I will spend the time on Google to find a summary.

 

[oivavoi]

Photographs of magazine pages in a foreign language are pretty useless to me and probably most others who read this forum - we need something that we can use with Google Translate.

I've pushed photographs of text through OCR in my native language, and that has turned out to be very difficult and with limited benefits because of all of the typos. Doing this in a language that I have no familiarity with is pretty much mission impossible.

I've worked with listening tests from this Swedish Audio group in the past.

Here is a partial translation of some of their work that I could still find: http://www.sonicdesign.se/amptest.htm

Their experimental procedures were debatable, and what was represented to me initially as being a DBT turned out to be a SBT.

Please try to save yourself time by not spending it by processing useless information.

Last comment on this for the day, before I need to start doing work I actually get paid for doing:
The test you are referring to was done by them in their previous iteration as a group, many years ago. You are correct that it was only single blind at that time. They changed their testing methodology around 2010 (I think?), when Svante Granqvist (who has a PhD in acoustics and compouter science) came on board and wanted to make their methodology more scientific. Now it's double blind: A computer program generates random files they listen to. It's not foobar though, they do some kind of AB testing, not ABX testing. Based on what I have read on Swedish forums, it does indeed seem as if they are able to detect fewer devices with this new methodology, which may imply that experimenter bias indeed accounted for some of the positive results in the previous SBTs.

As for their dac testing, Svante describes their methodology in Swedish in this forum post: https://www.faktiskt.io/phpBB3/viewtopic.php?f=9&t=66684&full=1#p1960394

What they do for dac testing is some elaborate scheme where they compare the output from the dac under test with the output from a super-duper dac, and they also do some resampling of the signals to be sure that any difference is due to the dac-part of the device, and not sampling differences or other things. (please don't rely on my sketchy summary here for details, it's a simplified version)

In their latest testing, it does seem that they are in fact able to detect several dacs using this test - meaning that they are not transparent in the absolute sense of the word. Still, differences are very slight, and it's questionable whether it would be noticeable when listening to music in normal mode. And they also find that they have indeed been unable to detect some very cheap gear in their DBTs (such as the Yamaha WXC-50).

My understanding is that this group is mainly a collection of audio enthusiasts of a perfectionist bent, some of who happen to be audio professionals (either in manufacturing or in research). They seem to be animated by a desire to find audio gear that is absolutely transparent - in much the same way as our very own @amirm . All their activities are on a voluntary basis. But they don't seem to be animated by a desire to prove everybody else on the internets wrong, as strange as that may seem for some of us.

 

[restorer-john]

...if there is some established level of THD/IM distortion which is generally acceptable as "hearable"...

Someone here (hear) may be able to put their hand on Matti Otala's work in the late 70s on TIM (transient inter-modulation distortion) and its audible effects at relatively low levels. Some tried to dismiss it, but the basics, I have always considered sound (another pun-sorry). In summary, it ended up being wide bandwidth, low(ish) NFB and well regulated, low impedance PSUs with high current output stages. I'm sure there are others here with knowledge on this subject.

 

[Krunok]

I wasn't suggesting you were. There has been much research done over decades re this matter and I would definitely say, yes. Others can possibly provide a brief reference to typical levels. If not I will spend the time on Google to find a summary.

I tried Google before asking the question but no luck, at least not from a reliable source. Here is what I find confusing: few days ago I posted THD data for a 8500 USD worth B&W 803 D3 speaker and they are claiming in specs <1% of THD for 2nd and 3rd harmonics in the range of 70Hz - 20kHz and <0.3% in the range of 100Hz-20kHz.
http://www.bowers-wilkins.com/Speakers/Home_Audio/800_Series_Diamond/803-D3.html

Now, how would I be able to hear 0.38% vs 0.308%? I can hardly imagine 0.08% or 0,008% making any difference when played by a speakers with 0.3%.
And those a really good speakers with really low THD..

 

[Krunok]

Someone here (hear) may be able to put their hand on Matti Ottala's work in the late 70s on TIM (transient inter-modulation distortion) and its audible effects at relatively low levels. Some tried to dismiss it, but the basics, I have always considered sound (another pun-sorry). In summary, it ended up being wide bandwidth, low(ish) NFB and well regulated, low impedance PSUs with high current output stages. I'm sure there are others here with knowledge on this subject.

I'm not talking about TIM here, my Rotel doesn't use high loop gain and my tube amplifier has 0,2% THD (preamp has 0.3%) which is excellent figure for a tube device.

Modern solid state amps don't suffer from TIM as lessons were learned from the past.

 

[Wombat]

Unless I hear something amiss with a system of reasonable specs I don't go looking further. Old fashioned I guess. I think about those individuals who become educated in fine-wine and are destined to be unhappy drinking what they can afford.

Trivial difference as an academic interest, yes, but time-wasting on already disproven stuff is not for me unless credible challenging evidence is provided. Been around a while. Other things to waste time on.

Warning: I might say so.

 

[Krunok]

Unless I hear something amiss with a system of reasonable specs I don't go looking further. Old fashioned I guess. I think about those individuals who become educated in fine-wine and are destined to be unhappy drinking what they can afford.

Trivial difference as an academic interest, yes, but time-wasting on already disproven stuff is not for me unless credible challenging evidence is provided. Been around a while. Other things to waste time on.

Hahaha -well said!

Btw, it has been proven on many occasions that expensive wines cannot be differentiated from a decent budget wines in a true blind test even by experts.

 

[Wombat]

Hahaha -well said!

Btw, it has been proven on many occasions that expensive wines cannot be differentiated from a decent budget wines in a true blind test even by experts.

As in HiFi wine price is not a reliable indicator.

 

[Krunok]

As in HiFi wine price is not a reliable indicator.

Very true.

 

[oivavoi]

Concerning wines, it is not totally correct that there is zero correlation between quality and price, or that expensive wines cannot be differentiated from cheaper wines. I looked into the research on this some time ago, spurred by the fact that blind-testing of wines sometimes became a part of some hifi discussions. I found that some of the media coverage on this was somewhat incorrect.

In short: Individual cases of ratings of wine are heavily influenced by things like color, reputation, prices, etc. Nevertheless, there does seem to be a modest correlation between wine prices and quality ratings in blind tests. Which means that there are objective quality differences between wines after all. This article sums up much of the evidence:
https://academic.oup.com/ajae/article-abstract/97/1/103/2737501

I found this article interesting too:
https://www.cambridge.org/core/jour...d-replicates/BBF3DCCD599F587F4E7AE19F72EAE412

Even though individual wine judges are far from having a perfect replicability score, there is nevertheless a higher probability that they will rate a "good" wine as better than a "bad" wine. Which means that a 0-result in an individual blind test between two wines doesn't invalidate that one of the wines may in fact be better, and that it's somewhat more likely that wine drinkers will prefer that one over the other.

For me, this seems roughly on a par with where I think things stand in audio. I trust measurements and objective data over sighted listening, and even though I like DBTs, I don't think individual negative results in a blind test is the final word (if there are in fact substantive objective differences that can be measured).

 

[Krunok]

Concerning wines, it is not totally correct that there is zero correlation between quality and price, or that expensive wines cannot be differentiated from cheaper wines. I looked into the research on this some time ago, spurred by the fact that blind-testing of wines sometimes became a part of some hifi discussions. I found that some of the media coverage on this was somewhat incorrect.

In short: Individual cases of ratings of wine are heavily influenced by things like color, reputation, prices, etc. Nevertheless, there does seem to be a modest correlation between wine prices and quality ratings in blind tests. Which means that there are objective quality differences between wines after all. This article sums up much of the evidence:
https://academic.oup.com/ajae/article-abstract/97/1/103/2737501

I found this article interesting too:
https://www.cambridge.org/core/jour...d-replicates/BBF3DCCD599F587F4E7AE19F72EAE412

Even though individual wine judges are far from having a perfect replicability score, there is nevertheless a higher probability that they will rate a "good" wine as better than a "bad" wine. Which means that a 0-result in an individual blind test between two wines doesn't invalidate that one of the wines may in fact be better, and that it's somewhat more likely that wine drinkers will prefer that one over the other.

For me, this seems roughly on a par with where I think things stand in audio. I trust measurements and objective data over sighted listening, and even though I like DBTs, I don't think individual negative results in a blind test is the final word (if there are in fact substantive objective differences that can be measured).

First of all nobody said that there is "zero" correlation, we only agreed that price is not a good indicator.

Now, quote from your first article:
"The research identifies that the relation between the price of wine and its sensory quality rating is a moderate partial correlation of +0.30"

If I remember correctly statistics from university correlation factor 0.3 was called "weak", not "moderate".

If unfamiliar with statistics look here: http://www.dummies.com/education/math/statistics/how-to-interpret-a-correlation-coefficient-r/

 

[Wombat]

Concerning wines, it is not totally correct that there is zero correlation between quality and price, or that expensive wines cannot be differentiated from cheaper wines. I looked into the research on this some time ago, spurred by the fact that blind-testing of wines sometimes became a part of some hifi discussions. I found that some of the media coverage on this was somewhat incorrect.

In short: Individual cases of ratings of wine are heavily influenced by things like color, reputation, prices, etc. Nevertheless, there does seem to be a modest correlation between wine prices and quality ratings in blind tests. Which means that there are objective quality differences between wines after all. This article sums up much of the evidence:
https://academic.oup.com/ajae/article-abstract/97/1/103/273750

I found this article interesting too:
https://www.cambridge.org/core/jour...d-replicates/BBF3DCCD599F587F4E7AE19F72EAE412

Even though individual wine judges are far from having a perfect replicability score, there is nevertheless a higher probability that they will rate a "good" wine as better than a "bad" wine. Which means that a 0-result in an individual blind test between two wines doesn't invalidate that one of the wines may in fact be better, and that it's somewhat more probable that wine drinkers will prefer that one over the other.

For me, this seems roughly on a par with where I think things stand in audio. I trust measurements and objective data over sighted listening, and even though I like DBTs, I don't think individual negative results in a blind test is the final word (if there are in fact substantive objective differences that can be measured).

Audio signal quality can be accurately measured. Wine quality is much in the subjective arena. E.g. I love chardonnay, can't stand Reisling.

Oivavoi, my impression is that you are attempting to apply soft-science principles to long established hard-science topics. Back up your views with credible results and you will be listened-to.

https://en.wikipedia.org/wiki/Hard_and_soft_science

Affable score?

 

[Krunok]

Oivavoi, my impression is that you are attempting to apply soft-science principles to long established hard-science topics. Back up your views with credible results and you will be listened-to.

+1

 

[oivavoi]

Guys, I really struggle to understand why you start arguing like this. Why not just argue against what I'm saying and leave it at that? Why do you need to include your "impression of me", Wombat? And the implication that I'm unfamiliar with statistics, Krunok? I have published peer-reviewed articles using statistical methods, if that matters.

This is audio we're discussing here! It's not important! It's supposed to be fun.

Concerning "moderate" - this is a forum and I write quickly. I don't define every term to avoid any ambiguity. I used "moderate" in an intuitive sense. The point is that there is a correlation between price and quality ratings, but other factors have a stronger impact when it comes to individual tastings of wine.

EDIT: rereading I see that I actually said "modest", not moderate, which is closer to "weak" than "moderate" semantically

 

[Krunok]

No hard feelings, I didn't post the link about correlation factor specifically for you.

The point is that 0.3 correlation is weak, so it really shouldn't be used a solid argument to prove that it makes sense to buy expensive wines. The point is also that your style of discussion is quite "unprecise" which tends to provoke response from technical people. For example: you're again claiming "there is a correlation between price and quality ratings". But the truth is this: yes, there is a correlation, and it is weak, which doesn't really justify spending more money into expensive wines.

 

[Wombat]

Guys, I really struggle to understand why you start arguing like this. Why not just argue against what I'm saying and leave it at that? Why do you need to include your "impression of me", Wombat? And the implication that I'm unfamiliar with statistics, Krunok? I have published peer-reviewed articles using statistical methods, if that matters.

This is audio we're discussing here! It's not important! It's supposed to be fun.

Concerning "moderate" - this is a forum and I write quickly. I don't define every term to avoid any ambiguity. I used "moderate" in an intuitive sense. The point is that there is a correlation between price and quality ratings, but other factors have a stronger impact when it comes to individual tastings of wine.

Science-based forum. Credible research results are welcome. Evidence of practical application is also welcome. Soft-science has to work harder to get predictable results, i.e. the bar is much higher, here.

I don't know why you make this a personal issue. Impressions are basic to soft-science.

Peer review in soft-science is not the same as peer review in hard-science - very different standards of factuality and predictability. I don't know why you put the two fields in the same arena wrt equipment performance.

Don't put me down to elevate your position. Oh, not affable - yeah, I know.

 

[oivavoi]

To be honest this is starting to become annoying. That should not be necessary, as my interest here is factual: I put forth arguments, and I'm more than delighted to get good counter-arguments in return. That is why I discuss on this forum. I have no idea why it has to become heated. I don't like it, and I try to avoid it.

My person is absolutely and completely irrelevant to the matter we're discussing. Focus on the arguments, please. It is you who started making this personal. I'm not putting you down, Wombat, and I struggle to see why you interpret it that way. I have not made any characterization of you at all here. But both of you are making general characterizations like "impecise discussion style" etc.

Also, the distinction between hard and soft sciences is not drawn once and for all. Soft subject matters can be investigated with "hard" methods (which I often do). I have used both hard and soft methods in my work, and I'm quite familiar with the possibilities and limitations of the different kinds of methods and research designs (as a matter of fact I teach research methods to bachelor students every fall, which includes research designs from the hard sciences).

But again: this should not be relevant. If I make an argument, or post a link to a scientific article, not only do I have nothing against it, I actually enjoy to be challenged on rational and empirical grounds. But I really dislike the kind of person-focused argumentation that you're engaging in here.

 

[Wombat]

To be honest this is starting to become annoying. That should not be necessary, as my interest here is factual: I put forth arguments, and I'm more than delighted to get good counter-arguments in return. That is why I discuss on this forum. I have no idea why it has to become heated. I don't like it, and I try to avoid it.

My person is absolutely and completely irrelevant to the matter we're discussing. Focus on the arguments, please. It is you who started making this personal. I'm not putting you down, Wombat, and I struggle to see why you interpret it that way. I have not made any characterization of you at all here. But both of you are making general characterizations like "impecise discussion style" etc.

Also, the distinction between hard and soft sciences is not drawn once and for all. Soft subject matters can be investigated with "hard" methods (which I often do). I have used both hard and soft methods in my work, and I'm quite familiar with the possibilities and limitations of the different kinds of methods and research designs (as a matter of fact I teach research methods to bachelor students every fall, which includes research designs from the hard sciences).

But again: this should not be relevant. If I make an argument, or post a link to a scientific article, not only do I have nothing against it, I actually enjoy to be challenged on rational and empirical grounds. But I really dislike the kind of person-focused argumentation that you're engaging in here.