• WANTED: Happy members who like to discuss audio and other topics related to our interest. Desire to learn and share knowledge of science required. There are many reviews of audio hardware and expert members to help answer your questions. Click here to have your audio equipment measured for free!

Audio Science Review

Status
Not open for further replies.

Simon 13th

Member
Joined
Mar 18, 2021
Messages
95
Likes
8
I’m considering doing a video on ASR and what people take from it. Can someone help me , have I got it right you use measurements to inform your subjective opinions, on listening, not use the measurements to say what’s good or not.

I get lots of comments to my YouTube channel 13th note hifi reviews saying that a particular dac I have reviewed is really poor based on ASR but I think it’s subjectively and comparatively very good. I don’t prop up bad products to anyone who knows me (eg Auralic Aries g1) but appreciate most hifi gear is competitive at price, by the same token. I also understand there are subjective (listening based) consensus groups around what people like in hifi, which is the right way to do it on experiences.

This fills me with dread because putting off people buying or trialing based on what might be using measurements solely, isn’t right.

Im not making this post to put down your site because I know a measurement approach can get lots of benefit for people eg amp power, slew rate, damping factor and so on. And this is why ASR is popular. But it should inform your listening not the other way around. Is this very much for the view of people who participate here, Am I right Or misunderstanding something. As I say I’m not trying to put down but understand.
 

Koeitje

Major Contributor
Joined
Oct 10, 2019
Messages
2,292
Likes
3,880
Reviews are useful for build quality impressions, functionality and the user experience. But listening? Only useful when you do it in a volume matched double blind test. Otherwise you will be listening with your eyes and preconceptions in addition to your ears.

With regards to price and objective performance, its all about if any issues in performance are in the audible range and what the price is for that performance. I don't know what DAC you reviews, but I wouldn't pay €3000 for a DAC if I can get superior performance with the same connectivity for €300.

I'm purely speaking about DACs here by the way. You can 100% buy a DAC based on measurements alone.
 
OP
S

Simon 13th

Member
Joined
Mar 18, 2021
Messages
95
Likes
8
Reviews are useful for build quality impressions, functionality and the user experience. But listening? Only useful when you do it in a volume matched double blind test. Otherwise you will be listening with your eyes and preconceptions in addition to your ears.

With regards to price and objective performance, its all about if any issues in performance are in the audible range and what the price is for that performance. I don't know what DAC you reviews, but I wouldn't pay €3000 for a DAC if I can get superior performance with the same connectivity for €300.

but do you assume the dac ain’t good to not try it based on measurements?
 
OP
S

Simon 13th

Member
Joined
Mar 18, 2021
Messages
95
Likes
8
Why not? It's more "right" than some person using flowery prose. Only one of those is verifiable.

i don’t do that. I deliberately avoid words that are too florid but when you make comparisons you achieve factual difference by comparison.
 
OP
S

Simon 13th

Member
Joined
Mar 18, 2021
Messages
95
Likes
8
If the DAC measures well, is well made (according to a proper engineering tear down) and others have reported no issues then I’ll consider it. I’m not happy to buy based on someone else’s subjective evaluation without these criteria being met

so if I said x dac costs $500 and another $1500 or similar multiples at lower prices, and I said one appears to me to have more detail and it’s very obvious, you wouldn’t trust that the view I had was a factual objective comparative one based on comparison?
 
OP
S

Simon 13th

Member
Joined
Mar 18, 2021
Messages
95
Likes
8
I wouldn't look to buy /audition a dac that measured poorly on ASR. If it measured OK but had a unique killer feature I wanted, then I would try it.

how do you determine all the variables that makes a component measure poorly or not to equate that to sound quality of say one dac against another. Isnt it too complex
 

Koeitje

Major Contributor
Joined
Oct 10, 2019
Messages
2,292
Likes
3,880
but do you assume the dac ain’t good to not try it based on measurements?
Why try it if I can just buy better regardless? I'd rather not pay more. I get the cheapest that has no issues that could be audible and matches my preferences in terms of looks and connectivity/features.
 

BDWoody

Chief Cat Herder
Moderator
Forum Donor
Joined
Jan 9, 2019
Messages
6,948
Likes
22,625
Location
Mid-Atlantic, USA. (Maryland)
I also understand there are subjective (listening based) consensus groups around what people like in hifi, which is the right way to do it on experiences.

Suggestion for a video...

Make a sincere post that you would like to make a video of yourself taking a properly controlled listening test, to clearly demonstrate to us here how our approach is missing the 'right' way, and what we can learn from it.

We have been waiting for someone to do this for a long time. Interested?

Take the input, take the test (DAC's would be easiest), and do it all on video.

The premise here is basically that beyond a fairly low competence threshold, good luck telling them apart once you control for volume differences, filters, etc. and any sighted bias by not knowing what's playing, in a normal room, listening to music.

Electronic devices aren't part of the music chain. They do their jobs as designed, with no magic beyond the realm of normal physics.
 
Last edited:

bogart

Active Member
Forum Donor
Joined
Sep 10, 2019
Messages
248
Likes
304
so if I said x dac costs $500 and another $1500 or similar multiples at lower prices, and I said one appears to me to have more detail and it’s very obvious, you wouldn’t trust that the view I had was a factual objective comparative one based on comparison?

I'd believe you heard it. I would also believe that it's possible that expectation bias is nearly all of the reason you could hear it, not that you were deceived or 'heard wrong.' Since we're talking about signal processing, fidelity of decoded output compared signal input is the sole measure of quality.

The sound of a DAC should be "none" among all well-engineered DACs. None of them should have a sound signature of their own.

Tolstoy said it best: “All happy families are alike; each unhappy family is unhappy in its own way.” Replace "happy" with well measuring, and family with DAC, and it's well stated. Unhappy DACs can easily impart a sound signature; but measurements tell me I can easily rule them out, as I'm not interested in listening to my DAC.
 

eddantes

Addicted to Fun and Learning
Joined
May 15, 2020
Messages
706
Likes
1,380
i don’t do that. I deliberately avoid words that are too florid but when you make comparisons you achieve factual difference by comparison.

Unless you do your subjective review repeatedly, consitently, in a DBT - it's not "factual".
 

JustJones

Major Contributor
Forum Donor
Joined
Mar 31, 2020
Messages
1,726
Likes
2,391
so if I said x dac costs $500 and another $1500 or similar multiples at lower prices, and I said one appears to me to have more detail and it’s very obvious, you wouldn’t trust that the view I had was a factual objective comparative one based on comparison?

Since it's not objective but a subjective opinion I would trust it as that.
 

threni

Major Contributor
Joined
Oct 18, 2019
Messages
1,277
Likes
1,519
Location
/dev/null
so if I said x dac costs $500 and another $1500 or similar multiples at lower prices, and I said one appears to me to have more detail and it’s very obvious, you wouldn’t trust that the view I had was a factual objective comparative one based on comparison?

So you're not even saying "trust me that I did a proper double blind test and couldn't tell the difference/thought one sounded better"; just "trust my sighted opinion that this is better than that because it has 'more detail'"? And specifically about DACs, which are solved and which don't require more than around £$100? No, that video would be absolutely meaningless to me. For reasons which are probably spelt out hundreds of times all over this site.
 
Last edited:

BDWoody

Chief Cat Herder
Moderator
Forum Donor
Joined
Jan 9, 2019
Messages
6,948
Likes
22,625
Location
Mid-Atlantic, USA. (Maryland)
so if I said x dac costs $500 and another $1500 or similar multiples at lower prices, and I said one appears to me to have more detail and it’s very obvious, you wouldn’t trust that the view I had was a factual objective comparative one based on comparison?

If you did it under controlled circumstances, I would believe it might apply to someone other than yourself. Bias has to be controlled for...there's no way around it, other than for broken/gross kinds of problems.

If it's just the standard casual, sighted listening comparison, I wouldn't pay much attention.
 

sergeauckland

Major Contributor
Forum Donor
Joined
Mar 16, 2016
Messages
3,440
Likes
9,100
Location
Suffolk UK
Reviews are useful for build quality impressions, functionality and the user experience. But listening? Only useful when you do it in a volume matched double blind test. Otherwise you will be listening with your eyes and preconceptions in addition to your ears.

With regards to price and objective performance, its all about if any issues in performance are in the audible range and what the price is for that performance. I don't know what DAC you reviews, but I wouldn't pay €3000 for a DAC if I can get superior performance with the same connectivity for €300.

I'm purely speaking about DACs here by the way. You can 100% buy a DAC based on measurements alone.
I would extend that to any electronics. The measurements will show whether an item is transparent or not (almost everything is) so as long as those numbers come out OK, and the item has the looks and facilities (and price) desired, then there's no need for any other consideration.

There may be exceptions made for specific items that are not transparent, like some valve amplifiers, and of course all transducers, whether microphones, loudspeakers or pickup cartridges which are very seldom transparent. Then, which one chooses depends on the balance of imperfections or certain aspects of their performance that makes them more suitable for the application. Again, measurements will act as a guide.

Listening tests have to be done level matched and blind, or they're completely pointless.

S.
 

ahofer

Major Contributor
Forum Donor
Joined
Jun 3, 2019
Messages
4,947
Likes
8,694
Location
New York City
Speaking for myself,

1)inaccurate measurements (within audible thresholds) would definitely preclude a purchase. For some here a *lack* of measurements would preclude a purchase. For me it is a warning.

2) I am comfortable buying electronics without listening, based on measurements, quality reviews, reputation, features, and returnability. I believe measured transparency is audible transparency in electronics. I also believe the evidence is heavily on my side in that regard.

3) I am increasingly narrowing my speaker criteria to emphasize spin measurements as I’ve come to understand them and can hear and name the differences these measurements reveal. But I take a more subjective approach to speakers due to the actual complexity of measurement and room interaction.

One thing the “listen to everything” mindset gets wrong is the value of my time. We all need a proper Bayesian way to narrow what we audition. Measurements are vastly superior to subjective reviews in this regard.

I don’t know if I’m typical, but this is my approach.
 
OP
S

Simon 13th

Member
Joined
Mar 18, 2021
Messages
95
Likes
8
Suggestion for a video...

Make a sincere post that you would like to make a video of yourself taking a properly controlled listening test, to clearly demonstrate to us here how our approach is missing the 'right' way, and what we can learn from it.

We have been waiting for someone to do this for a long time. Interested?

Take the input, take the test (DAC's would be easiest), and do it all on video.

The premise here is basically that beyond a fairly low competence threshold, good luck telling them apart once you control for volume differences, filters, etc. and any sighted bias by not knowing what's playing, in a normal room, listening to music.

Electronic devices aren't part of the music chain, they do their jobs as designed, with no magic beyond the realm of normal physics.

I’d maybe like to do it, but then again.....you’d get loads of posts like this doubting it, and anyway for me it would be like setting out to tell why I like fresh spag bol over mouldy spag Bol. It’s testing something I don’t need to do. if I can’t tell the difference between a musical fidelity mix-dac and dCS bartok it’s time to.....
 

Koeitje

Major Contributor
Joined
Oct 10, 2019
Messages
2,292
Likes
3,880
@Simon 13th I think the most important lesson for you would be that your senses are easily fooled and everybody is fallible. It is a hard thing for people to admit they are fallible, but such is life.


I would extend that to any electronics. The measurements will show whether an item is transparent or not (almost everything is) so as long as those numbers come out OK, and the item has the looks and facilities (and price) desired, then there's no need for any other consideration.

There may be exceptions made for specific items that are not transparent, like some valve amplifiers, and of course all transducers, whether microphones, loudspeakers or pickup cartridges which are very seldom transparent. Then, which one chooses depends on the balance of imperfections or certain aspects of their performance that makes them more suitable for the application. Again, measurements will act as a guide.

Listening tests have to be done level matched and blind, or they're completely pointless.

S.
I agree, but for amplifiers for example you also need to take into account the use case. Your 5 watt amplifier isn't going to drive Salon 2's to decent levels. Of course power output is also a measurements, but whether an amplifier will always operate within its operating range depends on the use case. A DAC (or cd player, network player, etc) will always perform at the same level.
 
Status
Not open for further replies.
Top Bottom