Master Thread: Are measurements Everything or Nothing?

[solderdude]

A couple days ago I was playing with the affect of slight level differences. When I setup the two test files in an ABX, onmost runs I could tell the different right away, but on several, I think four, I couldn't. I had to listen far longer on those to hear it, though I did get 10/10. That's something I don't often experience - being able to differentiate something so clearly in a few seconds, and then all of 5 seconds later not being able to.

How slight were the changes ?

 

[ahofer]

Some say it's butyric acid. That most North American chocolate was originally made with rancid milk, and that the manufacturers these days add butyric acid (giving the taste of vomit) on purpose. Either to get a longer shelf life, or because people demand the flavour. Other say that butyric acid is no longer found in North American chocolate, but it's just terrible quality in a number of other ways.

Either way, IMO, it give a funny link to the mechanisms of nostalgia and/or "burn-in" and how it defines our ideas of quality.

I wonder how many North Americans, who have lived in Belgium or Switzerland, still prefer their native variety of chocolate.

One of our German Au Pairs went on and on about how much better German candy was. Then her Dad sent her a care package of her favorite stuff and it was full of Mars and Cadbury products.

(I lived in Switzerland, you don't have to convince me there is better stuff than Hershey's, but the market is very global now).

 

[navin]

Did you make an effort to avoid things like confirmation bias when you compared the two? Or just straight listening?

Not really but I am a cynic by nature. I also enjoy experimenting and tinkering (as a hobby I design and build my own amplifiers, loudspeakers, and guitar pedals for example). I try things and listen. Sometimes I am surprised by what I hear, often times I am not. I did not expect a change in power supply to make a noticeable audible difference because we shouldn't if everything is in clean DC. Maybe the wall wart I got wasn't great. Who knows. It's just one empirical observation (2 if you include my friend's P50).

Since I have 3 systems in my home, I have ordered 2 WiiM Pro Plus that are scheduled to arrive in India on September 15th and 30th (via 2 different friends). I will connect these 2 with the same linear power supply (I have 2 5V DC / 1A linear power supplies and can easily build a third) I have and check.

 

[JP]

How slight were the changes ?

That one was 1.5dB on the source file - I don't know what level that manifested through on my system. The intent was to make it enough to be readily apparent there was a difference but not obvious what the difference was.

ABX

Listening tests in the browser

abxtests.com

 

[DonR]

Not really but I am a cynic by nature. I also enjoy experimenting and tinkering (as a hobby I design and build my own amplifiers, loudspeakers, and guitar pedals for example). I try things and listen. Sometimes I am surprised by what I hear, often times I am not. I did not expect a change in power supply to make a noticeable audible difference because we shouldn't if everything is in clean DC. Maybe the wall wart I got wasn't great. Who knows. It's just one empirical observation (2 if you include my friend's P50).

Since I have 3 systems in my home, I have ordered 2 WiiM Pro Plus that are scheduled to arrive in India on September 15th and 30th (via 2 different friends). I will connect these 2 with the same linear power supply (I have 2 5V DC / 1A linear power supplies and can easily build a third) I have and check.

So you don't know what you hear or at least cannot rule out influences from your biases. That is why blind testing is conducted.

 

[IPunchCholla]

This is easy enough with digital files. A bit more difficult with cables ad a lot harder with electronics if not nearly impossible with speakers and headphones.
In the latter case, by lack of reference, this comes down to preference at best.
This requires quite some knowledge, measurements and even test equipment and the involvement of others and or computer programs and interfaces.

Even then you can only prove things to yourself. That last part is the important part though. Then still the question will remain... was the test valid ?

Demanding evidence from people that claim things such as: something being inaudible because of this or that, all the way to 'things being audible without there being a clear reason for it'.

All one can do is remain friendly and inspire folks to do some testing and advise people on how this can be done and what would be needed.
Best to start with some audibility tests (digital files)

Personally I prefer the former option. It is ok not to have an opinion. It is even more acceptable to not share an opinion (usually not an opinion, but a incorrectly believed falsehood). And I don't think asking someone to state how they came to their opinion (or again incorrectly believed falsehood) is the same as demanding evidence, but trying to get them to go through their own methods of logic, or at least provide specifics so that conflating factors can be elucidated.

Basically, I am tired of people thinking all they have to do is state something as a fact. "I heard a difference" is a statement of fact. All statements of fact should be supported by either citation, or evidence. If we grow a culture of having to support facts, if I hear a difference, and want others to believe I hear a difference, I need to show that I hear a difference. I can do this by diving into the literature to see if hearing a difference in this case is actually feasible, or by doing some sort of test as you suggest.

I hope I was being kind, though I see how the quoted sentence is abrupt. But it is really about civil (not polite or friendly) discussion in the sense of coming to an understanding, instead of weaponizing information to be right. Not calling out unsupported claims leads to all of the issues we see in todays talk culture: Gish galloping, Russel's teapot, sea lining, whataboutism, etc. And yes. Proving something can be very difficult. So what? Do the work or see if others have done the work and shared it. Or maybe just actually state it as an opinion. "I believe I heard a difference" is one way to do this as it explicitly opens up the belief to questioning, The same goes for other types of opinions. "Rap music is bad" is a statement of (unsupportable) fact, I have seen often on this forum. "I have never heard rap music that I have enjoyed" is an opinion. The first is a crap statement since it implies a fact that demeans people who do enjoy rap, the second is fine.

 

[ads_cft222]

One thorough set of measurements can say something about a component but not everything about the perceived sound quality , since there is not a one to one function for that.

One subjective opinion on how something sounds like says nothing . 100 opinions averaged in a context of system setups that have reasonably good measurements can confer some confidence and could act as a complement to measurements.

 

[solderdude]

maybe just actually state it as an opinion. "I believe I heard a difference"

That is basically how I read those statement I fill in the 'I believe' part for myself when reading the claims
They just do not say it that way and omit the 'I believe' part simply because they heard it and thus it is as real to them as anything else.

You cannot convince people that are already convinced about something to see things in another way. Even if you present them with evidence.

So probably best to just explain and motivate them to test in another way. They usually don't do this.
Just because they clearly heard it, they have excellent ears and gears and know about their carefully selected gear and why they bought it. Measurements and blind tests say nothing to them and are flawed. They read articles by 'experts' they adore that say so.
A no-win situation.

Instead of winding yourself up just post the counter arguments (without getting personal), explain things and on the next claim that will be made the following day by someone else who does not agree with 'ASR' and their deaf inhabitants that only look at graphs and listen to test tones.

 

[presence]

So you use a effects box to master stuff? That seems a bit drastic when the option for linear signal processing is available.

I guess you do not know a much of mastering process? It's nothing but effects. Effect after effect, quite subtle though. Final touch. Emotion. Technically quite advanced of course, if you referring that.

 

[IPunchCholla]

You cannot convince people that are already convinced about something to see things in another way. Even if you present them with evidence.

Citation needed.

It is actually my day job to do just that.

I basically agree with the rest. Though I don’t think I’m wound up.

 

[IPunchCholla]

They just do not say it that way and omit the 'I believe' part simply because they heard it and thus it is as real to them as anything else.

Yes, but pushing for the non abbreviated version forefronts doubting our senses. Which is important. It also reduces ambiguity and conflation. In a public, written forum, I believe that those are good goals.

 

[presence]

When both amps do not distort the FR significantly and making up stuffs, please explain how it would sound different…
And simple question: Neve is an engineer, if sound isn’t represented by the measurements or calculations, please explain how he designed the amp and mixer etc. to have the same sound signature….

By hearing? You still have your senses, beside calculations? That is the biggest mistake by mixer or mastering engineer to trust just numbers and hope the best.

 

[solderdude]

It is actually my day job to do just that.

Probably not in the audio business
And I bet convincing people to do things differently is a long process.

In a public, written forum, I believe that those are good goals.

It is a good thing to state facts and encourage people to look at it from another angle.

 

[solderdude]

I'm a mastering engineer and I have tested between RME adi-2 and RNHP. They are complitely different animals.

RME is ultra clean power, straight what input is, someone could say sterile and dynamically flat. RNHP is like a compressor. It change the dynamics of sound, but not in a bad way, similar to Neve's preamps. If one cannot hear the difference, one has wrong career. RNHP sound colored but is well balaced, when RME can be a bit confusing, because it reveal (too much perhaps) details.

So, Neve's aim with RNHP is to give an engineer better focus to sound I guess to help with mixing/mastering process, which it does well.

Sound is not just a couple of measurements out there, but they do give confidence. In the end in professional level sound is what you hear, feel and sense.

There is no evidence of this headphone compressing sound, at least not with high impedance headphones.

The frequency response is flat so there is no tonal change. The output resistance is low so there can be no tonal changes meaning it does not color the sound.

Since you are a mastering engineer it would be quite easy for you to record the output of both headphone outs (under load preferably) and post them here so we can all hear the differences and analyze the crap out of those recordings.
All 'we' have now are measurements and Amirs findings.

 

[Gorgonzola]

I get that. Being told it is all in your imagination is similar to being told you’re deaf. It’s dismissive and a simplification.

True.

So when someone says it is in your head or you are biased, it can sound insulting, but what is being said, too simply and perhaps too dismissively, is that yes, you are hearing what you are hearing, but it is an artifact of how our brain/ear system functions and not a change in the sound waves.

I think such diagnoses of why a person thinks he/she hears differences is quite presumptuous.

So if you are sick and tired of hearing it’s in your head, you can either stop making the claim, or show that it is not.

As for me, I always clearly state that what I'm hearing is my personal impression; one is free to consider it or dismiss it as one sees fit.

 

[Doodski]

I guess you do not know a much of mastering process? It's nothing but effects. Effect after effect, quite subtle though. Final touch. Emotion. Technically quite advanced of course, if you referring that.

I've been in 3 recording studios but I was never involved in the process and I was a friend of the band and I was simply observing the stuff and looking at all the gear and facilities. I made a mistake and thought the mastering process was all about levels, balance and not effects. Sorrrrry! >@^_*@<

 

[atmasphere]

Thanks, Don, for a response that is evenhanded and civil - but more than that, for a response that is useful. I think your post is useful for two reasons:

1. You identify and start to discuss a clear, specific mechanism by which distortion could be more or less easily heard (because of differential masking of various orders of harmonics). This is of course central to what Ralph discusses in the linked comment, and it's quite reasonable and interesting as a topic.
2. Your comment also helps us focus on the problem with the OP's apparent belief, and with the "opinion" that Ralph seems to be promoting here: even if certain harmonics are more easily detected than others, that doesn't matter if all orders of harmonic distortion products produced by an amp are well below the threshold of human audibility. In @amirm 's amp tests, we routinely see that 2nd and 3rd order harmonics are at the highest levels (usually by far), and that higher-order harmonics - the "dangerous" ones - are almost always at much lower levels, even in amps deemed "competent" or mediocre.

It's the super-common problem in audiophile culture: someone - in this case Ralph and our OP - focuses on a mechanism or a phenomenon and acts like if the mechanism or phenomenon is real, then it must automatically be audible. The part where the magnitude or significance of the mechanism/phenomenon is evaluated compared to human hearing ability gets overlooked. This has been the formula for the sales pitch for almost every piece of audio snake oil, although folks can of course also employ this formula in good faith or at least without any specific agenda of selling something.

When discussing the audibility of distortion, we have to keep in mind that the distortion products are not a separate signal. If they were, and as low as many amps are capable of, they would be easily masked. Distortion is different in that its a modifier. IOW it changes the sound of musical instruments by adding harmonics. Harmonics define that sound of a musical instrument, in case their role isn't clear... ask any luthier about that.

I like and respect Ralph as a designer. But I think he's wrong here. Not so much that higher order distortion is more audible, but that human ears are super-sensitive.

For the record, I did not say the ear is super-sensitive! To many things it is not. What I did say is that its keenly sensitive to the higher orders.

One could use Pkane's Distort software to see how much distortion becomes audible. Or to see if lots of 2nd and some 3rd mask other distortion to give a smooth sound.

A 3rd harmonic in loudspeakers is well-known to mask higher ordered harmonics. I'd be interested in a study that shows that somehow this phenomena does not happen in amplifiers as well

Has a version of Distort been issued that allows one to model rising distortion with frequency? The author mentioned on this site that he had a version that could do that but didn't release it because the idea was 'too arcane' (paraphrasing). Since this is a common problem with most amplifiers employing feedback (which is most amplifiers in general) an audibility test that does not include this behavior is oranges and apples.

If you've not read it, Bruno Putzeys has a great article on feedback:
https://linearaudio.net/sites/linearaudio.net/files/volume1bp.pdf
If you read this article you'll see how important this aspect of distortion can be. We see that, as frequency rises, when the circuit meets its Gain Bandwidth Product limit, feedback decreases on a 6dB slope so distortion rises on a similar slope. As frequency continues to increase, its not uncommon for the distortion to rise at an even faster rate than that (because the feedback can fall off on an even faster rate depending on circuit variables).

If we take the example of 1KHz as a turnover point, we can see that at 7KHz the distortion can be quite a lot higher than the THD otherwise suggests! Our ears, as pointed out earlier, will assign a tonality to this and 7KHz is at the upper end of our ear's most sensitive range. In a nut shell if Distort does not model this you can't rely on the result.

If distortion rises with frequency, I think you'll find that the THD figure that an amplifier might have in practice isn't really accurate. IMO it really isn't a useful spec.

 

[coonmanx]

We talked about this subject a few years back.

Excellent. I was about to post up the "time to get the popcorn out" comment. I have to say that at first I was just looking at the snide comment that the OP made to another member. That seemed uncalled for. And it is also good to know that this has already been discussed. Frankly, my thoughts are, as I get older, that none of this really matters because I grab a CD, throw it into the player and play it through a decent vintage system that sounds good. So far @tmtomh has brought some civility to the discussion so thanks to him for that well thought out post.

 

[IPunchCholla]

I think such diagnoses of why a person thinks he/she hears differences is quite presumptuous.

You're right. I should have said may be an artifact.

As for me, I always clearly state that what I'm hearing is my personal impression; one is free to consider it or dismiss it as one sees fit.

You said earlier: However the fact remains the many people can hear differences between amp 'A' and amp 'B'. You state it as a fact. It is an unsupported fact.

 

[coonmanx]

I get that we come towards issues like this from different perspectives as well. I have a background in both Math and Physics but I don't get too hung up on distortion numbers. I have always owned equipment with low distortion numbers. And those amps have always sounded good. I don't know. For me I kind of feel like if I get too hung up on the numbers or the technical aspect of some things then the enjoyment goes away. And I have dabbled enough to have some very good basic knowledge as well. I have built two different sets of speakers in the past few years. That was fun and I learned a bit. But that started with good components as well.

Maybe for me it comes down to the question of how many times one can split a hair...