Does what we hear correspond to what we measure?

[pablolie]

I have two likes on it so far. That's a 3:1 ratio (you are the 1). So I win.

But hey keep showing everyone that you prefer jiving to answering direct questions, and maybe I'll do even better.

And that's without the ability to do down-votes.

 

[pablolie]

Excuse me, currently 13 people have taken the trouble to vote, and out of respect, I have taken their votes into account.I am not claiming that 2+2=5, but that among the votes cast, the percentage distribution is majority (>50%) for No. 2, and in relation to the votes, this is "significant." ...

Let's say you ask a group of people if they prefer the taste of a blood orange or a navel orange. And you randomly manipulate the available fruit, including some rotten fruit or include some with worms in it. Could be intentional or accidental, doesn't matter. And on top, some of the test participants are desperately hungry and dangerously undernourished, while others just gorged themselves on perfect fruit just prior to the test.

That's about the statistical significance of your test, there is zero common baseline, zero control.

I know what you were trying to do, but it was done in a way that had the majority of the potential audience wondering "why should I waste my time in a totally uncontrolled test?" and then, as you listen through the tracks your are "Come on. The music is good, both sound good, and I am not familiar with the original track to begin with, this is silly."

I have gone through many of these tests before, and didn't enjoy them even when they were rigorously conducted (Archimago did many of them). Hence I replied that I don't care about the differences - which you most definitely should have included in your list of possible responses, because actually THAT would have been the most interesting answer to track in this case.

 

[DVinylfan]

It seems to me that immediately saying that recording No. 1 came from a Bluetooth receiver would have irreparably biased the listening. For me, the best tests are when you don’t know you are taking a test... then the responses have a chance of not being biased.

For you yes perhaps, but for the world in general there's no 'the best tests are when you don’t know you are taking a test'.
You're alluding to blind testing but swerving the diligence that involves.
You've got a Bluetooth device you want investigated.
If you have the means, test it, if not surrender it for testing to NTTY or amirm.
In all other cases hold your counsel.

 

[pablolie]

Buddy ... Is anybody near you to apply immediate help?!

FYI this dude seemingly has been banned (thank heavens). But he'll be back. He used to go by another name, the style is unmistakable. Watch up for the pattern of provocation masquerading in a mix of picking up random statements and turning them into questions about the mental sanity of established contributors. His last stint made me quit ASR for a few months.

 

[Timcognito]

FYI this dude seemingly has been banned (thank heavens). But he'll be back. He used to go by another name, the style is unmistakable. Watch up for the pattern of provocation masquerading in a mix of picking up random statements and turning them into questions about the mental sanity of established contributors.

I think that comment was sent in response to me and I have ASR near me for immediate help. I hope.

 

[solderdude]

You and I are fans of the magnificent reviews done by Amir, NTTY, and others, where they show us measurements that seem to predetermine what we are going to hear.

This is already an incorrect statement as measurements can only show signal fidelity.
This is linked to what we hear (sound) but there is no 100% correlation because of perception..

But what if we did the opposite : let's listen and try to correlate what we hear with objective measurements.

This is 'fun' for some but then there should not be multiple changes to the signal which was the case nor the question he asked.
He wanted to know if people had a preference and not what was provable with measurements.
Could be trolling, could be lacking a correct understanding of all aspects involved.

Heated discussions are usually the result.

 

[DanielT]

Regarding this with BT. Hasn't the same Bluetooth codec been tested, measured via different OS? It would be especially interesting to see via Android vs Apple.If you go a step further, the same Bluetooth codec via different versions of Android (all the way up to version 16). Preferably measured and investigated via different brands, models of mobile phones and tablets. Although maybe that's overkill but you get the point I'm making.

 

[whereisbrian]

I have a pragmatic point of view: I'm offered a test, I like the tests, and I do it.

Leaving aside the hardware aspect of the test, it's obvious that the result can only be subjective, regardless of the tester's hearing.

Then, I used Spek to see if there was a match with what I had perceived, then DR Meter to have objective elements.

But the most interesting thing to analyze isn't the test itself, but the type of reactions it generates!

 

[Vintage02]

Regarding this with BT. Hasn't the same Bluetooth codec been tested, measured via different OS? It would be especially interesting to see via Android vs Apple.If you go a step further, the same Bluetooth codec via different versions of Android (all the way up to version 16). Preferably measured and investigated via different brands, models of mobile phones and tablets. Although maybe that's overkill but you get the point I'm making.

Here is an interesting idea. I have a Samsung mobile phone running Android 14 and also a laptop with Windows 11 that has Bluetooth functionality (I have never used it, but since I was advised to do the tests from a PC, I checked and saw that I could do it). I do not have a Macintosh, but I should be able to test with an Ubuntu system. This is going to involve a lot of work because if we consider the three operating systems (Android, Windows, Linux) and the three connection modes of the Bluetooth receiver (AptX-HD, LDAC, USB), that will make 8 measurement sessions. I will check with NTTY (a friend of mine) the measurements to be taken to get reliable information. I will follow up with another topic on this receiver.

Edit : LDAC is not natively supported by Windows as it seems to be by Android 14. However, there is a software solution that bypasses A2DP and allows LDAC selection. This is the A2DP Alternative Driver program. However, it is not clear what this program does to the signal.

 

[DVDdoug]

Here is an interesting idea. I have a Samsung mobile phone running Android 14 and also a laptop with Windows 11 that has Bluetooth functionality (I have never used it,

IF you're testing Bluetooth, just make sure Bluetooth is the ONLY variable. It has to be the same recording, level matched, etc.

Then you can make measurements and/or do Controlled Audio Blind Listening Tests.

Measurements are usually "easier" with test-tones than with music/program material.

With lossy CODECs, measurements don't always correlate well with what we hear so blind listening tests are often better*.

But we don't all have equal hearing so the results of the listening test may only apply to you. And some program material is easier to compress than others so you might hear a difference with one song and not another...

When it's lossy, so we KNOW there is a difference from the uncompressed original but we don't always know if the difference is audible.





* MP3 (and most other lossy compression) is a based on psychoacoustics and it was tested and optimized using listening tests rather than by using measurements.

 

[Vintage02]

IF you're testing Bluetooth, just make sure Bluetooth is the ONLY variable. It has to be the same recording, level matched, etc.

In this thread, I was specifically notified that the source operating system (here Android) did not allow bit-perfect, so a reference was needed for that and it was necessary to use another system like Windows, which should allow bit-perfect, and therefore, as you said, to have only Bluetooth as a variable.

Furthermore, I was told that listening tests of a piece of music could not be used to qualify a device and that standardized signal measurements were necessary, which I am going to do as I explained in my previous message.

 

[Multicore]

Android phones have a poor reputation for sound out of the box. There are lots of things you can do to fix this.

Would you please point me in that direction?

 

[MaxwellsEq]

Would you please point me in that direction?

This is a very helpful post:

Can the Android 48khz sample conversion be measured in the form of distortion+noise, has anyone done this?

I've been reading a lot about how people use bit-perfect drivers or players when using a phone as a DAC amp or as a DAC transport with external Dongle DACs. My understanding is this is to avoid 44.1khz and other rates being converted to 48khz as people say this degrades the quality of the...

www.audiosciencereview.com

 

[Svensson]

In any meaningful scientific comparison (and we are here because of scientific measurements), one must remove all major variables between two or more options, even for n=1 assessments. This applies to clinical practice where patients very, just like us and our hearing and hearing equipment hardware. The variation between patients is 'normal' so we can meaningfully consider the difference in the treatment options and outcomes, even where those outcomes are perceptual. For me (personal view, not a criticism), the refusal to consider or to advise level matching was the end of the line for me.

 

[whereisbrian]

In any meaningful scientific comparison (and we are here because of scientific measurements), one must remove all major variables between two or more options, even for n=1 assessments. This applies to clinical practice where patients very, just like us and our hearing and hearing equipment hardware. The variation between patients is 'normal' so we can meaningfully consider the difference in the treatment options and outcomes, even where those outcomes are perceptual. For me (personal view, not a criticism), the refusal to consider or to advise level matching was the end of the line for me.

i
It's important to keep things in perspective; it's simply an aural comparison of two files.

If we want to be more scientific, we can add the participants' audiogram, the bandwidth test of the headphones used, their possible equalization, the listening level, and other circumstances.

 

[Svensson]

i
It's important to keep things in perspective; it's simply an aural comparison of two files.

If we want to be more scientific, we can add the participants' audiogram, the bandwidth test of the headphones used, their possible equalization, the listening level, and other circumstances.

Agreed, trying to take the time to provide helpful constructive feedback to OP.

 

[MaxwellsEq]

It's important to keep things in perspective; it's simply an aural comparison of two files

Indeed, and no harm was done by posting the clips and getting feedback. People enjoyed listening and commenting.

But the title is wrong. The OP implied measurements were involved (in the title). But we've established errors in levels, compression, balance etc. So there's no chance of learning anything from listening.

 

[MAB]

i
It's important to keep things in perspective; it's simply an aural comparison of two files.

It sure wasn't presented as such.

If we want to be more scientific, we can add the participants' audiogram, the bandwidth test of the headphones used, their possible equalization, the listening level, and other circumstances.

Perhaps, but first the files needed to be level-matched, which is critical. These additional considerations may be useful in certain studies, but the foundation of the study needs to be correct.

 

[Timcognito]

You poor victims AI maybe all these predictions of havoc are true. 12 pages, really?

 

[Vintage02]

Indeed, and no harm was done by posting the clips and getting feedback. People enjoyed listening and commenting.

But the title is wrong. The OP implied measurements were involved (in the title). But we've established errors in levels, compression, balance etc. So there's no chance of learning anything from listening.

The title is:
Does what we hear correspond to what we measure?

As you can see, there is a "?" at the end, so it is a question and not a statement.

Furthermore, I also wrote:
But what would happen if we did the opposite: listen and try to correlate what we hear with objective measurements.
First of all, I suggest you listen to two recordings of the same piece and already share your opinions on which one seems to be the "best."

It seems to me that I left room for doubt and I wanted to get your opinion on listening to this piece of music.This was certainly misunderstood because I expressed myself poorly, for which I am truly sorry. However, I have taken the feedback into account and next time, I will present things differently.