Any correlation between measurements and perceived sound quality?

[JustIntonation]

Are you referring to the Bob Stuart study? I consider the methodology of that study to be flawed due to the authors' decision to use rectangular dither, and I'm not aware of any other studies that have given similar positive results (but am open to it if you know of any?).

No I did a quick read of one of the post on the blog jsrheta linked to http://archimago.blogspot.com/2014/06/24-bit-vs-16-bit-audio-test-part-ii.html
Haven't looked into if it was a good test. Just mentioned my take/opinion on the argument mentioned there against 24-bit.
Btw, pretty sure that if one has a very dynamic capable system playing at 100dB SPL with very dynamic music (preferably not real recorded due to the high noise floor) that the 16-bit dither floor should be audible in a dead room. I have no problem hearing the (recording microphone) noise floor on most acoustic recordings on my current substandard system and I'm not even in a dead room right now.

 

[jsrtheta]

Apparently some people have (proveably) showed they could hear the difference.

Who, exactly? And where?

 

[JustIntonation]

I suggest you read this.

Not only is it a quite comprehensive introduction to the field of psychoacoustics, but it also outlines the landmark studies into virtually all aspects of the human auditory system. For me, it provided a solid basis for interpreting evidence from the kinds of studies that I think you're talking about.

I will! Looks like the perfect place to start. Thanks!

 

[JustIntonation]

Who, exactly? And where?

The blog you linked to:
http://archimago.blogspot.com/2014/06/24-bit-vs-16-bit-audio-test-part-ii.html
Didn't read the whole thing. Don't know if it was proveable / they used ABX comparator or something like that for proof.

 

[andreasmaaan]

The blog you linked to:
http://archimago.blogspot.com/2014/06/24-bit-vs-16-bit-audio-test-part-ii.html
Didn't read the whole thing. Don't know if it was proveable / they used ABX comparator or something like that for proof.

I didn't interpret the results that way. What part of the results/conclusions are you referring to that show this?

 

[JustIntonation]

I didn't interpret the results that way. What part of the results/conclusions are you referring to that show this?

On the main page that links to the article it is introduced like this:
"I've heard over the years some people wondering whether there really is a point to 24-bit DACs. After all, there is little if any evidence that "hi-res" 24-bit music actually sounds any better - in fact, you might recall that way back in 2014, here on the blog we ran a blind test and the results did not show significant audible benefit among respondents. More recently last year, even with Dr. AIX's "HD-Audio Challenge", very few people were able to experience benefits to "hi-res" audio (no surprise of course!). "
Reading it again and reading the article I now see that I probably misinterpreted the meaning of the above text. Combined with the article I now read it like he previously thought 24-bit didn't make sense but now with the latest poll he thinks it does.
I should stop quick reading things..

 

[amirm]

So good for you if you heard differencs between mp3 bitrates on headphones.

Headphones are *the* standard for evaluating lossy audio compression. They allow one to block outside sound, keep channels from masking distortions in another, and you can turn them up to very high volumes to hear the smallest level of distortion. My audiophile playlist sounds a lot worse with headphones that with speakers.

Now, there are some aspects that are better with speakers but that is a very small list compared to what headphones are good for.

Just about every difficult double blind test I have passed which countless others fail (or are scared of taking the test) have been with headphones.

 

[amirm]

I have taken both Archimago and AIX Records double blind tests using foobar ABX plug in and passed most of them:

foo_abx 1.3.4 report
foobar2000 v1.3.2
2014/08/02 13:52:46

File A: C:\Users\Amir\Music\Archimago\24-bit Audio Test (Hi-Res 24-96, FLAC, 2014)\01 - Sample A - Bozza - La Voie Triomphale.flac
File B: C:\Users\Amir\Music\Archimago\24-bit Audio Test (Hi-Res 24-96, FLAC, 2014)\02 - Sample B - Bozza - La Voie Triomphale.flac

13:52:46 : Test started.
13:54:02 : 01/01 50.0%
13:54:11 : 01/02 75.0%
13:54:57 : 02/03 50.0%
13:55:08 : 03/04 31.3%
13:55:15 : 04/05 18.8%
13:55:24 : 05/06 10.9%
13:55:32 : 06/07 6.3%
13:55:38 : 07/08 3.5%
13:55:48 : 08/09 2.0%
13:56:02 : 09/10 1.1%
13:56:08 : 10/11 0.6%
13:56:28 : 11/12 0.3%
13:56:37 : 12/13 0.2%
13:56:49 : 13/14 0.1%
13:56:58 : 14/15 0.0%
13:57:05 : Test finished.

----------
Total: 14/15 (0.0%)

As you see, 14 out of 15 right which is almost perfect.

And Mark's test tracks he produced for a test on AVS:
foo_abx 1.3.4 report
foobar2000 v1.3.2
2014/07/10 18:50:44

File A: C:\Users\Amir\Music\AIX AVS Test files\On_The_Street_Where_You_Live_A2.wav
File B: C:\Users\Amir\Music\AIX AVS Test files\On_The_Street_Where_You_Live_B2.wav

18:50:44 : Test started.
18:51:25 : 00/01 100.0%
18:51:38 : 01/02 75.0%
18:51:47 : 02/03 50.0%
18:51:55 : 03/04 31.3%
18:52:05 : 04/05 18.8%
18:52:21 : 05/06 10.9%
18:52:32 : 06/07 6.3%
18:52:43 : 07/08 3.5%
18:52:59 : 08/09 2.0%
18:53:10 : 09/10 1.1%
18:53:19 : 10/11 0.6%
18:53:23 : Test finished.

----------
Total: 10/11 (0.6%)

Again, 10 out of 11.

This doesn't mean high-res sounds better than CD. It however means I know what to listen for and pass such difficult tests.

 

[Kal Rubinson]

Headphones are *the* standard for evaluating lossy audio compression. They allow one to block outside sound, keep channels from masking distortions in another, and you can turn them up to very high volumes to hear the smallest level of distortion.

I grant you that but..........................................

Now, there are some aspects that are better with speakers but that is a very small list compared to what headphones are good for.

The list may be small but I find them critical for enjoying music.

 

[andreasmaaan]

I have taken both Archimago and AIX Records double blind tests using foobar ABX plug in and passed most of them:

foo_abx 1.3.4 report
foobar2000 v1.3.2
2014/08/02 13:52:46

File A: C:\Users\Amir\Music\Archimago\24-bit Audio Test (Hi-Res 24-96, FLAC, 2014)\01 - Sample A - Bozza - La Voie Triomphale.flac
File B: C:\Users\Amir\Music\Archimago\24-bit Audio Test (Hi-Res 24-96, FLAC, 2014)\02 - Sample B - Bozza - La Voie Triomphale.flac

13:52:46 : Test started.
13:54:02 : 01/01 50.0%
13:54:11 : 01/02 75.0%
13:54:57 : 02/03 50.0%
13:55:08 : 03/04 31.3%
13:55:15 : 04/05 18.8%
13:55:24 : 05/06 10.9%
13:55:32 : 06/07 6.3%
13:55:38 : 07/08 3.5%
13:55:48 : 08/09 2.0%
13:56:02 : 09/10 1.1%
13:56:08 : 10/11 0.6%
13:56:28 : 11/12 0.3%
13:56:37 : 12/13 0.2%
13:56:49 : 13/14 0.1%
13:56:58 : 14/15 0.0%
13:57:05 : Test finished.

----------
Total: 14/15 (0.0%)

As you see, 14 out of 15 right which is almost perfect.

And Mark's test tracks he produced for a test on AVS:
foo_abx 1.3.4 report
foobar2000 v1.3.2
2014/07/10 18:50:44

File A: C:\Users\Amir\Music\AIX AVS Test files\On_The_Street_Where_You_Live_A2.wav
File B: C:\Users\Amir\Music\AIX AVS Test files\On_The_Street_Where_You_Live_B2.wav

18:50:44 : Test started.
18:51:25 : 00/01 100.0%
18:51:38 : 01/02 75.0%
18:51:47 : 02/03 50.0%
18:51:55 : 03/04 31.3%
18:52:05 : 04/05 18.8%
18:52:21 : 05/06 10.9%
18:52:32 : 06/07 6.3%
18:52:43 : 07/08 3.5%
18:52:59 : 08/09 2.0%
18:53:10 : 09/10 1.1%
18:53:19 : 10/11 0.6%
18:53:23 : Test finished.

----------
Total: 10/11 (0.6%)

Again, 10 out of 11.

This doesn't mean high-res sounds better than CD. It however means I know what to listen for and pass such difficult tests.

Very impressive Amir What equipment were you using for these tests?

 

[JustIntonation]

Headphones are *the* standard for evaluating lossy audio compression. They allow one to block outside sound, keep channels from masking distortions in another, and you can turn them up to very high volumes to hear the smallest level of distortion. My audiophile playlist sounds a lot worse with headphones that with speakers.

Now, there are some aspects that are better with speakers but that is a very small list compared to what headphones are good for.

Just about every difficult double blind test I have passed which countless others fail (or are scared of taking the test) have been with headphones.

I really can't agree to this.
Yes, speakers in a room means the room direct reflections and reverb+modes masks a LOT of details. But I was talking about speakers in a very well treated room, like a big treated studio or treated mastering room where care was taken in placement etc, or best: speakers in an anechoic room.
Then there is simply no comparison in my opinion. You get not only all the detail that you get with headphones but you actually get a lot more detail. A headphone driver is no match for a good dedicated tweeter at all. On top of that you can never get the head related transfer function right with headphones which means the audio is always unnatural and this prevent hearing "into" the audio in my opinion. And lastly those single driver headphones all have serious frequency problems especially bad in the treble, with associated ringing etc. I'd also be scared to see IMD distortion measurements of headphones I can already guess they won't look pretty.
I've heard a lot of headphones, Sennheiser HD800, HD600, HD590, HD650, AKG K1000, K701, Beyerdynamic DT1990 and more and really really given them a try to get the frequency response right based on many pro independent measurements and many different correction curves, with good measuring DACs and headhone amps etc etc. I could only conclude the more I learned about headphones that it will simply not get there no matter how much you spend.
Room treatment is the biggest path to real quality audio, the more the better in my opinion. After this speakers and their placement are the most important, then DAC and amp it seems to me. Room treatment is what makes speakers better and hell in an anechoic room even much tighter and deader than any headphone I've ever heard.

edit: well I do understand headphones are the standard, but only because almost nobody has a good room!

 

[JustIntonation]

I mean. In my opinion the people who have high audio quality are the studio users. If you read for instance the Gearslutz forums you'll see a lot of talk about room treatment. And these people are often in medium to large rooms (the ones that aren't can forget about good results), treat them fairly well according to a system of usually absorbing all direct reflections (except floor) and big bass traps for treating the room modes in the bass. And they're listening near-field, about 1 meter distance on average ear to speaker or so. And they are STILL complaining about the rooms effects on the sound. And rightly so!
It really takes a lot to take the room out of the equation enough. Massive amounts of thick low gass flow resistance absorption and room dimensions that allow for good treatment (high ceiling, big enough). But once you do this there is no comparison. Headphones are a toy compared to this. And living room systems in untreated living rooms are basically never high resolution in most ways in my experience no matter the cost.

See here what it takes: http://www.acousticmodelling.com/porous.php
And you must also get the bass otherwise the result is very uneven sounding. Thick, low GFR, and full walls and ceiling covered (I'm personally also doing the floor this time). There is no better investment in audio quality in my opinion. (And it usually still means listening near-field, don't get me wrong. To go anechoic enough for far field listening requires even more)

 

[andreasmaaan]

I mean. In my opinion the people who have high audio quality are the studio users. If you read for instance the Gearslutz forums you'll see a lot of talk about room treatment. And these people are often in medium to large rooms (the ones that aren't can forget about good results), treat them fairly well according to a system of usually absorbing all direct reflections (except floor) and big bass traps for treating the room modes in the bass. And they're listening near-field, about 1 meter distance on average ear to speaker or so. And they are STILL complaining about the rooms effects on the sound. And rightly so!
It really takes a lot to take the room out of the equation enough. Massive amounts of thick low gass flow resistance absorption and room dimensions that allow for good treatment (high ceiling, big enough). But once you do this there is no comparison. Headphones are a toy compared to this. And living room systems in untreated living rooms are basically never high resolution in most ways in my experience no matter the cost.

See here what it takes: http://www.acousticmodelling.com/porous.php
And you must also get the bass otherwise the result is very uneven sounding. Thick, low GFR, and full walls and ceiling covered (I'm personally also doing the floor this time). There is no better investment in audio quality in my opinion. (And it usually still means listening near-field, don't get me wrong. To go anechoic enough for far field listening requires even more)

I don't disagree with you on principle, but I have to say that when it comes to my preference, significant room reflections are a large part of it for me, particularly side wall reflections. I've never had a great time listening to a stereo setup in an acoustically dead room (multichannel is a different story of course). When it comes to stereo, getting the room "out of the equation" is the last thing I want to do if musical enjoyment is a goal. I would happily sacrifice the better imaging and resolution of detail that a dead room provides in favour of the spaciousness and immersiveness of a well treated, but still relatively reflective, room.

Couldn't agree more when it comes to small rooms though; they are awful IMHO. A big room, a good balance of reflection and absorption (in the right places), and listening in the far field are what do it for me. Just a personal opinion here though of course.

 

[JustIntonation]

I don't disagree with you on principle, but I have to say that when it comes to my preference, significant room reflections are a large part of it for me, particularly side wall reflections. I've never had a great time listening to a stereo setup in an acoustically dead room (multichannel is a different story of course). When it comes to stereo, getting the room "out of the equation" is the last thing I want to do if musical enjoyment is a goal. I would happily sacrifice the better imaging and resolution of detail that a dead room provides in favour of the spaciousness and immersiveness of a well treated, but still relatively reflective, room.

Couldn't agree more when it comes to small rooms though; they are awful IMHO. A big room, a good balance of reflection and absorption (in the right places), and listening in the far field are what do it for me. Just a personal opinion here though of course.

Agreed that reflections can be a veary pleasant colouration. And even required as you say for enjoyment of certain music. Especially if it's close mic'd acoustic music (sounds like your ear is right next to the instrument in an anechoic room), old music or music mixed without a satisfactory amount of reverb and depth in it.
But it is still colouration that masks a LOT. If the music itself has the spaciousness and immersiveness then in an anechoic room you get it fully in spades. In fact my most immersive and spacious music experiences are in an anechoic room. Your listening space can sound like being in a cathedral or an artifical moving space etc etc.
I did place the speakers farther apart than an equilateral triangle and plan to do so again. Somehwere halfway between a headphone and regular speaker setup. But the imaging can be second to none with such a setup.
I do ask if you've listened to a dead enough room? If you can't hear an echo when you clap your hands this does not mean it's dead enough for a truly amazing sound experience. With dead I mean dead including at least the upper bass. I think such a system can be very musically enjoyable, but it is critical though of errors in the music production and errors in the speakers etc.
But it's not for everybody of course. But in relation to testing the human ears / brain, I'd say this is how to test things. And one can afterwards say that the audibility of certain things will be masked by most rooms in most listening situations.
Anyhow when my room and new speakers are finished (still waiting for the Hypex amps before I can continue.. there's been delays from their factory) I'll do a lot of ABX tests in Foobar with every meaningful file I can find out there and share the results.

edit: as far as "dead" goes. When I had the big semi-anechoic room. The thing that bothered me most besides errors in speakers (crossover/tweeter quality/distortion/port etc) was the single reflection that was left. The floor. It was SO audible and got in the way. I tried a carpet and a few small bass traps on the floor but it didn't do enough and only coloured the reflection. Also couldn't stand a desk in front of me. Making a small acoustically transparant desk this time (mesh wire with some soft fabric on it). It's really unbelievable how much and how precise you hear when you go truly dead. Anyhow this time the floor will be ~40cm thick absorption with a mesh wire on it

 

[amirm]

Very impressive Amir What equipment were you using for these tests?

Usually my Etymotic IEMs.

 

[amirm]

edit: well I do understand headphones are the standard, but only because almost nobody has a good room!

That is not it at all. Hearing non-linear/small distortions is not about enjoyment. Nor does frequency response matter much. We are trying to hear the smallest details. I have tried to pass the same blind tests that I pass on headphones with speakers and it is often much more difficult. Speaker do well if the impact is in low frequencies as you can better feel that difference. Above those frequencies, sound isolating headphones are hugely superior. We used to have a $350K listening room at Microsoft but we still did vast majority of our testing with headphones.

So to be clear, for enjoying music I far prefer speakers to headphones. But again, that is not the point. Indeed, the more you enjoy the music, the less critical you will be in analyzing it for distortion. In video, we often turn off the color for example to see distortions in black and white. Same with music.

 

[JustIntonation]

Above those frequencies, sound isolating headphones are hugely superior. We used to have a $350K listening room at Microsoft but we still did vast majority of our testing with headphones.

But you're talking about a room with reflections and reverb right? Vs headphones.
I'm talking about a dead room and nearfield / midfield listening. No reflections, no reverb, preferably close to no room modes in the bass. Giving actual flat measurements like a speaker measurement shows in an anechoic room.
Really such a listening condition has far superior detail to any headphone I've heard.

 

[amirm]

Really such a listening condition has far superior detail to any headphone I've heard.

When you say this, did you try to pass difficult double blind tests of distortion? That is the ultimate test.

 

[JustIntonation]

When you say this, did you try to pass difficult double blind tests of distortion? That is the ultimate test.

I will do this when my new room and speakers are finished.
When I had my previous room I didn't do ABX blind tests yet. I did listen to it daily for almost a year and had various speakers in there and did measurements of the speakers in the room. But that was basically it.
Never got it "right enough" though overall (it gets reaaaly critcal). But learned enough from it to go for a second go

 

[JustIntonation]

Oh but as far as the comparison to headphones go. I'm very certain I heard more detail with speakers in that room. Far more.
There's really no comparison. Overall in smooth frequency response is a big difference. But as I said before, the treble quality is just not comparable. In comparison I've never heard a headphone that does that anywhere near the level of a speaker. With a speaker you can tell if the treble sits wrong in a mix etc, don't even nead an anechoic room for that, a fairly well treated room and nearfield listening will already give that with the right tweeters. And the treble extension isn't there either for me with headphones. Last time I tested my treble range I belief I heard till about 16k (I'm 41 so that's ok enough), but while headphones can reproduce treble it just doesn't have anywhere near the info to me. Though I've only heard dynamic headphones, perhaps electrostatics are an improvement in this area.