• WANTED: Happy members who like to discuss audio and other topics related to our interest. Desire to learn and share knowledge of science required. There are many reviews of audio hardware and expert members to help answer your questions. Click here to have your audio equipment measured for free!

TEST FILES. How to verify the quality of your audio system and other audio tests.

Soniclife

Major Contributor
Forum Donor
Joined
Apr 13, 2017
Messages
4,500
Likes
5,417
Location
UK
None of the files were clipping (stayed well below clipping levels) but they all differ in amplitude, peak levels and harmonic contents.
I had assumed they were not level matched, you can always tell when it's not explicitly stated how they were matched. Thanks for checking.
 

solderdude

Grand Contributor
Joined
Jul 21, 2018
Messages
15,891
Likes
35,912
Location
The Neitherlands
I guess in real life you are not able to differentiate between two red fruits such as strawberry and watermelon :)

What's it have to do with the discussed files and what they really represent ?
 

solderdude

Grand Contributor
Joined
Jul 21, 2018
Messages
15,891
Likes
35,912
Location
The Neitherlands
I had assumed they were not level matched, you can always tell when it's not explicitly stated how they were matched. Thanks for checking.

The strange part here is that the guy who made the files (not Maty) mentioned all the files were 'normalized'.
I looked at the original article from the guy who made this comparison (= not a test).
Of course one can 'hit' the normalize button and assume this makes all files have the same peak level and subsequentially assume also the same average levels, but when the peak levels and average levels are not the same the average levels will differ.
This easily happens when using a microphone and speakers in a room so the 'normalizing' does not seem to have had the desired effect.
 

Wombat

Master Contributor
Joined
Nov 5, 2017
Messages
6,722
Likes
6,459
Location
Australia

solderdude

Grand Contributor
Joined
Jul 21, 2018
Messages
15,891
Likes
35,912
Location
The Neitherlands
I know the SF10 plots do not show the exact peaks zoomed out over an entire song BUT when differences look this big the differences are really substantial.

Below all the plots of the different files in an animated GIF in a row. Big enough to be audible ? Caused by differences in amps ? You can even see the 180 degrees phase shift with some files.

 
Last edited:

Sgt. Ear Ache

Major Contributor
Joined
Jun 18, 2019
Messages
1,894
Likes
4,151
Location
Winnipeg Canada
I guess in real life you are not able to differentiate between two red fruits such as strawberry and watermelon :)

So what you are wanting us to do is to prove we can hear the difference (or at least say we can or can't) between these objectively different files? Then, if we say we can't, it will validate your claims that you can hear all sorts of "golden ear" differences that don't show up in objective measurements. I see. Here's the problem... if we could compare two files as Solderdude has above and see those sorts of differences between them in measurable terms, none of us would claim it isn't possible to hear those differences. But when you compare for example an mp3 at 320kps and an flac file you don't see those sorts of differences (assuming good level matching of course.) So when people claim they can hear a difference between two files that measure nearly identically (unlike the files you are asking us to listen to) but they haven't done any actual blind testing of those claims, that's when we call BS on it. In other words, nobody has an issue with folks claiming they can taste the difference between a strawberry and a watermelon. We have a problem with people claiming they can taste the difference between a strawberry and a slightly smaller strawberry.
 
Last edited:

SIY

Grand Contributor
Technical Expert
Joined
Apr 6, 2018
Messages
10,386
Likes
24,752
Location
Alfred, NY
Why would anyone waste time with a badly implemented "experiment"? Set files up correctly and maybe someone will try it. What you have there would be tossed out of a 6th grade science fair.
 

Blumlein 88

Grand Contributor
Forum Donor
Joined
Feb 23, 2016
Messages
20,524
Likes
37,057
Why would anyone waste time with a badly implemented "experiment"? Set files up correctly and maybe someone will try it. What you have there would be tossed out of a 6th grade science fair.
Why indeed. It made me think someone wasn't careful when they were MP3. I didn't check, but I'm not surprised they aren't level matched.

I wish everyone would learn, if you don't match levels you have nothing in comparison files. Nothing. I've wasted time on this so many times that is why I saw MP3, probably not level matched, life is too short. I'm only bothering to type this so maybe Maty will understand why we didn't listen to his files. I don't think Maty was trying to mislead anyone. It is too easy to mislead oneself. So check for level matching on things like this.
 
OP
maty

maty

Major Contributor
Joined
Dec 12, 2017
Messages
4,596
Likes
3,161
Location
Tarragona (Spain)
You do not want to understand. Although the two red fruits are very different and not normalized, their taste is so different that a person immediately appreciates the difference. Same here with the clips from the three DIY amps.

In music, regardless of the sound quality, Elvis rock and roll and current reggaeton are easily differentiable.
 

Blumlein 88

Grand Contributor
Forum Donor
Joined
Feb 23, 2016
Messages
20,524
Likes
37,057
You do not want to understand. Although the two red fruits are very different and not normalized, their taste is so different that a person immediately appreciates the difference. Same here with the clips from the three DIY amps.

In music, regardless of the sound quality, Elvis rock and roll and current reggaeton are easily differentiable.
Okay let us go with this. I compare files from a 250 watt SS amp and 3 watt SET both driving an average low efficiency speaker. Could I hear a difference in those without level matching? Most likely I could. Does this surprise me or inform me in some new way? No not at all. I would have expected that difference. A waste of time on something obvious.

I'm continually amazed and mystified as to why it is so hard to get people to simply match levels. It is so easy to do. I also find if someone balks at this thinking it unimportant, then I really am wasting time on any of their experiments.
 
OP
maty

maty

Major Contributor
Joined
Dec 12, 2017
Messages
4,596
Likes
3,161
Location
Tarragona (Spain)
What you comment would be the required if the comparison were from FLAC vs MP3 or WAV or 16/44 vs 24/96 or similar tests.
 

Blumlein 88

Grand Contributor
Forum Donor
Joined
Feb 23, 2016
Messages
20,524
Likes
37,057
What you comment would be the required if the comparison were from FLAC vs MP3 or WAV or 16/44 vs 24/96 or similar tests.
No level matching is a job #1 critical step in comparing anything in audio. Plus if you skip this step nothing else you do is telling of anything involving listening to things.
 
OP
maty

maty

Major Contributor
Joined
Dec 12, 2017
Messages
4,596
Likes
3,161
Location
Tarragona (Spain)
The only valid conclusion of the test, with those three DIY amps, is if we are able to differentiate sound profiles, with the same audio quality, low.

Not if one is better than another, for which yes it would have to be a much more rigorous test, of course.

The song Always on my mind is easily distinguishable between the Elvis version and that of Pet Shop Boys, even if it was an MP3 at 128 kbps or worse, the rhythm is different.
 

SIY

Grand Contributor
Technical Expert
Joined
Apr 6, 2018
Messages
10,386
Likes
24,752
Location
Alfred, NY
The only valid conclusion of the test, with those three DIY amps, is if we are able to differentiate sound profiles, with the same audio quality, low.

They're different levels, so of course they can be differentiated.

How many people have to tell you that you have not set up a correct experiment to answer the question you have, before you start thinking, "Maybe I made a mistake and ought to listen to the good advice I've gotten?" If the answer is, "I will never, ever admit error and try to do things correctly," then I think you should not be participating in an actual technical forum.
 

pkane

Master Contributor
Forum Donor
Joined
Aug 18, 2017
Messages
5,632
Likes
10,207
Location
North-East
The only valid conclusion of the test, with those three DIY amps, is if we are able to differentiate sound profiles, with the same audio quality, low.

Not if one is better than another, for which yes it would have to be a much more rigorous test, of course.

The song Always on my mind is easily distinguishable between the Elvis version and that of Pet Shop Boys, even if it was an MP3 at 128 kbps or worse, the rhythm is different.

They are different because they are at different levels. That’s all anyone here needs to know. Any other comparisons or conclusions cannot be made until you match levels.
 
OP
maty

maty

Major Contributor
Joined
Dec 12, 2017
Messages
4,596
Likes
3,161
Location
Tarragona (Spain)
But nobody has commented yet that differentiates them or not.
 

pkane

Master Contributor
Forum Donor
Joined
Aug 18, 2017
Messages
5,632
Likes
10,207
Location
North-East
But nobody has commented yet that differentiates them or not.

Because everybody who has done controlled testing before knows that it’s not worth the time to A/B non-level matches tracks. It is a foregone conclusion that the louder track will sound better/different. Why would anyone want to waste their time?
 
Top Bottom