• WANTED: Happy members who like to discuss audio and other topics related to our interest. Desire to learn and share knowledge of science required. There are many reviews of audio hardware and expert members to help answer your questions. Click here to have your audio equipment measured for free!

General debate thread about audio measurements

Esotechnik

Member
Joined
Mar 9, 2019
Messages
72
Likes
5
Location
Russia
Noise floor is also lower in APx555 analyzer:
1 000 000 point averaging is a bad idea for detect occasional "glitches".
Digital oscilloscopes has flash or SAR ADC, not delta-sigma.
Old phonograms (<1995 yr.), digitized via SAR ADC is more preferred vs. modern, generated by D-S ADC (with equal music styles and dynamic range).

This is other test system:
MSB_DAC-Measurement-Flow-Chart-Pico.jpg
 
Last edited:

Shadrach

Addicted to Fun and Learning
Joined
Feb 24, 2019
Messages
662
Likes
947
Well, I've waded through this one.:confused:
I've really only got one point to make but it's really important; there is no music!
People keep talking about the effects of various parameters on the music. These boxes of electronics don't make music, they don't transport music, they don't decode music, they wouldn't know music if you took them to a concert. There is no music on the medium, no music comes out of the speakers. The music is all in your head.
It's signal processing. The measurements are about how well, or badly a particular unit does this. The comparisons are to other signal processing units. Some measure better than others. You may well not be able to tell the difference by ear. That's not the point.
 

amirm

Founder/Admin
Staff Member
CFO (Chief Fun Officer)
Joined
Feb 13, 2016
Messages
44,368
Likes
234,381
Location
Seattle Area
1 000 000 point averaging is a bad idea for detect occasional "glitches".
There is no million point averaging. The averaging and FFT lengths are set exactly to what was measured in the link you provided. For one test it is 20 averages and the other 10. And if that gets rid of occasional "glitches," that much the better. It makes the measurement system immune to issues you imagine to be there (but not demonstrated with any measurements).
 
Joined
Jan 8, 2019
Messages
17
Likes
3
Question: is the ESS "hump" audible?
 

Shadrach

Addicted to Fun and Learning
Joined
Feb 24, 2019
Messages
662
Likes
947
Interesting thread.;)
A point about measurement. It doesn't matter which yardstick one uses as long as you use the same yardstick for every unit measured. It's a very basic rule when measuring anything. If the divisions on your stick are very small then your accuracy is better. The point that to some amirm's tolerances are too tight, would have some merit if none of the units fell within tolerance. That hasn't been the case here from what I've read. If company A can make a product that falls within these tolerances for X amount of money, then it seems fair to point out that company B's product at Y amount of money doesn't.
To the best of my knowledge amirm only applies a value judgement to the product briefly at the end of his reviews. This is his subjective evaluation. You can dismiss, it or not bother reading it as I do.
The measurements are about design competence; they are not about whether or not it's going to sound good to you.
Part of amirm's mission (I'm sure he will correct me if I'm wrong) is to demonstrate that often the subjective evaluations of equipment are not born out by the measurements. So, if we talk about a building brick for a moment, the measurements will tell you its size, how much compressive force it can take before deformation, its colour etc. In your building project you may not need a brick that can take 40 tons per square inch and a 5 micron dimensional tolerance and is yellow; a red brick with 1mm tolerance at 20 tons limit may well do the job. Buy the red brick then but it's pointless trying to argue that one brick is better than the other.
 

sergeauckland

Major Contributor
Forum Donor
Joined
Mar 16, 2016
Messages
3,440
Likes
9,100
Location
Suffolk UK
Interesting thread.;)
A point about measurement. It doesn't matter which yardstick one uses as long as you use the same yardstick for every unit measured. It's a very basic rule when measuring anything. If the divisions on your stick are very small then your accuracy is better. The point that to some amirm's tolerances are too tight, would have some merit if none of the units fell within tolerance. That hasn't been the case here from what I've read. If company A can make a product that falls within these tolerances for X amount of money, then it seems fair to point out that company B's product at Y amount of money doesn't.
To the best of my knowledge amirm only applies a value judgement to the product briefly at the end of his reviews. This is his subjective evaluation. You can dismiss, it or not bother reading it as I do.
The measurements are about design competence; they are not about whether or not it's going to sound good to you.
Part of amirm's mission (I'm sure he will correct me if I'm wrong) is to demonstrate that often the subjective evaluations of equipment are not born out by the measurements. So, if we talk about a building brick for a moment, the measurements will tell you its size, how much compressive force it can take before deformation, its colour etc. In your building project you may not need a brick that can take 40 tons per square inch and a 5 micron dimensional tolerance and is yellow; a red brick with 1mm tolerance at 20 tons limit may well do the job. Buy the red brick then but it's pointless trying to argue that one brick is better than the other.

For many years now, I've applied the principle of 'adequacy' to any judgements I make about anything. What do I need the item to do, and how well, then look at the specs / measurements to decide which item comes closest. I then buy the cheapest that meets my requirements. The important part is to specify one's requirements completely, including such intangibles as 'build quality', serviceability' and 'appearance' which I accept are subjective, but it's my spec I'm buying against. What this means is that I don't necessarily by the cheapest of any one item, as even if the measurements are adequate, the item fails on the build quality or other 'subjective' parts.

It can therefore be said that I'm no different to subjectivists who buy on perceived (non blind tested or level matched) sound quality, but in reality, because evaluations are not done blind, their choice is perhaps made on anything but sound quality, whereas in my case, I never bother to listen first to anything I buy in audio. I just need a detailed set of measurements, which is why I find this forum so useful. It's a pity that it's impractical for Amirm to do loudspeaker measurements....what's wrong with an anechoic chamber at home......

I've also been criticised for buying what's 'adequate' rather than what's excellent, but what gets missed is that what I consider adequate may be their excellent, or indeed vice versa.

As very little in the way of HiFi electronics isn't transparent these days, I don't see the problem with buying on facilities required and perceived build quality.

S
 

JJB70

Major Contributor
Forum Donor
Joined
Aug 17, 2018
Messages
2,905
Likes
6,148
Location
Singapore
Indeed, in some ways the debate is often wrongly framed. If you like something because it looks nice, it is made of nice materials, you lust after a status symbol or you just think it sounds nice regardless of what anyone else thinks then nobody has to justify their choices to anyone else. It'd be a boring world if we all thought the same and acted entirely rationally all the time (how many of us chose our partners for rational and logical reasons?).
However, what is wrong is to deny the value of measurement in objective terms. If you buy something which is less accurate than alternatives or which costs 10x more than an alternative that measures the same then that's a personal decision but don't try and argue that measurements are wrong or that the magical and wondrous properties of audio signals cannot be measured. There is nothing especially unique or challenging about audio measurement if you have the necessary equipment and know how to use it. There is truth in the adage that if you can't measure it you can't improve it. All good engineering I have observed was based on sound application of good scientific and engineering principles and analytical tools are there to be used. Yes, you can like something or think it is nicer, but that isn't the same as trying to claim it is better in objective terms.
The danger I do see in measurement is that it is easy to become fixated on figures. Once you achieve audible transparency then while continued improvement is not a bad thing neither will it do much for the listening experience.
 

FrantzM

Major Contributor
Forum Donor
Joined
Mar 12, 2016
Messages
4,335
Likes
7,717
Indeed, in some ways the debate is often wrongly framed. If you like something because it looks nice, it is made of nice materials, you lust after a status symbol or you just think it sounds nice regardless of what anyone else thinks then nobody has to justify their choices to anyone else. It'd be a boring world if we all thought the same and acted entirely rationally all the time (how many of us chose our partners for rational and logical reasons?).
However, what is wrong is to deny the value of measurement in objective terms. If you buy something which is less accurate than alternatives or which costs 10x more than an alternative that measures the same then that's a personal decision but don't try and argue that measurements are wrong or that the magical and wondrous properties of audio signals cannot be measured. There is nothing especially unique or challenging about audio measurement if you have the necessary equipment and know how to use it. There is truth in the adage that if you can't measure it you can't improve it. All good engineering I have observed was based on sound application of good scientific and engineering principles and analytical tools are there to be used. Yes, you can like something or think it is nicer, but that isn't the same as trying to claim it is better in objective terms.
The danger I do see in measurement is that it is easy to become fixated on figures. Once you achieve audible transparency then while continued improvement is not a bad thing neither will it do much for the listening experience.
Is there an emoji for liking a post a lot!!!?
Great Post.
 

Shadrach

Addicted to Fun and Learning
Joined
Feb 24, 2019
Messages
662
Likes
947
For many years now, I've applied the principle of 'adequacy' to any judgements I make about anything. What do I need the item to do, and how well, then look at the specs / measurements to decide which item comes closest. I then buy the cheapest that meets my requirements. The important part is to specify one's requirements completely, including such intangibles as 'build quality', serviceability' and 'appearance' which I accept are subjective, but it's my spec I'm buying against. What this means is that I don't necessarily by the cheapest of any one item, as even if the measurements are adequate, the item fails on the build quality or other 'subjective' parts.

It can therefore be said that I'm no different to subjectivists who buy on perceived (non blind tested or level matched) sound quality, but in reality, because evaluations are not done blind, their choice is perhaps made on anything but sound quality, whereas in my case, I never bother to listen first to anything I buy in audio. I just need a detailed set of measurements, which is why I find this forum so useful. It's a pity that it's impractical for Amirm to do loudspeaker measurements....what's wrong with an anechoic chamber at home......

I've also been criticised for buying what's 'adequate' rather than what's excellent, but what gets missed is that what I consider adequate may be their excellent, or indeed vice versa.

As very little in the way of HiFi electronics isn't transparent these days, I don't see the problem with buying on facilities required and perceived build quality.

S
I wish I was as utilitarian, but I'm not.
Joining this forum (always easier to blame someone else) encouraged me to buy the first bit of new kit I've bought for maybe 15 years.;)
I bought the JDS Labs Atom, purely on the specifications kindly supplied by amirm. I was curios to see if a state of the art product (I know it's cheap and plastic but it's measured performance is excellent and comparable to any hi end headphone amp I've read specifications on) sounded any different to my current tube driven effects box. I was able to make an informed choice thanks to amirm's measurements.
Out of curiosity I scoured the internet for reviews. Interesting to read there were a number of reviewers who while stating that it was a good value product, still implied that a high end (higher priced) product could be an upgrade.:facepalm:. Through impatience I cancelled the order and bought the Topping DX3 Pro which was instantly available. The Atom had already been dispatched when I cancelled so now I have both.
I've been listening to all three units a lot. I can perceive an audible difference between the WAD 83 and the other two very easily and I'm a bit disturbed that I have been content to listen to what to considerable distortion for many years now. So much for any audiophile credentials.:p
For the first time for many years I can sit and listen and think to myself 'so this is more like what that recording is supposed to sound like'.:cool:
I wouldn't have been able to do this without amirm's work here on ASR. I'll ignore the fact that he is just costing me money and undermining my complacency.:rolleyes:
I bought my Exposure amplifiers because I heard them in a friends system, like how they looked and had the money to spare. I've had them, with other stuff in between for over 20 years (I'm not hitting the consumer bracket very well either)
My loudspeakers are totally unsuitable for my current location, but I built them (a number of iterations now) and have got used to the sound they make.
I've tried other speakers, but I just don't have the same level of attachment and therefor, they don't sound as good, even if they are more accurate.
My complaint about ASR is amirm will insist on trying to have a life rather than measuring stuff all day.;)
 

JJB70

Major Contributor
Forum Donor
Joined
Aug 17, 2018
Messages
2,905
Likes
6,148
Location
Singapore
I bought the JDS Atom, partly on the basis of how it measured, partly on the basis of my confidence in JDS as a company, a lot on the basis of it having achieved such great performance for peanuts. And I haven't been disappointed, for sure I would prefer a nicer metal case and would pay more for them to use something like the March Audio style casework but in terms of a headphone amplifier doing what a headphone amplifier needs to do then I honestly see no point going any further. I think it is a terrific product and a prime example of why excellent engineering and performance do not need to be expensive.
 

Krunok

Major Contributor
Joined
Mar 25, 2018
Messages
4,600
Likes
3,065
Location
Zg, Cro
The danger I do see in measurement is that it is easy to become fixated on figures. Once you achieve audible transparency then while continued improvement is not a bad thing neither will it do much for the listening experience.

Very much so. And if you don't realise that simple truth you easilly end up spending your money at a component with better numbers which you won't be able to hear instead of another component of your listening chain where you would immediately notice improvement. Spending money on a DAC/amp with better numbers instead of buying better headphones/speakers is a typical example.
 

Krunok

Major Contributor
Joined
Mar 25, 2018
Messages
4,600
Likes
3,065
Location
Zg, Cro
And I haven't been disappointed, for sure I would prefer a nicer metal case and would pay more for them to use something like the March Audio style casework

Well, as @March Audio is quite focused on the quality caseworks and is not affraid to put another company's electronics in them there is always a hope we will one day see March Audio Atom Edition. Pun intended. :D
 
Last edited:

sergeauckland

Major Contributor
Forum Donor
Joined
Mar 16, 2016
Messages
3,440
Likes
9,100
Location
Suffolk UK
I rather like this. Matrix have been around for some time doing something similar to ASR.
http://matrixhifi.com/ENG_marco.htm
Click on the Philosophy link on the left.
It was their blind test that first alerted me to the Behringer A500. That and Peter Aczel's measurements confirmed that the A500 would be 'adequate' for my needs to drive my active 'speakers. Not been disappointed.

It's a pity that they have only documented the one blind test, and have little other activity, even on their forum.

S.
 

josh358

Senior Member
Joined
Jun 13, 2017
Messages
493
Likes
387
Nice but a bit strange discussion about distortion...

I think that the sound system at home should not add anything to the original "data" on the media. But we live in an imperfect world. The greatest source of distortion is the loudspeaker, no doubt. But still we can hear and measure distortion and other faults of other gadgets between source and the speaker too. Loudspeaker is a very unlinear device with many problems, of which distortion is not the greatest. When we measure it's acoustic performance one big problem is noise floor - at low spl it masks all distortion. Distortion % is not constant, it gets bigger when spl is raised, and this is nonlinear.

I can hear differencies in amplifiers sometimes, even with moderate spl and midrange speakers. I don't know where it comes but one of my suspects is the amount and type of feedback used. Power rating and distortion specs don't tell it, unless there is something really strange like most tube amps have.

I understand Kal well and respect the honesty! Sorry, I always skip the impressions text of reviews in S'phile and check Atkinon's measurements and comments first. Then I look at Specs and price. Then only sometimes I read the conclusions/final words of the review and usually get surprised or confused.
LOL, John Atkinson would agree with you that measurements and subjective impressions can diverge, he gave an address about that which is online somewhere. I think it becomes more understandable as you look into the nature of how we perceive these things, for example, Harman developed an algorithm that they say correlates subjective impressions of speaker quality with on axis and polar response, but while Stereophile does some of these measurements we don't have that algorithm. There's also a published formula for the audibility of harmonic distortion components, which increases in the higher order harmonics. Again, not something we have access to when we see a distortion plot. So I've found that it's not a good idea to "listen with your eyes," and go for the measurements that look better.

Here's another example -- I have an Exasound 9038Pro DAC here and a Gungnir Multibit as well. The Exasound measures better than the Gumby and in many respects, it is audibly better -- more detailed, more solid imaging, more resistant to jitter, etc. But personally, I prefer the Gumby, because it doesn't have the high frequency glare that multibit delta-sigma DAC's do. It sounds more analog and, more importantly, more like live music, on my system, anyway (but might not on less revealing speakers or on different kinds of music). So where's the measurement for that? I'm sure there is one, but I don't know what it is, just that multibit delta-sigma chips tend to have digital glare in the highs while ladder DAC's don't (though they have their own issues).

Curious that you can hear these differences on speakers that have much higher distortion than the electronics, isn't it? Though lower distortion speakers are I think more revealing -- at the extreme, electrostatics, which will show up just about anything.
 

Purité Audio

Master Contributor
Industry Insider
Barrowmaster
Forum Donor
Joined
Feb 29, 2016
Messages
9,045
Likes
12,143
Location
London
LOL, John Atkinson would agree with you that measurements and subjective impressions can diverge, he gave an address about that which is online somewhere. I think it becomes more understandable as you look into the nature of how we perceive these things, for example, Harman developed an algorithm that they say correlates subjective impressions of speaker quality with on axis and polar response, but while Stereophile does some of these measurements we don't have that algorithm. There's also a published formula for the audibility of harmonic distortion components, which increases in the higher order harmonics. Again, not something we have access to when we see a distortion plot. So I've found that it's not a good idea to "listen with your eyes," and go for the measurements that look better.

Here's another example -- I have an Exasound 9038Pro DAC here and a Gungnir Multibit as well. The Exasound measures better than the Gumby and in many respects, it is audibly better -- more detailed, more solid imaging, more resistant to jitter, etc. But personally, I prefer the Gumby, because it doesn't have the high frequency glare that multibit delta-sigma DAC's do. It sounds more analog and, more importantly, more like live music, on my system, anyway (but might not on less revealing speakers or on different kinds of music). So where's the measurement for that? I'm sure there is one, but I don't know what it is, just that multibit delta-sigma chips tend to have digital glare in the highs while ladder DAC's don't (though they have their own issues).

Curious that you can hear these differences on speakers that have much higher distortion than the electronics, isn't it? Though lower distortion speakers are I think more revealing -- at the extreme, electrostatics, which will show up just about anything.
‘High frequency glare of multibit delta sigma dacs’!
Please save that shit for other forums.
Keith
 

josh358

Senior Member
Joined
Jun 13, 2017
Messages
493
Likes
387
‘High frequency glare of multibit delta sigma dacs’!
Please save that shit for other forums.
Keith
Sorry if I violated some kind of forum taboo, I've enjoyed the measurements but I haven't contributed here before so don't know what they are.

That said I could amend this to the intentionally underdamped impulse response of the filters in the ESS chips, assuming that's what causes the issue (and I'm not sure that it is, although it certainly has an effect on the sound, which is why ESS does it even though they say it isn't technically correct).

As an engineer, I'm always curious about the relationship between measurements and subjective impressions, as the former are of little use without first establishing that correlation.
 
Last edited:

josh358

Senior Member
Joined
Jun 13, 2017
Messages
493
Likes
387
Interesting discussion. Subjectivists usually say that it is impossible for live music to be reproduced faithfully. This makes limited sense because there are all kinds of things that may colour live performances -- from concert hall (acoustic space effects for symphonic performances) to amplification effects (amplifier distortion in contemporary performances).

I tend to lean the other way... give me sources, preamps, and amps that allow the least possible colouring and work with loudspeakers and spaces to achieve the necessary effect. The ABH2 fits within this model nicely, whereas a great many tube and solid state consumer and so-called pro shite do not, according to objective measurements.

The measurements taken by Amir and others elsewhere on the web demonstrate that Benchmark has done a very credible job designing and building the ABH2. Since these measurements are pretty much irrefutable at this stage, one may conclude that Benchmark has come closer to reaching amplifier excellence than a great many manufacturers of audio equipment (past or present).

If transparency is not the goal, then perhaps there's room for another conversation like source selection, decoding, room set-up, equalization etc. But these lie outside the role of an amplifier, which is to provide gain and gain only, to an audio signal.
I used to believe that, but I've learned over the years that the issues are a bit more complex. For example, I referred earlier to the masking of offensive higher order harmonics by the purposeful addition of lower order harmonics. The more colored tube amps would be an example of this.

Another consideration is *what* you measure. Studies suggest that harmonic distortion is inaudible on a sine wave at the levels in any good amplifier. And when you look at DAC's, the measured steady state distortion is typically so low that it is almost certainly inaudible. And yet it has also been demonstrated that there are other forms of distortion that are very audible in small amounts. Crossover notch distortion, which as I mentioned is audible in very low amounts, is an example. A couple of years John Siau told me that he'd successfully ABX'd crossover notch distortion at a remarkably low level, I forget what it was but it was impressive. And what is inaudible on a sine wave is *not* necessarily inaudible on music, even though it is composed of sine waves. But that's at a single frequency. Play those sine waves together, and they will intermodulate. And intermodulation distortion sounds awful.

The conventional measures, the ones in product specifications and reviews, are typically only a subset of significant measurements, so can only describe some aspects of the sound.
 

Shadrach

Addicted to Fun and Learning
Joined
Feb 24, 2019
Messages
662
Likes
947
That's why (since it isn't practical for me to do blind tests) I always seek verification for my subjective impressions, either by having a non-audiophile who knows live music listen and tell me what they've heard, or reading reviews *after* I listened to see if the reviewer heard what I did. In my experience, they almost always did, although we don't always agree on what we like.
If they're not ABX, they don't count, sorry and all that.;)
 
Top Bottom