Amp comparison by in room measurements. Advice? Conclusion?

[GM3]

To briefly sum things up, got a new Fosi V3 amp to replace a 40lbs pro amp, "clean power is clean power", yadayadayada, heard differences, this being ASR, measurements. So goal of this thread being whether or not we can conclude anything about the amps in these set of measurements. (mostly in bass region)

Spoiler: links to inital posts, not required, but links to amps, speaker and stuff

https://www.audiosciencereview.com/...-audio-v3-amplifier-review.45757/post-1787112
https://www.audiosciencereview.com/...ents-everything-or-nothing.29062/post-1787282
https://www.audiosciencereview.com/...ents-everything-or-nothing.29062/post-1788252

Basically, I just set up a Blue Yeti mic a bit randomly; mostly relatively on axis, 1-2m away, at around woofer height, as I just wanted to measure the relative bass differences I was expecting seeing. (Although reflecting on this, tweeter level would have been better...). Room is quite lively...

Used REW, pink periodic noise, 1 speaker, RTA averaging. The SPL is likely off as no true dBmeter for calibrating REW/Yeti, started off moderately loud, last set was loud enough that it was getting annoying & I could see the bass woofer move quite a bit, but not super loud.

(The QSC has 30Hz protection activate which rolls the bass, but apparently might be kicking it at ~35Hz, as QSC rolls off far quicker <35Hz)

Attempt #1 With open port, V3 in orange.

Attempt #2, With blocked ports. Same location as #1.
This time, the QSC is in orange...

I realized that me being about 2.5 meters away might be affecting the results... So at this point, moved further away...

Attempt #3, moved mic & speakers a bit..
Green is QSC

Green = QSC (same as above, 1/12 smoothing, sorry REW kept switching colors...)

Attempt #4, moved mic around again a little..
Orange is V3

1/12 smoothing

In all honesty, I'm not entirely sure what we can conclude. I think mostly, results might be within the margin of error, and in some way, likely quite a bit of interpretation too in reading the graphs...

My 2c, YMMV, as I said, with ports blocked, I commented that I heard the V3 'struggle' with the ports blocked, seemed to have less bass. And #3 and #4 shows more bass between 35-80Hz with ports blocked for the QSC...

The graph where both speakers are closest together seem to be #1; bass ports open, which again corresponds to my earlier comment. But it could be coincidence as I didn't notice any consistency between the graphs for the amps... (ex; if blocked port altered V3 between 200-400 Hz or 1KHz or whatever, we'd see a consistent effect between different graphs, which I'm not seeing.)

So seems there could indeed be a bass difference; what the measurements seem to illustrate, but think I'd need more measurements to really confirm... Too big margin of error; more samples could help, but yeah, maybe the best I could do with my current setup... So yeah not likely super convincing...

Thoughts? Insight? Advice?

 

[sam_adams]

First off, your images are not showing up. Second, the Blue Yeti is hardly what one could call a measurement mic.

 

[GM3]

First off, your images are not showing up. Second, the Blue Yeti is hardly what one could call a measurement mic.

Thanks! Images should be fixed. (stop looking at me like that!!! ) And yes, aware for the yeti; not using it for any sort of absolute measurements, I'm just trying to demonstrate that there is a bass response difference between two amps. As you can now see, the FR graphs are very jagged, very likely due to the lively room. And as I was thinking, looking back, I probably should have tried to use a sweep as to minimize effects from room/reflections... I'll have to look into how/if REW can do it...

I thought RTA would be good enough, but evidently, maybe not... Though, with 100+ averages, seemed fairly consistent... First time using REW, definitely learning experience for me.

 

[kemmler3D]

First off, your images are not showing up. Second, the Blue Yeti is hardly what one could call a measurement mic.

True, but any mic with remotely linear dynamic response (read: almost any half-decent mic) should be OK for relative measurements like this. We don't know the actual FR but the differences in curves should be roughly/directionally valid as long as the mic didn't move.

@GM3 to my eye this looks like a possible 1-2 dB deficit in the bass region for the V3. Could be audible. Could be that it's running out of power or is poorly suited for your speakers. These things happen.

Questions:

• Did you leave the room or stand in the same EXACT spot for the measurements? The position of your body can affect these things.
• Did you use the random or periodic pink noise? I think for these things you want to use the periodic one.

Personally I would be more interested in seeing a slow sweep (like 2M or something) for this. If the V3 is running out of power, holding a 20-40hz tone for a time should reveal that better than PN.

To draw out differences more, I would also crank the volume as high as you're willing to go.

You also need to very carefully level-match as well as you can before starting, was that part of the process here?

 

[DVDdoug]

To me they look pretty-darn close and and it's probably measurement error/variation. The high frequencies can be especially variable. If repeated the same experiment with the same amp you might get similar results.

I was doing some high frequency measurements once with my SPL meter on a tripod and as I was moving around behind the meter I could hear the reflected and left-and-right going in-and out of phase, making huge variations.* Moving around behind the meter also made significant variations in the measurement, but I don't remember how much.

I'm not worried about the mic. Even though the response isn't flat or calibrated, a 1dB change is a 1dB change (at the same frequency) and mics are very linear unless you overdrive the head amp inside a condenser mic by sticking it in front of a kick drum or loud guitar amp, etc.



* Very noticeable with constant test-tones but usually not noticeable with muisc/program material.

 

[GXAlan]

Run your recordings through DeltaWave.

It’s interesting that the bass is the difference. I saw that exact issue comparing the Fosi V3 against the Marantz PM-10 although the V3 had more bass not less.

Integrated Amplifier Comparison: $90 vs $9000

I continue to hear differences between amplifiers that might be expected to be inaudibly different. That is, for two high-SINAD systems that I'm not running into clipping, I hear differences. I also believe in measurements. My perspective has been consistent: "Everything that can be measured...

www.audiosciencereview.com

 

[GM3]

True, but any mic with remotely linear dynamic response (read: almost any half-decent mic) should be OK for relative measurements like this. We don't know the actual FR but the differences in curves should be roughly/directionally valid as long as the mic didn't move.

Yeah, for recording it's pretty decent, was using cardiod which has apparently quite a bit bass roll-off starting at 200Hz as well as >4Khz... (bottom right)

to my eye this looks like a possible 1-2 dB deficit in the bass region for the V3. Could be audible. Could be that it's running out of power or is poorly suited for your speakers. These things happen.

Questions:

• Did you leave the room or stand in the same EXACT spot for the measurements? The position of your body can affect these things.

Yeah for the first two I was at the same spot, sitting 2.5m on the side of the mic. But the last 2 I moved further away, ~5 meters, sitting still for 1m. And actually, true.. I should have hooked up the laptop's audio, and connected remotely to the laptop, to control the sound + mic with 1 device from another room...

• Did you use the random or periodic pink noise? I think for these things you want to use the periodic one.

Yep, periodic pink noise, full range.

Personally I would be more interested in seeing a slow sweep (like 2M or something) for this. If the V3 is running out of power, holding a 20-40hz tone for a time should reveal that better than PN.

To draw out differences more, I would also crank the volume as high as you're willing to go.

You also need to very carefully level-match as well as you can before starting, was that part of the process here?

Kinda... The V3 vs QSC sound difference I perceived was at all low/med/high volume levels, so I wasn't super worried about getting 0.1dB accuracy, probably +/- 1dB...

Thanks for the help btw! I might try all of that over the weekend.

Run your recordings through DeltaWave.

It’s interesting that the bass is the difference. I saw that exact issue comparing the Fosi V3 against the Marantz PM-10 although the V3 had more bass not less.

Integrated Amplifier Comparison: $90 vs $9000

I continue to hear differences between amplifiers that might be expected to be inaudibly different. That is, for two high-SINAD systems that I'm not running into clipping, I hear differences. I also believe in measurements. My perspective has been consistent: "Everything that can be measured...

www.audiosciencereview.com

Woa, nice comparison!!

Weird! Yeah I'll try with mic at tweeter level, was kinda foolish having it so low, tweeter level shouldn't really have affected bass, as it's much less directional, would have been more interesting measurements! Below mid & tweeter like that likely skewed mids/highs...! response; which tbf didn't think should really vary between both amps, but yeah would have been interesting!

 

[sam_adams]

There are too many differences and unknowns in these tests to reach a valid conclusion as to the performance of the amps using this method. The unknown technical capability of the mic, changing the position of the mic, and presentation of the data with different stimulus levels and graph smoothing will lead to assumptions that are not valid from the data gathered.

 

[Blumlein 88]

Why not just measure voltage of sine waves every 1/3 octave up to 200 hz with voltmeter at speaker terminals? More precise and fewer ways for interference.

Also, without touching anything, run your measurement 3 or 5 times to see how much run to run variability you are getting. Your results are not useful for the purpose you have in mind. That is okay, everyone learns thru doing.

And of course REW does sweeps, that is how it normally is used. It also would do stepped sine waves. No need to do the higher frequencies if you are looking for bass response differences. 200 hz and down would be fine.

 

[TurtlePaul]

Can you test the QSC with the bass roll off disengaged? I had an old QSC which added bass boost if the high-pass was engaged but don’t know if that is the case for your model.

 

[TurtlePaul]

Good stuff, congratulations on your first test. This supports with data your supposition that there is a difference in the bass. That nearly 2 dB difference should be audible.

By the way, set the y-axis on your charts to 45 or 50 dB, the difference is bigger than first appearance due to the wide axis used. Also, I know your mic isn’t calibrated, but can you use an SPL meter in your phone to confirm the volume of the sweep? It should be 60-80 dB. I see 40 dB on the chart which would have background noise issues in the bass, but you may be testing louder and the mic isn’t level calibrated.

I agree you should try measuring a slower sweep (512k or 1M) rather than pink noise.

I think that unplugging the port should raise the impedance of the speaker on the low end substantially. I wonder if that is the difference. Or I would appreciate if you could test the QSC with output filters removed. I believe there are dip switches to remove the high pass and the limiter. Without limiters a 60 dB sine sweep should still not hurt any reasonable speaker as it is a fraction of a watt.

 

[GM3]

There are too many differences and unknowns in these tests to reach a valid conclusion as to the performance of the amps using this method. The unknown technical capability of the mic, changing the position of the mic, and presentation of the data with different stimulus levels and graph smoothing will lead to assumptions that are not valid from the data gathered.

As mentioned, as I'm not aiming for an absolute measurement, using the same mic to compare two different amps should be fine. Relative differences. If one amp measures the same 3 times in a row, and you switch amp, and it shows a +2dB in bass ................ That's a valid measurement. Not sure how anyone reasonably could argue against this. I'll post a few consecutive measurements to confirm the consistency of measurements, but yeah, I kinda agree for the rest, my initial set of measurements could be considered at best questionable; fair criticism.

Personally, pretty much confirmed that there's a measurable (audible) difference... Due to what... Cause unconfirmed.. (<30Hz QSC bass cut boosting bass? V3 issue driving hard to drive speaker with blocked port? etc..). Interesting open port measurements matched best. But yeah, I'll try to dive in later, cool that others have posted similar experiences with V3/classD! I had read quickly through 100+ Fosi V3 review pages, didn't notice any comment implying similar issues!

Great that ASR is technically driven, but as I said in the Master measuring thread comment here, if you go too overboard and dismiss everything that doesn't correspond to your untested possibly invalid assumptions from a limited review...! Not better off... (Can't just dismiss everything as "sounds the same, no blind test, not real"; some cases can be real difference)

(comment from me from Master measurements are everything thread)

Yes, I should hope so. I'd accept measurement differences in lieu of a blind test.

Measurements are hard though! Tedious...

My first attempt at capturing the difference:

Amp comparison by in room measurements. Advice? Conclusion?

To briefly sum things up, got a new Fosi V3 amp to replace a 40lbs pro amp, "clean power is clean power", yadayadayada, heard differences, this being ASR, measurements. So goal of this thread being whether or not we can conclude anything about the amps in these set of measurements. (mostly in...

www.audiosciencereview.com

I'll definitely make a second attempt, though it'll probably be in a few days... But I'm quite confident that there were significant (audible) differences, and they did show up in the measurements, although there were more variations in the mids than I would have liked... And those mid variations didn't seem to be super stable between different attempts; so while it's logical that moving the mic and speakers around will affect the FR, the 1st set of measurements (amp 1 vs amp 2), seemed to fit closer together.

But for some reason, take #2 and #3 showed more variations in mids... It might be a trait of the blocked port; the speakers as per original design measurements showed 3.3 Ohms min, so ~4 Ohms average, but that might hurt the V3 more than the QSC. What's strange, most seem to believe that this isn't possible. And that for some reason the V3 MUST SOUND EXACTLY like the QSC, which isn't the case. (as per my imperfect measurements, and imperfect ears/brain..) lol

Anyway, lots of great ideas/recommendations from the linked thread, I'll have to dig deeper maybe if I can next few days or weekend!

Thanks!

 

[joqueval]

Interesting test.
The means for all measurements are not always available.
I was thinking something similar to check out out of curiosity.
Regarding the comments, the Yeti microphone distorted the test a little because its behavior can really vary.
If at the outset the comments you indicate have a certain logic, any professional power amplifier will have a greater power reserve than a domestic amplifier.
That's why you can appreciate deeper bass, because the volume of the QSC is superior.
The important thing is to enjoy the music and the equipment

 

[ctrl]

Basically, I just set up a Blue Yeti mic a bit randomly; mostly relatively on axis, 1-2m away, at around woofer height, as I just wanted to measure the relative bass differences I was expecting seeing. (Although reflecting on this, tweeter level would have been better...). Room is quite lively...

With the huge measuring distance without the use of a gate, the influence of reflections is enormous. Even a change in the position of your own body can influence the result.

If you are only interested in the frequency response in the low and mid-range, then bring the mic as close as possible to the woofer 1cm / 0.5'' (adjust the measurement voltage according to the microphone characteristics and LS sensitivity, 2V or below should be okay in any case). Then the influence of room reflections is minimal.
If the loudspeaker has a BR port, you should plug it.

When changing the amplifier, make sure that the speaker and mic position do not change. Make sure that the test sequence is always identical, for example to avoid errors caused by compression (e.g. 30s noise, 10s pause and then the measurement).

Then the probability is very high that measured differences are really caused by the amps.

 

[GXAlan]

@GM3 There is a bias/reasonable belief that all amplifiers sound alike provided that you aren’t pushing it into clipping so you will get a lot of skeptics chiming in, but good job on these measurements. I get the same skepticism when I have posted about amplifier differences going into speakers. Most of the people complaining haven’t tried to take their own measurements, etc.

A couple of good tips for next time,
1) you should take repeated measurements of the same system. This shows that the microphone is precise even if it isn’t accurate. In general, I would assume because the mic is less sensitive at bass frequencies, I would imagine that it is non linear and the real difference is even BIGGER than your 2 dB measurement.

Something like the E1DA Cosmos is sensitive to temperature, for example.

2) DeltaWave is a nice way to do null testing. There’s a lot of data generated and a higher chance of false positives (saying there is a difference when none exists) but it is a good quantitative approach. It takes a lot of effort to record something using something like Audacity, and you should do your best to trim the front of the recording as close as possible to help the alignment in software.

The benefit of DeltaWave is that you can compare the sweep BUT also musical recordings.

 

[NoMoFoNo]

For the OP, you posted in another thread that you were convinced you heard differences between these two amps, so in this case it seems that you're trying to chase measurements to confirm what you thought you heard. That seems backwards to me. In a simple case of two amps would its not be more useful to set up blind testing of the two amps to see if you can actually identify a difference between them that isn't biased?

For the V3 at least, we've already got a full suite of tests conducted here at ASR by Amir.

 

[kemmler3D]

For the OP, you posted in another thread that you were convinced you heard differences between these two amps, so in this case it seems that you're trying to chase measurements to confirm what you thought you heard. That seems backwards to me. In a simple case of two amps would its not be more useful to set up blind testing of the two amps to see if you can actually identify a difference between them that isn't biased?

For the V3 at least, we've already got a full suite of tests conducted here at ASR by Amir.

If you think you hear something you shouldn't be hearing, I think doing measurements to confirm or disconfirm what you might have heard is a great step. As long as you are open to ruling out differences as opposed to simply proving yourself right, I think OP's process is entirely proper. Of course the devil is in the details, but if OP improves his measurements a bit, it could turn out to be useful.

Amps all sound the same (except when they run short on power) - it's not a super rare event to push an amp to its limit.

 

[Blumlein 88]

As mentioned, as I'm not aiming for an absolute measurement, using the same mic to compare two different amps should be fine. Relative differences. If one amp measures the same 3 times in a row, and you switch amp, and it shows a +2dB in bass ................ That's a valid measurement. Not sure how anyone reasonably could argue against this. I'll post a few consecutive measurements to confirm the consistency of measurements, but yeah, I kinda agree for the rest, my initial set of measurements could be considered at best questionable; fair criticism.

Personally, pretty much confirmed that there's a measurable (audible) difference... Due to what... Cause unconfirmed.. (<30Hz QSC bass cut boosting bass? V3 issue driving hard to drive speaker with blocked port? etc..). Interesting open port measurements matched best. But yeah, I'll try to dive in later, cool that others have posted similar experiences with V3/classD! I had read quickly through 100+ Fosi V3 review pages, didn't notice any comment implying similar issues!

Great that ASR is technically driven, but as I said in the Master measuring thread comment here, if you go too overboard and dismiss everything that doesn't correspond to your untested possibly invalid assumptions from a limited review...! Not better off... (Can't just dismiss everything as "sounds the same, no blind test, not real"; some cases can be real difference)

(comment from me from Master measurements are everything thread)


Thanks!

Not trying to be mean, but no you didn't pretty much confirm anything. Using the same mic in the same position in the same situation should work. You have to be persnickety however. Low frequency garbage can come from 100's of feet away. As someone else posted, for low frequencies, put the mic much closer.

And you'll rule out all kinds of stuff to use a voltmeter at the speaker terminals. If a speaker's output changes, the input voltage changed. There is a direct correlation to that. So no worry about low end noise, no worry about mic not being in exactly the same place or you being in exactly the same place, all of that and more is eliminated.

If you are trying to show something very unlikely or unexpected you'll need good clean data to convince someone. Or find out for yourself and know you've shown something to be so or not. Already, you have shown the QSC is not flat response in the low end. Very high probability that is what you are hearing if anything. 85% of hifi is frequency response. A few minutes with a voltmeter and tones every 10 hz will do it. Plus, you can make sure you are feeding the speaker about the same amount of power by matching voltages at least reasonably close.

An excellent learning experience is to do the voltmeter measurements, and then do the microphone measurements and see if they match.

 

[SIY]

I think mostly, results might be within the margin of error

Exactly, and kudos for recognizing this. As Blumlein said, you want to do an electrical measurement, which is more repeatable, more precise, and more pertinent.

 

[pogo]

The output impedances of the amplifier over the entire frequency range could provide an indication. A power cube measurement or a reactive load test could also provide further information.