Probably very long post incoming. Apologies in advance.
First of all, thank you so much for this. This blind means so, so, so much to me! These have literally been my end game dream music speakers for the past 5 years. I had already somewhat resigned myself to the fact that I'd probably end up having to waste a ton of money by buying both of them to do a blind comparison myself in order to truly be satisfied with my decision. Measurements are great, but blind tests are king for me. I bought my current mains(JTR Reference series) based entirely off blind tests(one that I attended, and one that I read online). There were no measurements at all of the JTRs at the time, but that's how much faith I place in blind tests. Given the price of these speakers, I figured the odds of 1. Someone having both speakers, and 2. Being willing to setup a blind match between them, was basically zero. The fact that you're doing this comparison here has the potential to save me a ton of money if one ends up winning convincingly.
I do have some suggestions based on my past experience with hosting 4 blinds, and attending 1. Hopefully these suggestions don't come across as criticism. I just care deeply about this particular blind, and so there's a lot of passion. Ultimately, only consider the suggestions that you're completely comfortable with, as no matter what you do, I'll be very grateful that you're doing it in the first place.
1. Do not tell the participants ahead of time ahead what speakers are going to be involved in the test. This greatly diminishes the value of the test. This was by far the biggest critique of the M2 vs Salon2 blind that was done on AVS, and many people disregard it completely for this reason.
2. Don't do an ABX text. Do a preference test. ABX is more useful for electronic equipment, but we know for a fact that speakers can sound very different.
3. Consider adding a third speaker. Adding a third speaker (ime) makes it harder to identify individual speakers and gives you another reference point, and it's also the recommendation of Harman and Floyd Toole. Further, if it's a speaker of lesser quality, it might help to differentiate these two beasts some in terms of ratings. For both the stereo and mono tests I hosted of the JTR 212 vs Revel M105, we used an Infinity Beta 20($400/pair when I bought them) as the third speaker. You could maybe add your Sierra towers into the mix? If the Sierras by some chance end up beating the Genelecs, it may shed further light on the mono vs stereo debate. BTW, not saying the Sierras are bad or comparable to the Infinity Betas, but the other 2 speakers are SOTA.
Problems #2 and #3(if not accounted for) combined together can make it very easy to identify the speakers, even under blind conditions. It's part of the reason I no longer participate in the blinds I host with friends. I can easily identify all my speakers. Adding a third speaker and not telling them what speakers are under test makes it almost impossible for anyone(but you) to identify them.
4. Mono vs stereo. I think this is going to be a very hot topic. There's most likely gonna be lots of debate back in forth, and no matter what you choose, half of the people are gonna say your test is useless
. Do what you think is most logical.
I'll share my plebeian thoughts on the topic based on reading Floyd's book, reading the actual Harman studies, reading great forum threads on AVS, and doing my own mono vs stereo vs multichannel blind listening test at home. My overall knowledge and understanding of these topics is far below many others on this forum.
The Harman view is that mono vs stereo doesn't matter if the sample size is sufficient. The results will be exactly the same, but you'll need a larger sample size with stereo to come to a meaningful conclusion. Imo, this is a valid interpretation of the data represented in the study, but there are also other valid interpretations.
Looking at the study, the Quad's which have very narrow dispersion(and presumably the best imaging) sounded terrible in the mono test, but were very close to the others in the stereo test.
The Harman interpretation was that the Quad's improvement from mono to stereo was due to the fact that its flaws were more hidden in stereo. But, another interpretation of that data might be that the Quad's strengths, namely imaging, were absent in the mono comparison. I see no reason why this latter interpretation is any less valid than the former. In my opinion, more research is needed to settle the mono vs stereo debate. Even Toole admits that increasing the number of channels decreases the preference for wide dispersion(don't remember if this was in his book or a forum post).
Personal anecdote on this topic:
As mentioned earlier, one of the first blinds I hosted was a JTR 212 vs Revel M105 vs Infinity Beta 20. This is the only blind where I've ever done a mono vs stereo vs multichannel test, and the results were very enlightening.
In the mono test, the M105 was preferred overall by 3/3 people and preferred on ~80%+ of the tracks. The JTR actually came in last, losing slightly to the Infinity Beta 20.
In the stereo test, the JTR was preferred overall by 3/4 people(we had an extra person after the first day), though admittedly less strongly than the Revel was preferred in mono(I think it was high ~70%, but I don't remember exactly). The JTR also dominated the Infinity in the stereo test.
In the multichannel test, the JTR basically stomped, winning 4/4 overall and only losing 6 times total amongst 104 trials(26 tracks x 4 listeners). It also beat the Infinity Beta 20 104/104 times, and this was in spite of the fact that the center channel was an Infinity Beta C360, and the rear and side surrounds were Infinity Beta 20s. I've since acquired a full JTR surround setup, but this was before, when I just had the front L/R.
Based on these results, I'm fairly confident in saying that the mono vs stereo Harman view doesn't hold for my room. Whether or not that applies to other rooms and speakers, I don't know. As for why it applies in my situation? I don't know for sure, but my current best guess is that it's due to the difference in dispersion width of the speakers. The JTR is +/- 30, and the Revel is +/- 70. I'm not sure what the Infinity is, but I can tell it's closer to the Revel than it is the JTR.
All that said, I don't think the mono vs stereo factor will be as big as a problem for your test as it was for mine. I was comparing a +/- 30 speaker against a +/- 70 speaker. At +/- 60, the 8351b is still a wide dispersion design(the M2 is ~+/-50) in my view. We don't really know the Salon2's horizontal beam width, and we won't until
@amirm measures it(side note: I'd be willing to donate money to help
@amirm hire movers to move it downstairs and back up), but I would ballpark it in the range of my M105's. So, for you're test, we're probably comparing a +/- 60 dispersion width against a +/- 70-75 dispersion width(and maybe a bit wider if you include the Ascends
). I don't think it will be a huge factor in which speaker wins.
If I could get anything I want, I'd want you to do both a mono and a stereo test. Regardless of the results, I think it would be hugely beneficial for audio science. If the same speaker wins both tests, it would add credence to Harman's view that mono vs stereo doesn't change preference. If different speakers win, it would add credence to the notion that more research needs to be done exploring the effects of dispersion width and the relation to the number of channels.
Off topic: I plan to do the mono vs stereo vs multichannel test again(maybe after covid) with my now better knowledge and experience of hosting blinds. I'll probably start a similar thread to this one to ask for advice. The plan is to have the JTR 210 vs Infinity Reference 263 vs Revel M105 vs JBL 308p vs Genelec 8030c, but I still need to find a good way to switch between active and passive speakers.
In order of my personal preference for testing, I would vote:
Mono and Stereo test with subs > Mono test with subs > Mono and stereo test without subs > Stereo test with subs > Mono test without subs > Stereo test without subs.
5. Ask each participant to keep score on a 1-10 scale. We did this for the first blind test I hosted, but stopped doing it after that because it was hard, and the ratings were very inconsistent. I now really regret that we stopped doing it.
6. Have people take notes on what they like/dislike. You did this for your last blind, so I'm guessing you already have this in the plan.
7. Consider again the idea of placing the speakers more optimally. Ideally this would place the Genelec on a stand with the tweeter at ear height at 35" or so, and the Salon2 on the ground with its tweeter at 49"?. Nothing can be done about the Salon2 tweeter being too high(except a long listening distance, which you have). The Genlec though can be optimized better with extra work. Currently you've got the Genelec with a tweeter height of 58"?. Angling it down does help(imo), but it also takes the speaker beam off parallel with your ears, which comes with other disadvantages. Going with what you have I think will be mostly ok, but equal ground would be more optimal. Some may choose not to value the blind because of this. Personally, I don't see it as a huge deal.
FWIW, the two blinds I've hosted with my biggest towers(155lbs), we had 2 guys standing ready to lift the speaker and carry it backwards, and one guy to move the bookshelf + stand into place. We never had very much trouble with that setup. 155lbs in the shape of a tall skinny box is surprisingly light for 2 guys to lift and carry 2ft back. I've also recently been picking up and moving the speaker(about 6ft back) by myself almost every day to give myself more room for playing with my VR headset
. It's 9lbs heavier than the Salon2, so it can be done, though I admit it's hard(you really have to commit). The way I've found works best is to get behind the speaker, wrap my arms around and grab the front(not on the drivers), bend down to pick up the speaker with mostly my legs and back, then lean back(with it resting against my inclined chest) to walk with it. That said, my speaker has a truck bed finish, so I don't have to worry about damaging(or fingerprinting) the cabinet, so the Salon might present an additional challenge.
It may not be worth it for you, though, and I don't begrudge you for that. Better to do it sub-optimally than not do it all, I say. Above all, I'm still immensely grateful that you're doing this.
8. On EQ. Please, no matter what, don't let people talk you into not EQing below the transition frequency. We know for a fact that below 100Hz, we're not listening to the speakers. Below 100Hz, we're listening to the dimensions and material of the room, and the placement of the transducers within those dimensions. The goal of this test should be to test the speakers themselves, not the speaker placement. If you don't EQ below the transition frequency, you're not testing speaker A vs speaker B, you're testing placement A vs placement B. That latter is still useful, but it's only useful for those who have the exact same room as you(which is probably no one but you).
9. On subwoofers. Please use subwoofers!!! Don't let people talk you out of this. Not using subs would make it a cake walk win for the Salon2 at critical listening levels(especially at 20ft). Given that we know bass is 30% of the battle, it would make the test basically meaningless for those of us who use subs. Whether I end up buying the Salon2 or the 8351b, I'll be crossing them over to 4 JTR Captivator RS2s at 100Hz, so I have zero care for which speaker has better bass. If you don't use subwoofers, you have to do a hard cutoff of everything below 100Hz or so, but even that will be far less useful than how they integrate with subs.
10. If you do decide to do a stereo test, don't do it the way Harman does. Harman, puts the speakers in the exact same spot with the same toe in, and it's the biggest reason why I don't consider the Harman stereo tests as all that valid. This was the way I did my first stereo blind test, but I'm now of the opinion that it's a flawed method of testing. Different speakers in a stereo pair will require different amounts of distances between them, and different amounts of distance from the listener to achieve optimal imaging. The reason Harman does it this way is most likely because they have an automated speaker shuffler and they have to do it this way(designing a more sophisticated shuffler would be much harder and expensive), but it's definitely sub-optimal(imo). Doing it this way has a decent chance of leading to incorrect results.
For example, my M105s sound and measure best about 10.5ft apart, 10.5ft from the listener, and toed in directly on axis. My 212s sound and measure best about 12ft apart, 12ft from the listener, and toed in to cross 3-4 feet in front of the listener. My JBL 308/5ps sound and measure best pretty close to where the Revels do, but aimed straight ahead. If I measure stereo pair A in the position that stereo pair B sounds best, stereo pair B will be at a major advantage. If I measure stereo pair A and B at a random location, the stereo pair with wider dispersion will be at a major advantage, as wide dispersion designs are far more tolerant of positional changes.
In my experience, it takes a solid week's worth of work of moving, listening, and measuring to figure this all out, and given that most people don't do this(including the Harman shuffler), it's a big reason why I much prefer a mono test over a stereo test.
The way we've accounted for this so far is that we'll place colored tape(silver, yellow, purple) on the corner of each speaker top, and 4 pieces of the same colored tape at the base of each speaker to mark its position and toe in. It's easier to show with a picture than it is to explain so I'll post that below.
Using the tape pieces, the 2 speaker movers will move each speaker into the same colored tape position when the switch is called, and the previous speaker will be moved back behind and out of the way.
I very much look forward to your results! Might be fun to start a forum bet beforehand to see where people's opinions lie. I'll throw my own guess into the mix.
I think the Salon2 will win, and I think it will be very similar to the M2 vs Salon2 blind. The M2 has very similar measurements to the 8351b(both are noticeably better than the Salon2), and the Salon2 beat the M2 by 2 points out of ten pretty consistently. The only explanation I can think of is the wider dispersion.
If it's a mono test I'll say Salon2 90% chance
If it's a stereo test, I'll say Salon2 70% chance
PS: You have no idea how jealous I am of the fact that you own both of my dream speakers that I'm trying to decide between.
