# Audio Engineering Society 2023 Conference

#### MayaTlab

##### Addicted to Fun and Learning
You can reproduce these plots in Matlab or GNU Octave softwares just by the *.m code script below and the originally downloaded *.csv data files.

As you can see, all data processing is faithful to the original data. No extra selection, filtering or error correction procedures were adopted.

Due to the lack of specific experimental details, only a general boxplot is plotted to describe the distribution of all subjective rating data for each target curve.

No further investigations into the influences of the assessors, the pieces of sound samples and the experiment trials were included in the code and the plots. I am not familiar with Sean Olive's data processing and logical inference procedures either. Happy to see further discussions and your opimions.

Code:
``````clear;clc

freq0 = table1{1,2:end};
targets = struct;
count = 1;
for i = 2:height(table1)
targets(count).name = table1{i,1};
targets(count).freq = freq0;
targets(count).gain = table1{i,2:end};
count = count + 1;
end
target_names = [targets(:).name];
figure(231011003);clf(231011003);
for i = 1:length(targets)
semilogx(targets(i).freq, targets(i).gain);
grid on;
hold on;
end
xlim([30,25000]);
xlabel('Freq (Hz)');
ylabel('Gain (dB)');
legend(target_names, 'Location', 'eastoutside');

figure(231011004); clf(231011004);
ylabel('Rating');``````

Thanks a lot for the answer, amazing work. I am not familiar with Matlab, but reading the documentation, am I understanding it correctly that the box plot command produces the median value, as well as the 25th and 75th percentiles ?

#### ThomasXia

##### Member
Thanks a lot for the answer, amazing work. I am not familiar with Matlab, but reading the documentation, am I understanding it correctly that the box plot command produces the median value, as well as the 25th and 75th percentiles ?
Yes, a simple boxplot provides the 25, 50(median), 75 percentile values.

Violinplot is an alternative to get a better view of the actual rating distributions. Here is a violinplot showing the distributions, the means, the medians and the 25 and 75 percentiles of the ratings at the same time.

The HP3 curves remain the most satisfactory targets, regardless of the parameter being considered.

You can reproduce it in Python3.
Python:
``````import matplotlib.pyplot as plt
import pandas as pd

fig, ax = plt.subplots(figsize= (24, 8))
violin_parts = ax.violinplot(violin_data,showmeans=True, showextrema=False, showmedians=False, widths=0.85)

for pc in violin_parts['bodies']:
pc.set_color('azure')
pc.set_edgecolor('black')
pc.set_alpha(1)

quartile1, medians, quartile3 = np.percentile(violin_data, [25, 50, 75], axis=1)
inds = np.arange(1, len(medians) + 1)
ax.scatter(inds, medians, marker='o', color='steelblue', s=50, zorder=3)
ax.vlines(inds, quartile1, quartile3, color='skyblue', linestyle='-', lw=5)``````

Last edited:

#### MayaTlab

##### Addicted to Fun and Learning
Yes, a simple boxplot provides the 25, 50(median), 75 percentile values.

Violinplot is an alternative to get a better view of the actual rating distributions. Here is a violinplot showing the distributions, the means, the medians and the 25 and 75 percentiles of the ratings at the same time.

View attachment 318687

The HP3 curves remain the most satisfactory targets, regardless of the parameter being considered.

You can reproduce it in Python3.
Python:
``````import matplotlib.pyplot as plt
import pandas as pd

fig, ax = plt.subplots(figsize= (24, 8))
violin_parts = ax.violinplot(violin_data,showmeans=True, showextrema=False, showmedians=False, widths=0.85)

for pc in violin_parts['bodies']:
pc.set_color('azure')
pc.set_edgecolor('black')
pc.set_alpha(1)

quartile1, medians, quartile3 = np.percentile(violin_data, [25, 50, 75], axis=1)
inds = np.arange(1, len(medians) + 1)
ax.scatter(inds, medians, marker='o', color='steelblue', s=50, zorder=3)
ax.vlines(inds, quartile1, quartile3, color='skyblue', linestyle='-', lw=5)``````

Thanks again. I have seen enough box plots that did not adhere to these definitions that I wasn't sure what I was looking at.
Generally speaking, it seems that Harman's method for processing preferential data is to find the mean across all listeners and test tracks.

#### ThomasXia

##### Member
Maybe the answers could only be avaliable in the final paper.

#### MayaTlab

##### Addicted to Fun and Learning
Maybe the answers could only be avaliable in the final paper.

There's a lot riding on the methodology used to know how to interpret the data indeed.

The csv file containing the FR curves might have some errors, I and others have noticed that there may be a shift in values on the x axis for some traces. Hopefully this only happened when publishing the csv file to Zenodo and not during the actual study.

#### ThomasXia

##### Member
There's a lot riding on the methodology used to know how to interpret the data indeed.

The csv file containing the FR curves might have some errors, I and others have noticed that there may be a shift in values on the x axis for some traces. Hopefully this only happened when publishing the csv file to Zenodo and not during the actual study.

In fact, these may not be shifts errors in the x axis as they look like, but the actual frequency responses of these headphones!!!!

Maybe it is just the fact that some headphones with responses deviating from usual frequency targets heavily are still preferred by the listeners.

Just for example, HP3 vs APM:

reference: https://www.soundguys.com/apple-airpods-max-review-44975/

Update2, Example2, HP4 vs DT700PX. (Deviations look like a quality control or positioning issue more than a simple horizontal shift on x axis.)

Last edited:

#### Thomas savage

##### Grand Contributor
The Watchman
Forum Donor
One challenge with the conference is that a number of talks are concurrent. So not possible to attend all of them.

On my session, they put one of the top Pro youtubers in parallel with mine!
I'm almost sure that's a unintentional and regrettable oversight in scheduling .., almost .

#### MayaTlab

##### Addicted to Fun and Learning
In fact, these may not be shifts errors in the x axis as they look like, but the actual frequency responses of these headphones!!!!

Maybe it is just the fact that some headphones deviating from usual frequency targets heavily are still preferred by the listeners.

Just for example:

View attachment 318719
reference: https://www.soundguys.com/apple-airpods-max-review-44975/

To me it seems that not all the curves are affected, and the APM one indeed isn't.

From what I understand the APHarm targets originated from that thread : https://audiosciencereview.com/foru...easurements-using-brüel-kjær-5128-hats.15352/
Mad_Economist has noticed that the curves in the csv file are shifted on the x axis relative to the curves in that thread, and that they match nearly perfectly when you correct for the shift.

In a similar way, using the list of headphones featured in that study :

I'm starting to think that a plausible matching list would be something like this :
• HP1 = Epos Adapt 600
• HP2 = jabra
• HP3 = apm
• HP4 = 700proX
• HP5 = Aonic
• HP6 = Q45
• HP7 = XM4
• HP8 = B&O HX
For some of these, some recognisable features of their FR, particularly the ones that are quite insensitive to positional variation or fixture, seem shifted a bit.

I don't have the time this WE to spend too much time writing here. But, for example, the DT 700 Pro X features a dip at 4kHz that you'll find on all fixtures and that is quite resilient to positional variation, ex here : https://www.soundguys.com/beyerdynamic-dt-700-pro-x-review-60868/
If it is indeed HP4 (and I think that one would struggle to make the case that it isn't), then this is what we get from the csv file :

For some of the other ones these individual features also match better once you scale / shift the FR on the X axis a bit.

Re the question of the preference scores, I don't have a particularly strong opinion on them as we have no idea of the methodology used, or even, if these shifts happened during the study, of what was actually evaluated.

#### ThomasXia

##### Member
For some of the other ones these individual features also match better once you scale / shift the FR on the X axis a bit.
This difference may simply be due to individual variations in quality control or positioning as in the updated figure above, since one single shift do not promise a perfect match.

Thanks for figuring out all these models, really a good job.

Last edited:

#### MayaTlab

##### Addicted to Fun and Learning
This difference may simply be due to individual variations in quality control or positioning as in the updated figure above, since one single shift do not promise a perfect match.

Not for that 4kHz dip . And neither for the response in the feedback range for some of the ANC headphones, or some other individual features.

#### beefkabob

##### Major Contributor
Forum Donor
One challenge with the conference is that a number of talks are concurrent. So not possible to attend all of them.

On my session, they put one of the top Pro youtubers in parallel with mine!
Totally different audiences.

#### Travis

##### Senior Member
I don't think they have a specific agenda and the work simply depends on interest of the students and faculty.
Highly doubtful. They are a research university, so they rely on funding from Canadian Govt. and private industry for research. The agenda probably depends on what interests those providing the funding. They have an entire lab devoted to nothing but emersive audio,

I’m sure the lab director can guide where the research goes, the next step (and it might even interest faculty and students) but they are accountable to whichever research council is providing them with funds to do research.

There are 22 people signed up for my session. So I am out of the woods as far as presenting to an empty room.
I think you will end up with more than that. I think you drew an excellent time slot and a good track.

Did I read that correctly, all of the presentations on you track are 15 minutes each?

OP

#### amirm

Staff Member
CFO (Chief Fun Officer)
Did I read that correctly, all of the presentations on you track are 15 minutes each?
Yes. Indeed majority of the talks are 15 minutes. I think only a couple are 30.

#### Travis

##### Senior Member
Yes. Indeed majority of the talks are 15 minutes. I think only a couple are 30.
Well that will work to your advantage based on your YouTube videos. You can get 2x the info. in 1/2 time as most.

#### Sean Olive

##### Senior Member
Audio Luminary
Technical Expert
View attachment 318109
It might be easier to interpret if you order the mean ratings from highest to lowest - I did this and removed the 2-3 modifications they made to the 8 headphones as I was less interesting in that , although it might be the focus of a different question,

The graph below is the mean based on all ratings ( all listeners, programs, repeats). They didn't code the data according to location (Dane vs Japanese) so we can't analyze it based on culture.

The ratings of the headphone curves is remarkably similar to our work in terms of Harman Target being preferred, diffuse and free field rated progressively lower. It is amazing to me and one of the co-authors I spoke with that two separate experiments using different subjects and methodologies can some up with similar conclusions.

They did use two of our standard tracks (Jennifer Warnes "bird on a wire" and Tracy Chapman "FastCar" so there was that commonality in methods. They also used a virtual headphone method applying equalization filters to a replicator headphone so that rapid A/B comparisons can be made using a MUSHRA method. They used a closed headphone which to me is surprising because open backs generally eliminate the issue of leakage among subjects.

Last edited:

#### alumnicesar

##### Active Member
It might be easier to interpret if you order the mean ratings from highest to lowest - I did this and removed the 2-3 modifications they made to the 8 headphones as I was less interesting in that , although it might be the focus of a different question,

View attachment 319570
finally, the original way it was presented is horrible lmao. Thanks

is there a way to isolate how the soundguys graph looks in comparison to hoe the estimated harman 2018 v2 graph looks? Not the preference rating but rather just where it differs in frequency

Last edited:

#### Sean Olive

##### Senior Member
Audio Luminary
Technical Expert
It might be easier to interpret if you order the mean ratings from highest to lowest - I did this and removed the 2-3 modifications they made to the 8 headphones as I was less interesting in that , although it might be the focus of a different question,

The graph below is the mean based on all ratings ( all listeners, programs, repeats). They didn't code the data according to location (Dane vs Japanese) so we can't analyze it based on culture.

The ratings of the headphone curves is remarkably similar to our work in terms of Harman Target being preferred, diffuse and free field rated progressively lower. It is amazing to me and one of the co-authors I spoke with that two separate experiments using different subjects and methodologies can some up with similar conclusions.

View attachment 319570
Here is a table of the means, median, etc. in descending order from highest to lowest mean.

#### Sean Olive

##### Senior Member
Audio Luminary
Technical Expert
AES conference program is finally available. I thought I post it for members to see what the buzz is about and whether there are any talks of interest: https://aesny23.sched.com/

You might see some familiar names there from ASR.

I will be attending.. See you there..

Replies
276
Views
18K
Replies
13
Views
4K
Replies
21
Views
2K
Replies
37
Views
7K
Replies
2
Views
835