• WANTED: Happy members who like to discuss audio and other topics related to our interest. Desire to learn and share knowledge of science required. There are many reviews of audio hardware and expert members to help answer your questions. Click here to have your audio equipment measured for free!

Developer Help With Review Index/Search

amirm

Founder/Admin
Staff Member
CFO (Chief Fun Officer)
Joined
Feb 13, 2016
Messages
44,376
Likes
234,526
Location
Seattle Area
Hello you all. As some of you know, we have been using Tableau to index our reviews. Alas, it is a crude system for this use and highly limited as far as what we can do with their free service. The paid version is hugely expensive and it is not clear it solves the problems we have with it. Performance is horrible and presentation is poor despite tremendous work our few members have put in to make it work.

So I thought I get help from membership to create a back-end that I can then embed in the forum to provide a much better user experience. For version 1, I like to just get the index in a more legible and performant way. For version 2, we can look at comparing measurements/graphs and such. For the former, @pozz has been kind enough to maintain an extensive database. The data for the latter doesn't exist yet in ready form and hence not focusing on it now.

@pozz can explain how he is maintaining the data right now. Perhaps he can create a sample we can work against. Hopefully we have some developers willing to put in the time to do this. There is no pay though and what you write becomes our property to use. :)

This would also be a good thread to discuss what we want out of the new service/interface.

There is an option to use forum software API to develop the UI but I am not clear if that is a good choice. It can open up security issues and using the API may be limiting.

I will leave it to @pozz to lead the project at high level but he would need a technical lead as well.

Anyway, we really need a good system here given the mountain of reviews we already have to organize and present in a better way. I hope I can count on your help to get us there.

Thanks,
 

Racheski

Major Contributor
Forum Donor
Joined
Apr 20, 2020
Messages
1,116
Likes
1,699
Location
Chicago
I have offered to help streamline the data and improve the analytics user experience previously, and I'll offer my help once again. I'll PM you & @pozz about my thoughts.
 
Last edited:

pierre

Addicted to Fun and Learning
Forum Donor
Joined
Jul 1, 2017
Messages
963
Likes
3,048
Location
Switzerland
I do not have the time or the inclination (*) but I see 2 easy ways to do it:
1. XenForo has a good documentation on how to create add-on (s): I would make a classical project with a SQL table (denormalized) and a few forms to add data and correct them. Then you only need some simple template to render the data in any way you want. There are some security risks as always when you have a frontend talking to a DB but nothing that can be taken care of if you program with care.
2. Just use javascript and load a json metadata file. Then you have 1000s of library to do it client side, from smart tables to search stuff. Less integrated with the forum but likely easier to do.

If you know PHP go for 1. if you know javascript go for 2. That's also a nice way to learn either of them.

Pierre

(*) I am currently overloaded and I do not have the time to work on my new pet project :(
 

MZKM

Major Contributor
Forum Donor
Joined
Dec 1, 2018
Messages
4,244
Likes
11,482
Location
Land O’ Lakes, FL
Ignoring the coding:

The collecting and maintaining the data would be the most challenging part (see how I bulk update my preference score spreadsheet).

The other challenge is how to display “vs level” measurements. Knowing the members, they’d want the full power vs level data, meaning you need to digitize/trace the plots and not just record a max value for wattage/voltage (which has no defined definition as you pick somewhere in the knee).

And then you have tests that some units get that others don’t, like some AVRs measured have a SINAD vs level measurements for the pre-outs, but others don’t.
 
Last edited:

pozz

Слава Україні
Forum Donor
Editor
Joined
May 21, 2019
Messages
4,036
Likes
6,827
I don't have the time at the moment to address this project properly. But here's a look at the data, which @RickSanchez helped bring to it's current form.

This is the electronics portion of it. The same basic format is applied to speakers and headphones: tabs in Google Sheets linked by joins.

Master tab
1618279907822.png


Tags tab
1618279986976.png


Reviews tab
1618280036261.png


La pièce de résistance, the measurements tab (the first 16/53 columns of it):
1618280145585.png



Ignoring the coding:

The collecting and maintaining the data would be the most challenging part (see how I bulk update my preference score spreadsheet).

The other challenge is how to display “vs level” measurements. Knowing the members, they’d want the full power vs level data, meaning you need to digitize/trace the plots and not just record a max value for wattage/voltage (which has no defined definition as you pick somewhere in the knee).

And then you have tests that some units get that others don’t, like some AVRs measured have a SINAD vs level measurements for the pre-outs, but others don’t.
If this reaches version 2, which is a lot beyond what I'm collecting for electronics and what @RickSanchez is doing for speakers/headphones, we'll need more people to participate. Whatever ambitions we have they have need to be practical (less data collection, less complicated interface), and give members and nonmembers the right sort of flexibility (more data, more complication). The more we move towards the latter, the more professional the project will have to get, and hopefully a core team will form at some point to keep it going.

Feels nice to get this point.
 

ascorbyl

Member
Joined
Dec 16, 2018
Messages
27
Likes
21
I think it would be helpful to be able to access the available data in its current state to see what we can do with it.

Would the forum API allow for something simple like just embedding a React app in an iframe? I assume that’s what’s currently happening with the Tableau embed.
 
OP
amirm

amirm

Founder/Admin
Staff Member
CFO (Chief Fun Officer)
Joined
Feb 13, 2016
Messages
44,376
Likes
234,526
Location
Seattle Area
As admin, I can embed any HTML or object within the page. The forum software API is not needed for that since you are not interacting with the forum itself. And yes, for Tableau we are just embedding their object.
 

someguyontheinternet

Active Member
Joined
Apr 16, 2021
Messages
194
Likes
334
Location
Germany
I can help out, but probably can't act as a "lead developer" or similar for time reasons. Spending a couple of hours per week should probably be possible for me though.
My work experience so far has been fullstack development with Java/JS+Vue and GCP/TS+Vue projects.
 

Racheski

Major Contributor
Forum Donor
Joined
Apr 20, 2020
Messages
1,116
Likes
1,699
Location
Chicago
I can help out, but probably can't act as a "lead developer" or similar for time reasons. Spending a couple of hours per week should probably be possible for me though.
My work experience so far has been fullstack development with Java/JS+Vue and GCP/TS+Vue projects.
Interesting...have you done any development work on data analysis projects?
 

someguyontheinternet

Active Member
Joined
Apr 16, 2021
Messages
194
Likes
334
Location
Germany
Interesting...have you done any development work on data analysis projects?
A pretty far fetched kind of, but not really. I've been involved in projects that collect usage data and process them with google Data Studio, but that's about it. There are some workshops on Data Engineering which Google offers for GCP developers that I've been planning to do, but I haven't done any really meaningful work involving data analysis and processing so far.

TL;DR the interest is there, but the experience is not
 

sandymc

Member
Joined
Feb 17, 2021
Messages
98
Likes
230
You might want to look at some of the "no-coding" environments. Generally, they have a spreadsheet style with lots of sorting, etc options. The challenge will probably be to find one that allows unlimited read access. The one I have experience of, Amazon's HoneyCode, so far as I know, requires a license for every individual using the environment, which would not work here.

And btw, yes, Tableau is slow. Even inside a a corporate intranet, it's slow.
 

pozz

Слава Україні
Forum Donor
Editor
Joined
May 21, 2019
Messages
4,036
Likes
6,827
A pretty far fetched kind of, but not really. I've been involved in projects that collect usage data and process them with google Data Studio, but that's about it. There are some workshops on Data Engineering which Google offers for GCP developers that I've been planning to do, but I haven't done any really meaningful work involving data analysis and processing so far.

TL;DR the interest is there, but the experience is not
Are you interested in helping with data entry? We need a lot of that.

To anyone else reading this: please volunteer for data entry. We have three streams:
  • Speakers
  • Headphones
  • Electronics (highest learning curve)
You need to know something about these devices or be interesting in learning how they work. The actual work of examining charts and picking out important figures helps a lot. Not sure how to read charts or how to pick your next DAC? Try your hand at helping with the database and watch your knowledge blossom.
 

P_M

Active Member
Joined
May 31, 2019
Messages
131
Likes
51
Location
San Francisco Bay Area, CA
I can help with data entry, whichever area or stream needs it most. I consider myself an advanced hobbyist with experience from vintage analog to modern digital , and from Vacuum Tubes to Class D, DIY loudspeakers, and also RF (radio) into the mix (hence my avatar).
I am a software engineer by profession although more on the database side of things like SQL, data modeling, movement, and such.
 

Newman

Major Contributor
Joined
Jan 6, 2017
Messages
3,454
Likes
4,217
Hi guys,

I had previously reported, about 9 months ago, the non-functionality of the Speaker Review Index (and IIRC the others were no better) for iPad here:
https://www.audiosciencereview.com/...dex-v2-updates-and-feedback.14166/post-542461

I just noticed that they are now working beautifully on iPad.

I just want to give a huge high-five/thumbs-up/you-name-it to whoever was involved in making this happen. Well done to all!

o_Oo_O:):):p:p:p

cheers
 

respice finem

Major Contributor
Joined
Feb 1, 2021
Messages
1,867
Likes
3,775
If misplaced, please redirect - no developer here, just an idea for the search: Sometimes it would be nice to be able to search reviews "by panther".
 

dsnyder0cnn

Addicted to Fun and Learning
Joined
Dec 14, 2019
Messages
540
Likes
800
Are there any plans to update the index? Looks like it's not been updated since mid October, which is kind-of a long time ago…
 

pozz

Слава Україні
Forum Donor
Editor
Joined
May 21, 2019
Messages
4,036
Likes
6,827
I unfortunately can't participate this time. Apologies.
 
Top Bottom