• WANTED: Happy members who like to discuss audio and other topics related to our interest. Desire to learn and share knowledge of science required. There are many reviews of audio hardware and expert members to help answer your questions. Click here to have your audio equipment measured for free!

Using AI for Review Thread Summaries?

Fahzz

Senior Member
Forum Donor
Joined
Nov 27, 2020
Messages
464
Likes
471
Location
Outside Providence
Ok. I haven't read all the posts above,so forgive me if this has been answered, but is AI being commonly used for obituaries (which are sort of reviews, I guess) these days? I see an awful lot of deceased people who were described as a "beacon of light" while living. It's kinda disturbing.
 

PierreV

Major Contributor
Forum Donor
Joined
Nov 6, 2018
Messages
1,448
Likes
4,813
Ok. I haven't read all the posts above,so forgive me if this has been answered, but is AI being commonly used for obituaries (which are sort of reviews, I guess) these days? I see an awful lot of deceased people who were described as a "beacon of light" while living. It's kinda disturbing.
Scam AI-generated fake obituary spam is a thing these days...
 
Last edited:

GXAlan

Major Contributor
Forum Donor
Joined
Jan 15, 2020
Messages
3,917
Likes
6,048
It has already been trained on the ASR forum content up to its cut-off date and updates itself "live" (see below: its response is near instantaneous, but it gets my current number of posts/likes correct as of today).
More precisely GPT is trained on the internet up to a cutoff. “CoPilot” merges GPT4 with Bing, so it has more knowledge. You shouldn’t “need” to retrain it, because the little bit you can won’t solve the problem.

It was wrong summarizing the V6 review which is super interesting (it comes to the opposite conclusion and erroneously reports the % distortion). But this would require GPT5 level training. That is, asking it to summarize a specific webpage is what SHOULD be possible but is still not perfect.

That is surprising to me.

But if you haven’t looked at my post about asking CoPilot to create a movie about ______ it’s really good at that.
 

kemmler3D

Major Contributor
Forum Donor
Joined
Aug 25, 2022
Messages
3,351
Likes
6,854
Location
San Francisco
is it probable that AI will have taken over sufficient tasks - such as design, how things work, repair, and manufacturing - that no one knows how to do anything when the AI breaks down?
I'd say very probable, but in this scenario, wouldn't the AI also be better at keeping itself running than people are?
 

Keith_W

Major Contributor
Joined
Jun 26, 2016
Messages
2,655
Likes
6,057
Location
Melbourne, Australia
My concern is that AI will not give proper weighting to contribution from experts and can not distinguish evidence based answers from the random bleatings of people who do not know anything (like myself ;) ). Take this classic ASR thread for example - there is some very useful information in there, but after 88 pages it started becoming circular and it was eventually closed. Some of the responses are very high quality, cite published references, some others are no more than hand waving.

I suggest that if we were to implement AI to summarize threads, we ask the AI to give more weighting to posts with a lot of likes (suggests that more ASR members agree with the contents of that post), and any post by an "Audio Luminary". The downside of this is that the majority of ASR members could be wrong, and so could the audio luminary.
 

Graham849

Active Member
Forum Donor
Joined
Jul 14, 2022
Messages
233
Likes
204
Location
Australia
Both Gemini and CoPilot remain at an immature level, I would suggest that this use of AI be reviewed in 9 months.

I have been using a standard question over the past 14 months: "please review SMSL DO 100 DAC." The various models still conflate configuration with HO100 and HO200 due to reviews that consider all three gears. Even when told the models still persist for 2-3 iterations in providing false configuration. AI at present is like a 4 or 5 year old child.
 

voodooless

Grand Contributor
Forum Donor
Joined
Jun 16, 2020
Messages
10,384
Likes
18,319
Location
Netherlands
Ok. I haven't read all the posts above,so forgive me if this has been answered, but is AI being commonly used for obituaries (which are sort of reviews, I guess) these days? I see an awful lot of deceased people who were described as a "beacon of light" while living. It's kinda disturbing.
The thread is about AI thread summaries. Why would you assume obituaries would come up at all? :facepalm:. I know ASR can be bad at times, but not this bad ;)
I read an article where it was claimed that AI carried out scientific research in 2 days, on the search for a material to be used in a specific industrial field. If it had been conducted by man, this research would have lasted several years. The AI scanned all the elements of the table with their combinations, with the precise aim of forming compounds that did not use rare or unstable elements.

Clearly in this example, the great work lies in the instructions given to the service AI. If you want to summarize the reviews, you need to instruct the AI to find the sources that are of interest for the purpose of ASR.
Note that this is not an LLM AI model that did this. It was a model purely "bred" for this single-minded task. It's not as if there was an LLM, that was given this task in text form: "Hey dude, go, and do this fancy research we've been at for ages... and hurry up, we'd like to be finished before lunch, please!" :oops:. We are still a very long way off from that goal.
 

-Matt-

Addicted to Fun and Learning
Joined
Nov 21, 2021
Messages
679
Likes
569
I suggest that if we were to implement AI to summarize threads, we ask the AI to give more weighting to posts with a lot of likes (suggests that more ASR members agree with the contents of that post),

No, this is a terrible idea. Unfortunately, rude, insulting and stupid posts (with zero useful input) frequently attract a high number of likes on the basis that people find them amusing.
 

voodooless

Grand Contributor
Forum Donor
Joined
Jun 16, 2020
Messages
10,384
Likes
18,319
Location
Netherlands
No, this is a terrible idea. Unfortunately, rude, insulting and stupid posts (with zero useful input) frequently attract a high number of likes on the basis that people find them amusing.
Then ask it to ignore those as well.
 

Jeromeof

Addicted to Fun and Learning
Forum Donor
Joined
Jul 25, 2021
Messages
623
Likes
996
Location
Ireland
It has already been trained on the ASR forum content up to its cut-off date and updates itself "live" (see below: its response is near instantaneous, but it gets my current number of posts/likes correct as of today).

Retraining it on the forum's content is almost pointless imho.


summarize the activity of the PierreV user on the audio science review forum.


ChatGPT
PierreV is an active participant on the Audio Science Review forum, contributing to various discussions related to audio equipment and technology. Here's a summary of PierreV's activity:

  • Joined Date and Contribution Level: PierreV, a major contributor and forum donor, joined the forum on November 6, 2018. They have posted 1,435 messages and received 4,682 likes as of February 8, 2021.
  • Interests and Purchases: PierreV has purchased several items based on recommendations from the site, including a dacmagic plus, a KTB, and Hypex kits. They highlighted the dacmagic plus's functionality and expressed a desire for improved volume control features.
  • Contributions to Discussions: They have engaged in discussions on various topics, including the value of high-resolution audio, the Dunning-Kruger effect, and the comparison of high-end passive bookshelf speakers versus mid-range active studio monitors.
  • Technical Insights and Reviews: PierreV has shared insights on specific audio equipment and technology. For example, they discussed the potential overestimation of competences related to the Dunning-Kruger effect within the context of audio science, and provided feedback on the performance and features of audio products like the dacmagic plus.
  • Forum Engagement: They actively participate in various threads, reacting to posts, sharing opinions, and contributing to the community's knowledge base. PierreV's contributions span across technical discussions, product reviews, and audio technology analysis.
PierreV's activity on the Audio Science Review forum indicates a deep interest in audio technology, a willingness to engage in technical discussions, and an appreciation for the community's insights on audio equipment. Their contributions reflect both a pursuit of quality audio experiences and a commitment to the forum community.
While not retraining - if @amirm wants a review thread summary that is up to date (say daily) the LLM needs to be fed the content - in my experience - while it can read a page of content - it typically won't read beyond the first page of the thread - so programmatically submitting the thread posts (or probably selected thread posts e.g. 'liked' posts ) will allow a better summary of the thread.

With the chatGPT API there is also a 'temperature' setting - see below (from their API playground)

1708013203427.png

Which controls how 'creative' the generated response will be. In these types of summaries, it probably best to 'deterministic and repetitive'.

There are other use cases where the content generated does not need to be as up to date (as your example above) and I did think a 'compare' reviews capability might be very interesting, e.g. take just the review content itself of 2 DAC or 2 Amps (or more than 2) and generate a 'textual' comparison for those you don't like comparing graphs.

But a 'chatbot' to answer audio science questions that only uses content from AudioScienceReview (and doesn't start pulling content from other pages that it was trained on ) would need IMO at the very least need to use 'continous pretraining' capabilities. This is where an existing trained LLM (and it doesn't have to be ChatGPT, CoPilot or Gemini) is 'updated' with domain specific content which it prioritises over other content. But while not as expensive as full training this still a more involved process that using the trained data.
 

voodooless

Grand Contributor
Forum Donor
Joined
Jun 16, 2020
Messages
10,384
Likes
18,319
Location
Netherlands
While not retraining - if @amirm wants a review thread summary that is up to date (say daily) the LLM needs to be fed the content - in my experience - while it can read a page of content - it typically won't read beyond the first page of the thread - so programmatically submitting the thread posts (or probably selected thread posts e.g. 'liked' posts ) will allow a better summary of the thread.
Generally, the more you pay, the more tokens it will be able to digest at once.
 

NiagaraPete

Major Contributor
Forum Donor
Joined
Jun 23, 2021
Messages
2,199
Likes
1,962
Location
Canada
Hello everyone and especially those of you with experience deploying AI in specific cases.

I have noticed Amazon including an AI generate summary of all the user reviews for products. I find them quite useful and accurate. I am thinking such a thing might be useful for the reviews as well. What I don't know is how one goes about using these AI tools to generate thread/site specific results. Do you have to become a commercial user and pay high licensing fees to do so?
I think when Xenforo releases 2.3 with a more open API it may become quite possible.
 

pseudoid

Master Contributor
Forum Donor
Joined
Mar 23, 2021
Messages
5,185
Likes
3,529
Location
33.6 -117.9
At about this time in this thread, Little Johnny - in the back of the room - shouts out "Why not go BIG?"

Summarizing may be attractive and desired due to the continued reduction of our attention-spans.
But wouldn't this 'summary' be more appropriate; if there was an audio version of it, that would be more inline the mission of ASR?
Text-to-speech AI specialists, such as ElevenLabs provides such a feature that can even be trained to duplicate/clone @amirm' speech, and also has models for about a dozen other languages
 
Top Bottom