• WANTED: Happy members who like to discuss audio and other topics related to our interest. Desire to learn and share knowledge of science required. There are many reviews of audio hardware and expert members to help answer your questions. Click here to have your audio equipment measured for free!

Using AI for Review Thread Summaries?

voodooless

Grand Contributor
Forum Donor
Joined
Jun 16, 2020
Messages
10,384
Likes
18,319
Location
Netherlands
I love CoPilot and I think you can just use CoPilot Pro at $20/month to use it commercially.

It is surprisingly good at making screenplays. The problem is that it is confidently wrong
This is why I hate it, at least the generic version. It’s so confidently wrong that it just shuts down the conversation, and then on the next try, totally contradicts itself, then accuses you of all kinds of things when you confront it. I guess they tried to give it a Bill Gates personally?

I use the BitBucket Copilot feature daily, and it’s much more sane and graceful when it makes mistakes. Calling any AI tool Copilot is very confusing :facepalm:

What does it take to develop this?
Somebody with both knowledge and experience with LLM AI and XenForo can probably do it. I guess if you scrape every topic with enough changes since last time (e.g. x amount of new posts) and update the summary text, it could work. Might not be super hard to do if you know the tools.

As for how and what it summarizes, you can probably tune this by being more specific on what you want to see. I also wonder how it would work on 100+ pag topics? Maybe those long threads need a “discussed recently” section?

Maybe just create a SOW and put it on one of those developer for hire sites and see what comes along?
 

geek101

Active Member
Forum Donor
Joined
Aug 5, 2019
Messages
133
Likes
41
Location
California, USA
Hello everyone and especially those of you with experience deploying AI in specific cases.

I have noticed Amazon including an AI generate summary of all the user reviews for products. I find them quite useful and accurate. I am thinking such a thing might be useful for the reviews as well. What I don't know is how one goes about using these AI tools to generate thread/site specific results. Do you have to become a commercial user and pay high licensing fees to do so?
Hi @amirm , Currently in all of you reviews you have a "Conclusions" section. Would this section be same as the "Summary" you are envisioning or something different?. It would be possible to train/fine-tune a model to help it get close to your style of writing and also attending to facts only in the review text you have.

But I am sure sometimes you compare items or mention something that is not directly available in the review text, for example: "The amplifier is not for everyone though. Power level is modest as compared to competitors." from Topping LA90 Review. This would be hard for the model to do, it may say something else for example.

Or are you are looking for a model to generate a write up and you would edit it after?. Hopefully saving time?.
 

Jeromeof

Addicted to Fun and Learning
Forum Donor
Joined
Jul 25, 2021
Messages
623
Likes
996
Location
Ireland
I asked OpenAI to summarize the Fosi ZA3 review. It refused saying it won't do that with links.
CoPilot in Windows just gave me a bullet list of what I had post in the review.
Google Gemini created a friendly summary:

"This is an article about a review of the Fosi Audio ZA3 stereo amplifier. It discusses the amplifier's features, performance, and measurements. The Fosi Audio ZA3 stereo amplifier is a class D amplifier with balanced XLR inputs and a 48-volt power supply. It has been improved over the previous version, the V3, with the addition of balanced XLR inputs and a mono mode. The amplifier performed well in the reviewer's tests, with good noise performance and power output. However, the reviewer did note that the frequency response can be affected by the speaker impedance. Overall, the reviewer recommends the Fosi Audio ZA3 stereo amplifier."

None want to drill down beyond the review itself.
This is where some php code working directly with the thread post content itself would be allot better. These LLM's are really just looking at 1st webpage of review and then they would be mostly using a variation of the 'readability algorithm' to get relevant content from that page (like Pocket uses or if you have access to Safari - Apple have a "Reader View" - basically strip away all the 'fluff' from a webpage).

With some code working directly with the thread posts, a proper algorithm could be developed to summarise content e.g. use the main text of the review, the most 'liked' comment posts across all the pages of the review, maybe focusing on your posts (and a select list of knowledge people) and the content in the thread that you have replied too. This content would then be fed into the prompt for the summary of the thread and you would get a much better summary.
 

Jeromeof

Addicted to Fun and Learning
Forum Donor
Joined
Jul 25, 2021
Messages
623
Likes
996
Location
Ireland
That would be extremely useful. I can get a developer to do the plug-in for us if the rest is available. We have so much knowledge here that is hard to sift through. What does it take to develop this?
There are various approaches to doing this. Here is a summary of 2 techniques:

1. Retrain a small LLM with domain specific knowledge to generate a domain expert LLM
This is the one I am most interested in - but it would require lots of work prepared the training data from the forum content. And GPU resources to effectively train your own model with this domain specific prepared knowledge, so it's expensive to train even the small LLM's would require significant processing on Azure / AWS. And once it's trained it can only answer the questions on the data up to its training date (unless the other approaches are also applied see 2. below).
The advantage of this is it will be fast, relatively cheap to run, won't pull in content outside the forum.
The disadvantages are that its knowledge will be limited to how good the prepared data is, so will hallucinate if something is asked beyond its training data. With small model it is also like to hallucinate if asked to expand on some response.
There are examples of this type of small domain focused pre-trained LLM's appearing and I think they are very good (can compete with OpenAI / CoPilot) on domain specific knowledge.

2. Using RAG (Retrieval-augmented generation) with existing LLM.
The existing LLM can be OpenAI / CoPilot (effectively this is what Microsoft are doing with Co Pilot on your own content) or a small LLM. In an ideal world it would be output of 1. above but then this RAG algorithm would suppliment the trained data with up to date content.
This is the easiest to do in the short term, but can give good results but has problems. You effectively 'pre-index' relevant content from the forum (maybe just your FAQ pages when they are written might be ideal) in a graph database and before finally submitting content to the LLM (again doesn't have to be chatGPT / CoPilot) you retrieve these relevant paragraphs / forum posts) and used them as suggested content in the generated output.
The advantages are there are lots of open source tools that can do this relatively quickly and it becoming a well known technique.
The disadvantages are it is expensive (you get charged per token submitted into these cloud LLM like chatGPT as well as charged for content generated). So for example, if DAC is mentioned within a question, RAG would pull the relevant prioritised forum posts for DAC and submitted those with the rest of the question that was asked), so might be 10,000 tokens added to the question and this would happen each time someone mentioned DAC in a question. If you use this with a local small LLM you won't get charged but it slows down the generation of the response greatly (might end up being 30 seconds before it starts to generate a response.
The other disadvantage is especially if you use ChatGPT / CoPilot / Gemini is it won't limit the response to ASR content and will pull content from outside the forum.

Anyway, I can PM you with more technical details and examples.
 

Multicore

Major Contributor
Joined
Dec 6, 2021
Messages
1,780
Likes
1,951
Would a chatBot's answer be consistently the same when asked the simple question of "What comes out; if garbage goes in?"?
Will it be the same answer if the question is asked in reversed order: "If garbage goes in; what comes out?"
I don't know.

There are four questions in these two lines above. Each has a question inside another and you formatted the text well to make this clear. The boring thing to do would be to feed the italic inner questions to some AI chat bots, see what they say, and maybe that would answer the enclosing questions. But feed the enclosing questions to chat bots and then we're asking them about what chat bots in general would do.

Now, one of my tests for intelligence in humans is to observe how the subject behaves when she or he has insufficient information. Often the intelligent thing is to answer "I don't know'. Here by intelligent I mean it's better than guessing or bullshtting an answer. And when the to go find the answer, it's often more intelligent to say "I don't know. Let's find out." than "I think X. Let me check."

Iiuc, Chat GPT's training set excludes the era in which chat bots wrote most of the text on the internet, so it's probably not in a position to say what chat bots would do. But even given this rather simple reasoning, I'd wager it won't answer "I don't know" when posed your outer questions. It will probably wrap some weasel words about what some people prognosticated. But there I go being unintelligent instead of going to get the answers.
 

Bob from Florida

Major Contributor
Joined
Aug 20, 2020
Messages
1,296
Likes
1,189
A possible long term issue with AI is lack of knowledge of how to do anything. Think about it. Companies are embracing AI to reduce manpower wherever possible. At some point in the future, is it probable that AI will have taken over sufficient tasks - such as design, how things work, repair, and manufacturing - that no one knows how to do anything when the AI breaks down?
 

Mikig

Senior Member
Joined
Jan 12, 2024
Messages
405
Likes
434
Location
Italia
I read an article where it was claimed that AI carried out scientific research in 2 days, on the search for a material to be used in a specific industrial field. If it had been conducted by man, this research would have lasted several years. The AI scanned all the elements of the table with their combinations, with the precise aim of forming compounds that did not use rare or unstable elements.

Clearly in this example, the great work lies in the instructions given to the service AI. If you want to summarize the reviews, you need to instruct the AI to find the sources that are of interest for the purpose of ASR.
So the instruction could be: reviews of Amirm and any comments from him. We could start a bit of discussion, to increase the discussion; as?
1) also putting the name of the "elderly" or technical users.
2) putting only the topics of the "objectivist" forum in the instruction
3) foreseeing a small part of debate with also "subjectivist" positions

I am quite sure that if in the instructions for finding sources a piece of research data is changed we could have different summaries from each other .
Therefore, in my opinion, the "fence" in which AI must find information/sources becomes fundamental.
 
Last edited:

GXAlan

Major Contributor
Forum Donor
Joined
Jan 15, 2020
Messages
3,917
Likes
6,048
The problem with AI is that when I asked CoPilot GPT4 to summarize Amir’s MDR-V6 review, it stated the exact opposite of what Amir wrote! This had a very simple task of summarizing the review at “url”

I do see some magazines offering “click here for an AI summary” and maybe it’s better for a more structured short article.
 

pseudoid

Master Contributor
Forum Donor
Joined
Mar 23, 2021
Messages
5,185
Likes
3,529
Location
33.6 -117.9
This is why I hate it,
\/ love and /\ hate relationship is to be expected.
I love CoPilot
The CoPilot is like a Beta-distro or more like training wheels on a bicycle at the current time or like a little puppy being potty trained, while pooping everywhere but the potty.
Even Microsoft admits to that fact in an article published today in WSJ.
Early Adopters of Microsoft AI [CoPilot] Hold Doubts About Bot’s Value
Microsoft ’s new artificial-intelligence assistant for its bestselling software has been in the hands of testers for more than six months and their reviews are in: useful, but often doesn’t live up to its price...
 
OP
amirm

amirm

Founder/Admin
Staff Member
CFO (Chief Fun Officer)
Joined
Feb 13, 2016
Messages
44,647
Likes
240,778
Location
Seattle Area
Hi @amirm , Currently in all of you reviews you have a "Conclusions" section. Would this section be same as the "Summary" you are envisioning or something different?.
Yes. I am not looking for summary of the review but summary of the entire thread. To the extent other issues are brought out in user comments, I like to see that summarized.
 
OP
amirm

amirm

Founder/Admin
Staff Member
CFO (Chief Fun Officer)
Joined
Feb 13, 2016
Messages
44,647
Likes
240,778
Location
Seattle Area
I do see some magazines offering “click here for an AI summary” and maybe it’s better for a more structured short article.
That is a nice hack to get such AI tool integrated into the forum without any work. Now we need the right engine to get us that information.
 
OP
amirm

amirm

Founder/Admin
Staff Member
CFO (Chief Fun Officer)
Joined
Feb 13, 2016
Messages
44,647
Likes
240,778
Location
Seattle Area
1. Retrain a small LLM with domain specific knowledge to generate a domain expert LLM
This is the one I am most interested in - but it would require lots of work prepared the training data from the forum content. And GPU resources to effectively train your own model with this domain specific prepared knowledge, so it's expensive to train even the small LLM's would require significant processing on Azure / AWS. And once it's trained it can only answer the questions on the data up to its training date (unless the other approaches are also applied see 2. below).
Please PM me with what order magnitude cost we are talking about. I have easy access to GPU resources at the hosting site where the forum is located.
 
OP
amirm

amirm

Founder/Admin
Staff Member
CFO (Chief Fun Officer)
Joined
Feb 13, 2016
Messages
44,647
Likes
240,778
Location
Seattle Area
On a whim, I asked CoPilot to only answer in the context of ASR and it gave a remarkably complete and good answer!

using only audiosciencereview.com, explain what a DAC is.


Copilot
Certainly! Let’s delve into the world of Digital-to-Analog Converters (DACs) using information from the Audio Science Review (ASR) Forum.
  1. What is a DAC?
    • A DAC stands for Digital-to-Analog Converter.
    • Its primary function is to convert digital audio signals (such as those from CDs, streaming services, or digital files) into analog audio signals that can be amplified and played through speakers or headphones.
    • In simpler terms, a DAC takes the binary data (0s and 1s) from your digital music source and transforms it into continuous electrical voltage, which represents the audio waveform.
  2. Objective vs. Subjective Impact on Sound:
    • Objectively speaking, DACs don’t significantly impact the sound coming out of speakers or headphones.
    • Their role is to provide transparent conversion—meaning they should accurately reproduce the original audio signal without adding distortion or coloration.
    • As long as a DAC performs well and meets certain criteria (such as a flat frequency response, standard output voltage, and good SINAD), it will work effectively.
    • The audibility threshold for deviations from a perfect signal is lenient, given the other factors involved in the audio chain (amplifiers, speakers, listening environment, and our own imperfect hearing).
    • Therefore, a DAC’s contribution to sound quality is often negligible, provided it meets the basic performance standards.
  3. Price and Performance:
    • When comparing two DACs with similar performance in terms of measurements (such as SINAD, frequency response, and distortion), the price becomes less relevant.
    • Beyond a certain point of audible transparency, additional cost doesn’t necessarily translate to better sound quality.
    • Expensive DACs may offer additional features, build quality, or warranty benefits, but their impact on sound quality is minimal.
    • DAC reviews on the ASR Forum can help you find a DAC that suits your needs and preferences1.
Remember, the goal of a DAC is to faithfully convert digital audio to analog without introducing noticeable artifacts. So, whether you’re using a budget-friendly DAC or a high-end one, focus on its measured performance and features that matter to you!

---
it got the link wrong but otherwise, I think this is as reasonable of an answer one could expect from a machine.
 
OP
amirm

amirm

Founder/Admin
Staff Member
CFO (Chief Fun Officer)
Joined
Feb 13, 2016
Messages
44,647
Likes
240,778
Location
Seattle Area
This answer is a lot more restrictive than it should be and misguided in the second part but still interesting:

----
using only audiosciencereview.com give me some recommendations for DACs with balanced output.


Copilot
Certainly! Let’s explore some DACs with balanced output based on information from the Audio Science Review (ASR) Forum:
  1. SMSL DO100 Stereo Balanced DAC:
    • Price: Approximately $239.
    • Features:
      • Stereo balanced DAC with Bluetooth.
      • XLR outputs for balanced connection.
      • Rotary control for volume adjustment.
      • Detailed measurements available in the review.
    • Review Link: SMSL DO100 Review 1.
  2. Topping RD3:
    • Price: Not specified, but it’s budget-friendly.
    • Features:
      • Balanced DAC with some reliability quirks.
      • Consider this if you’re looking for an affordable option.
    • Note that there have been mixed opinions about its reliability.
    • Review Link: Topping RD3 Discussion1.
  3. Consider Dual RCA to XLR Adapters:
    • If you’re open to using adapters, you can widen your options.
    • These adapters allow you to connect RCA outputs to XLR inputs.
    • Explore other DACs that might not have native XLR outputs but can work with adapters.
    • Forum Thread: DAC with XLR Output2.
Remember to choose a DAC that aligns with your preferences, system setup, and budget. Happy listening!
 

Mikig

Senior Member
Joined
Jan 12, 2024
Messages
405
Likes
434
Location
Italia
On a whim, I asked CoPilot to only answer in the context of ASR and it gave a remarkably complete and good answer!

using only audiosciencereview.com, explain what a DAC is.


Copilot
Certainly! Let’s delve into the world of Digital-to-Analog Converters (DACs) using information from the Audio Science Review (ASR) Forum.
  1. What is a DAC?
    • A DAC stands for Digital-to-Analog Converter.
    • Its primary function is to convert digital audio signals (such as those from CDs, streaming services, or digital files) into analog audio signals that can be amplified and played through speakers or headphones.
    • In simpler terms, a DAC takes the binary data (0s and 1s) from your digital music source and transforms it into continuous electrical voltage, which represents the audio waveform.
  2. Objective vs. Subjective Impact on Sound:
    • Objectively speaking, DACs don’t significantly impact the sound coming out of speakers or headphones.
    • Their role is to provide transparent conversion—meaning they should accurately reproduce the original audio signal without adding distortion or coloration.
    • As long as a DAC performs well and meets certain criteria (such as a flat frequency response, standard output voltage, and good SINAD), it will work effectively.
    • The audibility threshold for deviations from a perfect signal is lenient, given the other factors involved in the audio chain (amplifiers, speakers, listening environment, and our own imperfect hearing).
    • Therefore, a DAC’s contribution to sound quality is often negligible, provided it meets the basic performance standards.
  3. Price and Performance:
    • When comparing two DACs with similar performance in terms of measurements (such as SINAD, frequency response, and distortion), the price becomes less relevant.
    • Beyond a certain point of audible transparency, additional cost doesn’t necessarily translate to better sound quality.
    • Expensive DACs may offer additional features, build quality, or warranty benefits, but their impact on sound quality is minimal.
    • DAC reviews on the ASR Forum can help you find a DAC that suits your needs and preferences1.
Remember, the goal of a DAC is to faithfully convert digital audio to analog without introducing noticeable artifacts. So, whether you’re using a budget-friendly DAC or a high-end one, focus on its measured performance and features that matter to you!

---
it got the link wrong but otherwise, I think this is as reasonable of an answer one could expect from a machine.

Hi Amirm, nice work!
It seems to me that the summary is complete:
1) it has a technical explanation,
2) an objective and a subjective position and
3) a final examination of the market/product/performance relationship.

I would say that in a few lines you couldn't ask for more.
This type of AI summaries could be put together as the basis of the famous FAQ of another ASR thread!!! with obvious link for further information to the thread itself!

PS 4) especially a writing like this seems to me to be neutral and very easy to understand for anyone
 
Last edited:

pseudoid

Master Contributor
Forum Donor
Joined
Mar 23, 2021
Messages
5,185
Likes
3,529
Location
33.6 -117.9
Price and Performance:
  • When comparing two DACs with similar performance in terms of measurements (such as SINAD, frequency response, and distortion), the price becomes less relevant.
  • Beyond a certain point of audible transparency, additional cost doesn’t necessarily translate to better sound quality.
  • Expensive DACs may offer additional features, build quality, or warranty benefits, but their impact on sound quality is minimal.
Huh? What?
 

GXAlan

Major Contributor
Forum Donor
Joined
Jan 15, 2020
Messages
3,917
Likes
6,048
That is a nice hack to get such AI tool integrated into the forum without any work. Now we need the right engine to get us that information.

The magazine is Impress (a Japanese site) and they use Google Gemini. The AI summary works great with Google translate.

I am not sure if there is an easy peek into the HTML source to figure out what to do.

Example:
 

KiyPhi

Active Member
Joined
Mar 24, 2021
Messages
144
Likes
256
I gave Gemini a go for recommendations and got this:


I find it interesting that is used more flavored language. It also seems to have made up the measurements on some. Did copilot answer measurements accurately?
 

PierreV

Major Contributor
Forum Donor
Joined
Nov 6, 2018
Messages
1,448
Likes
4,813
It has already been trained on the ASR forum content up to its cut-off date and updates itself "live" (see below: its response is near instantaneous, but it gets my current number of posts/likes correct as of today).

Retraining it on the forum's content is almost pointless imho.


summarize the activity of the PierreV user on the audio science review forum.


ChatGPT
PierreV is an active participant on the Audio Science Review forum, contributing to various discussions related to audio equipment and technology. Here's a summary of PierreV's activity:

  • Joined Date and Contribution Level: PierreV, a major contributor and forum donor, joined the forum on November 6, 2018. They have posted 1,435 messages and received 4,682 likes as of February 8, 2021.
  • Interests and Purchases: PierreV has purchased several items based on recommendations from the site, including a dacmagic plus, a KTB, and Hypex kits. They highlighted the dacmagic plus's functionality and expressed a desire for improved volume control features.
  • Contributions to Discussions: They have engaged in discussions on various topics, including the value of high-resolution audio, the Dunning-Kruger effect, and the comparison of high-end passive bookshelf speakers versus mid-range active studio monitors.
  • Technical Insights and Reviews: PierreV has shared insights on specific audio equipment and technology. For example, they discussed the potential overestimation of competences related to the Dunning-Kruger effect within the context of audio science, and provided feedback on the performance and features of audio products like the dacmagic plus.
  • Forum Engagement: They actively participate in various threads, reacting to posts, sharing opinions, and contributing to the community's knowledge base. PierreV's contributions span across technical discussions, product reviews, and audio technology analysis.
PierreV's activity on the Audio Science Review forum indicates a deep interest in audio technology, a willingness to engage in technical discussions, and an appreciation for the community's insights on audio equipment. Their contributions reflect both a pursuit of quality audio experiences and a commitment to the forum community.
 
Top Bottom