Using AI In Audio Debates & GR Research [Video]

[RexrothPigeon]

I have used AI to help me make right decisions I may not otherwise have made or attempted without it and that is broadly in terms of law, irs, audio, books i may like, education, finance, accounting, taxes...that is pretty darn powerful tool not a toy.

Interesting, let's look at some of those and see if LLMs are accurate enough for mission-critical applications:

law:

Lawyers know AI can hallucinate. Judges have warned them. Courts have sanctioned them for it. They keep citing fake AI cases anyway.

The trend of attorneys getting caught citing AI-hallucinated cases points to a broader problem: instead of checking AI’s work, people keep trusting it

www.scientificamerican.com

High court tells UK lawyers to stop misuse of AI after fake case-law citations

Ruling follows two cases blighted by actual or suspected use of artificial intelligence in legal work

www.theguardian.com

Massachusetts Lawyer Sanctioned for AI-Generated Fictitious Case Citations

In a well-reasoned opinion, a Court sanctioned a Massachusetts lawyer for citing fictitious cases in court pleadings that were produced by an AI tool. The case highlights some of the real risks of using AI in the legal profession.

www.msba.org

taxes, finance:

Experts are warning against using AI for financial advice. Here’s why

Experts recommend taking extra caution when asking AI for financial help.

www.kplctv.com

https://www.nerdwallet.com/taxes/studies/doing-taxes-with-ai

https://eyfs.ie/wp-content/uploads/2024/04/AI-in-Tax-Article-2024-V9-FINAL-10.04.24.pdf

(anecdotally, my accountant just gave me my tax bill yesterday and it bears zero resemblance to the predictions I got from (paid) Claude and (paid) ChatGLP as well as (unpaid) Gemini and Grok. Fortunately it was much lower than any of those predictions so I'm happy.)

My experience seems to be validated by these links: LLMs can be useful, but they simply aren't accurate enough to rely on anything important unless you supplement them with your own research. Some may quibble about where the boundaries of "very accurate" should be set, but to me, it's a non-starter if I can't use it for important applications without skepticism and supplemental hands-on research of my own.

 

[AudioX3]

Interesting, let's look at some of those and see if LLMs are accurate enough for mission-critical applications:

law:

Lawyers know AI can hallucinate. Judges have warned them. Courts have sanctioned them for it. They keep citing fake AI cases anyway.

The trend of attorneys getting caught citing AI-hallucinated cases points to a broader problem: instead of checking AI’s work, people keep trusting it

www.scientificamerican.com

High court tells UK lawyers to stop misuse of AI after fake case-law citations

Ruling follows two cases blighted by actual or suspected use of artificial intelligence in legal work

www.theguardian.com

Massachusetts Lawyer Sanctioned for AI-Generated Fictitious Case Citations

In a well-reasoned opinion, a Court sanctioned a Massachusetts lawyer for citing fictitious cases in court pleadings that were produced by an AI tool. The case highlights some of the real risks of using AI in the legal profession.

www.msba.org

taxes, finance:

Experts are warning against using AI for financial advice. Here’s why

Experts recommend taking extra caution when asking AI for financial help.

www.kplctv.com

https://www.nerdwallet.com/taxes/studies/doing-taxes-with-ai

https://eyfs.ie/wp-content/uploads/2024/04/AI-in-Tax-Article-2024-V9-FINAL-10.04.24.pdf

(anecdotally, my accountant just gave me my tax bill yesterday and it bears zero resemblance to the predictions I got from (paid) Claude and (paid) ChatGLP as well as (unpaid) Gemini and Grok. Fortunately it was much lower than any of those predictions so I'm happy.)

My experience seems to be validated by these links: LLMs can be useful, but they simply aren't accurate enough to rely on anything important unless you supplement them with your own research. Some may quibble about where the boundaries of "very accurate" should be set, but to me, it's a non-starter if I can't use it for important applications without skepticism and supplemental hands-on research of my own.

I sorry to say this as my intentions are pure and not trying to be antagonistic but your tone and responses are dismissive and so I have to point out that they are also a disservice. I think you are doing exactly the same strawman assertional analysis that this whole thread started with on how Danny Richie misused AI and is making false claims with links to broad ancdotal "evidence". You do not show what your question is, if it was give me evidence that LLMs can be dangerous and bad information, well of course. Can you site this as broad in every area i mentioned, absolutely. Can it happen across all AI chat platforms no matter the model or mode, absolutely. Regarding your tax example, maybe you gave the AI wrong information or it misinterpretted your prompts. Also, maybe your tax accountant is wrong but if you got lower taxes and don't get audited that works out for the better.

But you have proven nothing in terms that definitively proves AI in 2026 is nothing but a toy and completely untrustworthy for critical decisions or research. Your experience however for your expectation of results leaves you uncomfortable and that is fair enough for your risk tolerance.

Regarding my use cases, without giving you details and you don't have to believe me, but I think folks can see how these use cases can indeed make great valuable use of AI as a tool when care is taken to use properly.

I had tax questions, I used AI to find the answers from the IRS code. It gave me the answer. Was I going to just believe it, no. I looked then at the IRS code it sited and verified. I thus filed my taxes correctly.

Regarding, law, I had some situations I needed legal understanding, AI went into nexus/lexus and gave me case law I would never have found. I put some effort in to verify it to be true and applicable to my situation. That was very valuable and very quick without engaging a lawyer. At worst, I could have then engaged the lawyer simply to verify for 1 hour rather than 4 hours to research and verify. I then went in and negotiated a resolution to an legal issue that was very much in my favor.

Regarding finance, I developed a portfolio manager and AI helped with some complex formulas I may not have come up with. It also quickly taugt me how to use the finance features of Google sheets and that it was better than excel. I verified the formulas and the outputs to be true. We traded ideas on improvements. I now have a portfolio manager that I am using that automates signals (not trades) and position to risk tolerance. It is based on solid financial principals for manged accounts.

Regarding Audio, and most relevant to the ASR forum and maybe not as critical as the other areas but certainly critical to this audiance. I asked AI to help me replace my Adcom 545 ii with a more modern amp with protections to speakers because I love my speakers and can never replace them if i blow the woofers/tweaters. It asked me some solid questions and in the end we refined down to Buckeye and then down to which Buckeye based on the parameters I value. Now in this case if I did it on my own, I would have come to the same conclusion. But that AI took me to the same conclusion in an area I do know, that was very re-assuring on many levels including confidence in AI when used right.

My examples demonstrate Very Accurate, not necessarily Very Precise but good Precision.

We can agree to disagree, but my goal is to those in the forum who are interested in AI, don't dismiss it based on fear and strawman logic. And if you do use it, use it wisely and give it queries that can be verified to solve a specific problem, gain knowledge, perform research, get a second or confirming opinion. You have a very valuable tool in 2026 that you have access to potentially for free or a reasonable cost that can enable you to accomplish things efficiently, accurately with due dilligence, and save you money.

 

[RexrothPigeon]

don't dismiss it based on fear and strawman logic

I offered up several links, including a study, showing that when people rely on LLMs for mission-critical applications, they get burned. You seized on my one personal example, that I explicitly labeled as anecdotal, as a "straw man." Not an indication of good faith discussion.
Anyway, as I've repeatedly said, I don't dismiss LLMs. I use them all the time. But I don't trust or rely on them. Their convenience is that they are easy and fast, so they are good for first-stage applications. They are unreliable for anything important, though, so anyone not wishing to get burned must also do their own research to verify (or debunk). This, to me, should be completely uncontroversial to anyone making meaningful use of LLMs circa 2026.

*of course there are also environmental and economic concerns, but that's just too big and contentious a topic for this thread imo

 

[AudioX3]

I offered up several links, including a study, showing that when people rely on LLMs for mission-critical applications, they get burned. You seized on my one personal example, that I explicitly labeled as anecdotal, as a "straw man." Not an indication of good faith discussion.
Anyway, as I've repeatedly said, I don't dismiss LLMs. I use them all the time. But I don't trust or rely on them. Their convenience is that they are easy and fast, so they are good for first-stage applications. They are unreliable for anything important, though, so anyone not wishing to get burned must also do their own research to verify (or debunk). This, to me, should be completely uncontroversial to anyone making meaningful use of LLMs circa 2026.

*of course there are also environmental and economic concerns, but that's just too big and contentious a topic for this thread imo

I say peace, and we move on. Take care.

 

[xnor]

Let me be blunt, lots of people never even learned how to use Google properly, how to research a topic, so LLMs must feel like a miracle to them.
It's like showing autocomplete to a person that just learned to type. Ironically, LLMs are closer to autocomplete than to traditional web search.

 

[Mnyb]

The problem is in it’s actual use on for example this audio forum , disgruntled or otherwise agitated people clutching at AI for a ”counter argument” ( they think you have to ”win” discussions ) .

And just supercharge brandolinis law , putting up walls of gish gallop . It would take literally eons to painstakingly counter that.

I think ASR is about a personal experience , learning stuff about audio and have fun . Do the AI participate in personal and fun ? No it does not .

So I just think all copy paste AI responses should be banned. I use it for basically ”improved search” professionally and to make summarises of long meetings or ongoing topics , so it’s a great research tool make a good prompt and follow up the source’s it quotes.
99% of problems AI sorted for me has been about finding the original unambiguous source of something .

Do your research with AI , but for audiophile affairs with specific disclaimers that almost all training data is BS . I would get equal nonsense answer if a trianed my astronomical AI on Astrology data ?

But copied wall of AI text are either done in affect or maliciously !

This post is made entirely by my tired self sipping coffee writing in bad English, a second language for me

 

[antcollinet]

Interesting, let's look at some of those and see if LLMs are accurate enough for mission-critical applications:

law:

Lawyers know AI can hallucinate. Judges have warned them. Courts have sanctioned them for it. They keep citing fake AI cases anyway.

The trend of attorneys getting caught citing AI-hallucinated cases points to a broader problem: instead of checking AI’s work, people keep trusting it

www.scientificamerican.com

High court tells UK lawyers to stop misuse of AI after fake case-law citations

Ruling follows two cases blighted by actual or suspected use of artificial intelligence in legal work

www.theguardian.com

Massachusetts Lawyer Sanctioned for AI-Generated Fictitious Case Citations

In a well-reasoned opinion, a Court sanctioned a Massachusetts lawyer for citing fictitious cases in court pleadings that were produced by an AI tool. The case highlights some of the real risks of using AI in the legal profession.

www.msba.org

taxes, finance:

Experts are warning against using AI for financial advice. Here’s why

Experts recommend taking extra caution when asking AI for financial help.

www.kplctv.com

https://www.nerdwallet.com/taxes/studies/doing-taxes-with-ai

https://eyfs.ie/wp-content/uploads/2024/04/AI-in-Tax-Article-2024-V9-FINAL-10.04.24.pdf

(anecdotally, my accountant just gave me my tax bill yesterday and it bears zero resemblance to the predictions I got from (paid) Claude and (paid) ChatGLP as well as (unpaid) Gemini and Grok. Fortunately it was much lower than any of those predictions so I'm happy.)

My experience seems to be validated by these links: LLMs can be useful, but they simply aren't accurate enough to rely on anything important unless you supplement them with your own research. Some may quibble about where the boundaries of "very accurate" should be set, but to me, it's a non-starter if I can't use it for important applications without skepticism and supplemental hands-on research of my own.

On the other hand - I asked Chat GPT for help creating a retirement lifetime cashflow calculation in excel. I'd expected it to tell me how to structure it, and which formula to put in which cells.

Instead it just created it. It looks like the attached image (after import into apple numbers). (Data randomised and made small enough to be illegible) You can see how many rows and columns of formulae make it up. Two sheets - one with all the setup data - one with the calculation. Chat GPT also looked up all the tax rates and allowances to insert into the setup sheet.


Worked first time. I didn't trust it. I validated it though a number of what/if scenarios comparing with the data provided by my FA. I then had a few iterations of prompt to get it to add three charts that I wanted. Total time invested less than two hours. I spent probably 15 hours in total creating something similar but not as good when I first retired.

 

[testp]

I think we have different understandings of "highly accurate." To me, a calculator is "highly accurate." A well calibrated thermometer or barometer is highly accurate. Klippel NFS is highly accurate.
I use LLMs all the time. They're useful. But they remain toys: sometimes they can be used as tools (I can make a plate or something out of Legos; it will work) but since--as you say--you have to verify, they're too risky for anything mission-critical since they cannot be trusted (I wouldn't send Lego tanks into war).

machine learning models (already trained or your own data) -> can and is used very accuretly. it's at least consistant, so verification isn't necessary as long confidence was high.

 

[AudioX3]

On the other hand - I asked Chat GPT for help creating a retirement lifetime cashflow calculation in excel. I'd expected it to tell me how to structure it, and which formula to put in which cells.

Instead it just created it. It looks like the attached image (after import into apple numbers). (Data randomised and made small enough to be illegible) You can see how many rows and columns of formulae make it up. Two sheets - one with all the setup data - one with the calculation. Chat GPT also looked up all the tax rates and allowances to insert into the setup sheet.


Worked first time. I didn't trust it. I validated it though a number of what/if scenarios comparing with the data provided by my FA. I then had a few iterations of prompt to get it to add three charts that I wanted. Total time invested less than two hours. I spent probably 15 hours in total creating something similar but not as good when I first retired.

Miraculous and Auto Complete. Touche! (Plus your typing skill is impecable. Remarkable)

 

[antcollinet]

I offered up several links, including a study, showing that when people rely on LLMs for mission-critical applications, they get burned

But you only offered links where the tool had been used incorrectly - you missed the (probably) many more examples where lawyers have used AI to find case law, and have then verified that case precedence by looking it up in the authoritative sources - probably saving themselves hours, or days in finding relevant case law.

 

[antcollinet]

Miraculous and Auto Complete. Touche! (Plus your typing skill is impecable. Remarkable)

Not at all sure what you are saying (or perhaps implying).

 

[xnor]

He's just salty because of my reply.

 

[338h10]

I have used AI to help me make right decisions I may not otherwise have made or attempted without it and that is broadly in terms of law, irs, audio, books i may like, education, finance, accounting, taxes...that is pretty darn powerful tool not a toy.

I am a tax attorney, often involved in the most cutting-edge transactions where you need to know the last ounce of tax regulations and case law to make the right judgment. I periodically test reputable, subcription-based professeional tax AI engines (e.g., Bloomberg) on a number of subjects where tax laws are uncertain on the surface but the actual answers are clear and established -- AI is incorrect 1 out of 3. To me, it is an efficient search engine that can point me to the right sources faster (even when its answers are wrong), but I cannot rely on its answers (yet).

 

[RexrothPigeon]

But you only offered links where the tool had been used incorrectly - you missed the (probably) many more examples where lawyers have used AI to find case law, and have then verified that case precedence by looking it up in the authoritative sources - probably saving themselves hours, or days in finding relevant case law.

Because that was my point: That LLMs are not appropriate for mission-critical applications, that if you need to verify what they tell you, and that as a result, claims of their being very accurate are misguided. They are complex autocomplete machines, and if you think they are accurate and use their output as they offer it, you will get burned.

 

[AudioX3]

Because that was my point: That LLMs are not appropriate for mission-critical applications, that if you need to verify what they tell you, and that as a result, claims of their being very accurate are misguided. They are complex autocomplete machines, and if you think they are accurate and use their output as they offer it, you will get burned.

 

[Axo1989]

But you only offered links where the tool had been used incorrectly - you missed the (probably) many more examples where lawyers have used AI to find case law, and have then verified that case precedence by looking it up in the authoritative sources - probably saving themselves hours, or days in finding relevant case law.

Because that was my point: That LLMs are not appropriate for mission-critical applications, that if you need to verify what they tell you, and that as a result, claims of their being very accurate are misguided. They are complex autocomplete machines, and if you think they are accurate and use their output as they offer it, you will get burned.

I’ve had mixed results personally the very few times I’ve tried these things but don’t doubt they can be useful. It’s not just a case of ‘incorrect use’ the generative LLM-based tools simply can’t be entirely accurate by virtue of the underlying probabilistic tech. A new or hybrid foundation will be required for that.

 

[AudioX3]

I think we all agree LLMs are not for mission critical tasks. I doubt anyone here is involved in mission critical tasks. But they can make use of Ai for critical tasks to them and several have given examples of their successes in critical areas to them.

The point of contention is you called it a toy and not a tool. I would think since you use LLMs often you are using it as a tool not a toy. Maybe you may want to rephrase that so those who have no experience with AI can understand use it but be careful with it.

But really I shared my tips and experiences as so have others. Hope others can find that of value and I be glad to help anyone who reaches out to me. I think it could make for a useful channel to add where folks can go and share.

This is last I gonna post on this thread and we can agree to disagree if that as close to common ground we come to np.

 

[Sal1950]

Just posted a video about general use of AI in audio debates with specific example of Danny Ritchie from GR Research using it to prove he is "right."
I also get into why we have our policy regarding use of AI at ASR.

I'm sure someday in the future AI will greatly improve. But in my experience over the last few years, whenever I attempt an internet search on a subject, the first thing that comes up is an AI answer. And in better than 50% of the cases on something I already have a little knowledge in, AI's answer is WRONG. Grrr.

 

[RexrothPigeon]

I would think since you use LLMs often you are using it as a tool not a toy.

I am using it like a toy that can sometimes have the usefulness of a tool. Like legos. Or a slot machine.
Everything it does for me is done at a level well below what I would do on my own, after all. It just encourages my laziness, which makes sense since its basic goal is to keep me using it.

 

[Mnyb]

When it improves it’s not going to be better for us , it will be better at deception and following the agendas setup by thier owners we will never know truth again.

[edit] is my response more fit for the general AI tread ? Is this to off topic?