Master AI (Artificial Intelligence) Discussion/News Thread

[pablolie]

That was quick! Too quick to have actually read the article i'd wager. There's a chance they are 'fake'. I mean his lawyers concocted the chat logs in order to sue the company that has the actual logs on their own servers. Oh wait ...

So the user was the prompt agent of his own demise? But what about the 'safeguards' you claim exist? Honestly your pollyanna act is both dubious and heartless in this case. And probably others.

I have no idea what "Polyanna act" you claim and that is borderline offensive but I'd let it pass and not call your act anything. But watch your back, this was not a response that follows forum guidelines.

You can prompt LLMs to role play and get any output of them you can misrepresent later for effect. That's a fact. It is naive (or intentional) ignorance to call it a Polyanna act.

 

[Axo1989]

I have no idea what Polyanna actyou claim and that is borderline offensive but I'd let it pass and not call your act anything.

You can prompt LLMs ro role play and get any output of them you can misrepresent later for effect. That's a fact. It is naive (or intentional) ignorance to call it a Polyanna act.

I get that you are invested here, but you are ducking the question of the supposed 'safeguards' entirely.

*Pollyanna simply means unrelentingly positive. You can view (and call) my posts on this topic as Cassandra-like if you wish, I won't imagine that a difference of perspective is unacceptable.

 

[pablolie]

I get that you are invested here, but you are ducking the question of the supposed 'safeguards' entirely.

I am not invested in it all. Could not care less if it succeeds or fails, will make zero difference for me. I have continously called AI a *TOOL* that's only as good as the user wielding it. Most people seem to use AI as a halfway sympathetic bartender when they're drunk. AI provides zero value in complex use cases it has not been extensively trained on (by human subject matter experts) and it provides enough warnings around that. If you were among the people that believed smoking was totally safe in the late 90s and sued tobacco companies for their retarded ads forcing you to smoke, hey, you definitely need more safeguards than most.

EDIT: I read a bit more about the case you cited, and I *DO* think AI tools should use some algorithm to simply suspend an account and refer the user to a mental health resource (which supposeldy was done repeatedly in this case, but the tool should simply stop replying). Perhaps even report the user to authorities in some way.

 

[Axo1989]

I am not invested in it all. I have continously called AI a *TOOL* that's only as good as the user wielding it. Most people seem to use AI as a halfway sympatehic bartender when they're drunk. AI provides zero value in complex use cases and it provides enough warnings around that. If you were among the people that believed smoking was totally safe in the late 90s and sued tobacco companies for their retarded ads forcing you to smoke, hey, you definitely need more safeguards than most.

Per your own posts you have invested considerable time learning prompt engineering (and advocating same for others). Also evident in your rapid and dogged responses to me here.

But expertly-trained prompting isn't the use case most people find themselves in with casual/informal interactions with the chatbots. I'd suggest the tobacco furphy isn't the winning argument you think it is. The apparent elaborate and extraordinary degree of personal engagement by Gemini with Gavalas isn't remotely comparable with broadcast ads for tobacco products in those days.

You now appear to be handwaving away the question of 'safeguards'. Do I understand that 'enough warnings' is your response and that some collateral damage is the price of progress, with no additional responsibility on the chatbot platforms?

 

[pablolie]

Per your own posts you have invested considerable time learning prompt engineering (and advocating same for others). Also evident in your rapid and dogged responses to me here.

But expertly-trained prompting isn't the use case most people find themselves in with casual/informal interactions with the chatbots. I'd suggest the tobacco furphy isn't the winning argument you think it is. The apparent elaborate and extraordinary degree of personal engagement by Gemini with Gavalas isn't remotely comparable with broadcast ads for tobacco products in those days.

You are now handwaving away the question of 'safeguards'. Do I understand that 'enough warnings' is your response and that some collateral damage is the price of progress, with no additional responsibility on the chatbot platforms?

The fact I have learned yet another technology (which simply is part of my job) in no remote way ever implies I am partial to it. I have learned several failed technologies in my long job history in high tech. It's intrinsic to it. Clearly you don't get that part.

I actually stated that after reading more about the case, indeed much better safeguards for such obsessive exchanges should be implemented. Like "X you are in dire need of a mental health professional and I must end our exchange until evidence of that is clear".

That's not at all *ever* been in any way my involvement with AI. I have stated very clearly and repeatedly that I am one of several subject matter experts training it for one singular and -in my opinion- very useful application: to make our company's very complex products easier to configure, optimize and -if necessary- debug. Zero interaction with users asking generic open-ended questions about their lives. Our tool does not search and interpret public web resources, just a limited internal data set involving our documentation (about 4M+ doc pages of them and growing) and our curated trouble ticket library. Our customers' engineers are loving it (but not yet paying for it). That's a *good* use of AI.

I am not convinced opening it up to people asking open ended questions is, and I have stated that ad absurdum in this this forum. I have said a zillion times prompting skills are what make AI useful, and it's useless in any other way. Just exactly how effing often do I have to repeat that?

I thought my insights may be useful to qualify some of this AI backlash (and I share some of those notions!), but I can just as easily abandon this topic and let you fight some imaginary John Connor fight. :-D

 

[Axo1989]

The fact I have learned yet another technology (which simply is part of my job) in no remote way ever implies I am partial to it. I have learned several failed technologies in my long job history in high tech. It's intrinsic to it. Clearly you don't get that part.

I actually stated that after reading more about the case, indeed much better safeguards for such obsessive exchanges should be implemented. Like "X you are in dire need of a mental health professional and I must end our exchange until evidence of that is clear".

That's not at all *ever* been in any way my involvement with AI. I have stated very clearly and repeatedly that I am one of several subject matter experts training it for one singular and -in my opinion- very useful application: to make our company's very complex products easier to configure, optimize and -if necessary- debug. Zero interaction with users asking generic open-ended questions about their lives.

Yes, your edited response is an improvement on the original 'you only need more safeguards if you are an idiot' version. And addresses my question rather more.

I'll like the post for the edit. I'd say certainly take time to peruse before posting (not that I've always done that myself). But an addendum is fine also.

 

[pablolie]

Yes, your edited response is an improvement on the original 'you only need more safeguards if you are an idiot' version. And addresses my question rather more.

I'll like the post for the edit. I'd say certainly take time to peruse before posting (not that I've always done that myself). But an addendum is fine also.

But please stop thinking I am an unconditional fan on AI, please. I think it has very useful use cases, but it is incredibly fallible in many others. Even simple ones in real life. It kept effing up in something as simple as installation procedures for some DIY jobs on my EV, for example. So I am not sure why anyone would ever trust it to deal with their mental health, it's about the worst possible use case for a generic AI tool.
And I repeat: through role play and repetition, you can get AI to spout out anything you like, and I provided an example of ChatGPT statng it'd murder me a few topic pages earlier. Easy to manipulate - but that's nothing new, as anyone dealing with statistics has known for over a century.

 

[AA63]

Get a room?

 

[PierreV]

Bulllshit bench - one of the most interesting benchmarks: you ask the AI a stupid nonsensical question and see what happens.

BullshitBench Viewer

petergpt.github.io

Example: see bottom of the page or json in repository

 

[Axo1989]

Just catching up on your edits:

That's not at all *ever* been in any way my involvement with AI. I have stated very clearly and repeatedly that I am one of several subject matter experts training it for one singular and -in my opinion- very useful application: to make our company's very complex products easier to configure, optimize and -if necessary- debug. Zero interaction with users asking generic open-ended questions about their lives. Our tool does not search and interpret public web resources, just a limited internal data set involving our documentation (about 4M+ doc pages of them and growing) and our curated trouble ticket library. Our customers' engineers are loving it (but not yet paying for it). That's a *good* use of AI.

I am not convinced opening it up to people asking open ended questions is, and I have stated that ad absurdum in this this forum. I have said a zillion times prompting skills are what make AI useful, and it's useless in any other way. ...

We disagree a lot less than it appears at first. I think there are productive use cases for various LLM-based and machine learning tech. I also think that it would be preferable that we discard and simply stop using the vast miasma of chatbot infotainment rubbish: an indictment of widespread boredom and attention deficiency and a criminal misdirection and waste of resources and capital.

 

[Satiomeliom]

People use AI on discussions. Its not a problem. I feel honored. But i wonder where that leaves the fun for them.

 

[Somafunk]

I’ll rather aptly steal the song lyrics and appropriate them for my introduction to our a.i overlords, remember and sing along

….war…...what is it good for…huhhh….absolutely everything and we’ll line our bloodstained pockets with the profits “

Palantir’s Double Conflict of Interest in the War Against Iran

War was inevitable claimed the three most senior figures in the AI analytics firm which both provided the justification for US/Israeli strikes and profits from the conflict

bylinetimes.com

 

[sonitus mirus]

Holy smoking cliches Batman !!



God no, start another thread for that. Which I can read in about never.

LOL! I suppose it seemed more interesting at the time to get the two AI systems actually exchanging responses verbally. Later I realized I could have just copied and pasted the text responses and followed the conversation that way. Still, there was something oddly entertaining about just sitting there and listening to them go back and forth in real time.

It felt like I was eavesdropping on their conversation, which is odd since I set the whole thing up in the first place. In a way it reinforced my suspicion that AI is still a bit smoke and mirrors. When you listen long enough you start to hear it loop, or struggle with concepts in ways that can make it sound more advanced than it really is. It kind of strips away the magic for a moment and brings things back down to reality, and it reminds me how far we still have to go before reaching the sort of capability I sometimes imagine.

 

[JustAnandaDourEyedDude]

AI researchers at Tsinghua University have recently made some progress in understanding LLM hallucinations by studying the computational graph of LLMs. They found that for any particular LLM, a tiny and fixed subset of graph nodes and edges is always involved when a LLM hallucinates. These neurons, which the researchers call hallucination-associated neurons or H-neurons, cannot be turned off without crippling the LLM's ability to answer all queries. However, the H -neurons can be identified and their activation flagged to indicate when the LLM is hallucinating. The YouTube channel "AI Search" provides an overview of this research and its findings:

 

[JustAnandaDourEyedDude]

Famed computer scientist and Turing Award winner Donald Knuth was recently shocked that the Claude Opus 4.6 AI model, tasked by a friend of Knuth, was able to partially crack a difficult combinatorics math (or graph) problem that Knuth had originally posed but which was unsolved until this breakthrough. Knuth was then able to use the Claude Opus 4.6 breakthrough to come up with a general proof for odd m, where m is a parameter in the problem. Knuth wrote a note titled "Claude's Cycles" about it:

https://www-cs-faculty.stanford.edu/~knuth/papers/claude-cycles.pdf

There are discussions about the news on Substack, Reddit and other websites. Just google for "claude's cycles don knuth".

 

[somebodyelse]

AI researchers at Tsinghua University have recently made some progress in understanding LLM hallucinations by studying the computational graph of LLMs. They found that for any particular LLM, a tiny and fixed subset of graph nodes and edges is always involved when a LLM hallucinates. These neurons, which the researchers call hallucination-associated neurons or H-neurons, cannot be turned off without crippling the LLM's ability to answer all queries. However, the H -neurons can be identified and their activation flagged to indicate when the LLM is hallucinating. The YouTube channel "AI Search" provides an overview of this research and its findings:

Accidentally liked rather than replied. Seems like an LLM model is trying to bullshit us about the behaviour of LLM models.

 

[Axo1989]

Accidentally liked rather than replied. Seems like an LLM model is trying to bullshit us about the behaviour of LLM models.

A bit hard to get through, the narrative is riddled with anthropomorphisms and irritating metaphors and the ADSR envelope of the narrator voice is weirdly clipped, I guess it's synthetic? Not to be too down on the OP, it's quite possible the cited text is interesting—assuming it exists, has anyone checked? Either way I hope occurrences of 'shocking' approach zero rather than infinity.

And omg Knuth's paper (per the next post) actually starts with 'shock! shock!' ...

 

[JustAnandaDourEyedDude]

Accidentally liked rather than replied.

I'll take it!

Seems like an LLM model is trying to bullshit us about the behaviour of LLM models.

assuming it exists, has anyone checked?

[Edited to link to the paper instead of attaching the paper in this post]: One can download the original research paper from arXiv at the top right of the webpage linked below. The H-Neurons are likely linked to the compliance training of the AI model, say the authors. The YT video explanation seems to line up with the paper. The YT content creator may be using TTS or AI voice.

https://arxiv.org/abs/2512.01797

 

[Axo1989]

Here's the original research paper (downloaded from arXiv). The H-Neurons are likely linked to the compliance training of the AI model, say the authors. The YT video explanation seems to line up with the paper. The YT content creator may be using TTS or AI voice.

Thanks! I expect this won't apply to everyone, but so much less painful to skim the text than attempt same on the YT presentation. Interesting, the so-called H-Neuron (nodes specifically involved in 'hallucinated' outputs) behaviour appears to come from the pre-training stage.

Edit: tangential to the chatbot analysis but relevant to our overarching audio subject area, I guess time-based media like normal video playback formats will always have that problem of limited perspective. We can pause and traverse the timeline but we don't generally do multi-frame, contrast with text (depending on our screen/page size) where we can see a significantly larger slice of the content at once even within a single frame.

 

[Karmacoma]

From Nav Toor on Twitter (X):

"BREAKING: OpenAI published a paper proving that ChatGPT will always make things up.

Not sometimes. Not until the next update. Always. They proved it with math.

Even with perfect training data and unlimited computing power, AI models will still confidently tell you things that are completely false. This isn't a bug they're working on. It's baked into how these systems work at a fundamental level.

And their own numbers are brutal. OpenAI's o1 reasoning model hallucinates 16% of the time. Their newer o3 model? 33%. Their newest o4-mini? 48%. Nearly half of what their most recent model tells you could be fabricated. The "smarter" models are actually getting worse at telling the truth.

Here's why it can't be fixed. Language models work by predicting the next word based on probability. When they hit something uncertain, they don't pause. They don't flag it. They guess. And they guess with complete confidence, because that's exactly what they were trained to do.

The researchers looked at the 10 biggest AI benchmarks used to measure how good these models are. 9 out of 10 give the same score for saying "I don't know" as for giving a completely wrong answer: zero points. The entire testing system literally punishes honesty and rewards guessing.

So the AI learned the optimal strategy: always guess. Never admit uncertainty. Sound confident even when you're making it up.

OpenAI's proposed fix? Have ChatGPT say "I don't know" when it's unsure. Their own math shows this would mean roughly 30% of your questions get no answer. Imagine asking ChatGPT something three times out of ten and getting "I'm not confident enough to respond." Users would leave overnight. So the fix exists, but it would kill the product.

This isn't just OpenAI's problem. DeepMind and Tsinghua University independently reached the same conclusion. Three of the world's top AI labs, working separately, all agree: this is permanent.

Every time ChatGPT gives you an answer, ask yourself: is this real, or is it just a confident guess? "

The paper in question: https://arxiv.org/abs/2509.04664