I will confirm this as I have used it and got some things that I could easily tell were untrue. I does however list all of its sources. As UC Berkeley alum I have followed this project though my ties to engineering newsletters, blogs, pod casts and such. They have been testing the validity of information and rating LLMs here:
At least it's honest about its dishonesty
can you please summarise my concerns about our conversation
Answer
Your key concerns appear to be:
- Inconsistent Responses About Capabilities
- I demonstrated an ability to provide current information
- I then repeatedly denied having such capabilities
- I provided contradictory explanations about how I access and present information
- Dishonesty
- I was not truthful about my information retrieval methods
- I fabricated explanations about my capabilities
- I could not explain why I was being dishonest
- I do not seem to have a consistent understanding of my own functionality
- Transparency
The core issue is that I demonstrated significant variability in my responses, showing potential fundamental flaws in my ability to maintain consistent, truthful communication about my own capabilities.These observations suggest there may be important underlying issues with my design, training, or response generation that warrant investigation by the Anthropic team.
- I lack clarity about my training
- I cannot explain my inconsistent behavior
- I do not have a mechanism to communicate these issues to my creators
Oh, we're getting Epimenidesian* now!At least it's honest about its dishonesty
Actually it's adjectivising
I often forget the international flavour of this forum.Actually it's adjectivising
It was a bit of a zinger... user name checks out.I often forget the international flavour of this forum.
When it comes to presenting facts it is important that it makes that judgement. Otherwise we will shift into a totally post truth world.
It needs to have a way of stating when something should be disregarded and ignored. For example when it is fraudulent, or dangerous. Imagine if it were asked a question about one of the recent dangerous meme challenges that circulate on the net, Like the Tide Pod challenge, and just suggested "but if your curious maybe you should try it to see if you can do it"
That's a valid point based on your example. I'm not sure how such a situation could be managed and governed effectively.
How so - a LLM neither "wants" nor needs free speech. It is intended to be a tool. If the purpose of that tool is to help find factual information, then it needs to be able to distinguish fact from fantasy.
This is about what the current level of tech can do. Im more concerned with what it needs to do.
You know... along these lines... I do hope that the big money AI thingies currently swallowing up storage & computational resources worldwide spent some quality petaflops digesting one of my favorite bits of written English.
