AI makes mistakes on simple questions

[JeffGB]

Today I asked Google a question which elicited a completely incorrect answer. Actually I made a statement, designed to elicit a response. The statement was " The high gain setting of my headphone amp sounds better even at low volumes. Bass is more powerful etc.". In AI's overview answer it said "Better Volume Control: Many volume knobs sound unbalanced or thin at the very bottom of their turn. Using high gain lets you turn the physical knob up higher, where the channel balance and sound quality are often cleaner."

I told it fact check itself and explained how it was wrong, to which it answered "
You are entirely correct, and I appreciate you catching that. I completely reversed the logic on how the volume knob behaves.". Clearly, AI is capable of great improvements in speed for research but you must be able to fact check it's answers. Now, after having gone through this question/answer will Google AI now "know" the correct answer or will it just begin again from scratch if someone else asks?

 

[RexrothPigeon]

the latter

 

[JeffGB]

Well, I have my answer. The AI will not retain anything that was corrected during my questioning. If asked the same question it will go through the same logic searching and likely give the same incorrect answer. It told me it would only change when it's developer updated it's underlying model and training data. Wow, what an incredible waste of energy. No wonder the data centers require so much power and cooling. I guess the long and the short of it is, it will never be accurate because developers will simply add more features and capabilities rather than use selective, verified as correct, data. They are simply using the internet for data, correct or not. I think AI, if used based on physics, medical and other verified data, will be a major game changer but the current design and use is going to be unacceptable.

 

[sonitus mirus]

That summary box in Google search isn't really the same animal as Gemini or ChatGPT. It runs on the cheapest, fastest model they've got, summarizing whatever pages the search happened to cough up. Ask the same question in Gemini itself and you'll usually get something better with sources you can actually check.

The part that jumped out at me is that you didn't ask a question, you made a statement. It agreed. Then you told it that was wrong, and it agreed with that too. It followed you both times, so the corrected answer isn't really any more trustworthy than the first one.

Funny thing is the first answer wasn't all wrong. Channel imbalance at the bottom of an analog pot is real and measurable, and bumping to high gain to get the knob into a better spot does help on some amps. The bass part was nonsense, and it caved on the good part.

On your last question, the chat versions do remember you now if you switch it on. Getting your correction to fix things for everyone else is a much harder problem, mostly because plenty of corrections people type in are wrong too.

 

[amirm]

Well, I have my answer. The AI will not retain anything that was corrected during my questioning. If asked the same question it will go through the same logic searching and likely give the same incorrect answer.

This is very common in both Gemini and ChatGPT Pro that I use. In the short term, after correction, it will know it. But wait a bit and it forgets again. Good practice calls for repeatedly and frequently prompting it what you want it to know.

 

[amirm]

Above is why you have to know more than AI and only use it to accelerate/multiply your understanding.

 

[sonitus mirus]

Above is why you have to know more than AI and only use it to accelerate/multiply your understanding.

Agreed, multiply is the right word. Like managing a team, it multiplies what you already know, and does nothing for what you don't.

 

[JeffGB]

Above is why you have to know more than AI and only use it to accelerate/multiply your understanding.

Exactly. I'm fairly sure AI is partially responsible for most of the new battery chemistry in China. It can look at vast numbers of combinations in a fraction of the time required by people. Once ideas are presented by AI, the actual testing begins.

 

[nerdstrike]

LLM training costs millions of bucks, and is a one-time effort. It's very difficult to do additive training (such as ingesting a correction into the model). When you correct it, they probably add it to the big pile of issues they might consider in the next iteration.

Ultimately the responses are driven by probability, so the chances are it will be correct sometimes. Frankly, the source material for HiFi is so saturated with audiophilia that you can expect these models to spout florid nonsense at every opportunity.

 

[Holmz]

Today I asked Google a question which elicited a completely incorrect answer. Actually I made a statement, designed to elicit a response. The statement was " The high gain setting of my headphone amp sounds better even at low volumes. Bass is more powerful etc.". In AI's overview answer it said "Better Volume Control: Many volume knobs sound unbalanced or thin at the very bottom of their turn. Using high gain lets you turn the physical knob up higher, where the channel balance and sound quality are often cleaner."

I told it fact check itself and explained how it was wrong, to which it answered "
You are entirely correct, and I appreciate you catching that. I completely reversed the logic on how the volume knob behaves.". Clearly, AI is capable of great improvements in speed for research but you must be able to fact check it's answers. Now, after having gone through this question/answer will Google AI now "know" the correct answer or will it just begin again from scratch if someone else asks?

People like stories and AI has been trained by stories.
It will make it answer sound like a convincing story, and then almost always say, “ You are entirely correct, and I appreciate you catching that.”

I have saved pages of “Conversations with Claude”.

In one of them IT said, “i think you’re right…”
Me: “when did you begin to start ‘thinking’? “

 

[gzost]

I recently decided to use a LLM to help me compile a list of all recordings of complete cycles of Mahler symphonies (I'm a collector). I knew the correct answer to be at least 40, and I had learned of the existence of all of them via the Internet (across years of growing various forums and other sites though) so I had no data available to me that should not have been in the training sets for the models.The first LLM gave me a too short list where every entry at least was a cycle, the supposedly better model a longer one which contained a few non-existent cycles. Neither was remotely close to a correct result. Don't trust these things!

 

[fatoldgit]

That summary box in Google search isn't really the same animal as Gemini or ChatGPT. It runs on the cheapest, fastest model they've got, summarizing whatever pages the search happened to cough up. Ask the same question in Gemini itself and you'll usually get something better with sources you can actually check.
people type in are wrong too.

But that's the danger... Google's AI summary is the AI that most "normal" people will see so by using a "quick and dirty" engine people will see dis-information and as we know, people are generally lazy so they head off into the sunset with "facts" that aren't reality.

99% of any google search I do (I still prefer it to duckduckgo) are deep dive C/C++ programming/Linux queries (being an older retired gentleman, I sometimes can't remember stuff that my younger self would simply because I don't program as much as I did)

Anyway, I do pursue the AI results and in > 90% of cases the answer is wrong or not relevant. For example I always preface my query with say "Linux" but it will return some result for Windows or I might say Ubuntu 24.04 and it will return some BS reddit post from 16 years ago (the 24 means 2024 and not 2008). I think reddit has the lowest quality info on the web but Google AI seems to use it a lot in it's results.

And it will hallucinate answers like suggesting some Linux utility that doesn't exist.

Finally, I can post the same exact query text in and the AI will return a different result set so very inconsistent.

Peter

 

[Timcognito]

Summarizing and regurgitating data isn't really AI. It is the first step towards AI where a decision must be made from the data. That is why the world wide web is one the worst sources of polluted data because of its vastness. Real Al takes a smaller set information supplied from a proprietary source, say, cardiac patient data from a health care provider and use the trained AI model to apply a certain criteria to find possible diagnosis. Most of those planned data centers are for hire operations for those with proprietary data but lack crunching power they are not specific purpose or project built AI delivery factories, but warehouses where others can download and analyze there own projects. Not to say that Microsoft, Google, Meta, and Amazon Web Services (AWS) are not in an arms race to power massive LLMs and generative AI applications. Most are being planned by a relatively new but fast-growing class of customers includes specialized "AI cloud" providers like CoreWeave, Lambda Labs, and Crusoe Energy. Because traditional enterprise servers can't handle high-density AI workloads, these companies purchase immense data center capacity specifically to pack them full of high-powered, power-hungry NVIDIA GPUs. They want to sell to Fortune 500 companies, Health Care conglomerates, Streaming Networks, Government and Defense all who have loads of proprietary data. But a lot of them are real estate ventures that are in search of those with big piles of data and understand what it will take to buy land, build a facility and get permits.


[1, 2]

 

[keithm55]

Summarizing and regurgitating data isn't really AI. It is the first step towards AI where a decision must be made from the data. That is why the world wide web is one the worst sources of polluted data because of its vastness. Real Al takes a smaller set information supplied from a proprietary source, say, cardiac patient data from a health care provider and use the trained AI model to apply a certain criteria to find possible diagnosis. Most of those planned data centers are for hire operations for those with proprietary data but lack crunching power they are not specific purpose or project built AI delivery factories, but warehouses where others can download and analyze there own projects. Not to say that Microsoft, Google, Meta, and Amazon Web Services (AWS) are not in an arms race to power massive LLMs and generative AI applications. Most are being planned by a relatively new but fast-growing class of customers includes specialized "AI cloud" providers like CoreWeave, Lambda Labs, and Crusoe Energy. Because traditional enterprise servers can't handle high-density AI workloads, these companies purchase immense data center capacity specifically to pack them full of high-powered, power-hungry NVIDIA GPUs. They want to sell to Fortune 500 companies, Health Care conglomerates, Streaming Networks, Government and Defense all who have loads of proprietary data. But a lot of them are real estate ventures that are in search of those with big piles of data and understand what it will take to buy land, build a facility and get permits.


[1, 2]

Good points. Additionally, when specialized AI applications process very large, high-resolution images—such as 3T MRI scans covering extensive sections of the body—and support multiple concurrent users, the bottleneck shifts to the bandwidth at which data can be retrieved from storage (where a metric might be GB/sec of bandwidth per GB of capacity). It becomes essential to focus more on the design of the storage system, favoring enterprise SSDs over HDDs, as concurrent usage generates a more random workload that HDDs handle poorly. These higher bandwidth demands also drive the adoption of optical interconnects instead of copper cables to increase bandwidth between chassis within a single rack and even between components within a single server, whereas optical connections were previously used primarily for long-distance communication. 800 Gb/sec data rate interconnect was designed for these kinds of large scale use cases. See: ConnectX-8

 

[ta240]

Someone the other day commented if you had a contractor at your house and they said "We will start by tearing out that wall" and you interrupted them with "Isn't that a load bearing wall" and they responded with "you are correct, we will start by tearing out that other wall" you'd not consider using them but we keep going back to the AI.

I was asking one about what fluids to use in my car and it started saying the transmission was the same as in newer Mazdas and so I'd need special Mazda ATF. The transmission in my car has nothing at all to do with Mazda and their fluid would likely be very bad for it.

I typically ask a couple AIs the same question for comparison. Unfortunately, I'm finding they don't include as many links to the source of their information, which used to be very helpful to tell if it was going off of a single reddit post or one person's youtube video compared to multiple reliable sources.
Instead of just having a warning that the information could be incorrect, it would be nice to have something like a score for how reliable its source is; is it from the manufactures website or a 3rd party 'expert' site or just some random post?

 

[audio_tony]

The high gain setting of my headphone amp sounds better even at low volumes. Bass is more powerful etc.

The answer I got: (ChatGPT - subscriber)

Yes — and that can be a genuine effect, although “high gain sounds better” doesn’t necessarily mean the amplifier itself performs better in high gain.

A few things could explain what you're hearing:

• Volume matching is extremely critical. Even a fraction of a dB louder will commonly be perceived as having better bass, greater detail and more impact. High gain makes tiny movements of the volume control produce larger level changes, so accidental level differences are very easy.
• The gain switch may actually alter the amplifier topology or feedback ratio. Depending on your headphone amp, high and low gain aren't necessarily identical signal paths with only an attenuator changed. Different feedback conditions can produce slightly different output impedance, distortion/noise behaviour or frequency response.
• The volume-control operating point changes. In high gain you'll use a lower position on the volume control. If it's an analogue potentiometer, relay attenuator, DAC-based volume control, etc., performance can differ with position.
• Headphone impedance can interact with the amplifier. If the output impedance changes between gain settings, some headphones can acquire a measurable bass/frequency-response difference.
• There's also the normal equal-loudness effect: even a tiny increase in actual SPL tends to make bass sound disproportionately more substantial.

Given the audio measurement equipment you've been working with, this is actually a rather nice thing to investigate rather than speculate about. I'd feed the amp a fixed tone/sweep, put a suitable dummy load across the headphone output, and match the high- and low-gain output levels very precisely. Then compare frequency response, THD+N/SINAD and output impedance in the two modes.

The particularly interesting measurement would be frequency response into a realistic headphone load, rather than unloaded. If the bass difference you're hearing is electrical, that should expose it very quickly.

If you tell me which headphone amplifier and headphones you're using, I can probably narrow down which explanation is most plausible.

As it happens - a while ago - I built a preamp and it sounded thin at low volumes (I proved this by feeding it into another gain stage to raise the gain) - it transpired that at very low volume settings, the source impedance was high enough to cause issues with the input stage.

So both answers are plausible IMHO.

 

[dpturner]

two days ago (thursday) i asked google "tnf tonight", to see who was playing on thursday night football. it replied that tnf airs on thursday, and wasn't on tonight, because it's sunday. i'm retired, and often don't know what day it is. but i don't know what google's excuse is. maybe it errs on thursday!

 

[ta240]

Well, I have my answer. The AI will not retain anything that was corrected during my questioning. If asked the same question it will go through the same logic searching and likely give the same incorrect answer. It told me it would only change when it's developer updated it's underlying model and training data. Wow, what an incredible waste of energy. No wonder the data centers require so much power and cooling. I guess the long and the short of it is, it will never be accurate because developers will simply add more features and capabilities rather than use selective, verified as correct, data. They are simply using the internet for data, correct or not. I think AI, if used based on physics, medical and other verified data, will be a major game changer but the current design and use is going to be unacceptable.

The is interesting, I had wondered the same thing about it learning from being corrected by users.

Another one I saw was that Google's AI will try to answer in ways where it can guide people to things that will generate advertising revenue for Google.
There is lady that does a series of videos called How to Cook That has a bunch where she debunks silly and sometimes dangerous tiktok videos. One viral video said putting aluminum foil in your blender would sharpen its blades. She asked google AI about it and it said "Yes it does work, here is a video showing it" and pointed to the video on youtube that she was looking into. When she questioned further if it was a false claim it then responded with "Yes, it is likely false, there is a lot of evidence showing it doesn't work" and listed sites that said it didn't. She researched how google handles responses and found that they will, whenever possible, respond with a link to a google related product.

 

[ta240]

two days ago (thursday) i asked google "tnf tonight", to see who was playing on thursday night football. it replied that tnf airs on thursday, and wasn't on tonight, because it's sunday. i'm retired, and often don't know what day it is. but i don't know what google's excuse is. maybe it errs on thursday!

I was using AI to study for an employment test and I chose answer #3. It responded that my choice was incorrect and that the correct answer was #2, however the text from its answer matched #3 not #2. I got the classic "You are completely correct, the right answer is #3" response when I corrected it.

At work I had it generate a list of items and it said it found 10 and listed them all out with numbers next to each. Except after the 10th one there was another listed with a dash next to it. After a moment it said "Woops, there are actually 11". At least it caught that by itself.

 

[RexrothPigeon]

Someone the other day commented if you had a contractor at your house and they said "We will start by tearing out that wall" and you interrupted them with "Isn't that a load bearing wall" and they responded with "you are correct, we will start by tearing out that other wall" you'd not consider using them but we keep going back to the AI.

If that contractor worked for free, they would be in very high demand despite that shortcoming