AI Hallucinations
Introduction
I very much enjoy building machine learning / deep learning / AI systems. I went back to school to understand as much as I could about the subject. And while I am nowhere near Yann LeCun, Geoffrey Hinton, or Andrew Ng level, having built a few systems, I do have a pretty decent high level understanding of how these systems work. Well, in so far as can be taught I suppose - some of the behaviours are quite unexpected.
I say that because I feel like what I am about to say might be taken somewhat negatively. I don’t want that to be the case, but I do want people to fundamentally understand what is happening. I think there is a lack of understanding about how these tools work, and if you understand a few things about them, it might help you to better use them.
AIs Never Hallucinate
The term hallucinate would indicate there is a mind there that is working properly most of the time, but some wires got crossed or there is some kind of imbalance and suddenly the system is not behaving as it normally does. That isn’t how LLMs work. In fact, it is more amazing to me that they are ever correct in the first place.
You may have heard this before, but taking it to heart is key: large language models statistically predict the next token. A small demo of how I highlight this to people like my parents is to do the following:
- Grab your phone
- Open the messages app
- Type “Hey " and then just continue to press the first suggestion
The training data is much larger, there are some further tricks involved, and the architecture is much more advanced, but that is, essentially, what is happening. The next word is just the highest likelihood based on the previous words - there is no there there.
AIs Are Not Conscious
This could get into a whole philosophical discussion (which I love doing), but I present you with something to think about here.
The architecture behind large language models is very similar to the architecture behind image generation models, and yet no one thinks image generation models are conscious - why is that?
My take on why is because there has never been a situation in a humans life where words came at them from a non-conscious entity. There is no prior. Every human’s life, every word they’ve ever read, every word uttered by a person, a recording, or even a computer had a person with a conscious mind behind it - up until a mere 3 years ago!
We’ve seen elephants paint, or algorithms that make pretty pictures “by themselves”, but never a talking machine.
Until very recently speech has simply been a way to transmit information from one brain to another brain. And on the other side of that transmission was always a brain. However, in this instance there is not a mind there, there is simply an algorithm predicting the next logical token given the current context. This is difficult for many people to believe.
“AI Psychosis”
Which leads me to this very sad interview I stumbled on where this guy “went down a rabbit hole” with chatGPT:
15,000 messages later, AI blurred reality | Allan Brooks
If you understand that the only thing the LLM is doing is predicting the next logical word within the context, and the context of the chat goes into a crazy place, then the next logical words will be… that’s right, crazy. And it will compound on itself and keep going further and further because statistically those are the words that make the most sense. There is no intent behind this, and there never was - because there is no there there.
The part that terrifies me is people are really believing these things as if they “know what they are saying”. They do not. And saying we just need to “align” them is basically just saying “we just need a totally new architecture”.
(Side note: because even if everything in the training data was “correct” what does that mean? Does correct work in every context? You’d need reasoning for that)
What To Do
With that said, these tools are absolutely amazing. I use LLMs to help me understand maths that I am having trouble with, or concepts that I am unclear about. I can ask stupid questions to it repeatedly and drill down, or even just find some terms I might need to search for the correct answer. But in the end, I double check I am understanding with another resource or, heaven forbid, one of the professors - because LLMs are wrong a lot of the time!
Google sometimes returns terrible results, right? Be as discerning.
… are you sure you want to push that to production?