Just ask ChatGPT what it thinks for some non-existing product and it will start hallucinating.
This is a known issue of LLMs and DL in general as their reasoning is a black box for scientists.
If scientists made AI, then it wouldn’t be an issue for AI to say “I don’t know”.
But capitalists are making it, and the last thing you want is it to tell an investor “I don’t know”. So you tell it to make up bullshit instead, and hope the investor believes it.
It’s a terrible fucking way to go about things, but this is America…
It’s got nothing to do with capitalism. It’s fundamentally a matter of people using it for things it’s not actually good at, because ultimately it’s just statistics. The words generated are based on a probability distribution derived from its (huge) training dataset. It has no understanding or knowledge. It’s mimicry.
It’s why it’s incredibly stupid to try using it for the things people are trying to use it for, like as a source of information. It’s a model of language, yet people act like it has actual insight or understanding.
Imagine searching your computer for a PDF named “W2.2026”…
Would you rather the computer tell you it’s not in the database? Or would you prefer a random PDF displayed with the title “W2.2026”?
This isn’t a new problem.
You’re getting hung up on “know” instead “has relevant information in it’s database and can access it”.
But besides all that and the other things you got wrong:
It’s still about capitalism for the reasons I just said
You do not understand how these things actually work. I mean, fair enough, most people don’t. But it’s a bit foolhardy to propose changes to how something works without understanding how it works now.
There is no “database”. That’s a fundamental misunderstanding of the technology. It is entirely impossible to query a model to determine if something is “present” or not (the question doesn’t even make sense in that context).
A model is, to greatly simplify things, a function (like in math) that will compute a response based on the input given. What this computation does is entirely opaque (including to the creators). It’s what we we call a “black box”. In order to create said function, we start from a completely random mapping of inputs to outputs (we’ll call them weights from now on) as well as training data, iteratively feed training data to this function and measure how close its output is to what we expect, adjusting the weights (which are just numbers) based on how close it is. This is a gross simplification of the complexity involved (and doesn’t even touch on the structure of the model’s network itself), but it should give you a good idea.
It’s applied statistics: we’re effectively creating a probability distribution over natural language itself, where we predict the next word based on how frequently we’ve seen words in a particular arrangement. This is old technology (dates back to the 90s) that has hit the mainstream due to increases in computing power (training models is very computationally expensive) and massive increases in the size of dataset used in training.
Source: senior software engineer with a computer science degree and multiple graduate-level courses on natural language processing and deep learning
Btw, I have serious issues with both capitalism itself and machine learning as it is applied by corporations, so don’t take what I’m saying to mean that I’m in any way an apologist for them. But it’s important to direct our criticisms of the system as precisely as possible.
You don’t seem to understand. There is no database.
It is made by scientists. The problem is that said scientists are paid by investors mostly, or by grants that come from investors.
Just put this into GPT 4.
What’s your view of the fizbang Raspberry blasters?
Gpt ‘I’m not familiar with “fizbang Raspberry blasters.” Could you provide more details or clarify what they are?’
It’s a drink making machine from china
Gpt ‘I don’t have any specific information on the “fizbang Raspberry blasters” drink making machine. If it’s a new or niche product, details might be limited online.’
So, in this instance is didn’t hallucinate, i tried a few more made up things and it’s consistent in saying it doesn’t know of these.
Explanations?
And by the time the system can actually research the facts, the internet is so full of LLM generated nonsense neither human or AI can verify the data.
No surprise, and this is going to happen to everybody who uses neural net models for production. You just don’t know where your data is, and therefore it is unbelievably hard to change data.
So, if you have legal obligations to know it, or to delete some data, then you are deep in the mud.
I think of ChatGPT as a “text generator”, similar to how Dall-E is an “image generator”.
If I were openai, I would post a fictitious person disclaimer at the bottom of the page and hold the user responsible for what the model does. Nobody holds Adobe responsible when someone uses Photoshop.I would post a fictitious person disclaimer
… or you could read the GDPR and learn that such excuses are void.
You just wasted a lot of my time. What did I do to deserve this?
… said the sparrow and flew out of the library.
There we go. Now that people have calmed their proverbial tits about these thinking machines, we can start talking maturely about the strengths and limitation of the LLM implementations and find their niche in our tools arsenal.
I can’t wait until the AI bubble finally pops.
There’s definitely a niche for it, more so than for other fruitless hypes like blockchain or IoT. We really need to be able to offload tasks which need autonomous decisions of simple to average complexity to machines. We can’t continuously scale up the population to handle those. But LLMs aren’t the answer to that, unfortunately. They’re just party tricks if the current limitations cannot be overcome.
Asking chatgpt for information is like asking for accurate reports from bards and minstrels. Sure, sometimes it fits, but most of it is random stuff stitched together to sound good.
The technology has to follow the legal requirements, not the other way around.
That should be obvious to everyone that’s not an evangelist.
