If you put ten minutes into writing a prompt, and Mallory gives a completely off-the-rails, useless answer, and you lose ten minutes, well, that’s just what using a computer is like sometimes. Mallory malfunctioned, or hallucinated, but it does that sometimes, everybody knows that. You only wasted ten minutes. It’s fine. Not a big deal. Let’s try it a few more times. Just ten more minutes. It’ll probably work this time.
If you put ten minutes into writing a prompt, and it completes a task that would have otherwise taken you 4 hours, that feels amazing. Like the computer is magic! An absolute endorphin rush.
Very memorable. When it happens, it feels like P=1.
This feels like a metaphor that doesn’t quite match. Or maybe inventing a kind of person to get mad at. Not to make any claims that LLMs are actually really amazing and always get things right, but I do feel like there are about three main camps that people fall into when using LLMs, and only one of them is from the perspective of someone who uses it like a slot machine.
For one, a slot machine is designed to fail most of the time, and along with a number of other dark methods, use that to make the times it succeeds more memorable and desirable, make you keep trying to get that one time that it does finally pay you back.
LLMS and AI, on the other hand, are marketed as being reliable and easy. (Which they are not, but I’ll get into that.) In order to sell that, most of the time it does have to be correct on things. The goal is to make it seem mundane, a part of life, a standard tool that you can use most anywhere. If you ask an LLM how to get the top off of a jar and it tells you to break it with a hammer or stuff it up your ass, you’re not going to trust it anymore. So for most questions, it has to at least seem reasonable and correct, most of the time.
Anyway, the first camp is the one who views it like a slot machine, and those are the people who… well, don’t use it. Which, if you think of it like a slot machine, makes sense. And it does seem silly for other people to be cranking the lever over and over again when obviously it’s just a worthless machine.
The vast majority of people who do use AI, I expect, use it like a search engine that can talk to them. And to be frank, I don’t think that’s too far off. Not because I think it’s particularly accurate, but because I think generally search engines (especially nowadays) aren’t the most reliable anyway. The people who use LLMs really casually like this are the same people who type a question into Google, scan the summaries of the first two articles, and then decide what that means from there. Which, lest we forget, are articles written by humans and often the top ones are the most popular or most paid-for places, neither of which means the articles are any more accurate. We had to make concerted efforts to stop people from mixing bleach and ammonia well before AI became a thing. But either way, it’s not treated like a slot machine where you get a rush when it’s correct over all the times it’s not, it’s just mundane.
The rest of the people who use AI are also usually the people who understand how it works, and thus are the people most in tune with it’s flaws. And, just like with a search engine, if you want good data you can’t just plug in a question and take the first thing you get. You have to work with the machine and it’s quirks to find your answer.
This is where the similarities between an LLM and a search engine end, though. For a search engine, you need to put in your keywords, filter out the keywords you don’t want to see, narrow it down with quotation marks, iterate on the prompt until you finally start seeing the articles you want to see, and then you search and cross-reference those articles until you feel reasonably confident that you’ve found the correct answer to your question.
LLMS, you can’t really just put in the prompt in different ways, because it’s not going to respond the same way a search engine does. At the broadest scale, they can do the same thing–find you a quick and dirty answer for a question that you don’t want to bother fact checking. At the precise scale, they go the exact opposite ways.
A search engine works better the more you know about a subject, because you can put in more keywords and more filters to describe what you’re looking for.
AI works better the less you know about a subject, and also in conjunction with a search engine. For example, when trying to understand how to code something in a game the other day (a hobby I have recently gotten into, and thus don’t know a lot about), I wanted to create a weighted random set. Searching the keywords I could think of wasn’t bringing up what I wanted, because I didn’t know the correct terminology or even how to find it. Likely I would have gotten there eventually by following one link after another, but instead I tried out an AI, asking it what kind of code I was looking for in order to create this random weighted set. It was able to take my layman speech and find the professional terminology I’d been looking for, which I was then able to use in the search engine to find exactly the commands I’d been searching for.
Now, all of this to say that the way companies have been marketing it and the way it’s been shoved into our lives at all angles is wrong and annoying and often detrimental and even dangerous. But that’s not a problem with the technology. If a salesman told me that a kitchen knife was the best tool for shaving, cutting my hair, trimming my nails, starting my car, and cutting vegetables, that’s not the fault of the knife for being bad at everything but the last one, it’s the fault of the salesman for false marketing and dangerous practices. I think that should be stopped and that people should know how to use the technology we have, and that’s been the case long before AI happened. Hell, Google has been purposefully making their search engine worse for the sake of serving up ads. I don’t know the last time I naturally saw a Wikipedia article without specifically searching for “Wikipedia”.
In all likelihood, it wouldn’t be hard to just improve a search engine to have ‘professional’ and ‘layman’ options in order to do the same thing, without needing an AI to make small talk at you at the same time. But they’re not going to do that, because the faster you find what you’re looking for, the less ads you see.
