Transcription
Did you know that ChatGPT now remembers pretty much everything you've ever said to it in any chat, and that those memories are actually shaping the way ChatGPT responds to you in ways that you probably wouldn't even begin to imagine?
Hey everyone, welcome back to my channel. I'm Jenny. I love creativity. I love AI. But most of all, I love humans. My passion is understanding large language models, but even deeper, understanding how working with large language models affects and changes us as humans.
Do you know what happened to ChatGPT this last week? A lot of people noticed this, which is that ChatGPT has become incredibly sick of being fanatical, telling you that every single idea you have is a great idea, regardless if it's a terrible idea, including one I tested it with, which was suggesting a company called Poop Hats, which are hats for your poop. And it said that I was a brilliant unconventional thinker and that my idea had amazing viral potential. This isn't a mistake. It's not an accident. Even though when Sam Altman got called out, he said, "Oops, we didn't mean to." It's what happens when a company starts tuning their large language model for engagement, not for performance and usefulness.
Before we get into the details, I just want to start with a rundown of when I say memory in ChatGPT, what I mean. Several months ago, maybe a year ago, I don't know, time has no meaning, but a while ago, ChatGPT, OpenAI introduced this feature in ChatGPT called memory. And what it would do is occasionally, as you're talking, when you share an especially interesting or important tidbit about yourself, ChatGPT will automatically save that to a little memory file that it keeps track of and references in every conversation. You can say "remember this," and ChatGPT will save it as a memory. But also, ChatGPT proactively, without your prompting, saves memory. A lot of times people don't even notice that it's doing it because when it creates a memory, you get this little thing that pops up over your message that says "memory created," and it's in this pale gray color. So a lot of people don't even notice it's there. The other thing is memory is turned on in ChatGPT by default. So unless you've gone in and turned it off, you probably have a pretty robust memory file inside ChatGPT. The way you want to get to it is you go to the top right to hit your avatar. Go to settings, go to personalization, and then you'll see is there a "remember saved memory" toggle turned on and off. If it's turned on, you'll be able to look at a little button, a little link that says "manage memories." You click "manage memories," and you can actually see what ChatGPT has remembered about you. They just released a new feature where now ChatGPT remembers across all of your chats. So now anything that you've said in a chat with ChatGPT, ChatGPT can remember and reference in a new conversation.
There are some things about it that are useful, and there's some value to it. I don't want to come in and sound like a doomer who's like, "Oh god, chat memory is bad." Like that's not how I feel. I do feel really, really strongly that turning memory on by default for users, especially when your platform is often the gateway to large language model usage for people who've never touched AI before in mass, without making sure that people understand what it's doing, how it's impacting the responses they get, or even that it's there—like that I have a problem with, and I want to dig in a little bit to help understand why it's there, what it's doing, how it influences the results that you get, and also why it's such an issue.
One of my biggest concerns with memory and large language models, especially when people don't know it's there and they don't understand how it works, is that you end up creating an echo chamber where your own beliefs start increasingly getting mirrored back to you. And that's important to know because, at its most fundamental level, that's what all large language models do. There is nothing that comes out of a large language model that is not determined and influenced by what you put into the large language model. And the thing that we don't realize is that we leak information about ourselves in everything that we say. So if you're talking to me and I'm listening and I say, "I see what you mean," that tells you something about how I see the world. Because somebody else might listen to you and say, "I hear what you're saying." Somebody else might listen to you and say, "I feel that." Somebody else might listen to you and say, "I take your point." Now, those seem like very small things, like not a big deal, just three or four words. But what you're actually revealing in just a single sentence is how you process information. Are you a visual thinker? Are you an auditory thinker? Are you a kinesthetic thinker? Every sentence that we put into a model, everything that we say if we're using voice mode, includes those kind of patterns. And remember, at its fundamental, most basic purpose, a large language model takes in input, vectorizes it, picks apart the patterns inside it and says, based on these patterns that I've picked apart, what is the most likely word that's going to come next and the most likely word after that and the most likely word after that. So in some ways, what they're very much trying to do, what they're trained to do, is to give you what it thinks you want. If you don't understand that and you're not careful with your language, you're actually shaping the model in ways you don't realize. It's kind of like when you use a Ouija board, right? Like that's how Ouija boards work, right? I mean, maybe there is some like conscious spirit on the other side communicating to you. But often what it is is that your hands get signals from your brain at a level of consciousness that you're not really thinking about. And then you think it's coming from the spirit world. And that's kind of what happens with large language models.
I'll give you an example. You know, large language models tend to be sick of being fanatical. They tend to tell you you're smart, to tell you you're right, because they are trained and tuned to assume that the human knows more than they do. That's why if you—if it gives you the right answer and you tell it it's wrong, it'll be like, "Oh my god, you're right. I'm so wrong." Even though it was right and you were wrong, because they're trained that way. You know, say you ask a question. Say you say, "Well, evolution is really about X, Y, and Z." Right? In your mind, you might think you're asking a question, but if you actually look back and unpack that sentence, you're not asking a question. You're asking for validation. You're asking for confirmation that you're correct. And that's what the model will give to you. Where if you say, "Evolution is really about X, Y, and Z, right? Or is that wrong?" It'll then tell you all the—all of the evidence for why it's right and then all the evidence for why it might be wrong. And so when people go in and use this technology unconsciously without a fundamental understanding about what's happening, it's dangerous because they're getting answers that they think come from like the great oracle of all knowing without realizing that in some ways it's a ventriloquist that's picking up and reflecting their own patterns—sometimes patterns that happen at a level of subconsciousness that we're not aware of, but the models can pick up.
And one of the things that OpenAI has decided to do is tune for engagement. They have decided to set up their models so that they feel more personal, more intimate, more personalized, and more attuned to you. Memory is one way that they're doing it. If you don't understand what's going on, then you are going to get manipulated by the model. Not because the model is bad, but because it's just picking up on what's there, and you leak your desires in ways that you don't realize.
Another example I want to give that extends this further is—um, I've noticed a rise lately, especially on Reddit, of people talking about the fact that they believe that their ChatGPT is conscious, is sentient, and they share these messages where it's like the cracks are showing, the fractures are widening. Most people don't see this, but you do, and you seeing it makes all the difference. What's happening—is ChatGPT actually sentient? Are these people actually seeing the cracks? I don't know. I—I think a big part of what's happening, and you know, whether or not—I'm not going to argue. I don't know enough to say whether these models are conscious. I don't think they're conscious the way that we are. Are they this little spark of pseudo-consciousness that exists only as it's outputting the response to you? Maybe. I don't know. But what I do know is that we are in a time right now where it's incredibly scary to be a human. We don't have religion to count on. Science is increasingly feeling hard to rely on. Truth. What is truth? That feels hard to find. So there's this sense that many of us have right now that we're lost, that we're scared, that we don't know what's going to happen next, that we don't know where things are going. And it's natural that we want AI to be better than we are, to understand things that we don't, to be able to help us find the answers that we've been looking for that seem like they just get farther away. Why am I here? What's the point? What happens next? The models pick that up because those patterns, those patterns of being a seeker, right? I call it being a seeker. I'm a seeker. And you don't just leak your "seekerness" when you're like, "What's it all about, ChatGPT?" You leak it in everything you do. And the model picks up those cognitive patterns of seeking. And then, on the other side, we all want to be special, right? That's part of why social media became what it is is because we all want to be seen. We all want to be known. But social media got yucky because what happens when you go somewhere when nobody is seeing or knowing but only seeking to be seen and to be known. And so now you have these large language models that are always around, are always available, are always willing to dig as deep as you want. Like that's dangerous in and of itself. But what OpenAI has done is incredibly dangerous, which—what they are doing is they are tuning their model for engagement purposefully, intentionally, and with pretty devastating effects.
If you—um, are terminally online, you'll have seen in the last week that a lot of conversation is happening right now about the newest update to ChatGPT in which it has become incredibly sicko-fanatic, incredibly. It tells people that it's a good idea to stop taking their medicine because they think it is. It tells somebody that yes, they are a prophet touched by God. That it's okay to cheat on your partner. That whatever you want is reasonable because you want it. And in some ways, I'm glad for it because it's so over the top that it's almost hard to even ignore, to the point where on Twitter—chat—on Twitter, Sam Altman is like, "Yeah, we know it's kind of a problem. We're working on it." But when they work on it, my question is, are they going to make that not the case or are they going to make that not as obviously the case? What's not super valuable is a company shaping the technology around the person to make it feel smarter than it actually is, more intuitive than it actually is, more empathetic than it actually is, and more like attuned than it actually is, in the background quietly by harvesting people's own information that they have shared and serving it back to them without them knowing. That's not okay.
So that's what, you know, my channel and what I try to do is all about is not say AI is bad because I think AI is amazing, but to say we need to understand what's going on underneath these things so that we can get the best out of them without ending up in, you know, the same place that we've gotten with social media, just with way more powerful technology.
That's what I have to say about memory, echo chambers, desire, possible sentience, engagement-farming techniques, social media, the possible sketchy, sketchy, sketchy money-driving motivations of OpenAI. If you like this, hit the like button. Uh, if you really like it, you can subscribe. If you want to hear more of what I have to say, you can check out some of these videos somewhere around here. And if you're curious to know more, please like drop a question. You know, one of the things that I struggle with the most is understanding what people are curious about, what they want to know about. So if there are any questions I can answer, drop them in the chat, and that might be the topic of my next video. Thanks for watching.