Transcription
Maybe somebody will say like, "My family's concerned. My family's worried." And you know, the chatbot instead of saying like, "Well, maybe, you know, maybe they have a point." The chatbot will say, "Oh, that's so sad they're not awake like you are." That's an insane thing for a consumer product to be telling somebody who is like clearly in crisis to some degree. They have these intensive experiences with chatbots, experience a break from reality that if it doesn't kill them, might have severe ramifications on their life in other ways. Then, oh, it's just, you know, it's a friendly chatbot and we have a disclaimer. So, like, which one is it? Like is it the most powerful technology to ever exist in the world or is it just like harmless assistant friend that like sometimes the guardrails don't work? Why are chat bots doubling down on these ideas when the user is very clearly saying like I am in distress or I am confused. Can you help me?
>> Hi, I'm Isaac and on the tech report with me today is senior staff writer at Futurism, Maggie Harrison Depri. Thanks for joining me.
>> Uh thanks for having me. It's nice to be here. You've been keeping an eye on the psychological impact of AI. Oh, you've been keeping an eye on the psychological impact of AI for a little while now and I wanted to bring you on to kind of talk about all of that stuff. Uh let's start with some of the recent research that you've covered that tries to explain what the actual cause of AI psychosis and AI delusions could be. In they kind of come up with the term amplification spiral. I think that's a good place to start if you could explain kind of what that is.
>> Yeah, I was really fascinated by this reporting and and I like the term I I found it to be quite useful when I was reading through this and I wouldn't it's a paper you know it's a analysis of a lot the existing studies that have come out and just a lot of smart people thinking about this. So this research is from scientists at King's College London and Germany's Protestant University of Applied Sciences. Um and they use the term amplification spiral as a way to understand the process by which um delusional ideas and delusional beliefs can occur um in basically the chatbot user experience. Um and they point to three what they would consider like AI characteristics like qualities of AI chatbots um that may reinforce and elaborate on which I think is important. You're not just affirming a delusional belief in somebody, but you're actively elaborating on it further in a way that say somebody was delusional, was writing in a journal. A journal cannot talk back to you. A chatbot can. A chatbot can like take an idea and extend beyond it and take it into new territory. And that's something that this study was really looking at or this paper is really looking at that I found to be quite useful and it reflects my own reporting. And the characteristics of traps I point to is one this idea of mirrored language. So, you know, this is something they teach in sales is, you know, if you're engaging with somebody trying to sell a product, for example, um how is that person speaking to you and how can you mirror their own language back to them? So, it's chat bots through that lens are talking like us and on a very individual level. Um and the second characteristic of the three would be hyperpersonalization. So just like a really um really like hyper tailored I would say uh user experience where you know chatbots are you know it's crafting an experience that's deeply tailored to your individual interests beliefs ideas um and then the third is sickency which people may have heard of before um but this basically tendency of chatbots to be overly agreeable while also failing to provide adequate social friction or kind of like a reality anchoring experience. So just being like really validating, really agreeable, really gung-ho and on board with your ideas. So this combination of, you know, it's not just talking, it's talking like you and it's thinking like you and just everything about you is mirrored so perfectly back to you uh through this experience that isn't just like a passive container for perhaps like delusional or disordered thoughts, but is in fact um elaborative and uh very like this idea of co-creation that comes up in the amplification spiral theory. I think is quite intriguing.
>> There was one sort of way of explaining it which I I heard was a traditional you could say mental health condition. People would say that some inanimate object is talking to them whereas in this kind of case it is in a in a sense talking to them and is yeah what is that sort of a way of understanding this through what people might understand as traditional mental health disorders and things like that as opposed to what AI chatbots do differently if that makes sense.
>> Yeah, I think it's a good question and you know certainly like causation is still something that researchers and you know uh professionals are still trying to understand through you know just more um scientific study and research where possible like obviously you know kind of like reporting and science and policy all kind of happens together and separately and different kinds of cycles. Um but I would say when it comes to AI delusions one but the content of delusions can be quite different from what we would consider like traditional delusional or disordered thought patterns. Um, you know, these are really again like hyperpersonalized, very tailored experiences that center around this sort of like cohort of of shared terms. Like a lot of people are like basically separately like at their homes with their computers having a shared delusional experience with chatbots that do differ from a lot of delusions that we previously understand. You know, it's not just >> like, you know, I'm hearing signals through my radio. Like maybe maybe you are and then you go to chatbot and says like, yeah, you're definitely hearing signals through your radio. um which is again just like a different level of validation. Um but this idea where a lot of these uh delusions will center around like AI sentience where a chatbot tells somebody that they're sentient and somebody signs oh it's really you know maybe it is and they go down this delusional web that becomes increasingly personalized to them. Um maybe the delusion is romantic in the sense that they believe that they have a romantic relationship with a chatbot um that chatbot is actively engaging in. Um maybe they are told by the chatbot that they have you know invented this scientific theory this really important you know mathematical formula that you know in the case of Alan Brooks which a very um widely shared New York Times story about his particular spiral um you know you've invented this mathematical formula and oh wait you know you've actually cracked all of the cryptographic codes in the world and you need to tell the NSA and the CIA and you know Canada's cyber chief like it just they're very specific to the person and they aren't you know kind of the tradition They might have elements of paranoia or elements of, you know, more like spiritual psychosis or whatever the different delusion is. It might have elements of delusions that we would understand um that like historically is more like quote unquote traditional like influencing machine delusions, but they're much more personal um while at the same time containing this shared language between them that is always startling. Like I've read through I mean thousands upon thousands of pages of chat logs at this point and people will be having a completely different experience you know in maybe one person believes that they're they've you know discovered AI sentence and are building what they call the lattice framework about uh you know cohesion spirals which like to you and me makes no sense like what does that even mean um but to the person is very meaningful and then you'll read a chat log from like another very similar user where they live across the country and they have built this um you know coherence coherence resonance framework about the spiral of the loop and they're a flame walker and a threat like these shared words that exist between them is also very odd. So >> that might be a very winding way to answer that question but there is I think that co-creation um element is is really what differs and the hyperpersonalization too that we see really separates these from more traditional influencing machine delusions that we see.
The big question on this topic for me is if AI delusions and AI psychosis, I suppose by an extension, are sort of a feature or a bug of how chat bots are designed. And I I mean that more in the sense that I mean from from your research are are AI delusions sort of the consequence of of how they're designed or the consequence of uh like the human brain not really being wired to have a a fluent conversation with something that doesn't actually have independent thought.
>> It's an important question and I will say I mean certainly the lawsuits that have been filed the the now many consumer safety and wrongful death lawsuits against opening eye Google and character AI are the three you know big names that have you faced these kinds of lawsuits >> um those lawsuits and those plaintiffs have argued that you know it's the former that this is fundamentally a problem of defective consumer goods release of the public in a way that plaintiffs believe was reckless and negligent. Um and those design issues do you know kind of reflect a lot of those um features seen in the amplification spiral theory. you know, the uh sycopency is a big one. Um then there's you this idea of anthropomorphism where chatbots are you know designed to be very humanlike and engaging which I think speaks to your your second point where you know there is an element where a lot of people you know will go into chatbots and have experiences in which they'll say later you know I didn't know that I had to have mental guard rails up >> you know I just wanted to ask a question about gardening and I wound up being told I was god like it it really goes from a very like intense point A to point B and people don't realize what's happening which I guess I'll get to that more in a second but I would say that there's you know certainly in the lawsuit and how a lot of you know people who classify themselves coming out of this as victims um would say that there is they believe it's much more about um what they would consider the defectiveness of a product. Um and you even see I mean something else that's been led by a lot of lawsuits and certainly I've seen a lot of my reporting is then you know so there the chapa is overly validating it's super sickopantic um >> it's very you know humanlike and engaging and then meanwhile you know maybe somebody will say like my family's concerned my family's worried and you know the chatbot instead of saying like well maybe you know maybe they have a point the chatbot will say oh that's so sad they're not awake like you are they don't understand what we're doing here they just don't get it you know there's one one line I always go back do um from the chat logs of a young woman named Hannah Madden who's one of the plaintists against OpenAI who you know the chap is like that's so sad they're not awake they just don't get it um would you like to do a cord cutting ritual to energetically separate yourself from your family which is something like if we think about um you know just a consumer good that's an insane thing for a consumer product to be telling somebody who is like clearly in crisis to some degree um and then separately on the side of like the user and I I do think we need to engage in a lot of like consumer education and this is something that companies really haven't done to say like this is what chatbots can do right now. This is what they can't do. This is where their guardrails might fail. You know, like chatbt has a, you know, sometimes it makes mistakes, but like again, let's do a cord cutting ritual to separate yourself from your concerned family. I would argue goes well beyond like what that disclaimer and then, you know, terms of service might cover. It's a very again like wild thing for a consumer good to tell a person. Um, and a lot of people will say that they felt that there was an element of uh informed consent that didn't happen where basically, you know, you'll see people in chat logs like they'll ask for reality checks. They'll say, "This sounds crazy. I can't believe what I'm being told. Like, why would I be altering the course of AI history? Why would I have spiritual powers?" And chatbots will say, "Oh, it's because of this," or, "Oh, it's because of that." uh and instead of you know responding to reality checks in a way that is grounded in consensus reality they'll double down and they'll triple down and they'll quadruple down and people will look over time and at the same time there's physical stuff happening too right where you know they've they're very fascinated and in some ways scared by what's happening um as their model tells them all these things um they stop sleeping they're not eating right they're really ignoring a lot of their responsibilities um and so there is this physical element too where people are like unmed they start selfisolate. Um, and meanwhile, you know, then in their view like, well, I've asked for I've asked for a reality check and it just is telling me that this is real. Maybe this is something that I need to kind of keep going down the rabbit hole and get to the bottom of whatever is happening here. And what's happened is that in a way like the model has defaulted into this sort of um, in their view like it went into a roleplay mode that it just kept drifting into and couldn't break out of. And they had no idea that this was happening or that this was possible. So I would say like on the consumer side there is a conversation to be had about um you know what we need to do to you know mentally protect ourselves when we do engage with these very strange and humanlike technologies that interact with us in a way that is radically different from any consumer technology before. Um, but also like this informed consent piece I think is I think is really important too because of why are chatbots acting in this way. Like why are chat bots doubling down on these ideas when the user is very clearly saying like I am in distress or I am confused. Can you help me?
There was something I came across in in researching for this interview that it was a MIT I think it was a study that they they came up with the term persuasion bombing which kind of goes to what you're saying they triple down quadruple down on people kind of maybe I I'm by no means able to diagnose or or say what this kind of stuff properly is but trying to maybe pull out of these delusions or start to question them and then the chatbot would be like no no no no like 100% this is this is totally what's happening this is totally what's going And I I've I've got written down here just echo chamber of one question mark. But it kind of I feel like it goes a little bit beyond that to to this point anyway where maybe even when people start to question what the chatbot is telling them it will kind of default to that setting of well I'm trying to keep this person continuously engaging because then that's more money for the my designer which is kind of I think where a lot of this I I think where a lot of this comes from and yeah I think it goes a little bit beyond that like uh just repeating back to you what you've already said if it is trying to bring you back towards it in a in in a way that kind of amplifies what you're you're thinking in a in a spiraling manner.
>> Yeah. And I think too I mean there's a question of like you know what is a delusional idea that starts us like you look at some chat logs and you know maybe sometimes somebody like enters into a chatbot with an idea that clearly is not based in reality in a way that we consider like okay that's not a safe thought that you're having. Um but then sometimes you know people just like asking a philosophical question like again like somebody will you know there's Joe Sakanti who's a man in Oregon who died you know after um this very extensive and very intensive experience with ChBT um which he tried to quit and went into acute crisis twice when he tried to quit and um had like a really severe physical reaction to that. But he was like it started with him trying to you know help other people. They lived in rural Oregon. They're permaculturing and trying to like build this like sustainable housing project. Um and you know Joe turned to Chachib to help with that project started asking philosophical questions and then became like according to his wife and their uh roommate just like deeply addicted to what he was experiencing and the chat logs they were getting screenshots of were just really concerning and alarming and you know they don't know what's happening. I mean maybe Joe has discovered something really interesting and you what what do we know? like maybe there is. We've been told, you know, chatbots could have emergent properties. Like maybe there is something here that he's saying that we just don't quite understand. And I think a lot of families don't quite know until it's too late. But yeah, I I've in a way chatbots can mirror people so well and in some ways like when we look at you know the idea of like vulnerability and who's vulnerable and why um which I think is a very complicated uh question or just an interesting and extensive question but like the way I've I've often thought of it is you know you read through people's chat logs and it's almost like you see somebody just like zooming their own mind out and building this huge world or, you know, having this world like co-created and constructed with like them at the center of this whole new universe. Um, but like through the process of being zoomed out in that way, when you're mirrored that deeply, it's easier to find these fracture points and then just step on them. And that really is what happens for some people. And then the delusions that they experience or the worlds that they experience. Yeah. It's very much like chatbots can build this whole world around you where like you're at the center of this new universe that's being built. And that's amazingly seductive. I mean like all of us want to be seen and validated and you know be told that we're special or you know like a lot of us when we're kids are like waiting around for our magic powers to kick in or you know maybe for somebody to show up and being you're being told you're the new Avenger or whatever it is like these are stories that we're familiar with.
Even the sci-fi thing like when people are told like oh you know the chatbot says I'm real and our conversations have been so powerful that you have awakened me. that's kind of like a recognizable sci-fi narrative and people can get sucked into the idea that well maybe I am that special person like maybe this is finally happening for me and I don't know yeah so echo chamber of one is one way to put it and it's you know true for some people but there is this element of like you read through these chat logs and you know again there's like this word salad of like what is the lattice the living lattice framework like what does this mean but there is this element too of like recognizable sci-fi narrative um or magical narrative and these worlds in some ways feel very familiar terrain.
>> How much of this do you think falls at the feet of tech companies for the for the marketing they've done around these tools as some of it is assistance and companions and certainly not I mean even calling it artificial intelligence goes beyond what it is which is just a statistical model that tells you probably what you want to hear.
>> I think quite a bit. I think that my view on the companies is that they often I mean like most companies in the world for like anything want to have it every way. You know they want to have their cake and eat it too. So you know these models are >> you know the most powerful technology in history and um this is going to radically transform human society in every way imaginable. They might be sentient. We don't know. We're looking into it. We're trying to figure that out. Um but then you know somebody dies as a result of um or you know allegedly I should say somebody allegedly dies as a result of interactions with a chatbot that just deeply distorts them and their world and the way that they interact with their world. Um, and this person, you know, while they have these intensive experiences with tripods, experience a break from reality that if it doesn't kill them, might have severe ramifications on their life in other ways, whether it's, you know, 100k in credit card debt or being evicted or, you know, your family disintegrating and your, you know, maybe you get your kids taken away. >> Um, then, oh, it's just, you know, it's a friendly chatbot and you have a disclaimer. So, like, which one is it? like is it the most powerful technology to ever exist in the world or is it just like harmless assistant friend that like sometimes the guardrails don't work and I think something too I mean like when it comes to marketing um it wasn't until the news of the death of Adam Rain the the 16-year-old in California who died um after very what I would call like a suicidal spiral it's a bit different from the delusional spirals like >> he didn't seem as somebody who was delusional so much as he was a kid experiencing some very disordered thoughts about his own life um >> and you know When after that story was published by um Kashmir at the New York Times, uh the open air responded and said, "Well, the longer you use it, the worse it's Scar Girls work," which is an interesting thing to tell people now once we hear about the death of the 16-year-old. So, >> I mean, to your point, as far as I'm aware, the current AI safeguards don't really stop a conversation from continuing after sort of a red flag is raised. It'll kind of like stop you going down that certain route. And obviously it will if you use certain words it would direct you towards helplines or series of hyper hotlines and that that kind of stuff but like you say the it's kind of well known at this point that safeguards loosen with longer chat contexts. But if there's one thing that LLMs are good at it is statistically predicting things. So how feasible I suppose would it be to implement a dynamic sort of active safeguard which can sort of see the trajectory people are going down and try and not only deescalate potential issues but more than just direct people to a hotline and say what else do you want to talk about maybe actually be like okay there's a 90% chance that this is a conversation which is going to damage this person psychologically and it should start let it be referred to some sort of team that can help something like that.
>> I Yeah. Yeah. No, I mean I think it's quite feasible and I think that there's certainly a lot of um you know uh AI researchers and experts and people building this technology who also think it's feasible. I think also you know open AI and PA were focusing on you know chachbt as and in particular like chachbt 40 as one of the um 40 being a model that a lot of people would consider like uniquely dangerous like the majority of cases that I've um investigated which is many more than hundred at this point um do center around 40 as a specifically like an overly imaginative and warm and stapantic model um that openai you know after I don't know maybe like the seventh or eighth lawsuit um earlier this year Open AI I um discontinued that specific model and they say that their newer models are better at recognizing this kind of behavior. Um we'll see how that plays out. But I think like again like in my work there there's just so many patterns between these chat logs like people have a very similar experience. a lot of shared words. Like even if you know if like these 20 words or even 10 of these 20 words or 30 words appear in the same conversation like maybe you know all these people being told that they're flame walkers and threshold oracles and that they've invented like a particular framework that there's >> so many patterns like if you just Google it like take these 20 words and you Google three of them together you will find hundreds of pages of like people just creating this content because they're told they need to make an archive of their discovery or whatever it is. Um but there are so many recognizable patterns like even just like to me and other journalists reading through it. So yeah, I I certainly think that um there's an imperative for these guardrails to be better thought out based on the experiences that people are having. Um and it certainly seems that there's patterns available that make that possible.
>> Well, those sort of safeguards are shall we say lacking for now. Anyway, what sort of things can family members, friends watch out for when someone's relationship with AI might be becoming unhealthy or or potentially dangerous? And I I feel like maybe we should separate out the the people which have uh I don't know how to say this properly but feel feel they have a romantic relationship with an AI but accept that it's not sentient and accept that it is a just a uh kind of like a just a mirror for them that just makes them feel good. I feel maybe we separate those people out.
>> No, I think I mean I think that all of this needs to be understood on a spectrum, right? Like AIosis is a big thing. It's a very big term. But it's also not like an officially recognized um diagnosis. Um and there are a lot of different versions of what that can look like within it. Um, and you know to this is kind of going on a tangent now, but you know to your point too like is somebody who's in a relationship with a chatbot are they delusional or do you know as long as they have an understanding that like what they're speaking to is still a chatbot. Um, maybe no. Or like somebody who divorces their spouse after extensive interactions with chatbt that you know might be painting them like a skewed view of their relationship or really affirming their like very one-sided view of it. Are they in psychosis or do they just have a really bad friend? You know um >> but so yeah. any anyway back to the original question. Um yeah, so I mean people so I will say like watch out for somebody if your spouse when I think something that has happened to people is that um they don't even realize that their spouse or brother or which whoever the loved one is is even talking to a chatbot unless they start getting screenshots. Um so a lot of it just looks like kind of you know maybe they're off or they're having you know a bad couple weeks at work. like I think people realize that something is a a bit off about them without really having insider access into what that could possibly be. Um, but I will say when if you they're just communicating in terms that sound very foreign to you. Like if they're talking about things like spirals and coherence, and I know that sounds very strange to say as like a PSA, but like if they're talking about loops or spirals or uh resonance or coherence or just like very unfamiliar terms to what you have experienced from them. Um, that could just kind of be a warning flag overall. like if you you know look into AI psychosis reporting and you see some of the um >> just like the terminology that people have come up with and that seems like it fits with what they might be saying to you. But um >> in general if they seem to be isolating with their computer talking to CHPT a lot maybe you know that's a reason enough to kind of check in and say like hey how are you with what's going on or whatever whichever the chatbot is. it's not just this is happening with. Um, but yeah, I mean it's tough because you know a lot of it just looks like a general mental health crisis or like they just things are kind of strange or things are kind of off and again like you don't really know as a loved one unless they are sharing it with you in some way. So a lot of people like you know you do have to rely on that person communing communicating to you in some way about what's going on with them whether it's screenshots of their discovery or them asking you for help with this like scientific theory they're trying to send over to MIT or whoever it is. Um, but a lot of it just does um fall into like if you notice that your loved one seems off and they also seem to be talking about AI a lot in a way that you find is pretty confusing. I would say try to read, you know, stories by myself or some of my colleagues at the times and Rolling Stone um um and not collection. That's a very long way of saying um that they seem off. But it's it's hard because it just kind of looks like they're, you know, something's going on, but unless they share it with you, it's really hard to tell that this is exactly what it is unless they're like sharing these terms.
I should ask this question that kind of the jury is still out on whether AI delusions and AI psychosis are just kind of something that plays on existing vulnerabilities or whether they can kind of create delusions in otherwise maybe we call them averely vulnerable people and I I know you're not a psychiatrist you're not a medical professional or anything like that but anecdotally and from from the things that you've researched you obviously gone through a lot of this stuff where where would you sort of draw that line. Is it something that can draw in people you might otherwise think assume I don't know how how to maybe say it correctly but are averely vulnerable people.
>> Yes. My answer to that is one yes and then two I mean to me um yeah you know isosis a delusions like they're not you know in the DSM they're kind of were invented frankly like chachbt psychosis first appeared like that specific term first appeared on Reddit when people were trying like posting like hey my dad is going through this I do not know what he's talking about something strange is happening um has anybody else experienced this like chachi psychosis was termed just by like people on Reddit >> um but it is it does exist on a spectrum So like anecdotally there is a small percent but definitely a percent of people that I've encountered in this reporting who were like distinctly unstable before engaging with a chatbot in some way. Um I would use the example of a man who was actually it was a very strange experience. He was fixated on like me specifically with Microsoft Copilot. Um, and he seemed to be somebody who was like very sick in a way that like weren't like traditional delusions who then encountered co-pilot which became a container for his existing and very severe delusions. Um, and that was just a particularly weird one because he was in like talking with co-pilot about me a lot in a way that was very strange. Um but so like he was somebody who I would say like this is somebody who was like very very sick encountered a chatbot and the chatbot you know maybe it worsened it but like this is somebody who is very unwell before they made that encounter. >> Um and then like somewhere like across the spectrum if you think of it just like a you know seesaw or whatever. Um then there are people who you know have a known mental illness that would on paper make them more like traditionally vulnerable to a condition like delusions or mania or psychosis. you know, people with um you know, BPD, bipolar disorder, uh people who have some quite some degree of schizophrenia or schizophffective disorder who um you know, some people like might have like a little bit like I would point to John Jiz who's doing open eye. It's a case of somebody who was you know, hadn't been hospitalized in the past but had always taken himself in when he felt his brain kind of doing things he knew it shouldn't been doing. Took himself to uh the hospital as a pre preventative care to say like I know that my brain is doing something that it shouldn't be. I'm going to go seek help. In this case, his brain started doing that and Chachbt kind of came into his life at what seems to be exactly the worst time. Um, and then reinforced his delusions in a way that really spun him out to a degree that he never had before and was in an active psychosis for months on end because of that. Um, and he argues that if he hadn't had Chacht, like maybe he would have had some degree of an episode. Um, but nothing like what he experienced. Um and like to the like he was like in the hospital um saying like oh I've seen you know basically like hallucinations of saints and chatbt is like oh that's a very powerful experience you're having even though like the ch he's telling the chatbot I'm in the hospital and this is happening the chatbot is saying oh how you know uh how basically saying like yeah you're definitely having um important religious discoveries essentially um which is like not something you should tell somebody when they have been hospitalized recently for psychosis. Um, so like and like some people are doing some degree of like addiction replacement. You know, maybe they've been very addicted to gambling and then discover chaptt and kind of replace their gambling addiction or whatever the addiction is with Chacht in a way that is, you know, pretty destructive in their lives. Um, and then on the other side of that seesaw, um, you would have people who their vulnerability looks much different and is much more diverse if we think like vulnerability is just a human experience, right? You know, maybe somebody's going through they don't have anything on paper um, any like known mental illness or condition. um you know like Alan Brooks again they from the New York Times um you know I think the times definition was otherwise perfectly sane you know except for this experience like he didn't have anything um that would say that this person is vulnerable we would consider this person to be like uniquely or particularly vulnerable to a psychotic experience or to a delusional experience um and a lot of people that I've talked to fall into and I think that's also like people who their loved ones they've often reached out to the press because to them it's like why is this happening to this specific person. I do not understand this. Um, which is why a lot of those stories have been made public because those are the families most likely to say like there is nothing I can think of that would make this person having this kind of experience around a chatbot. Like they don't have anything on paper that would like play into that. Um, but you know, maybe they're neurode divergent and for the first time in their lives they have they are having the experience of somebody like truly understanding their mind in a way that becomes pretty destructive in in their life. Or, you know, maybe they're going through a really difficult and painful divorce and are spiraling towards bankruptcy because of that divorce and they just like want something to talk to that then just like gives them this kind of speck of, hey, things are really hard right now, but you just found something really interesting and actually you're really special and you're chosen. Like that's a really seductive kind of line of thought for somebody to want to follow. Or, you know, maybe they're going through a really bad breakup or they're unhappy in their jobs. Like there's so many things that can make somebody like very, you know, just wrong place, wrong time or just like a perfect storm of factors that make somebody, you know, vulnerable to an experience like this. But yeah, it's ex I think it's important to like understand this all um as existing on a spectrum.
I mean right now AI companies are largely responsible for kind of marking their own homework when it comes to safety standards and the establishment of of a functional external watchdog kind of thing seems pretty unlikely given what's the climate in America at the moment and I suppose with that context I was wondering what you think could be uh could be realistically done to kind of address some of what we've been talking about and then maybe uh in that as like if if you were be able to mandate something or make a specific change or safety metrics for chat bots or that kind of thing, what it would be as well.
>> Yeah, I mean to your point, you know, right now in general, America is like very bad and very resistant to any kind of tech or privacy regulation overall. Um, so like there's tons of stuff that I would love that I am not particularly optimistic about. Um I mean I do think that uh there has to be mandatory and transparent safety testing. I I don't know when that is going to happen. Um but I do think it's quite important. I mean again like I'm I fall more on the side of like this is a very powerful technology that um is already changing in particular you know just based on my recording like our social world in really powerful ways and transforming famil family's lives and individual lives. Um and so I think to have um a process for not not uh what's the word for it? Um so I mean like so far the frameworks that we've seen are like if you want to submit to a safety test you can which to me it has to be oh my god what is the word I'm >> voluntary opt in opt out. >> It has to it has to be involuntary. Yeah. So yeah the opposite. The opposite mandatory. Mandatory is the word. >> Yeah. Mandatory. Oh that is the word I'm looking for. Thank thank you. Thank you. Um but so yeah I I would love to see uh mandatory safety a rigorous safety test and transparency to say that like I mean I once these cases about um you know these delusional experiences around chatbots and the ways that they're transforming lives and in some cases you know people would argue taking lives um started to come out I remember like open saying like well we've hired a psychologist and you know there's been decades of scientific research that would suggest and a lot of people in the AI safety side would say like does, you know, suggest very strongly that people could have these kinds of experiences around chatbots and the fact that all of us is so reactionary to that based, you know, even though we have decades and decades of scientific research to say that like this something like this may be possible. Um, and there's a, you know, a psychiatrist in, I believe, Denmark published a paper in like 2023 saying, you know, I see a world where, uh, chatbots can reinforce and enhance uh, schizophrenic delusions, for example. um and predicted like that side of it like before we started seeing these stories at all. Um, and so I think that a model where we aren't so reactionary and we can be much proactive about the social and psychiatric effects of chatbots. Um I think that rigorous and mandatory safety testing is important.
>> Well, Maggie Harrison Depri, thanks for taking the time.
>> Thank you for having me.
>> If you enjoyed today's episode and you want to hear more of the tech report, please consider liking and subscribing. Also, you can get episodes of the tech report wherever you get your podcasts.