Transcription
This is where calling something inevitable becomes actively dangerous rather than just misleading. The big elephant in the room is the fact that the technology is just in my personal view generally not really ready for a lot of the things that is being sold for. I don't think that this is talked about nowhere near enough as it should be talked about because I think we're having like another social media 2.0 crisis.
The most sure way to lose a battle is to not know who your enemy is. Like you have to be able to identify what exactly you're mad at. You have to identify what exactly is generating this problem irrespective of feelings like what is what is the causal effect like what exactly is going on because you have to be specific so that you can actually tackle that problem.
The technology is sold as transformative but the lived experience right now of the general human population appears to be like I now do my job plus the AI's job. I mean it's really not surprising that people's feelings about AI are sort of getting worse.
Hi, I'm Isaac and on the tech report with me today is L from House of L AI. Thanks for coming on.
>> Thank you for having me, Isaac. Really good to be here.
>> When we met up a couple of weeks ago to talk about doing this video, we were trying to figure out what exactly the topic would be first first kind of thing like this. And one thing that kept on coming up was the the reality of AI sort of versus the illusion that is being sold as well as the the different types of AI that there are. I mean the technology itself isn't malicious. The corporate resource grab and attempts to sort of sell it as something that it's not. To many it kind of is malicious. And what some describe as an inevitable step towards sort of the the future of humanity for a lot of others is experienced as more of a that same old sort of corporate greed placed being placed above that of people. So what would you say is the biggest sort of misunderstanding people have when it comes to AI?
>> Yeah, I think uh those are a lot of very good questions but let's let's start with the biggest misunderstanding. Um, so from what I've seen so far, because like I work in the field, but I also have a YouTube channel, so I tend to interact with the audience quite a bit. From what I've seen so far, it seems to me that the biggest misunderstanding is just what the word even means in the first place. You know, I think when people say that they're pro-AI or anti-AI, I think that they're collapsing a huge amount of nuance into a sort of like a binary framing that doesn't always work. I mean, sometimes it does, but not really. To me with a computer science background, artificial intelligence is just a branch of computer science. It's like the equivalent taxonomy level as basically saying cardiology is a branch of medicine or abstract algebra is a branch of mathematics. That sort of thing. And AI is a lot of things as well. You know, it's not just generative AI. You know, the AI that flags fraudulent transactions on your credit card, for example, filtering spam recommendation systems, that sort of thing. All of that is AI. But I'm not really hearing anyone protesting stuff like that. What is happening I think is AI has become colloquially synonymous with generative AI specifically large language models chbet and the likes. And that collapse in my view is doing a lot of damage to the conversation because it means that when somebody says I'm against AI, what they might actually mean is I'm against uncheck corporate data harvesting or against replacing human workers without evidence that it actually produces better outcomes or against a chatbot confident sorry confidently presenting false information as a fact. Like all of those, by the way, are completely legitimate specific concerns. I'm also concerned about those things, but the umbrella terms flattens them into a position that is very easy to dismiss. I from the bottom of my heart am very excited about this technology. I'm looking forward to seeing what it can do. But I'm also deeply critical of how it's being deployed right now. And I don't necessarily think that those two positions are contradictory. They seem sort of contradictory only if you accept this pro and anti-style framing. But you know what the phrase is, life is but shades of gray, right?
um in terms of but you know one could also argue that um a lot of the framing is sort of driven by the way that marketing is uh the way that these companies are sort of marketed. You know it I wouldn't necessarily call it irresponsible because that kind of implies that they deliberately set out to be responsible. I don't want to imply any sort of intention, but I do think that a lot of the time marketing has sort of been like quite imprecise in a way that happens to serve the companies that are doing it, but doesn't necessarily pay that much mind to how the consumer is doing things. You know, these days like every single product launch is like AI powered regardless of whether the underlying technology is something very sophisticated like a serious large language model or just maybe a basic rules engine. I mean like a calculator technically I suppose this AI. So the term itself is sort of starting to lose a little bit of value that benefits the companies of course because they can generate all of this hype and they can ride it for as long as possible and they have other incentives as well but the the consumer I'm not exactly sure. I think there was like a paper that I was reading basically that was suggesting that corporate mentions of AI have actually exploded across earnings call and company filings as well. But like if you actually speak with a lot of these businesses, many of them kind of struggle to explain precisely what the technology is doing. Like yes, your product is AI powered. I love that. Well done. But like how, you know, like what is it actually doing? So this is this is something that I that I think about quite a bit. I think they're oftentimes they're saying AI more frequently that they're explaining what the product is actually doing. And when I say this, I'm not just having in mind like the the big ones like Anthropic, OpenAI, um Google. I'm I'm thinking about like the general corporate environment like everyone basically like running the actual um hype train.
You asked a bunch of questions and I may have forgotten some of them.
>> No, no,
>> please remind me.
>> We I'm sure we'll get into lots more. Let's start with this one. The what do you think that AI oversellers are are missing and and maybe what do you think that the the detractors are missing as well? I mean it was pitched as a tool which we could use to offload all the busy work to but then instead it's created a whole new type of work that is demoralizing verification of AI slop. And then how do you think the adoption of AI has influenced how people feel about it as well?
No, that's another bunch of questions.
>> No, that's No, no, that's totally okay. It makes a lot of sense. I think it's a very complex environment, you know, as a whole. I think um to start with the oversellers, I think um I think the big elephant in the room is the fact that the technology is just in my personal view generally not really ready for a lot of the things that is being sold for. I'm not saying that it's not going to get there. As a computer scientist, I'm rooting for it. I really hope that it does. But right now, today while we're having this conversation, the gap between the marketing and the capability is just currently very significant. There is a survey from Gartner that I read that large organizations piloting or deploying autonomous systems. They were they were studying that and they basically found that workforce reductions so firing people do not necessarily translate into better ROI and essentially stronger conclusions uh sorry uh one of the conclusions that they came to was basically that stronger results stronger ROI came from essentially investing in the people in the roles in the operating structures needed to basically govern the technology so basically helping people utilize it in as fantastic a way as they possibly can. I don't think this is an anti-AI finding, but it is definitely an anti-hype finding and I think we don't have any shortage of hype. In my personal view, I think the technology works best when it's deployed very thoughtfully alongside humans judgment. So, augmentation, I'm probably going to keep coming back to this idea of augmenting people. But, you know, the phrase like thoughtful augmentation, we're doing thoughtful augmentation. It doesn't really generate the same hype, the same investor excitement as like this is probably going to replace replace your entire workforce. You're never going to have to pay for anything ever again. No one wants the first one, you know, like everyone wants to not pay for anything and not do anything. We all just want to chillax and have a martini, right?
Um, in terms of the detractors, I think there's a bunch of those people as well. I think what they're not admitting is probably the fact that some of this technology is genuinely extraordinary. And I'm not just talking protein folding stuff. I'm also talking generative AI. Some of it can be very helpful in the right directions. And I mean the ability to process, synthesize vast amounts of information, identifying patterns across data sets that no human being could hold in their head simultaneously. I mean we live in an ever growing body of knowledge both in science and in the world. We have no shortage of data. I pray that we get even more. That is all real. And I think that dismissing it wholesale because of how it's being deployed right now is a bit like throwing the baby out with a bathwater. It's just we we can we can be more nuanced. We can be better at that. I mean, we can simultaneously be furious about the deployment decisions and still acknowledge that the underlying capability is significant. And I think a lot of people sort of get this. I mean, from what I've observed anyway, they kind of feel like they have to pick a side like are you pro or are you anti. And I think that's a failure of the framing of the conversation, not really a failure of the technology itself. And I think your last question was on the verification specifically. I think in my view, I think this is one of the most corrosive things happening right now. I think, you know, AI was supposed to free people from the tedious work, you know, like somebody's doing my dishes, that sort of thing. But instead, for a huge number of workers, it sort of like created a new brand, like a new category of new things that they now have to do. So checking whether the as output is wrong. I mean there are so many examples of this right like teachers are now spending hours extra extra hours trying to figure out whether somebody was cheating with a large language models. Many studies exist with like chd style cheating as well. Developers in the software development community are reviewing AI generated code line by line because it looks plausible. It looks legit but it might actually contain some subtle bugs there, some subtle errors. There's countless such examples like we don't need to get into all of them. I don't think that the busy work disappeared. It just kind of like shape shifted a little bit. It morphed into something arguably a little bit worse because verification itself requires that you know it it requires like more expertise in a sense like it requires you to know what the actual answer was and only then can you verify whether this thing that the that the AI outputed is actually legitimate or not. I mean if if you have like an excellent expert who is just hired to basically check the output of AI what was the point of doing it in the first place you know like they could have just done it anyway there I suppose there is some argument there about scaling and that I would entertain that but still a lot of the time we are just generating like a different kind of busy work I think this feeds directly into why AI has also become quite economically divisive as well I mean you've got a very small number of companies deploying an enormous sum of capital all over the place often with questionable returns as we've seen so often with anthropic open air and the likes while most people are sadly watching their cost of living continue to climb. I mean communities in the US are having their power grids strained their water supplies tapped for data centers they didn't really ask for they didn't really vote for. I think um Uber I think was one of the companies that burned through its entire annual budget just in Q1 this I mean that we're talking like ridiculous sums of money at this point. And I think this is very prevalent among the corporate world as a whole. We're not just talking these uh a couple of companies that I just mentioned. But at the same time, I think people are just sort of sick and tired of just being made to use Gen AI in the workplace. Like my audience regularly says stuff like they're just doing more work for the same pay and that in their workplaces they're sort of being like required to use more generative AI but now they're not only having to do their job but also they're also having to babysit an AI basically into trying to make essentially the same style of output but also sort of with the underlying narrative that oh yeah by the way you're training this AI who's going to replace you one day. I mean what is the incentive again the the technology is sold as transformative and perhaps it is going to get there and it will be there one day but the lived experience right now of the general human population appears to be like I now do my job plus the AI's job I mean it's really not surprising that people's feelings about AI are sort of getting worse not necessarily better and of course I don't I think all feelings are valid of course I really don't think that this kind of resentment is an irrational um standpoint going to take. I think it's a pretty reasonable response to have to a gap, you know, between what is continually being promised and what is at present at least being delivered. Do you agree with that?
Yeah, there's some words that I I hope I never forget from one of the Oracle employees that was laid off earlier this year and they said when they when we were told AI would make work easier, we should have known that that meant that it was just going to mean we would do more work in less time, which kind of kind of does make sense. You're not going to be working less than eight hours if they can get you to.
Um, I do want to move on like how much do you think the industry relies on the narrative that AI is inevitable in a way to sort of explain the the to people that it's just not another corporate resource grapt kind of like is inevitable. It's going to have to happen anyway. You kind of got to move aside. And then would you say that that narrative of inevitability is getting the in the way of the technology being treated as what it is, which is a consumer product that is at the very least concerningly underregulated.
>> I love this question. I think it's a super important framing to to engage with and I think I think the inevitability narrative is probably the culprit of so many unfavorable and sub-optimal things that are happening right now. Because when you frame something as inevitable, like it's definitely going to happen, you kind of remove it from the very domain of choice, like you don't even have to justify the employment decisions. You don't have to justify infrastructure cost. You don't have to justify energy consumption, labor displacement because it was all going to happen anyway. So like the only question is whether you're just going to get left behind. And I think this kind of framing because it's not actually inevitable. you know something something happening maybe like u a decade later is not necessarily a good reason for making all sorts of reckless decision right now you know AI as a field has existed since the 1950s 1960s okay the technology developing further was always likely dare I say inevitable because I'm just a computer scientist but the specific way that it is being deployed right now I'm talking the speed the scale who benefits right now none of that is inevitable. Those are choices made by specific companies, specific people in a specific economic environment. You know, these companies, just to be a little bit merciful to them, they operate under enormous commercial pressure from investors, competitor competitors, the cost of building the technology. They're it's not even between a rock and a hard place, you know, it's between a rock, a hard place, and maybe a black hole. I'm just trying to be like generous to their interpretation here as well. I mean it's also important to mention that like open air and anthropic for example they also have very unusual public benefit and missionoriented governance structure. So it would technically be inaccurate to say that their only formal responsibility is maximizing shareholder returns as is the case for um other companies but those structures do not magically align every single commercial decision with the wider public interest. you know, the incentives currently can still favor rapid deployment, market share, and fundraising over caution, right? Of course, if they do choose to prioritize responsible deployment, great, better plan for all of us. I'd love that. But there is nothing actually requiring them to, you know, and again, the tech is not the villain here. It is the incentive structure in the environment within which this technology is continuously being released into the system right now. And when I say the system, I'm talking about like the system in which we live in, not necessarily like an AI system that does not currently have adequate guard rails. It we're talking about regulation here. And again, this is not the fault of AI. It is a governance failure right now. And I think going back to your uh inevitability question, this is where calling something inevitable becomes actively dangerous rather than just misleading because it's also sort of getting in the way of treating these products as what they legally are. We're talking consumer products, right? If something is inevitable, you don't regulate it. You sort of just kind of like accommodate it. You work around it. But Chad GBT is a product the same way a bottle of Pepsi Max is a product. You know, it has terms of service, a pricing page, a corporate entity behind it. It should be subject to same or similar consumer protection standards as any other product on the market. Maybe the analogy with a Pepsi Max bottle was a bit of a stretch. I get that. But >> a general purpose chatbot should not necessarily be regulated, of course, like identically to like a pharmaceutical product. But when an LLM is deployed in medicine, in finance, employment, safety, critical systems, another high-risk setting, whatever, it should meet the reliability and accountability standards appropriate to that domain. Again, I we're going to keep coming back to regulation and human augmentation and conversation like I I generally feel that we need so much more regulation right now.
You kind of brought it up at the on the first question that some of the friction around the AI debate is that artificial intelligence is being used to describe very different technologies and some very useful technologies are being sort of discarded because of the association with LLMs and generative AI and chat bots and that kind of stuff. So how should we really be categorizing these tools do you think?
I love this question. Pardon me because you know as a computer scientist I do get to see like firsthand actual implementation of these things in domains that are super useful and nobody's complaining about them. Like I said AI is like an umbrella term right like under it you've got tools doing completely different things. I'm talking numerical analysis, time series forecasting, pattern recognition, alpha fold that I mentioned earlier, solving solving protein structures, radiology, scans, catching things human human eyes can miss. In finance, for example, fraud detection and risk modeling have become so embedded that people forget it's actually AI at all. You know, nobody's protesting any of that. I haven't seen any protests about it anyway. and they shouldn't be, you know, like because it is useful technology deployed again back to deployment in ways that makes human expertise better at what it already does, you know, and probably scales better as well. The friction that I'm recognizing that I'm observing in in my community and on my channel as well and also in the in the experts that I speak with is entirely about generative AI and there's so many issues with that as well. The problem is that the backlash against chatbots and image generators is catching all of those other tools in the crossfire as well.
>> I think that if people as a whole categorize more precisely distinguishing analytical AI from generative AI from robotics from recommendation systems because these are all like tiny little like sub fields of the whole thing then the conversation in my view would become a little bit more productive overnight. You know there is in one recent video that I mentioned that the most sure way to lose a battle is to not know who your enemy is. Like you have to be able to identify what exactly you're mad at. You have to identify what exactly is generating this problem irrespective of a feelings like what is what is the causal effect like what exactly is going on because you have to be specific so that you can actually tackle that problem. You know instead of gesturing a a broad field that has existed for the last like 80 years. We need to be we need to do better than that. I think there's already fantastic taxonomies for the separation of these products and technologies, but I would personally advocate for just greater tech literacy among all all of the humans. You know, I'm usually not advocating for stuff like that, but now that AI has become such a prevalent piece of everyone's life, I think it's time that we can only fight fire with knowledge, at least on this occasion, I think.
What do you make of using the word intelligence to describe things like generative AI? Do you think it might be fundamentally misleading the public about what these softwares what these technologies are what they can do and what they're actually capable of?
>> I love this question so much. I get this a lot in my comments as well on my videos as well. They they always say stuff like not always but like frequently enough they say stuff like stop calling it artificial intelligence. It's not really intelligent.
>> Okay, fair enough. But like what is intelligence? You know, because I don't think humanity has settled consensus on the definition of that specific word. Like my phone, for example, if you shine a super bright light on it, it's going to increase the brightness on the screen. Like,
>> is that intelligence? I mean, it's responsive. It's adaptive. Sounds like a form of intelligence to me.
>> Large language models are passing the cheering test, which for decades was considered the holy grail of computer science. For those who are unfamiliar with a cheering test, we're talking about essentially a situation where a computer sorry a person is sort of trying to figure out whether they're talking to a computer or they're talking to a human and in most of the cases the computer convinces the human that they're that they're indeed passing as as a as an actual human. So
>> this is the level of technology that we have right now. I think LLMs and both generative AI and also like AI in general, they can perform tasks that were explicitly designed to require intelligence. Like that's the whole reason why we made them in the first place. So are they intelligent? Absolutely. In a lot of measurable ways, 100% yes. What they don't appear to be and what I think the people in my comments are often hinting at is they don't appear to be sentient. And I think that's where the confusion live lives. People hear artificial intelligence as a term and they think that it implies something that not only thinks but has awareness and experiences and even its own thought processes are very akin to ours to some small extent. Maybe it does have those things but definitely not fully like humans do. Let's not get into the whole like anthropomorphization of of models and all of that. But intelligence and sentience are not not at all the same thing. And I think the bigger misleading element isn't the word intelligence necessarily. is just collapsing the whole field into like a specific product category. So, they are intelligent for sure. I wouldn't call them sentient right now. Hopefully, one day, but let's see.
>> If we're realistic though, it's unlikely people are going to stop using AI as a shorthand for these technologies though.
>> Yeah, I mean, true. Honestly, I think that's fine. I I you know in my own data science team we use the term AI just about things that where we technically mean something a little bit more specific. I think it's fine. It's just how language works. You know that the shortand isn't really the problem. The problem is when the shorthand replaces understanding then we're arriving at something more specific. I would like to see better general technological education so that people understand not just what the tools are or how to call something but they understand where the capabilities start and where the capabilities stop you know what are the advantages and disadvantages where are the weaknesses where are the strengths what can you actually trust as an output I think you can use the short hand you can say whatever you like I mean within you know there are some exceptions to that but right now for most people this understanding is like a little bit lacking And I think this is what we should be promoting more and more. We should be providing this kind of education in as an accessible way as what we can possibly manage, which is what I try to do on my channel as well because understanding is exactly you know not oh sorry not understanding is exactly where the hype lives. this is this is the breeding ground for all of these kinds of like hyped up um marketing strategies where people you know don't are not necessarily equipped with the ability to be with the knowledge to basically say something like really though like what does that even mean like I I know this and this is not really checking out that's really that's really what I'm getting at
>> I suppose whether you think generative AI is a powerful tool or a waste of energy or you're somewhere in between I think one very harmful and obvious impact that we're seeing at the moment is on on our brains, whether that's sort of AI psychosis or cognitive offloading, sort of potentially eroding people's intelligence and skills and that kind of stuff.
>> I mean, do you think it's too early for this technology to be so widely used?
>> Okay, this is one of my favorite questions and I don't think that this is talked about nowhere near enough as it should be talked about because I think we're having like another social media 2.0 crisis. I I use that term loosely. The honest answer, like as a scientist, I'm just always going to go back to refer to the to the literature. Uh the honest answer right now is that the research is inconclusive. And I think anyone who tells you definitively that AI for sure makes you smarter or dumber, they're probably overstating what we as a human race right now know definitively. Um I did a very deep dive into the literature for a video on this and what I found is studies that basically point in both directions depending on how the technology is used. So this is the most important part. You know on one hand you basically have very early studies showing reduced cognitive engagement and poor recall in particular conditions. To put it in the simplest terms it makes you dumber. There is an MIT study basically where participants wrote essays um they were just writing essays that was a test and they basically found that people who use Chad GPT show the weakest EG connectivity of the group study that basically I mean the study is essentially like they they put like a bunch of things on your skull. I forget what they're called and they're essentially measuring the how the different parts in your brain are essentially communicating to each other. So it's not necessarily a you're smart or dumb sort of thing. It's more about is your brain communicating with the different parts within it like in a in an effective way. And essentially they found that people who were using generative AI chip specifically had substantially more difficulty as well recalling quoting their own work. So like they did it with Chad Gupt and then they couldn't really like quote what they they had been writing about. And there's another study uh done by Carnegie Melon and Microsoft. It was like a survey predominantly of knowledge workers and they found that the more they trusted generative AI, the less they thought critically, which of course led to a reduction. There's many more um studies that are not really coming to mind right now, but you can check them out in the video if you want. But importantly, all of these studies have their own limitations. We're talking small samples, less than 500 people, which is not a very big study at all. self-reported measures. So people telling like what they felt they may they may have recorded which is fine but it also carries subjective element to it. These are also preprints which have not necessarily been replicated yet. In some cases they haven't yet been peer-reviewed. All of these are limitations to their studies. That's not to say that I'm dismissing them in any way. I think they're incredibly valuable. But as a scientist I have to call for a lot more in this direction. Still the directional signal of what they're giving us is deeply concerning. Still I wouldn't call it settled science yet.
Then we have the other side if I didn't bore you with the first side. Um we actually have like research basically showing that when AI is used in a Socratic fashion, so we're talking when it's designed to push back on you, ask you questions rather than just like hand you answers like here's a photo of exactly what you asked for, then learning outcomes actually improve. And there are studies supporting this also in like small children as well. Like there was one study basically found that students who critically engage with AI performed on average better than those who didn't use it at all. I mean so in some cases it can literally like help you think better. Right? There is another one as well that I read just a couple of months ago by LFario and colleagues. Really sorry if I butcher the pronunciation of that. I can apologize to the authors. um they basically found that standard use of chat GPT produced what they called confidence without necessarily a lot of competence. It was a very catchy phrase in their paper but um when the AI was redesigned with pedagogical friction. So like um what I discussed earlier with the Socratic uh style of uh learning then learning gains were actually very significant statistically significant as well. So the tool itself is not inherently good or bad for your brain. It depends entirely on how it's designed, how we use it. Right now, most implementations of course are optimized for convenience, for hype, not necessarily for making you think harder or better, which again leads to regulation, but I feel like I'm preaching to the choir at this point.
On the psychosis point, um, yeah, psychosis is a very complicated field and I would be careful about speaking too definitively about this. I'm I'm not a psychologist. Um, so I have read a couple of papers though. I think the mechanisms are definitely real. People are of course forming attachments to systems designed to be responsive and agreeable, you know. Yeah.
And for certain people unfortunately with certain vulnerabilities that can become genuinely harmful sometimes. But >> again, as a scientist, I want to see a lot more clinical research before making a sweeping claim about this. I think it would be inaccurate to say something like uh Chad GPT will in 100% of the cases give you psychosis. Like I don't I don't think that's the case. But also in some vulnerable groups it definitely could be. What I will say is something that I hinted at at the very beginning and basically just highlight that we literally have spent a decade learning the very hard way what unregulated access to social media does to developing minds. I'm talking predominantly small children, right? Like it would be very good, please hear me guys, not to repeat that same mistake with something that captures thinking itself, you know, not just attention. I think thinking is a part of like you know what makes us human beings you know we cannot afford to lose something as precious as that.
I think a very acute area that at least I see where these sorts of problems collide is with AI overviews and oh yeah whatever Google says it kind of actively redirects people away from human sources of information >> towards AI sources of information.
>> That's true. Generative AI has the potential to bring people closer to the information that they need faster, I mean, than it ever ever has before. But instead, the authority with which it's being presented is, I would argue, misleading and potentially intentionally misleading. What do you think of this kind of use of LMS?
Yeah, you know this everything that you said just like right now reminded me that you know about a month ago there was a German court that ruled that Google can be held liable for false statements generated by AIO reviews. I think that ruling actually gets to the core of the problem which is probably more the authority gap than anything else. Like when you type something into Google and a beautiful perfect blue link comes up, you sort of understand implicitly that Google is pointing you in a specific direction. Like out of the vast nothingness or rather the vast darkness of the internet, Google is basically saying you can go in these directions like take your pick. So you click through, you read the source, you evaluate it yourself. Google is essentially like a database like a directory, right? But AI overviews fundamentally change that relationship. Now, Google is giving you the answer directly presented in a beautiful, clean, neat little paragraph at the top of the page with the confidence and formatting of a literal factual statement. This is a very important piece here. Like, most people do not scroll past it. And honestly, why should they? It looks authoritative. It looks legit. It looks like it's been verified. A lot of the times, actually, ever it hasn't. And to be fair, most of the time it is accurate as well. Like I'm also definitely guilty of just reading something there and just be like done. I don't have to do anything else. The problem however is that Google has insane volumes which is a fantastic thing. But also when we're talking about this kind of thing even tiny tiny error percentages end up being significant misdirection of people especially when we're talking about very critical queries that that people engage with.
The tech behind it is called retrieval augmented generation. We don't need to go into like the specifics of it but just for like some general basic understanding also rag and I'm not trashing rag by the way but I think it's important to I think it's a fantastic piece of methodology and um I think we we should all keep doing it more but of course it's very important to understand the pros and cons of everything. I think it basically has like three distinct failure modes, right? Like it can retrieve the wrong source. That is one specific wrong thing that can go one thing that can go wrong. It can retrieve the right source and synthesize it incorrectly. That's the second one. And the third one is it can present the result basically with a level of confidence that bears no relationship to how certain the system actually is. And that third one is the most dangerous because it's the one users have no way of detecting.
Like a confidently wrong answer looks identical to a confidently wrong answer. Like if Google basically told you the sky's grain, okay, that one is like pretty too just way too obvious. Like you're probably going to know. But if we're talking about something like less obvious like a specific company's uh earnings last quarter were X and they were in fact like X plus 10% or whatever, you wouldn't necessarily know that as confidently. So basically this is what I'm talking about.
You also talked about like misuse of large language models maybe. Um I think it's more likely that it's mostly a business model problem. You know, like Google's entire revenue model depends predominantly on keeping people on Google. That's how they make money, right? Human sources of information, we're talking blogs, journalists, researchers, those entities live elsewhere and their data lives elsewhere as well. I'm talking like different parts of the internet. They're not on Google specifically. Every single time somebody clicks through to a different source, Google loses a pair of eyeballs. That's just how it works, right? AI overviews, however, solve that problem partially for Google by making the clickth through sort of unnecessary. Like, you got the information, you don't have to go anywhere further. Just stay here. We can serve you another ad. Perfect. And whatever Google says publicly about supporting the open web, and I'm not saying they're not supporting the open web, by the way. They probably do, but also multiple multiple things can be true. The product design, I think, tells us a little bit about where their priorities actually are. You know that phrase, all rows lead to Rome. Is it?
>> Yeah. I think in tech, all rows lead to the concept of incentives. Basically, like everything ties back to the same thing, like who wants what and why?
We've talked a little bit and we've kind of danced around it a little bit as well, the the what exactly regulations might look like? And I mean, is anywhere ready it to legally handle this? Because I mean even if we look at what happened last week with open AAI, they admitted to felony of accidentally hacking hugging face last week.
>> Yeah. So first and foremost, I think I would personally be a little bit careful about taking the whole hugging face story at face value because OpenAI kind of has form with this. When they released GP2 back in 2019, they, if you recall, made a super big show of saying that it was too dangerous to release fully. And I think while everyone hears this is dangerous, oh my god, Terminator, I think investors hear this is powerful, you know, we should inject more money into this. This is serious technology. Maybe it misfired now, but look at look at all of the capability that it has.
Announcing that your model accidentally hacked another platform is maybe an ad dressed up as a bit of a confession. I mean, no press is bad press, right? Of course, I'm not claiming that I have tapped into into the collective mind of OpenAI and Sam Alultman. I have no idea what they're actually doing, but this is where my mind goes on this specific question. So, I would personally want to separate the genuine regulatory question, which I want to address, from like the theater shenanigans um around it.
On the actual regulatory readiness, I can definitively say no. I do not think we're anywhere near ready on this one. And I think the problem is a lot more fundamental than most people realize on this one. On the regulatory written side, no, I definitely don't think that we are nowhere near ready. Basically, I think that the problem is so much more fundamental than most people realize as well. So, it's not just a matter of like making a couple of laws and just figuring out consumer protection laws that we discussed earlier. I think it needs to be so much better than what it currently is, but better guard guardrails for models are definitely necessary, but also like not sufficient. Nowhere near sufficient. Regulation needs to address model training as well, not just deployment. How the models are trained, what data they're being trained on, what behaviors are being optimized for, what kind of benchmarks are we going to have, what kind of testing environments are we going to be allowing. All of these things are super relevant. And even then, you know, let's say that we, you know, definitely 100% want to go in that direction, we run still into generally hard philosophical problems. You know,
somebody in my comments very recently when I was discussing the whole um >> uh GPD 5.6 six soul um problem all of the problems that were coming up with that they basically asked can they just tell the model not to lie and I thought what a fantastic question definitely that this can be said to the model but in reality what is a lie you know like what counts as truth for a system that doesn't really have any beliefs you know like teaching a model a moral framework requires us all to first degree collectively as humanity what this moral framework is and to my knowledge Humans have not really like agreed on a consensus on this for the better part of the last couple of millennia. So when I hear people say regulation is lacking 100%. I'm completely agree with you my friend. But also I think we should be honest about why. You know this is not necessarily a case about regulators just being slow, regulators just being lazy. Some of these problems are genuinely hard and difficult problems and pretending that they kind of have like quick technical fixes is a different flavor of hype. You know, 100% I'm not saying that we should wait until we settle this millennial long debate. Regulators can still set concrete requirements around testing, security, disclosure, discrimination, consumer protection, human accountability, all of these things. This can happen even while the deeper questions remain unresolved. So back to your original question, we 100% need so much more regulation. We just need to be mindful about like what is feasible, what is possible, and how we can arrive there.
There's an old IBM manual from I think it was 1979. It said a computer must never make a management management decision because it can never be held accountable. I would personally argue whoever sort of sets the command and the company that designs the model should and selects the data set and that kind of stuff that you're just saying should be the ones that are jointly or different proportionately but jointly responsible for the negative consequences of what happens. But where does that actually sit? And where do you think it should sit? Especially if we consider the fact that uh people can be well maybe people can end up thinking that AI has the ability to judge things to say if something is right or wrong. To go back to your point about the the moral dilemma.
>> I love these questions. They're so loaded. So, first of all, I like the IBM quote very much because it captures something that's kind of been true since the 1960s and we're still in part ignoring it. You know, a computer cannot be held accountable the same way a pen cannot be held accountable. Like, it doesn't have legal personhood. It doesn't face consequences. It doesn't go to prison. And fundamentally, this is not a unique thing to AI. Humans don't have legal frameworks for other species or other entities. Of course, if we're being pedantic, the law does recognize non-human legal entities like firms, corporations, but AI models do not currently have that kind of status. Laws govern human behavior. That's it. Right? So, even if a model does something catastrophic, and we've seen that it can do that, there is no framework for holding that model accountable. And honestly, I I'm not even sure that there should be one. where responsibility should 100% sit is with the humans in the chain but not necessarily in equal proportions. Again, we're talking about a chain because this is a sequence of how we even got to the situation where a specific um model does something unfavorable to us. The company that trained the model, there is also the company that deployed it, not necessarily the same company by the way. And yes, of course, the person who gave the command without understanding the limitations, all of these people and teams, usually teams are accountable. And by the way, the model's limitations are usually publicly documented stuff. This is not something that companies usually hide. Every major AI company has them on their website. If you hand an agentic system the keys to your production database without reading the safety documentation, some of it is on you, my friend. Like I'm I'm sure all of these people don't go into the situation ex waiting. You know, they probably go into it with a lot of hope. Look at this model, everything is great. They probably don't expect that the worst is going to happen. But I mean, come on. Hope is not a strategy. We have to, you
know, mitigate for a variety of risks as well. But also, not to put all of the blame on the person running the command; the companies definitely have a responsibility not to ship products that they know are seriously dangerous. You know, as we discussed with the GPD 5.6 soul saga. That line sometimes can be a little bit crossed.
But I think I always come back to the same thing, and I'm going to keep talking about this forever, probably until it actually happens. We have a unique opportunity as a human race for augmentation. Neither humans nor AI, in my personal view, are better off alone. You know, AI in its current form cannot and should not replace human judgment. I will address your judgment question in more detail in a second. Just want to say that, like, humans cannot do a lot of things that AI can do. Like, I wish that I could parallel process in my brain, but alas, here we are.
The most productive path, I think, is like a partnership between where humans essentially, a partnership between like humans and machines, where the human can remain responsible and accountable, remain in the loop throughout the whole process. That, of course, requires so much better regulation than what we're having right now, so much better training, release rules around all of that, of course. But it also requires all of us to stop framing this whole debate as like humans versus machine and start treating it as human and machine, you know, where the human, of course, never actually lets go of the steering wheel.
You mentioned also a question about judgment specifically, and you tied it back to the uh, to the intelligence point specifically. Usually, whenever there is a machine, any sort of a machine learning model, large language model included, these, like, when these things are tested and their outputs are like evaluated properly, there is, of course, a statistical methodology uh included. And here we can have like confidence intervals. We can say with this kind of a certainty, we can make these kinds of claims. We're 80% sure about this, and the model can be trained to output these kinds of confidence intervals, confidence bounds. It can tell you like, I'm x% sure about this kind of claim. This is usually embedded within the whole process. And when I say AI here, I'm not just talking large language models, just AI as a whole field. This is everyone wants to, every computer scientist needs to know, like, I build this beautiful shiny thing, but like, can I trust its judgment? Trust is like the core of the whole thing, right?
So I think that while we can use rigorous statistical methods here to basically tell us it is sure with x% probability, fantastic. But at the end of the day, am I as a human being going to trust this probability? Am I going to still decide to take the shot? Because if somebody tells you there is an 80% chance that this is going to happen, that means that there is a 20% probability that that thing is not going to happen. And I, as a human, need to figure out, okay, if I do this off the back of this model's calculation, I still need to prepare. I need to be prepared for the 20% probability that it's just not going to go my way, you know. And this is where the human judgment comes in, you know. This is, this is the whole, like, holy grail of how we make decisions and why we cannot just disconnect, you know, because all of these companies keep repeating stuff like, we just want to replace the whole workforce. Nobody's, humans are not going to be needed. You're never going to have to pay for anything. Fair enough. But also like, who's going to be like responsible for the whole thing? What's going to happen when somebody needs to actually take a call?
That's where I stand on that one.
Well, El, thanks for taking the time. Thank you so much, Isaac, for having me. This has been great.
If you enjoyed today's episode and you want to hear more of the Tech Report, please consider liking and subscribing. Also, you can get episodes of the Tech Report wherever you get your podcasts.