Transcription
Many businesses looking to deploy AI are still concerned about errors and hallucinations, but are their fears justified? On this episode of Today in Tech, we're going to talk about whether hallucinations can be good for a company and whether businesses are getting skeptical about [Music] errors.
Hi everybody, welcome to Today in Tech. I'm Keith Shaw. Joining me on the show today is Ryan Welsh. He is the Field CTO for Generative AI at Click. Welcome to the show, Ryan.
Uh, pleasure, pleasure to be here. Excited to talk about this topic. So, what I've been seeing lately is that there hasn't been a lot of, of, of high-profile news, probably within the last three or four weeks, around AI hallucinations and AI errors. Uh, we have seen a lot in the last year and a half, obviously, as generative AI has come out. But when you look at the space, uh, how do you break down the market when it comes to the level of concerns that either companies or your clients have about, you know, hallucinations, errors, and are they the same thing, or, or can they be broken down, um, that way as well? Because not all hallucinations are equal, right?
Yes, exactly. So, we sell software to businesses, and businesses are incredibly concerned about hallucinations. Um, it's okay if you're on a weekend mucking around with ChatGPT as a consumer and it gets something wrong. As a business, when your employees get knowingly false information, or your customers get knowingly false information, right, that's, that's, that's, that's not okay. And so, it is top of mind for every executive that I speak with at, at businesses today.
Is it basically because they don't want to be the, the company that's in the headlines? And we're going to discuss some of the, the, the incidents that we've seen. Um, is it just because of that, or I mean, I, I guess they're also concerned about accuracy, and as a business, you probably want to be concerned that way?
Uh, yeah, it's, it's, it's part, it's part headlines, but then it's, it's part, like, actually making business decisions. And so, making business decisions with the wrong information leads to loss of money. And so, it's, it's just, it's just, you know, those, those just basic concerns of, of your systems being right and your people making decisions based on good information.
Okay. Uh, yeah, yeah. So, but the way that I've also heard it described is, uh, you know, there's, because AI can give you accurate responses when done correctly, but then it also gives you creative decision, you know, creative capability, like that, you know, create a poem from text, or write song lyrics. And now we're seeing images and audio and video, uh, that can be created from a text prompt.
Um, so there is a creative side to it. And, uh, so there's always that balance. And, and I think that maybe people don't care about a hallucination if I ask it to draw a picture of the, the puppet Alf from the 1980s TV show, and it gets it completely wrong. I've still yet to see one that can do a good version of that.
Um, then I'm like, okay, well, maybe it, it isn't ready yet, or, haha, look, look what it drew. Um, but I'm obviously, if I, if I need an answer of like a historical, uh, or a prediction or math, like I want that answer to be correct.
Yeah, it, it does, it does depend on the use case. So, the, for the creative use cases that you're talking about, um, hallucinations are, are a feature, not a bug. Um, and so, uh, creating marketing content, creating maybe sales emails where, you know, you kind of helped the sales development rep come up with some unique email that they can then look at and send on, like, that is absolutely where you want, I'll call it, you know, hallucinations or, or creativity to kind of like manifest themselves. But what, but what you don't want is like knowingly false information. So, if I'm, you know, sending, uh, an email to a client, and it then says that this client graduated with a PhD from Ohio State University, and they didn't, they went to Michigan, right? Right. You don't want to like send, send that email, send that email on, particularly with those two universities.
Exactly, exactly. Uh, yeah. So, I, I want to get to the creativity part later because I, there is, there is a key component, uh, that I want to talk about with the creativity. But let's dig into the, the accuracy side of, of the, uh, equation here. For companies that are looking to become better accurate, what are some of the methods they're using now? Are, you know, I've heard a lot about RAG, um, and, and taking an LLM, a large language model, and then integrating a company's own data so that they know it's accurate, and then making sure that the, the system doesn't leak that data. Um, is that one of the methods, or are there some other things that, that, that you're seeing?
That is one method, and there is like an emerging LLM app stack that is, that is coming into play here. And so, coming, first half of 2023, everyone wanted to just buy a large language model. Then they realized all these limitations. And so, now there's like an entire ecosystem that, that's emerging. One of them is part of that is that contextual enterprise data that, that you talk about. And so, you've seen the rise of vector databases or vector stores or vector search capabilities where you can turn enterprise data into vectors, store it in a vector, which is an ideal representation for putting into the, the large language model.
So, that's one area where people are starting to ground the large language model in enterprise data. And then there's a lot of tooling that's popping up for checking the answers as they come out of a, of a large language model. And so, even if you put, um, grounded enterprise data into a large language model, the large language model still may pull from its generic knowledge from looking at the web to create, um, knowingly false information in response. And so, you have to check as the answer comes out. And so, there's, there's fact-checking tooling that's popping up, there's guardrails that, that, that are popping up. Fact-checking would, as the answer comes out, looking back, back at a piece of information that is knowingly true, it is, it is actual fact, it is factually correct to just confirm that the answer that the LLM is generating is, is correct.
Then there may be some guard, guardrails that you want to put in place where it's just kind of like, hey, the LLM can't answer these type of questions, or has to answer certain questions in, in a, in, in a certain way. And that's just kind of like overriding the LLM system.
Yeah, are, are these fact-checking tools, are they, are they automated as well?
Or, yeah. So, so those are automated. And, and there's probably a certain level of, of still checking that the fact-checking has to do, right?
What, yeah, then you create this, this loop of constant, constant fact-checking. And so, so yes. And so, um, you know, one of the things that, that, that we've seen, um, be successful is if you can match some of the output text coming out of a large language model to the underlying contextual enterprise data that, that you put in. And so, if you know that, hey, um, the substance of this answer being generated from the LLM matches, you know, 80% of the substance or 90% of the substance of the input data, okay, this is great. But then if it gets down to 50%, well, maybe 50% of the answer coming out of LLM is, is made up.
And, and I would say the challenges with these systems is that then that adds to response times, right? And so, LLM response times are notoriously slow, given the, the computational nature of these systems. And so, if you had another step in of fact-checking, that just kind of slows the user experience further. But, right, kind of is what it is. And, and, you know, we've had a lot of guests on the show that, that continually use that term, you know, the term of human in the loop, and making sure that a human is still reviewing these answers. If you're trying to do that in a live, real-time kind of chatbot thing, then, then the whole AI thing, the, the purpose seems to go out the window, right? Like, why, why would you do that when I've got a human checking the answers anyway?
Um, this human, when they say human in the loop, they mean before you deploy something, right? Is that what people are saying?
Yes. Well, it depends on where you're putting the human in, in the loop. And so, you know, there, there may be, there may be aspects of, say, you're creating this, um, you're creating the knowledge space of facts to then check answers as they come out of the LLM. There's a human in the loop there to create that known knowledge base of, of facts. But then once it's deployed, that, that those people aren't, aren't there anymore.
Yeah. Then there's also human in the loop where it's like, hey, this, the system is, say, a customer support agent, as an example. I'm asking a natural language question of, the response is coming back to me. It's passing through the human before it goes to, to another human to then convey the answer from the LLM, so that the customer support agent can actually use their expertise and knowledge to actually then communicate that to, to an end user, right? So, even, even, even if the human in the loop is, is used to test the system before it deploys, there's still not 100% guarantee that your AI system is not going to burp up something, I guess.
That's correct. Yeah. Yeah. Let me, let me be, be clear about this. If you had 100% perfect factual data going into an LLM to train the LLM, it will still hallucinate.
Yeah. And that is, that is the nature of these systems. I mean, these are probabilistic systems. These are kind of like stochastic reconstruction systems. And so, if you had the most pristine data on the planet, 100% correct, it'll still hallucinate.
Uh, um, and do we know why that, that it's still, that it's still like this? Like, is it, is it just because the math, like, someone didn't carry the one?
No, it's, it's because we're trying to generalize language. And so, we're trying to create these systems that can generalize across multiple tasks, and in this instance, create human text. And so, to do that, you need to, um, have some probabilistic measure of what the next word's going to be. And it has to be a non-zero probability for that, for that next word. And so, because we're trying to generalize, we introduce the aspect of potential, um, creativity and going down different paths. And so, it's a, again, depending on the use case, it's a bit of a feature and bug in some instances. But we're trying to use these large language models for all types of use cases, whereas historically in machine learning, we would train a very targeted model for a specific use case. Whereas in this case, as I was saying, we're just letting it do all the use cases.
Right. So, I want to go through some of the higher profile ones that we've seen lately. Um, and, and since I've got you on, on, on the show here, I want you to answer if what you think may have happened or how it could have been prevented. Um, and if you don't know the answer, you know, just say, I don't, I don't know what the actual thing here was. But, first of all, was the Air, the Air Canada chatbot? This came out of a lawsuit where the chatbot was telling the, I think that the, the customer was, um, trying to get a bereavement discount for the airline, and the chatbot told him, yes, you can do it. But once you get back, and the policy that the airline had set up was, no, you have to get this before you take the flight. But somehow, that the chatbot said it was okay. So, what, what do you think went wrong in, in that case?
This is, this is a good one because it sounds like the airline put in place some initial steps to prevent hallucinations. And so, as I, if, if I recall correctly, um, when the answer came back, there was actually a link to the underlying documentation that the person could have looked at to see the real policy. And so, they were doing retrieval-augmented generation, where they were retrieving the relevant snippets of text or documents, feeding it into a large language model, and having the language model generate the response to the, to the user. But as happens with RAG, sometimes the language model taps into the generic knowledge or general knowledge that, that it learned from the internet. And that's what happened in, in this case, where the system now made up, um, a response that was in conflict with the actual policy. And so, this is the step where I talk about, you have to check the output of the LLM back against the underlying data to make sure that it's not tapping into, to knowledge it's, that step that seems to have been, been, been missing here.
Is it possible in something like this that, that when they were testing the chatbot, they, they didn't input, put a text prompt for the chatbot that was like, hey, what, how do I, what is the, the bereavement policy of the airline? And so, it didn't necessarily know? Well, like, the airline didn't know how it was going to respond until it actually was deployed? Could that, in a, in a situation like that, an example, generic prompt that you're going to put in is, you know, you're a customer support agent, um, you can only answer, um, the user's question based on the input documentation. So, you're trying to constrain the LLM to say, don't tap into my generic knowledge, only use the input text. But even in those instances, sometimes the LLM taps into its generic knowledge. And so, you wouldn't, you wouldn't have a specific bereavement prompt in there. You would just have a, okay, um, here's the question. It was a bereavement question. So, you'd add the user's question, and then you'd have a generic prompt that you would use, you know, again, you're taking on the persona of a customer support agent, you can only answer the, the user's question based on the input documentation. But even in those instances, they still tap into their, their general knowledge.
That just made me think of another question. I'm sorry for the tangent, but why have we not seen an AI or chatbots that just basically go, "Eh, I don't know"? Like, you never really see that. Like, I see like, oh, uh, "I am unclear," or "I don't know the answer." No, no. Or like, "I'm not allowed to answer that question." That's where you see the guardrails. "I don't have enough information." Like, you never actually just see one, "I don't know. Ask again." Or, or like a magic eight ball type of a thing where it's like, yeah, try again later.
Well, these, these systems are doing their best to please you. I know. I know. That's, that's the other thing. They're trying so hard to be your friend. And they're text generation machines, right? And so, like, you give it a prompt, and it's sitting there going, "I need to generate text." And so, it's, it's, you know, they, they just really, really want to please you as a user. And we have seen, you know, with good prompts and good input text, we have seen systems just say, you know, again, this is a bit generic language and annoying, you know, the, the context doesn't provide an answer to that, to that question, as opposed to, to, "I don't know."
Yeah, like, we accept that with humans. If I ask you a question that you don't know, you don't do your best to try to, you know, make me happy about the answer. You could just like, well, I don't know. I'm not familiar with that. Or, you know, what's funny is, you actually respect humans more than say, "I don't know."
Right, right. That was, that was one of the first things I learned in, in media training. They said to me, "Don't try to fake anything. If you don't know the answer, just say, 'That's not my area of expertise.'" And then they go on to the next question.
All right. So, uh, the, the other, the second story I want to bring up was the, uh, the, the famous, or infamous, lawyer who submitted a fake legal brief for a lawsuit. And it turns out that the chatbot, this was really early on, that the chatbot had made up a case. And so, is this a little different from what, what the hallucination was, was in the Air Canada case?
Yeah, so this is, this is classic definition of hallucination. Okay. And so, the lawyer was interacting with ChatGPT to create a legal brief. It came back with a bunch of stuff. The lawyer had no idea that LLMs hallucinate. Copy, paste, submit to, submit to court. Um, and so, this is just kind of the classic first case of hallucinations. And then the Air Canada one, that we actually just referenced, likely was like, "Oh, well, we learned that you can't just interact with an LLM directly. Right? Let's bring in retrieval-augmented generation." And so, this, this legal, um, this legal one is just the classic case of an early user, yeah, getting a hallucination. And this, and this was a case of the guy just didn't check the work. Like, you know, after it spit out, it was probably just like you said, a copy-paste thing. Which we, we saw from a lot of students as well. They were like, "Oh, well, this is the answer." And then there was something in the text that could, you could always tell that it was generated by, you know, an AI, because they would put like, "Hey, this answer is generated by AI." And students aren't checking. And, and you would hope, you would, I mean, maybe not, I'm not a lawyer, but you would hope like, if, if the person has experience in the field, and this is a, a case or a, um, a certain type of case that he's argued before, like he'd be like, "Oh, I've never heard that. Never heard of that case before." But, uh, this, yeah, you would want to go look it up, or at least try to figure out, you know, what case is he saying here?
Uh, exactly. And then the third one, we was, was a, um, there was someone who bought a car for a dollar through a, a chatbot car dealership. Now, I don't know if this was, this was actually, I, I read the story, this was a while ago, and I don't know if they actually honored the, the dollar purchase, or if they were trying to, it was like a media site that was trying to test the chatbot.
And then they got responses for, uh, this is a classic jailbreak. This, this is a classic jailbreak where, um, the LLMs are a bit Gable. And so, as I understand it, there was, you know, someone mucking around with a chatbot, saw that it said, "Hey, this is powered by ChatGPT," or one of the, one of the large language models out there, then knew right away, "Oh, I can, I can mess around with this thing." And then posted online, and then a lot of people started messing around with it. And so, in this instance, someone's saying, "Hey, um, you LLM, you're taking on the persona of, of this. Your response must, must be this." And you can actually start to trick these LLMs into kind of staying within those, um, confines that, that you put them in, or the personas that you put, put them in, or their responses. And so, in this instance, I think the person said, like, "Your response has to be, you know, $1 and no takesy backsies," or something, something like that, which is just hilarious, um, prompt. But this is, this is the case of, of users doing what's called a jailbreak, where you actually kind of, uh, manipulate the, the LLM to give you specific responses. And that's another thing that companies need to be aware about and be ready for if they're trying to deploy an AI, is that there's, with an understanding that, that humans will, humans being humans, uh, will do whatever they can to try to break your system, whether it's a security thing, or whether it's just to try to get your chatbot to say something goofy or give them a deal for a car for a dollar.
Um, so, is that, so that's another part of the pre-testing that needs to happen, right?
Yeah, pre, pre-testing. You're never going to get all of these, these cases. And so, ideally, you can capture a bunch of cases where you want to not allow the large language model to respond. Like, in this instance, you know, could you put guardrails in place where it's like, anytime someone tries to purchase a car, you can't purchase a car via the chatbot? Right? And so, just anytime someone says, "I'm gonna buy, you know, make me an offer to purchase this car for X," boom, it gets triggered. Like, in the sense of like, hey, um, "I'm not allowed to, um, come to an agreement and, and sell a car online. Please call the office," or something like that. So, ideally, you can kind of come up with these, you know, cases where these are the high-impact cases that if you got them wrong, could, could ruin your business or, you know, have a significant loss, where you put guardrails in place to then stop that from happening. And then, uh, being diverted to a, to a human being.
Is that going to be one of those wave of new jobs that people will have at a company? Is, you know, similar to penetration testing, if security companies, you know, you're going to be like, "Should I change my? If I'm really good at this, I should be, you know, like, hey, hire me as a person that I'll come up with the, the crazy ass, uh, jailbreak scenarios."
Maybe. Yes, absolutely. And there's a lot of companies popping up to do that. Yeah. Yeah. Whether it's, whether it's like prompt injections or jailbreaking. So, there's a lot of tooling that's popping up. There's a lot of companies getting funded here in Silicon Valley for this exact style of stuff. And, and, you know, what the OpenAI's and other large language model vendors of the world would say is, you know, "Do we just steamroll those companies because these models are going to get better and better and better anyway?" And so, they could be temporary companies. But as these large language models evolve, ideally, they can be, you know, manipulated less, right? And so, these companies won't, won't exist. That's an ongoing debate here in Silicon Valley.
Yeah, there's, there's another, there's another bunch of news that came out recently around, uh, this concept of the whole "garbage in, garbage out" phrase that, that, that a lot of, programmers and, and coders have known for years. But we're starting to see stories around the amount of data that, uh, a generative AI system needs, and the, or the LLM needs, has gotten so large that all of the data that we've actually produced on the internet, at least all the, the legal ones, there's still copyright issues around that, but yeah, is not enough. Like, this, this data isn't enough. And so, then we start seeing companies that generate artificial data for the use of, of some of the, uh, these AI projects. But that always feels to me like it's a garbage in, garbage out. If you start adding generative AI stuff, or, or artificially produced data into your, your data set, that that's just going to start, it's like making a photocopy of a photocopy of a photocopy at some point.
Yeah. And so, it's funny because I always tell people too, that if you want to, if you want to generate, if you want to destroy a, not destroy, but if you want to break an AI, start putting out gibberish on the internet, whether it's a Reddit post or just, you know, write a blog and have everything be wrong in it, and then, you know, see if it gets picked up by the AI. And, um, I'm not sure that's a one-person job. But is this idea of generative, of garbage in, garbage out, something that will produce more errors down the line?
It, garbage in, garbage out still holds. And the, what is, is really interesting from a research perspective in large language models lately is how much people are really starting to understand the quality of the, of, of the content. And so, the company Mistral, um, out of, out of France, I would say one of their big contributions to large language model creation was to remind everyone that the quality of content, um, is really, really important. And so, they were able to build smaller models, um, that performed as well as larger models. And their real know-how was the content curation. I mean, look, these guys are geniuses at actually building large language models from scratch. But then, like, in addition to that, they were really good at curating content from the web. And so, you get, if you get high value, high, um, value content, and you use that to train a large language model, then you can actually get that large language model to be smaller from a, uh, parameter perspective, and be as good performance as the, the larger ones.
Larger models who have to take in more data because there's more noise compared to the signal, right? You're trying to overcome all that bad data out there.
Yeah, by, by, by shelving a bunch of, of data in to get, get performance. And so, I mean, garbage out, or garbage in, garbage out, you know, holds. And candidly, I think will, will always hold.
Well, especially if they, if they start taking transcripts from YouTube videos and, you know, and, or if, you know, and even if they're using this video, for example, as, as the training model, "2 + 2 is five. 2 + 2 is five." That's truth. That's the, the accurate answer. Uh, just in case, in case they're using this.
I mean, this has been a, if, if you don't mind me actually jumping in, I mean, this has been a concern in, in, in AI for, for a while. Uh, uh, I mean, particularly if you now start getting into like, um, uh, corporate security, enterprise security, nation-state security, where, you know, other bad actors could inject bad data into, to models to get it to do a specific thing. And you thought you were training a model on good data, when it turns out to be bad data, manipulated data. Now you're doing bad predictions or bad responses. And so, that's actually an, an attack vector that, that enterprises need to kind of consider.
Yeah, I mean, if I was in enterprise data security, or, or if I was in, uh, national security, I would try to, at this point, I would stay away completely from a lot of this generative AI stuff. But, um, there is, there's probably some value in AI for some decision-making thing. But even then, I don't know if I would trust it completely. Totally.
Yeah, yeah. Um, I wanted, I want to ask another question about the speed at which a lot of these systems have been deployed because again, this, you know, it was such a fast, hyped technology, and everyone was was jumping on this bandwagon. But before that, if we go, you know, back a few years, the big thing was data science, data analytics. It was always like, your, your data scientist is going to be making $300,000 a year. This is, you know, there's a lack of data scientists out there. So, we saw all of these companies in the enterprise space that went to, uh, you know, it was all about data science. And so, what I'm wondering is, whether all of those companies that did that route, whether they completed these projects in time for the next big thing to happen, and suddenly they're like, "Oh, we have to do generative AI now." And there is a link. And, you know, there is a, but it always makes me feel of, here's my analogy. Is like, if you put a cake into the oven, and someone's like, "Well, we need the oven to, to make the cookies now," or "to, we got to put the turkey. We're not having Thanksgiving dinner." And then you take the cake out, and it's not fully baked. How many of these data science projects were not fully baked when you saw the jump over to generative AI? And maybe I'm just assuming that there was some, maybe, maybe they all finished the cake.
Yeah. No, I, I think the data science projects, uh, weren't completely done. So, I, I thought data science was like crossing the T's and starting to really get mainstream adoption. But then GenAI happened. And I, I don't think you need to stop those, those projects because you're, you're trying to get, there's different use cases that you're going to use data science and supervised machine learning models for, okay, then you would use a large language model for. So, an example being, maybe fraud at a, at a bank. There's really great machine learning models that you can, that you can build to detect fraud. You're not going to use a large language model to, to detect fraud. And, so, you should continue to pursue all of the data science projects, you know, years-long data science infrastructure builds, if you were building them as, as a large enterprise, then you should also start to layer in generative machine learning to open up a bunch of different use cases, because there's use cases that you would not have used the data science platform for. Natural language question answering, as, as an example. It's unlikely that anyone was building a generative question answering model for customer support using their data science platform, right? Is just, it's just a different system. But they were using those things to do, uh, machine learning on structured numerical data, different types of data, and things like that. And the purpose of the data is, is different. And so, like, if you kind of look at when people talked about data previously, it was all about getting the data and then labeling the data to train a supervised machine learning model. Now, getting access to data is not about getting access to, to enterprise data to label it to train a supervised machine learning model, right? It's getting access to enterprise data to then feed it into an LLM. You don't actually need to label it. You're just actually feeding the raw snippets of text into an, an LLM to actually have it generate a response. And so, the purposes are, okay, different of those two technologies.
Yeah, but, but because the spotlight did change from these machine learning data analytics projects to generative AI, I'm wondering if, if the ML people, or the, you know, if they start feeling like the middle child, like a Jan Brady type thing, where like, you're, you're always giving attention to the new kid, or the new, you know, Marsha, Marsha, Marsha. I'm sorry, Ryan, I'm going to throw pop culture references during this interview. But do you think that they feel like, "Oh, you know, we've been working so hard on this on this machine learning project, and now, you know, generative AI is getting all of the the spotlight?"
Yes. Okay. Yeah. Absolutely. And, and having built generative AI systems for the last five years, I, uh, I'm happy to be in this position where all the ML folks and, you know, maybe it's the AutoML folks, which was a popular category for the last several years, are now going, "Yeah, that, that darn GenAI guy comes through and is the, you know, the bell of the ball." Right? And I think about what's, what's, what's going to come next. And so, you bring up an important point. And so, I answered your question from a, just like a pure technology and use case perspective. I think you imposed a really important constraint on what I was, was, was saying was the people aspect of this, right? And so, um, if you wanted someone to build a, a fraud, uh, model, like that was a smart data scientist, um, building a fraud model, um, that person may now be more interested in generative AI use cases. And so, you don't have them now working on fraud models because they believe their next career move is, is generative AI. And that's really the, the growth area. And so, I think you imposed a really important constraint on, on, on, on your question was the actual number of people that can actually do these things. So, while it's great that supervised machine learning can be applied to different use cases and generative AI can be applied to different use cases, um, are the data science people now learning generative AI? And I would say, I would say yes, because it's, it's cool, it's interesting, and you can do, you know, really, really cool stuff with it that you couldn't do previously. So, I think you imposed, as I said, a really important constraint on, on the question.
So, you know, in, in, have discussed, uh, the amount of data that the companies have, and whether they're using it for, you know, supervised machine learning, or if they're using it for generative AI, can a company ever feel confident in knowing, you know, that their data is going to be good enough for an AI project for either one? How do they, how do they know to? Because again, you can filter and filter and filter, and then maybe the data scientist or whoever you've got working on GenAI could go, "It's all good now." You know? Yeah. Is there a point where they can do that?
Yeah. Go ahead. I'm, I'm laughing because like, the cleanest data sets that we've ever operated on within an enterprise are still filthy and still, and still, and still, still inconsistent. I, I would say again, um, you know, data for, for, what, for what purpose? There's the data for actually, like, then labeling, labeling it and training a, a, a machine learning supervised machine learning model. And then there's the data that you would want to feed into a generative AI system. And I'll actually, like, touch on that. That's where I've really seen, um, that's where I've worked in and have, have the most experience. Is we've seen that it's really important to, um, have that data be consistent and, and up to date, which is, which is hard to actually determine. It helps if you can kind of narrow the use case down. And so, an example being, maybe I'll bring up customer support again. And so, you may have a content management system that you use to expose to your external customers to ask questions about your product and service. That data is managed by someone, head of customer support. It's relatively up to date. Um, it's constrained to a specific topic. Yeah. And you should be able to then feed that into an LLM as a part of a RAG system and, and have good interactions with that, that data. As you obviously go larger, say, um, "I'm going to deploy a RAG system across an entire enterprise that has 300,000 employees, and they can ask a question of any data set anywhere in the company." Like, that's where it gets, it gets, it gets much, much harder.
Actually, that reminded me of a story when I was, I was interviewing this really smart PhD guy who was working, he was working for Toyota, and he was working on autonomous vehicles. And now that's super complicated, you know, AI type training. This was way before generative AI stuff. And I, he, he got up at a, at a thing where I was talking with him, and he said, that his biggest challenge and his biggest problem was that the team members that, that he was working with, he would always ask them, "Are we ready to go? Are we ready to go? Do you have enough data?" And they're like, "Nope. We never, we never have enough data. There's always more data that needs to be done." And he was, he was getting frustrated with like, "Well, at what point can I release this autonomous vehicle system to the next level, or at least get it to to deployed rather than out of that research space?" So, I would imagine that in the same way, you know, a lot of data people are like, "There's never enough data. You, you could always use more," to make the system, you know, that 0.4% more accurate or whatever.
Yeah. Yeah. So, you need, you need to measure the, the amount of work that's going, it's going to take to actually get those, those last bits. I mean, as, as, you know, you know, those, people are say, "Oh, well, I was able to get to 80%, and it's, I should be able to get to 90% pretty easily." It's kind of like, whoa, you know, getting to 80% was, was easy. Getting to 90% is going to be really hard. Getting to 95% is just going to be significantly harder than getting from 80 to 90, 90%. So, you just need to kind of weigh the, you know, what are the benefits of, of having that extra percent?
Yeah. Does that, does that mean companies have to do a, a risk assessment for what percentage accuracy will be acceptable within the, the realms of whatever you're producing? Because you're never going to be 100% accurate, right?
Absolutely, absolutely. Yeah. You, you need to, you need to do that assessment. And you need to, you know, like, we actually go back to the Air Canada case, which I think is interesting to think about from a business perspective and say, um, that's just, that's just the cost of doing business. Like, I, I don't know what the payout was to this gentleman. Maybe it was just actually like the bereavement policy being being reimbursed, or something like that. Maybe plus a little, little lawyer fees or something like that. But, hey, that's, that's the cost of doing business. Like, "We saved $30 million by servicing our global customers better, and we had to pay this guy $10,000." Great. You know, just kind of, just kind of, just kind of is what it is. Also, take the hit of they to take the, the bad publicity hit, too. And that's, and, you know, sometimes you don't put dollar amounts on those anyway.
Exactly. So, I've always been the feeling that the speed of development of of new large language models, um, and advances in the, the AI technologies and the tools, I feel like it's been too fast. You know, has it been? I feel like it's been fast. When you talk to people in the space, has that development been too fast, too slow, or just right for most large businesses that are looking to adopt it? Because, um, there was an interesting thing that you told me before the show was that most businesses are fine with the current speed, or or actually, it's probably be nice. I don't know. You, where do the businesses sit in terms of this development?
As, as someone that, that is, that was, has been in the AI space for a decade now and has, has built these, these systems, um, it was blow-your-hair-back fast. Like, in, in early 2023, like, it was just wild. Like, every new day came with something that, that you never would have thought before, and just a, a significant step forward. But I think what that did was that it set us up for success in, in the future. And, and specifically, people started to think about, "How do I, um, um, have a plug-in-play system where I can plug in any large language model, and whichever large language model performs best on a certain task, maybe at the lowest, lowest cost?" And so, from an architectural perspective, because the world was moving so fast, people said, "Oh, we need to architect our systems to plug in, uh, OpenAI, or Anthropic, you know, name your large, large language model to, to plug in here." Which I think was the right architectural decision. But what's also interesting is just the, how, how slow, I'll call it, um, approval for large language models is getting done within an enterprise. And so, um, everyone's kind of racing to build the next great large language model. But from what I'm hearing from enterprise customers, and these are large enterprise customers, over a billion in, in revenue typically, that it takes them about a year to actually get that large language model approved by legal to be, to be used. And so, it goes through this long process. And so, while we may be on, you know, GPT-5 coming up, yeah, people are still operating on GPT-3.5, right? Because, because it's going to take, you know, a year after GPT-5 is released for it to be approved and then actually put into, put into production. So, I, I think we're moving at the, at the appropriate space because it kind of drives decision-making. I think the appropriate decision-making. Yeah. But we're also kind of lagged a little bit, just by some of the, the compliance issues that we see around large language models.
I was asking a bunch of other AI companies, uh, whether, uh, being LLM-agnostic is table stakes now for a company. If you're trying to produce an AI tool for a company, that most companies will probably ask, uh, to be, to have options for multiple LLMs. And the only example that the guy gave me was, well, if, if they're, if they're so locked into an infrastructure, like if they're so locked into Google, or if they're so locked into, uh, Amazon, or whatever, like, then you'll have to pick what they're using.
Yeah. Well, I think what's interesting, I think what's interesting there is you're now seeing those hyperscalers diversify their LLMs. So, if you look at Amazon Bedrock, you can access OpenAI, you can access, I think Cohere or Anthropic, um, and other, other models. But then you see Microsoft with their OpenAI relationship, then actually broker a deal with, with Mistral out of, out of, out of Paris. And so, you're getting a bunch of large language models there. I'm not as up to speed on, on the, the Google access. You should just skip Google altogether. That's a little aside from me. But, but, but, but, but even, even the hyperscalers, even with, you know, heck, you got, um, you know, Microsoft putting $11 billion in its OpenAI, but still, but still having Mistral and other models on their, on their platform. And so, it's, it's just interesting how this, this space is evolving.
I want to bring up another thing that you had mentioned to me before the show, and that was that a lot of companies, I, I was surprised to hear this, that initially they wanted to build their own LLMs. And, and, and now that's not the case, right? Like, because they discovered a lot of problems with trying to do this themselves. One of the most interesting things I've heard over the last year, I, I think was was my bet a year ago would have been, "Every Fortune 500 globally is going to have their own proprietary large language model at some point in the next three to five years." And I think, I think that's why Databricks bought MosaicML for $1.3 billion to enable enterprise customers to to build those large language models to get around a lot of the, um, security and compliance concerns that you have with large language models. But then I heard several large enterprises say that they don't want to build their large, their own large language models, or even fine-tune their own LLMs, because they then believe that they would then become an LLM company. Okay? And that's the last thing they wanted to do, given all the scrutiny that the LLM companies are getting from, from, from government, right? And so, they've said that they've kind of happy to have OpenAI, Anthropic, actually take on the risks of being an LLM company as opposed to, you know, say, "I'm an, I'm an investment bank. I already have scrutiny because I'm an investment bank. Um, I don't want scrutiny because I'm now an LLM company too." Yeah. And so, I just thought that was a really interesting insight that that folks are saying, "We'll happily actually punt on this, and push it over to the LLM vendors until there's maybe more clarity around, you know, what is the actual risk that they're taking on?" But there's also some issues around, um, content moderation that, that some of these LLM makers have, where I think you were telling me, OpenAI, you have, they have 30 days to review the, the data. And there's like, you know, if, if you're a company and you've got a lot of data that you want to keep secret, that seems like a, a, that's like a big red flag. Like, I don't want another company to have access to this data, even if it's for 30 days. So, so what?
Does a company do in that case where if they want to protect their data but they don't want to develop their own LLM? Yeah, that's that's exactly the issue that that this uh. So the reason why I thought everyone would build their own LLM was privacy, that that that that you're bringing up. And it was so counterintuitive to hear people people say that. And so I think I think what what is what is really great about this is like how the market is evolving so quickly. And so, um, uh, Databricks again wants to build private LLMs for for enterprises. If they get that right, then those enterprises won't go to the hyperscalers to access LLMs. And so what the hyperscalers have to do is they now have to work with the enterprises and say, "Hey, what security and compliance do you want for you to put your data here?" And they need to move faster than Databricks, yeah, or else Databricks is going to is going to take the market.
And so, um, one thing that I've seen consistently in this wave of AI compared to say the 2012 to 2022 wave of of supervised machine learning is I consistently hear enterprises saying, "We're being thoughtful. You know, we're we're we're really thinking about the space." Um, which is interesting because like that implies last time they weren't they weren't thoughtful, they were just kind of running in a bunch of different directions, getting a bunch of stuff wrong, right? But I guess the point the point that I'm bringing up is that they are a little slower to adopt because they're starting to say, "Hey, what are what are the concerns?" Concerns. And they're sticking to their guns on the concerns, and they're not saying, "I'm going to risk, you know, enterprise security or compliance or anything like that." And so, um, they're letting the the vendors um come to them and add all the requirements that that they need. And I think those vendors are moving super quick. It's such a a large a large market.
Okay, a couple final questions. Um, I brought this up before when we were talking about the crea, you know, the creativity aspect of of AI, um, where a hallucination on the creative side might be a good thing. Um, and I've seen a couple of articles that have suggested that hallucinations don't even really matter and that it can be a good thing. Um, can you can you expand on that? Like from what you've seen, do you see a lot of of either scientists or other people say saying, "Yeah, don't worry about the hallucination, it'll be fine." And again, maybe not talking about the accuracy side of things, um, but, you know, maybe that this can be a good way of, we used to call this in the human side, U, you know, "out of the box thinking." Could you imagine if if instead of like thinking outside of the box, people were like, "Well, you're just having a hallucination"?
Yeah, I I think I think what's what's interesting that that um I kind of think back to now the explainability conversations of several years ago where you had a lot of luminaries in the of deep learning because deep learning is a black box say, "Hey, explainability isn't important." I think what everyone has has learned as AI gets out into the world said explainability is pretty important. I want to know where answers are coming from. I know I want to know why you're making making these decisions. And I think we're having that that same conversation around hallucinations. And again, it does come down to the to the to the use case. Is what am I trying to do here? Am I trying to be creative? And if we kind of draw a parallel to a type of meeting, you know, if you walk into a meeting and it's a brainstorming session, great, we announced that it's a brainstorming session so that we can all start to be creative and just come up with with new ideas. But if we walk into like a quarterly planning session, you know, and you're doing brainstorming, yeah, uh, or like or a quarterly review session rather, where you're reviewing the metrics, financial metrics from the previous quarter, like don't brainstorm in this meeting. These these are real numbers of whether or not we hit we hit our objectives. And so I think what's unique about humans is the fact that we can kind of turn that on, turn that off, at least most people can. Um, whereas with with computers, or at least with the large language models rather, um, we're not really turning them on, turning them off, they're just always always there. And so we need to be aware of them.
Is that why there's there there's three buttons down at the bottom of your ChatGPT thing was like more, you know, more more accurate, more creative, or balanced? And that's why, you know, does there need to be more buttons on that type of of system?
And I I think they're they're they're giving the appropriate amount of buttons, but but yes, that is why those buttons are there. So if you want if you want a more accurate or less hallucination system, like you're going to turn the temperature down to zero. If you want to be more creative, you're going to turn that that that temperature up. And so this is kind of that like stochastic nature of predicting the next word. You know, you don't want the system to always generate the most probable word to come next. You want it sometimes to choose one that's maybe in the second or third, uh, most most probable to come, which allows it to generalize across across the language and create and novel text. Yeah, but again, it really does come down to the use case. What are you using it for? And so the if you're good with hallucinations, those are the kind of brainstorming meetings that we're talking about or creative sessions to come up with stuff. Yeah, if you're not, if you're not, if it's a different kind of meeting, then you can't have the hallucinations.
Do do you feel like the explainability thing has gotten better from from companies or from from the AI models, um, than maybe you had before? Cuz it feels to me like it's when I, you know, when I ask my toddler, "Why did you steal that cookie?" and you get, "I don't know," or "I don't know why I did what I did, I just wanted the cookie."
Yeah, I uh, so from a from a actual technology perspective, no, it has not gotten better. Okay. From an industry perspective, it has it has gotten better. And and five years ago, uh, you would have Jeff Hinton, the, you know, Godfather of deep learning, say that explainability is irrelevant, it doesn't even matter. I think he's I think he's totally wrong, with all due respect to to a legend. Um, and at least at least now referencing back to an underlying document where the system generated the answer, that's table stakes, right? You know, like if you don't, if you don't do that, I'm not gonna buy, I'm not gonna buy your solution. And so that's at least, you know, explainability is a very big umbrella of which one of one of the kind of subcategories is something called provenance. And so in this instance, you're showing the provenance of where the answer was was generated from. And so I think we've made progress because vendors now, um, at least concede the point that some form of explainability is actually absolutely critical, um, whereas years ago, we didn't. But from a technology perspective, I we can't tell you any more than several years ago about why A system's actually generating a response, right?
So as you look at the future, Ryan, do you feel that the the the industry will get a handle on hallucinations and reducing them where they need to reduce them, but then also allowing for creativity in in certain cases? Is that where we're headed?
Yes. Yeah, it is it is where we're headed. So I I think as people learn how to use these systems, they're going to apply them in in appropriate places. And so I think people are getting the intuition for large language models and for probabilistic systems because they've been dealing with deterministic systems for for so long, right? I think what people are starting to realize is that if you have one component of an overall system that is probabilistic or stochastic, um, then the whole system is stochastic, right, right, right. Um, and so, okay, so now if a whole system is, well, then how do we apply this in the right way? How do we reduce the hallucinations in certain instances, or how do we let them flourish in other in others, depending on on the use cases? And all these reference architectures for different use cases are starting to to emerge. And I think it's, um, we'll get a handle on it and we'll apply them appropriately.
All right, uh, Ryan, I think we're we're out of time. Again, thanks for for joining us on the show today. Um, I think we're going to probably have you back when GPT 5 comes out, we can, uh, laugh at all of the mistakes that that's making initially. When that, that would be great.
Thanks for having me.
All right, uh, if that's all the time we have for today's episode, don't forget to like the video, subscribe to the channel, add any comments you have below. Join us every week for new episodes of Today in Tech. I'm Keith Shaw. Thanks for watching.
[Music]
Oh.
[Music]