Transcription
The practice of law is the paradigm information work. Just talking about things and facts and you go and you push a paper somewhere and something happens. And more, we can guide people toward principles and systems and ways of thinking about adoption based on strategy first and foremost. Is this delivering value to the client? In some ways, that's the only metric that matters.
[Music]
Welcome to Between the Briefs, a podcast by Steno. We're here to bring you practical tips, expert insights, and real conversations about the pre-trial process, court reporting, and the legal technology shaping the future of litigation. I'm your host, Adrien Seo, and I'm your host, Joe Stevens. Whether you're an attorney, paralegal, or just curious about how technology is changing the legal industry, we've got something for you. Each episode, we'll break down complex topics, share behind-the-scenes intel, and talk to the people leading innovation in and out of the courtroom. So, grab a coffee and let's get into what's happening between the briefs.
Hello and welcome to Between the Briefs, brought to you by Steno. We are your hosts. My name is Adrien Seya and I'm your co-host, Joe Stevens. In this episode, we are thrilled to welcome Todd Itame, director of artificial intelligence and e-discovery solutions at Covington and Burling. Todd is a leader in the application of AI and machine learning to legal practice, particularly in e-discovery and information governance. His expertise spans complex litigation, large-scale legal tech projects, and developing bespoke software solutions. With a career dedicated to blending law and technology, Todd has helped reshape the way e-discovery and AI are used in legal work. He's played a critical role in advising global clients on the integration of AI tools and software, making him a key figure in legal technology. Todd, welcome to the show. It's great to have you with us here today. How are you?
>> Good. Thank you. Great to be here. Great to talk to you all.
>> Of course, man. So, I want to take our audience right to the beginning. Tell me what got you really involved with with law and what really sparked the passion to lead you to where you are today?
>> Yeah. Yeah, it's an interesting question I ask myself sometimes. I think it was like an unfortunately high score in the LSAT led me down this road. You know, I was always kind of a computer nerd. Grew up in rural Idaho, like, you know, no stoplight in town, rural Idaho. Um, and I had two things to do, you know, rock and roll music and computers. And my folks bought us this, I actually have keep one of these on my desk, 486DX computer back in, I think it was 1990, and I would just, you know, broke it, took it apart, fixed it, and just have been a computer nerd ever since. Just a total tech geek. And then in college, started a business first out of my dorm room and then kind of grew from there to have an office space doing like B2B, you know, small and medium-sized business, like full-stack development, whether it was web technology or, you know, custom software, those sorts of things. And I don't know, I got in a position in 2010 to sell both businesses and thought, like, I've always wanted to be a lawyer. I'm going to go do that now and, you know, wordsmith and stand up in court and leave the world of technology behind. But yeah, that didn't last long, you know, staying away from it. And so, um, yeah, found myself back at, at actually this firm right here, you know, big giant Washington D.C. law firm, and with discovery databases and people saying, "Oh, we're having a problem." I'm like, "All right, what's the architecture?" You know, and just kind of got sucked in there to it all. And I've, you know, over the course of the years, have been, you know, doing kind of smaller coding or workflow design to automate various tasks for lawyers and for our clients and, you know, saving all sorts of money. But it's, I'm happy recently, like all of a sudden software is cool, you know, like kind of like the kid in the corner over in the high school, like, "You guys talking to me? You like it?" You know, now with all this energy around the AI, and it's kind of a different world. And in some ways, you know, surreal that all of a sudden the industry like really cares, you know, and there's so much focus on integration of these technologies. And it's really in an industry that has been, I don't know, 10, 15 years behind, you know, on kind of where they're at with software, where they're at with automation. But now there's the kind of opposite energy. And so it's great. It feels kind of like, you know, the ignored kicker on the high school football team that everybody's like, "Hey, soccer boy, go inside, just kick the winning field goal on the state championship," and you're showing up at school next, you know, kind of what it's like for computer nerds now at law firms. And so it's a good moment, I think, for legal tech people, and it will continue to be for at least, yeah, the foreseeable future. And so that's that's where I'm at. I'm not sad about that. That's the state of things.
>> Todd, we're going to dive into so many things that you just talked about at greater length. I have got to ask though, you have a Metallica record behind you, you've got a guitar behind you. How do you pivot from that world, which seems to be in some ways the polar opposite of what's going on inside a large law firm when they're coming to vetting like technology?
>> Yeah. No, I love, I love all, you know, rock and roll music. Music from the, you know, pretty much the 60s on. I love it. But it is, it is different, you know. I kind of, I kind of think it's hilarious, you know, like the lawyer or the accountant that like thinks they're heavy metal. Uh, but no, I don't know. I just, I, I love it. I think that, um, you know, my musicianship to enjoyment level is like way skewed, right? Like I'm a very bad musician. And I play the guitar, drums, and I'm like, a very okay, but I love that, you know, and it's something about, I think there's something with that that really maps on to legal tech in that like I have this, you know, desire to create. And there's a part of me, like as previously a graphic artist and, and, um, in music, you can constantly create and improvise. And there's a creative aspect of it. But at the same time, you know, music's very technical and it's all mathematical and harmonies and chords and it's just, you can figure it out with math. And we've seen that with, you know, these deep neural networks really doing a good job of reproducing songs and those sorts of things. And so it's an interesting kind of mixture of the creative and the technical coming together to put something that people can really connect with, you know. If people don't know why the hell, you know, I sometimes I listen to things and I'll listen to literally anything, you know, I'm not a music snob where it could be as poppy or if I like it, I like it. I'm going to go buy. I'm going to go to the show. I'm going to scream. And so it's kind of like that, you know, these things come together to create a product we can look at. I think, you know, legal tech is the same way. So in some ways, like, wow, esoteric, and we can get to a big talk about my personal opinions on the proper type of embeddings for a RAG model to practice. You know, if we're if it's for a legal workflow, but at the end of the day, I think, you know, lawyers and your average person can look at and go like, "Wow, this is really helpful." And can realize, huh, tech's like having a moment here. You know, it's creating something. It's where we've built new products. The software is fundamentally different now than it was three or four years ago. And something, there's something great about that. We're still trying to figure out like, okay, is it like Terminator 2 Judgment Day cool or, you know, or Star Trek Utopia cool? Like which one? We're not sure. Neither am I. But, you know, I just, there's something, there's some parallels there that both of them combine these, you know, highly kind of creative elements and technical elements to produce something that people can just consume.
>> Definitely. And you mentioned something there about how the world of technology and how we're seeing it today embedded with our workflows is much different than it was in the past.
>> Yeah.
>> One of those large elephants in the room for everyone is AI.
>> Yep.
>> How have you seen it really revolutionize e-discovery? How are you using it currently to make your your flow better when it comes to discovery?
>> Yeah. Yeah, I think, um, you know, the large language models and, you know, there's this whole debate whether or whether core capabilities are actually expanding or whether it's just we're integrating the models better or structuring it better. And, um, but it strikes me that if today, you know, in May of 2025, it's, it's over, development stopped, large language models at their core capabilities will not improve. I still see just a huge, you know, long tail of applications to the practice of law in so many ways that, you know, the current integrations are really blunt instruments of the technology and it's still insanely helpful. Um, and I think, you know, it's hard for me to think of a legal workflow or a legal task that is not aided by AI. Uh, I think, you know, information, in some ways, the practice of law is the paradigm information work. Is, you know, you're you're talking about things and facts and you go and you push a paper somewhere and something happens and you try to convince people the words on the paper. And these, this is information work at its finest, you know, like maybe like you could say, oh, accounting, maybe something like that, as a professional service that's a little bit even more in that direction. But it's hard for me to to decide what isn't. And so, people ask this all the time, like, what, how do we, you know, how do we use it? And I think, um, you know, there's a big piece of, there's, there's a disconnect between developers and users of software, uh, in general. And I think for lawyers and legal tech, there's more of a risk that that misalignment is going to take place, just because like lawyers are so far away from the technology at bottom. And so it makes it hard sometimes to see like, what do you, so what do we do with this again, you know, or like, you've given me this chatbot, like, now what? And so I try to tell people, like, okay, take the easiest kind of example of just proofreading. You know, if all, if all you're doing is taking the many emails or the many documents that you're writing and feeding into an LLM and saying, "Hey, can you just issue a spot for me, grammatical errors or typos?" Um, you know, that takes like five seconds to control, control copy, control paste, um, into your properly secured and, uh, with properly, you know, walled off for privacy reasons, chatbot, and get something back. And it's just, it will catch errors. And there's no lawyer in this country who cannot benefit from that on some level. And it will not improve the quality of your work product. And just that one kind of very simple use case for lawyers. And I think, you know, you can take it and expand from there and say, like, okay, what are the weaknesses in this brief? You know, what do you have any ideas, computer, about better arguments or what are we missing? You know, be critical of it. Okay, now agree with it. Now take on the role of a judge, you know, you can ask it all these things. And it takes two seconds to fire it off. You maybe have three different chat windows open. Uh, and you could tell right away, you know, if it's good stuff or not. You could tell right away. Are these ideas good? Do they actually catch typos? You know, is this, do I agree with their word-smithing choices? And I think, you know, um, whether or how quality, how accurate, how good the outputs are, well, first, they're better than nothing. They're better than the old way. No matter what, that's conclusive. Um, and if not, you know, it's acting as this creative medium, as prompting us with ideas, because more times than not, you, I'll ask it, even if it's a fairly technical thing, maybe I'll be coding. I'll say, "Do you have any optimization ideas here for this piece of code?" And it will tell me things. And sometimes I'm like, "Yeah, wow." You know, never in a million years could I mean, Claude is a far better coder than I am. Um, but sometimes I'll be like, "Yeah, I don't really think you're not considering X, Y, and Z things in the cloud architecture and you're not thinking about this, but you did give me an idea." Yeah, you did kind of make me think about this critically. And so I think a lot of it is creating that relationship or that intuition about how do we use these tools effectively and how do we use them in a way where we're not like, you know, driving our car following our car into the ocean on our AI map guidance in a sense, and instead using it responsibly and being able to kind of put it through this filter of like, how do we use these tools? I mean, it's an intuition, it's a skill. It's something that takes experience to do. But once you get there and understand how to use them, like, wow, you know, kind of sky's the limit. But even these basic use cases are easy. It's easy to see how they deliver value and certainly worth whatever the $25 a month, you know, you're paying for XF Frontier Model Enterprise Edition.
>> Yeah, Todd. I mean, it, I love the idea of, I mean, there is this whole theory that really good ideas and great innovation comes from like the ability for multiple ideas to sort of collide together, right? And when you have that instant feedback from your AI, that's what you're doing. You're bringing your mindset to, you're kind of treating it like a thought partner, and then you never know what you're going to get.
Okay. Given that, if you can take us inside to the degree to which you can, I mean, you're talking about a law firm. You're talking about an industry that, as you said, is decades behind in some ways tech-wise, and you've got this incredible creative force and this productivity enhancer and with its limitations. How are you internally sizing these products up? What are the main considerations that you're looking for before you can sort of run free or encourage people to to turn loose in their use?
>> Yeah. So, it's a really good question. And I think, um, you know, the more we can guide people toward principles and, um, you know, principles and systems and ways of thinking about adoption that are more based on like, what is our strategy? You know, what are the things we consider good about this product? And then building in an expectation that, guess what, the tool is going to change because capabilities are so rapidly changing. So I think the more we can push toward thinking about systems, thinking about standards, rather than rules, um, those are the ways we should be thinking about things as organizations. And so then it's like, okay, so what the hell does that mean? It means like, all right, well, you know, we could think of tool selection kind of more along the lines of life cycle management and putting in place and saying like, "Hey, this is our process. Every year, or maybe your organization doesn't move that fast, every 18 months, um, just like a piece of hardware that we'd refresh, we're going to re-look at these tools." And I think, you know, don't let perfect be the enemy of the good. And more general tools are fine. You know, if it's, if it's a choice between like sitting around and waiting six more months to find the exact perfect assistant or agent to do X, and just pulling something off the shelf that we can implement yesterday. Wow. Do the other one, because, you know, capabilities are changing so quickly with this stuff that it's, it's hard. You know, for some use cases, that's great. The specialized software for whatever X practice you're doing really matters. Um, for other, for kind of general use, it's like, well, you know, vanilla XF Frontier Model today is better than the best, most legally tuned old model two months ago, by far, at doing legal tasks. And so, you know, I kind of think of it like, all right, how I would approach it, um, within any organization is is kind of being like, all right, what are our kind of general use cases that everybody could can use, you know, um, whatever it is, the transactional folks or the litigation folks, or maybe you have a small in-house department or something. What, what could everybody use? What are things that, you know, we use in common? Okay, let's focus on that first. And then really just, um, try to get things going in a way you could try more than one tool at a time. You could try tools that aren't, you know, without confidential information, is kind of say, okay, we're going to go try these tools in a way that we might not have to vet it completely, you know, into our security process. And just creating that, you know, kind of a culture of testing things, of adopting things. And of course, you know, then there are all the things you have to check off the box, the D, the infos, uh, information security, the privacy, um, those basic things that in a lot of ways I think have been like really just overblown and overplayed by lawyers of, you know, the fear, uh, around those things. But they're of course very essential. But you can get them right. There is a way to do it. You know, there are answers there. And so that's kind of my general approach is, you know, think about this more like, okay, we're going to pick a tool for today. But wouldn't it be better if we could just sit around for a few minutes saying, "What is a structure that would work for our organization, uh, for how we are going to manage the life cycle of these tools and move forward?" And, you know, do we, and using all the old things in the book for your organization, you know, a use case registry. Hey, let's designate somebody as the use case arbiter. Uh, and maybe that person is an AI chatbot. I don't know. But you're saying like, "Hey, if you're using, if you find something you're used, here's a tool. It's general use. If you find something you're doing and you think, hey, this is kind of creative, go add it to the registry." You know, go pump the idea into it. Simple web form and whatever your platform is, solicit these, you know, have people go around, "What did you use it for this week?" Or maybe have a bounty for whoever comes up with the, you know, the coolest idea this month. "Hey, we'll buy you a RGB mouse that you can use that." Just kidding. Lawyers don't want this. But it just kind of like, you know, what is set up ways to really go and collect what people are doing because, um, I don't know. This is all I think about every day. But I am constantly amazed when the people making the sausage, the people doing the hard work, have access to these tools, they will think up things that none of us here ever came up with. You know, the users are the most powerful source of development at your organization because they get it, they get how to do it. And so, you know, creating a registry of that and having somebody could edit it or whatever, but publishing that, um, important. You know, maybe you have a select group of people who are kind of your special ops and you challenge them to say, "Hey, we're going to empower these people to, um, have, you know, they're they're going to be the pilot group and we're going to let them go out and kind of on an accelerated pace, try to find solutions, but give them more free rein." And then they come and present it to us to kind of get out of a bureaucratic structure or something like that. But I think there are, you know, a number of those things that you could do to kind of think big picture about it of how to, how do we inspire innovation? How do we, uh, make this transition? And what does that mean for our tech or our, you know, organization? Because really, you're not trying to develop an AI culture, you're trying to use the culture of your organization and have AI match up to that. You know, you match the tools up to how you're doing things and really ask like, "Who are we? You know, what do we do? What makes us unique now? Where does AI fit to that?"
>> That is a great point of view and I I I love that. Um, you mentioned you want to really foster the culture, right? But the first step in this, and I think a lot of law firms may struggle with this, is actually understanding who would be in charge of that innovation. You mentioned having the ops team or someone in the firm that would be responsible for carrying the weight of implementing AI.
>> Yep.
>> Interviewing you, you're one of the first persons we've had with a title that actually says director of AI and e-discovery, right? So I'm curious for any law firm out there or anyone in a managerial position, partner, that wants to implement or create this new seat in their company or in their firm, how can they go about it and how can they pick the right individual to actually get the first steps to foster this type of culture?
>> Yeah, it is a real, um, it is a real mixture between, you know, you can get the purely technical people. You can go out and get, um, IT people who are, you know, DevOps folks, whether it's a small or large organization. You can find somebody who their whole job is to place enterprise applications and they sit around thinking about the best way to do that and to inspire adoption. And those things, um, those are really important skills you need. You need to, you know, figure out the security piece. That is step zero. Does this fit into, you know, our management framework? How does this fit? Um, great. But on the same token, you know, hey, if we didn't need legal judgment in implementing these tools and all you needed was IT, then there would be no gap right now. You know, lawyers would already have these things integrated. We would never have been behind in the first place and law firms would be functioning with all sorts of levels of automation. And so there really is, you know, a chasm there that you have to bridge. And so, you know, I would think like, well, you should expect to dedicate some attorney hours to this. You know, you, some, you need to clear the path so it's not an afterthought. Um, in terms of making communicating, "Hey, here's our culture, or here are the things we're doing, this is the legal stuff with it," and creating that partnership to move it forward. But it takes real time. You know, it takes time to go and use the tool and say, "Ah, I don't, I don't know if I get this." And maybe it's just a matter of them getting it. And that's what's important. But I'd say you need some glue to put that together, you know, uh, I think it's best if it's a lawyer because they'll get it. Uh, maybe it's not though, you know, maybe it's somebody from the technical side who's really good at translating lawyer, you know, talking to them and knows how to talk to these lawyers at what they'll say and, um, and what will be persuasive. But I, I think it's one, a recognition like, "Okay, this will take time." You know, this will take attorney time. And if we don't want to do that, there will there will be some consequence for that. You know, there'll be some fit issue. Um, but it's, that's a good investment, man. Like it's, you know, with, hey, the practice of law is going to change. And I think people are not going to sit around regretting like, "Ah, man, why did we, why did we invest so much time and like sitting around thinking about how AI applies to X organization?" You know, and so I, I think it's realization, "Hey, yo, this will be significant. It's a good investment. Have somebody from the legal side." And then marrying it up with, "We need somebody to facilitate this." And lots of IT teams. And it's, in some ways, you know, I, I talked to, um, some of my peers or others, law other law firms, and I see some patterns. And one of the patterns is, yeah, lots of, guess what? Uh, the legal industry that is kind of behind this adoption, uh, sometimes undervalues it generally, you know, or some, you know, at some, uh, places I have a firm in particular I'm thinking of, if my friend works, and it's just clear like, "Wow, yeah, it is, you know, understaffed or too busy or, you know, it's really difficult for them to kind of explain their needs." Um, and if that's the case, then going and telling these people, "Hey, you have a new job. It's on top of your old job. And it's figuring out how to transform our practice using a very disruptive technology that's, you know, going to be bigger than electricity, literally." Uh, "Okay, that doesn't sound like a recipe for success to me." So, kind of like, "Okay, you got to clear the path." And it's not just a matter of you have some "Totty Tommy" character. It's like, "I got a great idea. By the way, you want to listen to some Megadeth?" Like, "Okay, that no, no, he has to, that plus has to be provisioned. They have to have staff. They have to have the DevOps people in place. They have to have budget." And so, it's a, the idea of like, "Yes, we need the leadership. We need somebody paying attention, but we also need to give them what they need to succeed." And that could mean a whole range of things, a whole range of sizes of organizations, but it's all within reach. You know, it, in some ways, smaller places, "Okay, you have a smaller problem." So, you know, so it's okay that you don't have three DevOps engineers or whatever. Um, but yeah, that's that's what I think it is. Is kind of clearing the path, giving them the resources they need, and creating this culture, uh, for innovation, creating a way that's AI-friendly, AI-safe to move forward.
>> Todd, if I had to put you on the spot here for a second, because you're, and Adrian's question about culture, I think, is a really good one because obviously that's like,
>> That's sort of the mindset or the psyche of an organization.
>> Um, all right. Inside that culture. What if I said to you like one sentence or two sentences? Tell me who, how are you speaking to the AI skeptic? Like, what are you going to say to the practitioner who's like, "Yeah, yeah, yeah. I'm hearing all about AI. Great, great, great. It's fantastic. I've been doing this this way my whole life. I'm not changing. I don't trust it. Um, we're not disrupting it." Whatever it might be. What is your sort of like hard-nosed,
>> Like how are you proselytizing evangelizing AI for these people?
>> Yeah. Yeah, it's it's interesting. You know, I think of it, um, you know, I think there is like a good guy pitch and a bad guy pitch, you know.
>> Yeah. Yeah.
>> I think the good guy pitch is something like, um, really addressing head-on that, "Let's talk about your concerns about security and privacy." And, you know, and it's usually the case that it, you know, they're like, "Well, I, I don't know." In some ways, like every news story that you hear about some hallucinated case being filed in a brief and then the lawyer lawyers getting panned, like sets us back a couple weeks or a month or two on adoption. Um, but we've kind of seen this before, you know, with the whole cloud, uh, you know, the whole SAS revolution. For a while there, everybody's like, "Well, I don't, I all my stuff in that cloud. I don't, we don't do clouds around here." I'm like, "Okay, you know." And there is the whole debate, you know, the public private cloud security paradox, you know, "Well, is does X hyperscaler know security better or does your security team with kind of tweak it more control know it better?" You know, like, uh, it's a debate. You kind of guess where I fall out on that usually with depending on, you know, how well provisioned, uh, different organizations may be. But I think the good guy pitch is something like, "Yeah, let me, let's talk about your concerns. Like, those matter to me." And not portraying AI as something it's not. I mean, if you like go to some of these conferences and listen to enough salespeople, you'll be like, "Shit." Like, "We're the, you know, we don't even need lawyers anymore." Yeah. And it's kind of like, "Guys, wow." You know, that's really not how the technology works. And the truth is is that it's still, there's still a lot of prompt sensitivity involved that is unpredictable to us. You know, there are still just some facts or some questions that you ask it and it looks to you and I like something it just answered really well and it just doesn't do well. Sorry. Um, and so it's really kind of being realistic about the limitations and saying like, "Hey, I get it." You know, I get it. But think of it not like a crazy Microsoft Word that's going to do some crazy stuff sometimes, but think of it more like a junior associate, you know, that you have that you're saying like, "Hey, here, it's your first week here on the or your first year, like, okay, some of the things they're going to do are great. You know, they're first-year associates law firm that are great, smarter than me, but great writers, great thinkers. And there, and but there's lots of stuff that I'm like, "Okay, so this is wrong." You know, it would be malpractice for me to take that brief and be like, "File that without looking at it." You know, it's not how we do things. And so, it's kind of the good guy pitch is more of like, "Hey, let's kind of reframe how you're thinking about this. And I want you to give it a try. I, I want you to try it. I want you to ask it to proofread. And you tell me the truth. You know, like, did it, did you find something? Uh, was it helpful? Or come tell me your pain points. What do you hate most? Oh, you hate computers. Me too. What else? You know, like, what, what was the biggest pain points? You know, and try to use AI to kind of say, here's a way to do that." And so it's taking seriously, they do have a point. You know, the people who are like, "This sucks. This is not going to, this is overblown." Like the things they're reading and reacting to. I probably agree a lot of it. But it's like, okay, the good guy pitch is addressing that, saying, "Hey, we should do this. You know, our competitors, they're out there kind of doing this and people are reporting. There are these studies, I don't know, they say they're it's good." Um, that's the good guy pitch. The bad guy pitch is kind of like, "Oh yeah, like it's okay. Well, don't use it then. See what happens." You know, I mean, it's kind of like, "Well, okay. I don't really see how, uh, you know, this, it's going to be inefficient. Your work product is going to suffer, period. You know, you're going to have worse information retrieval." Uh, it's just kind of the bad guy pitch is kind of like, "Yeah, sink or swim. It's your choice. And the market doesn't care what your personal opinion is. Sorry." And we've already seen like, it's starting to shift. And from day one, you know, from like, whatever the Goldman study was years ago when this stuff came out, everybody's like, "Oh, wow." Like GPT actually works, you know, the second, you know, predicted number two industry most impacted, legal. You know, from day one, everybody's recognized it as kind of like, I say, the paradigm information work. And so, bad guy pitch is, uh, "Hey, train's leaving the station. You know, you don't have to get on, but I don't know what's going to happen." You can go on the internet right now and search for prominent Wall Street law firm leaders, you know, saying things like, "Yeah, in five years, I don't know about all these junior associates." Like, "Oh, wow, that was said out loud. Okay. Well, that's an opinion out there from somebody who's very well respected." And so, I think, you know, there are lots of opinions, but it does seem prudent to, uh, we should do something about this, probably.
>> Yeah, hop on that train before it leaves the station.
>> That's right.
>> Now, I would love to get your thoughts on e-discovery. You know, you've done more than your fair share of e-discovery. What are some of your favorite ways of conducting it or using technology to really enhance your e-discovery? What are some tips you would give out there to some of our our listeners?
>> Yeah, I think it is, um, you know, in terms of in terms of artificial intelligence, I think it's, hey, the tools that have been proven up in court, kind of, you know, you're the old deterministic machine learning technology-assisted review, uh, that there was, I don't know, a decade ago, people were like, "What is this new thing?" But now it's clear in court. There are even some opinions out there that are like, "No, you need to use this if you're going to like claim burden." You know, "Nope, you need to try the ML, the machine learning to reduce your burden." Um, but so it's, use that anytime you can. Uh, there, you know, we use it on every single case. Even if it's not, you know, kind of making production determinations for you, you can use it to for QC, you can use it, um, to structure the review, to see, uh, certain documents before others. And so it's, you know, get all your legacy, uh, machine learning tools, analytics, use them on every time. And if it's burdensome, if you find it burdensome to use them, okay, you need to standardize that process. Then you just create a template. Take the extra 20% of the time it takes to do the setup and create a template. This is how we're just going to do it every time. Maybe it doesn't fit everything perfectly, but it really lowers the barrier to doing these things. And certainly within even within those tools, you know, there is space for creativity and to say like, "Okay, we're going to run alternative models or maybe we're going to stratify the sampling and try to find a certain fact." Um, and, you know, there are ways of optimization and making it even more effective. But the key is, hey, you should probably use that every time. And then, um, you know, Gen AI is really presenting a whole new world of e-discovery and document review options. Um, still, we're still waiting for case one that says like, that has a challenged Gen AI document review process. Uh, nobody's challenged it yet because nobody's done it, um, without having permission of the, you know, opposing party and coming to some agreement first. So still waiting for that challenge. Still waiting for that first case to come out. And that, that's where, you know, I'm like, "Oh, yeah. Don't be like the test case and just go for it." That's that's my advice to my clients. I think it's super cool if your clients try that. Okay. Somebody else, you guys should do that because it'll be cool. See that case, but I don't advise my clients. Yeah. To be the test case. Um, but no, no thanks. But, you know, Gen AI is a powerful tool. That's different than technology-assisted review and machine learning because you don't have to train it. You know, before we would say like, "Okay, you got to go code a bunch of documents and come up with a model and then you can kind of see if the model works." But Gen AI, like, no, it works the same on one document as it does on 1 million. Um, the use case and so, or, you know, the effectiveness of it. And of course, there's lots of prompt sensitivity, all that stuff in there. But it's, you know, you could try it right now. It's, um, you can use it in a QC fashion if you don't trust it, you know. Well, okay, send it through. Maybe use a cheap model, not the expensive one, and say, "Do any of these documents look like they might be privileged?" Checking everything that's been coded not privileged. You know, maybe that's the subset you ask it for. And okay, or, you know, and maybe you give it a definition of privilege that's pretty strict. So if it comes back with any yeses, then you really want to look at those documents because they're probably miscoded. You know, or maybe the opposite. Or you're using it in a QC fashion, or, "Hey, read these documents. Are any of these sensitive?" You know, "Is there any PR risk here with what we're about to produce? Should any of this be spun in a bad way?" You know, I used to have, I still do and still use them these like search term lists that were just like the terrible words. You know, it was like from a professor at university, like every nasty word I've ever heard of. Then I had like a subset of those that I created that is like the really bad ones. And I think if you like give that to somebody, it might be against the law. Like you just harass everybody. They're terrible.
>> It's just like the worst things you could think of. But I'm like, we're running those before we send docs out the door because if one of those words appears, we really want to know about it for our clients. And so kind of doing those sorts of checks, those QC checks, because, you know, it's moving everything we do, even, you know, the civil procedure as it stands, the rules, the tempo, the pace, um, how we, how we conduct it, is built on this assumption that every day that goes by is a more and more flawed assumption. And the assumption is this: there is a limited human capacity to carry out certain analytical tasks. And the more time that goes on, that's starting to disappear. And so there are all these things that we've kind of left off the table, you know, kind of like the looking out the window. How many drops of water are on the window? Like, "I wonder." Well, guess what? Like, literally go give it to these models. They can tell you in five seconds now. You know, and it's kind of like those sorts of tasks are back on the table. And realizing that, "Wait, we can ask it this thing. We can ask it about this fact." Or maybe we're there's some weird issue that we make sure this weird issue isn't in these docs. You know, those are all on the table now. And if you're using it in this additive fashion, in a supplemental fashion, there's kind of nothing to lose. Like, either it finds it or it doesn't. Check what it's found. Does it look right? Okay. Yeah. And the, the privilege piece and like being able to just put it in there and be like, "Are any of these documents privileged?"
>> Joe, have you experienced any type of prompting kind of like that yet? Have you tried?
>> I, I love that use case.
>> I mean, you, I truly, you talked about treating it, Todd, as a junior associate. Um, I always talk about just inviting it into the room to make it play the role that you want it to play. Is it information recall? Is it quality control? Is it, you know, detailed guidance? Um, is it just like, build me a timeline of things that exist in this document? I mean, but that leads to my question, Todd. Like, okay, help me understand this, especially from a bigger law firm perspective. You know, when you, from an IT perspective, you're implementing these new tools and you're tracking their success. So, and one first part of this is what are those metrics that you're using to track the success? The second piece is very broadly, one of the metrics seems to be like productivity enhancer, time saver. If that is a metric of success, doesn't that cut directly against the fundamental business model of a law firm, right? Like you now are doing things so not not 5% quicker, 85% more efficiently, 95%. So then the bottom, how that seems to undercut the bottom line of the business itself. The more successful the tool is, theoretically, the less successful the law firm could be financially. So, help me understand like the metrics you're using and then how law firms are discussing what the future of their bottom line is going to look like as a result.
>> Yeah. No, that's that's a good question. So, in measuring the success, or failure, of a tool, I think it's, it's a very nuanced thing to whatever tools in place. You know, to the extent that usage of the tool or time spent using the tool is diminishing, and it's just, you know, copy and pasting into, um, into another, uh, the proofing example, you know, I think I would test that by saying, you know, like, "Did you find this useful as a whole?" "Yes." "Okay, did this save you time?" "Yes." You know, kind of detecting because you want to make sure what you're doing matters for the client still, you know. I think most of it is so fast that it seems like it would really deliver good value. You know, in that case, like, "Okay, this is worth it to the client." But you want to measure that first and foremost. Is this delivering value to the client? In some ways, that's the only metric that matters. And if it's, if it is, if it's a yes, if it's positive, okay, you know, like then we should use it. We should use some form of it. And by delivering value, is it worth it? Is it accurate enough enough of the time to be worth it to use it? And you, there's like a million grades of where that is. So I think that's question number one. I'm measuring it purely in client value. Is this delivering value to client? Is this good for our clients? Is that, is that what we're doing here? We're representing clients. We're doing what's best for them, not us. And so don't forget that, lawyers. Um, so that's the answer number one. Number two is this, uh, this idea like, yeah, so kind of a, there's like a perverse incentive problem with efficiency in these law firms, billing by the hour. How about that? You know, this total, you know, this first, I'll recognize like, yeah, this is not a new idea, you know, within the practice. There's a lot written on this before AI, efficiency improvements, period. You know, created this problem like, "Oh, you don't need the shepherd's guide to go look something up." And, you know, in my clerkship, we verified every quote of Texas law physically with reporters from the Texas law library, and we went and laid eyes on the books. And by the way, caught a lot of errors from the digital edition, that's another story. But, you know, that takes more time than using a database. And so the same question, you know, comes out, "Well, okay, you know, isn't that kind of against, um, yeah, it is. There's no denying it. I think, um, it will make things more efficient. It, it for sure is a yes to question one, which is the one I said that matters. Does it deliver value to a client? But I kind of see the future as, and, you know, I just wrote an article on this, um, that came out last week. I think anybody who says that the whole pie is not going to get smaller, I can't follow that. You know, I, I can't go along with that. I think it is getting smaller. It's going to change things. And just like technology-assisted review changed things before, like contract attorneys are lawyers too. You know, those smart lawyers too. And, and for sure, TAR, uh, cut jobs for sure, you know, TAR cut review workflows by at least 50% before the, you know, Gen AI came along. So, yeah, there are real consequences.
Do I think it's going to happen? No doubt. And to people who, you know, there are some people, oh no, it'll, the GA law is a gas and it'll fill the size of a container you put it in. And to some extent, I think, okay, for some things, yes. But, uh, if we look at it a whole, as a whole, take a step back. Will Gen AI reduce the number of hours spent on the in the PRA law? For sure, it'll reduce the number of hours and it will make those hours that are spent so much better. It'll deliver so much more value to clients. Um, and so then the follow-up question is like, how should law firms think about this? You know, and I think that's above my pay grade, first off.
>> Yeah.
>> I'm like the tech guy. Uh, but my view is kind of like, okay, you know, I think question number one is the focus of good law firms. How do we deliver value to clients? Is this thing that we're thinking about doing, does this help us do that? Does that improve that? If yes, then you should do it and the consequences will follow. But I think clients will appreciate it. And I think if you're going to soon have the choice of, do you want to use really smart lawyers that are AI-enabled lawyers, or do you want to use the really smart lawyers that are not AI-enabled lawyers? And so that will be the choice in the market coming up. And I don't see a lot of adage for the people who are like, yeah, but we're going to be the last ones to sign up for that one, you know? Like, maybe that's a good strategy, maybe it is, I don't know. But I think, you know, question number one, delivering value to clients, that's really the one to focus on. And then from there, like, I don't know, there's other business decisions, other ways of dealing with it, kind of on a micro level. But yeah, the macro question, there's no way around it. I, I don't know, maybe somebody else has a persuasive view of this that we'll still need more lawyers. But, >> it's a very good question overall and we, uh, are coming up on time here. So I really wanted to ask you this, what is your hottest take on the industry right now?
>> Yeah, my hottest take, wow, this is just like opportunity to get in trouble. Um, I don't know. You can imagine I have many hot takes, uh, that do not speak for my organization or my clients. Um, but what is my hottest take? Um, I don't know. I think my hottest take is some is asking lawyers to fundamentally rethink what sorts of things are technical and what sorts of things are legal. And for a long time, we've gotten away with pushing things into the technical bucket that really don't make sense, you know? And if we went and pulled all of big law and said, "How many here know how to use the fundamental lawyer program of Microsoft Word?" You know, how many of you know how to use styles within Word? You have this very simple tool. Like I think you'd be surprised at, you know, the level of kind of Microsoft Word competency across the board for the things they use all day long. And there's a certain culture that has enabled that and a certain culture that has valued other skills and pushed lawyers into focusing on other areas to improve themselves as lawyers and say, this is what makes a good lawyer. And I think that that has taken a turn. I don't know if it was 20, 30, if it was 40 years ago, somewhere along the line, the culture took a wrong turn. And that we are going to experience a lot of slap back from that culture now that there is energy focused on using technology. And so I guess my hottest take is kind of like, wow, yeah, we need to radically rethink what sorts of things, how close we are to technology as lawyers, how much we need to understand. Because the lawyers can do it. I mean, you can go out and figure out some convoluted statutory scheme or some great transaction that I'm like, what is the why? Why are there eight entities, you know, involved in this in this transaction? We can figure out these really hard questions, but then we're like, I could not possibly learn Python like you could, you know, faster than you figure out this case for sure. And so I think it's just not that I think every lawyer is probably, but I'm just saying like, I think that we need to kind of take a fresh look at like, okay, what is our minimum viable kind of tech IQ? And that needs to change because it's that, that's what the market's going to force you to do.
>> Yep.
>> It's the future.
>> Yep.
>> Completely agree. Well, thank you Todd for for joining us here today and thank you to our audience for tuning into Between the Briefs, brought to you by Steno. Special thanks to our guests Todd and Tom for sharing his invaluable insights on the future of AI, e-discovery, and legal technology. And of course, big thank you to my co-host Joe Stevens for another fantastic conversation.
>> Yeah, I couldn't agree more, Adrian. This is a a good one. Be sure to subscribe so you never miss an episode and stay tuned for more thought-provoking conversations at the intersection of law, technology, and innovation. And we'll see you next time.
>> See you next time.
>> Up next on Between the Briefs.
>> Don't take client data and put it into public version of Chat GPT because it could be leaked. On the one hand, clients expect us to be on top of the latest technology and we are. I mean, we're investing millions of dollars. A year ago, people were talking about, oh, we need to be training everybody in prompt engineering. But as time has gone on, we've seen, no, we probably don't because the GI is learning how to do the prompt engineering for us. My hottest take in the industry is despite the fact that everybody's talking about AI and I still think >> Stay tuned for the full interview coming to you soon. Between the Briefs is brought to you by Steno. To find out more about Steno and how we combine exceptional court reporting and litigation support services to deliver a superior litigation experience, visit steno.com. That's S-T-E-N-O dot com. And then make sure to search for Between the Briefs in Apple Podcast, Spotify, or anywhere else you get your podcast. And click subscribe so you don't miss any future episodes. On behalf of the team here at Steno, thanks for listening. Let's go.