Transcription
Hey, welcome to the CMO Whisper Show. I'm your host, Steve Oinski, part marketing practitioner, part ad agency veteran, part journalist. I was a writer for Forbes for 10 years. I've had so many insightful conversations over the years with business leaders to athletes to celebrities to of course CMOs. The only difference now is instead of sharing those insights through written form, I'm doing it this way.
My guest this week is Brian Kotlar, the CMO at Hightouch, the leading composable CDP that helps companies activate their data warehouses to drive personalized marketing and business operations. With 20 plus years of experience at Intercom, Sprinkler, and New Relic, Brian knows how to cut through the martekch noise that makes one of us and build teams that turn data into real growth. Brian, my new found friend, welcome to the show.
>> Thank you so much for having me, Steve. It's great to be here.
>> It is my pleasure, my friend. I think we're going to have an incredible conversation and I know my listeners are going to learn a lot. And I want to start with I used to call this the elephant in the room, Brian. That two letters that everybody talks about. It's not the elephant, it's the whole damn zoo now. And that's AI.
And in marketing and differentiating using it. So let me just jump in and get right to the you're a perfect person to ask us ask this question about the real difference between LLMs and reinforcement learning and then how it comes to marketers actually use that.
>> Sure. So, as sort of this like tsunami of of AI technology has arrived in the last few years and we've all I think we've all deeply understood that like white collar labor, knowledge work, whatever you want to call it, is really never going to be the same. All kinds of things have started to be crammed under this moniker of AI. And the truth is that there's actually a lot of different disciplines and schools of technology that sit underneath this this broad term. And and so, you know, for a very long time, uh, we've actually all quietly and maybe not so quietly been using AI in our day-to-day lives through the moniker of what what is actually called reinforcement learning. And this been going on for years, by the way.
Um, basically, if you think about how Facebook works or Google works today, where, you know, when those platforms were first born, you know, you would be very precise. I want to market to young men in these regions with these interests or whatever. That's how it all work. And then few years ago, suddenly it really changed. Suddenly all that would happen is you would go to your Facebook interface or your Facebook salesperson and you'd say here's my creative, here's my money, here's my outcome, figure it out. And initially that was a little scary to do, but the reality was that actually worked better than any amount of specific targeting because Facebook's ability to learn and harness all their data given the breadth of platform they have is just outstrips any amount of very specific new targeting and media planning any of us could do. So that technology um that underpins that ability just to go to Facebook and say get me an outcome the rest is your problem is the underlying technology there is called reinforcement learning which is a branch of AI which we've all been using if any of us have bought a Facebook ad a Tik Tok ad a Snapchat ad or a Google ad in the last 8 10 years we've been using reinforcement just full stop and that predates this LLM revolution that we're all currently living in now where you know obviously most famously with like the chat interfaces of chat GPT and Gemini and things like that.
And so I think for us as marketers, what we're what we're kind of learning to do is to think about sort of the full workflow of what the job of marketing is. And now what we're doing is we're we're figuring out what's the right tool for the job from this broad AI toolkit for each facet of of work. Does that help answer your question?
>> It does. It does. I I want to keep going though because I think and I don't want to speak for all marketers, but to elaborate if there's more between that difference between LLMs and that reinforcement learning you're talking about, what can marketers like what should they be looking for? What's what are the main differences kind of thing?
>> Sure. Yeah, forgive me. I don't know if I answered that very well. So yeah, basically the thing to think about is that um LLM as we kind of famously know are essentially like these amazing engines for guessing the next thing to do and it allows them to take these creative leaps that mimic a human creative leap and are increasingly even better than a human creative leap um across all these different domains. And so essentially you can feed it an initial set of information and it'll take that next step and the next step and that next step you know and as the power of them gets better they can go further and further along in this chain of logic and and task they can do.
Reinforcement learning is a little different because what reinforcement learning is about is about trying things learning from what you tried and then feeding that learning back into the next initiative or try you're going to take. So that's why again using the Facebook example that because it's all so familiar to all of us as marketers. It's why they have these quote unquote learning periods where actually the algorithm is bad for a while. The reinforcement learning doesn't work initially because it needs to run those experiments and teach itself what it is you want and how to go get it. So you could think about it this way. An LLM is a great way. You give it a sort of some context and and a need and it'll generate something for you. But now, is that something actually the right um ad creative or the right copy or the right email subject line to engage your audience, get the outcome you want? Well, the ALM has no idea. It knows it generated something that fits the specifications of what you wanted, but will it quote unquote work? Well, that's where reinforcement learning comes in. Reinforcement learning is essentially the way now to in a very rigorous high-scale way test what's right.
And so when you pair these two things together, you use LLMs to generate things and then you use reinforcement learning to put them in front of real people, see if it's effective, and then tune it and tweak it and get better and better. Now you have this incredibly powerful system where you can generate vast amounts of of of creative or subject lines or ideas and then actually bring them and put them in front of customers and tune them and improve and optimize and learn. And so you you can create these really high-scale systems, net new creative testing, net new creative testing in a way that frankly human teams have just never been able to really uh achieve in the past. We've been very limited to simple things like you know art director, copywriter make a thing, then uh growth marketer or media planner or whatever tests that thing then some scientist does some reh read out of the AB test or something and brings it back to the media planner and they begin a new, you know, and that cycle is really slow um and difficult to learn from and frankly not particularly scientific. But this new world is incredibly datadriven and scientific and at a scale and a rate that just it boggles the mind.
>> Not to belabor the point, but I you bring up uh uh be remiss if I didn't ask this question. So large language models and reinforcement learning.
>> Mhm.
>> Can they operate independently or do they need each other?
>> For a marketer sake because there there's like a I'll be honest, there's a level of computer science here that's probably not interesting in the context of marketing. Yeah.
>> For as marketers, you don't need one to do the other. You can actually have like I could write creative and subject lines and work with my designers to generate things and then give them to a reinforcement learning engine like uh like what Facebook does for ads or what like what hight touch does. We have this product we call AI decisioning and that's like the category but the idea is that you can use AI decisioning to do this kind of same style of reinforcement learning across all your email channels and SMS channels and push channels too. It kind of brings that Facebook concept to all these other environments like your own channels like your websites and stuff. And so you can do that using the creative you have today on the shelf, sitting in your your digital asset management platform, sitting in your email service provider, sitting in your content management system without without touching an LLM. You could just use reinforcement learning technologies on ads and on own channels. The or you could be using LLMs to help your teams accelerate their creativity and accelerate what they produce and you could flight them into the market without touching a reinforcement learning technology. You would just do it the old way. Let me run this AB test on my life cycle program. let me run this AB test on my ads and that would be fine too. In fact, both are better than not doing it, you know, uh, in a vacuum. But what's really magical is when you marry the two. Now you have the workflow improvements of LLM's helping you be more creative uh, and much more and develop things much more quickly and you have the learning improvements, reinforcement learning, bringing this stuff to your customers in your emails, your SMS, etc. And the two together.
So, uh, we had a customer tell us that, and I'm quoting, um, they learned more in 6 weeks than they'd learned in the prior 12 months >> by doing this. You like, really think about that, right? That's an 11 10 and a half month improvement in in velocity, you know, just by turning on a piece of software. So, you're literally saving years years of effort.
>> So, yeah, and and thank you for elaborating on that. So, I want to stay in in the AI world for a few more minutes. um this is the most like uh let's just cut out the BS here. What in your opinion what's actually changing in a work in the daily workflow of a marketer versus what's just hype?
>> Sure. I mean, look, I would say that what I've observed is that if you think about sort of the the workflow of marketing ranging from like trying to understand what to do and figure out what to do initially, like almost like strategy classically, then this like period of making stuff in the middle and then this period of like exposing customers to it and running experiments and engaging with them. What I see is that basically the book ends, the beginning and the end of that process are being really heavily disrupted already like seeking to understand what to do, seeking to understand what's going on.
Um, LLM can really help you with that today as we speak. And then seeking to interpret that into briefs and strategies and things. LM are already really helping with that. They're starting to trickle into the middle in terms of helping with copywriting and editing and and and creative stuff, but it's still early days. Like very basic things LLM have really disrupted like generating subject lines, but like our whole creative teams being displaced. No, they're being helped, but they're not being disloed. And then at the end part, the part where the the materials actually meet the market, whether it's through an ad as we talked about on Facebook or Snapchat or whatever, or whether it's through an email or an SMS or a push experience like like what Highouch can help you with, again, that's being really heavily disrupted like that those core workflows of let me plan my AB test and send it out and study whether it's working is getting completely uh basically deleted and replaced by uh these these AI fueled experimentation programs. And so increasingly what I suspect will happen is the middle will start to collapse too. You know like beginning has been changed, the end has been changed. The middle is hard because it is so creative and brand sensitive and brand. But the AIS are getting better at that too. And so you're starting to see it move more and more towards the middle.
>> So what if anything is hype?
>> Well, I think what's hype as we sit here is there's a lot of this notion of the entire workflow collapsing and being run by robots. And as we sit here, that's just that's factually incorrect. Like we talk to customers every single day. I I run a marketing team, but I also interact with my peers and my customers every single day. And is like it is a lie. If you say, "Oh, marketing has been completely transformed from 9 months ago." It's not true. Just it's not true. But do you see these glimmers of how um AI is we weaving its way into each of the the elements of the workflow? Yes. And can you kind of squint and see a world where instead of building these comp complicated journeys like if this then that do this then that instead just like on Facebook you hand them some money and a picture and you say go.
Um just like in in not very long from now will you just hand a robot some words and some pictures and and a list of customers and just go for your email program and stuff. Yes, that's not very far away. There's this famous science fiction quote that the future is here. It's just not evenly distributed yet. And I and I I believe that we're in that kind of a mode where you see in isolated instances you see these very AI forward workflows present all over the place. But like these like I don't know how to describe them. These like apocalyptic or messianic depending on how you think about it statements about what's happening to marketing are not accurate. Just don't talk to anybody. It's like you know it's not true. So,
>> so that is I thank you for that because I think there's so much hyperbole and hype it's ridiculous. However, in the real world, there are still some skills, right, that marketers need to have in the AI world. What do you think those skills are?
So I think the a lot of the skills because the technology is adapting so quickly and changing so fast, a lot of the skills are actually soft and personality type behavioral things than they are like vocational training at this phase.
>> Mhm.
>> Because the like the specific vocational thing you would do right now to harness some of these technologies is going to change in a month. And so like I I don't mean to be trit because you folks may have heard this but I do believe it's true. Like sort of being very curious and and borderline naive to what's possible is is probably the most important thing like cynicism about oh robots can't do that robots shouldn't do that is is really dangerous because of the rate of improvement we're seeing. And so like coming to this as like a learner no matter where you are in your career is the is I'm speaking as an individual not as a leader but the beginning as an individual like I think is the critical thing. I could say oh learn to prompt. I could say oh learn to use this tool or that tool but um it's not clear to me that that prompt skill that you develop is actually going to be the necessary one in a month to get what you want out of AI. I'm not sure. So in in the near term yeah sure do that because you're learning and you're experimenting and you need to be open. But but is that like the critical thing that will futureproof your career? Nobody knows the answer to that question, you know.
Um what I think does future proof your career is that is that willingness to lean in uh and and learn and not get overly cynical about about these things. And then as a leader, I think it's a little different. It's about actually creating that culture across your team. You of course should be doing that. But the truth is most leaders, we don't quote unquote work the way that like our teams work. you know, like when was the last time a CMO listener to this actually edited a a script for a commercial, you know, like so but but they probably want their teams to use AI to help them do that. They probably want their teams to use AI to help them instead of hiring some expensive production company use AI to help develop that next ad that they're going to run or whatever. And so it's a lot about fostering the the the the curiosity and creating the space for teams to kind of the the old adage being like slow down to go faster. You might need to give your team permission to deliver something slower while they learn to use AI tools and explore AI tools for something um versus demanding that they you know they hit their deadlines that they always have because this is a learning moment for all of us.
>> Right. Right. Exactly. So how do you as a leader then the balance right between overwhelming them with AI but then reminding them hey don't lose the human touch.
>> Well I think the good news in in a sense right this second is that the technologies still require the human touch. I've never seen um and I like almost as a habit whenever I talk to any of my peers or or or marketer I just ask them what they're doing with AI and as they start to explain I will actually what I'll do is actually I'll pause them. I'll say, "Can we pause before you answer? And can you screen share me?" If we're doing this over Zoom, "Can I please see your screen? What actually did you type in? What actually did you click?" And um first of all, I think that's a good habit to your prior question about deflating hype because it lets me like actually see what's going on and learn. So, I think that's a good habit. Anytime somebody starts blthering on about AI, have them pause and say, "Oh, that's fascinating, but don't tell me about it at a high level. Show me what you're doing."
Uh, so that's a good habit, I would say. But um separate of that, what I would tell you is when they do show me their 0% of what these AIs are outputting is market ready. Just fact, all of it requires human curation and editing at this phase of the game. You can really get it's remarkable how close you can get to market ready, but 0% of what any of my peers have shown me is market ready. So the human touch, if you just set this on autopilot, you will not be happy with the outcome. you as a CMO will say, "Wow, my team is producing bad work."
Um, and and the reason is that the AI left to its own devices will produce bad work.
>> But the AI uh helping your team and then your team applying taste and judgment will produce good work much faster than it used to take to produce good work. If that makes
>> it does. It does. So, what are some ways brands are using AI in terms of um personalization for example?
Sure. So like um if I if I talk to you about sort of those that that end workflow, you know, from the strategy to the making stuff to the to the actual execution, a lot of the personalization happens in the middle and at the end. It's like you need a lot of variations of content and creative in order to to to have options of what to show people that are appropriate to them, appropriate to their journey. And then you need uh a system that's able to actually deliver all those options appropriately to the right people in the right spot. And so uh just as an example, we have a a customer um it's called Fundrise. It's like this alternative asset management platform. So if you want to invest as an individual in real estate or in you know obscure assets, they'll help you do that. And they are very AI forward. And so they have they produce these massive amounts of content variations. And they start always with what they call an investment letter. It's like this is their core point of view on the market. But that thing needs to be cut into countless variations for all the different kinds of assets they support and different types of investors that that might be using their platform. And so they use AI a lot to help them take that initial point of view and refabricate it into all these different assets and variations, visual ones, recorded ones, videos, whatever. And that's a very AI process, though they do apply human judgment there. Then they're like, "Okay, well now we have something to show Steve versus Brian versus Sally versus Stevie.
um what actually should we show them? We have no idea. That's really hard. And so that's where you get to the those end technologies like the the AI decision what I'm talking about where it's able to look at your data set and essentially propose these massive onetoone experiments like I want to show Brian this subject line and this asset. I want to show Steve this one and this one and then start to run those experiments at like a unprecedented unprecedented scale. And so what they've been able to do I mean this is in production right now. If you're a Fundrise customer, you're getting these communications is they they essentially market to every single one of their their massive customer base completely uniquely.
>> I I don't you and I would have completely different experiences of their marketing communications than anyone else that we know. And it's that pairing of a lot of variety with this massive onetoone experimentation engine that they operate uh using this AI decisioning concept.
>> Right. Are there um in terms of measurement and metrics, we know the usual suspect KPIs. Are there other metrics when using and optimizing through AI?
>> Well, I think that in the case of specifically that decisioning thing, it's actually kind of nice cuz the answer is no. Uh, you're still uh engaging customers through those same channels. It's still SMS, it's still email, it's still whatever. But the question is, is it better? So, what typically customers will do is they'll have a hold out based cuz they they're doing something. No company is just sitting on their hands, right? there's some existing email program or SMS program or web program or something. And so basically what they'll do is they'll do a hold out. They'll say this is the old way. Let's leave it alone cuz we want to make sure this is superior.
Um, and and they'll pick the metric on which they're going to judge if it's superior. Is it superior based on just engagement? Is it superior based on you know end lift in revenue or whatever and and then they'll they'll turn on the AI and start to to execute. And similar in a lot of ways to like as I mentioned kind of like how Facebook might work there's going to be a learning period. for a period it might be worse um because it has to guess and try and learn but what we see with customers is that given enough time it's basically always better.
>> Yeah.
>> But it does it may need time and it may depend on the metric you select because one of the things that's really challenging about marketing as we all know is that last click sort of wins everything. You know classically for the last 30 years of our lives, 40 years of our lives, we've been living with this world where just whatever the final point of conversion is sort of quote unquote wins all the credit. But we do a lot of stuff. Just because the four things that happened before that click didn't get credit doesn't mean they didn't help, you know, so you do need to have a broad in some cases a broader more open-minded perspective of what quote unquote works and helps. But I don't think that's unique to AI. I think we've been grappling with that for a long.
>> Yeah, I was just going to say that. Yeah.
>> So that I think that a lot of the more sophisticated thinking we've all tried to embrace of late, I think still applies. It's just using slightly different tools. Yeah. Hey, you've talked about a shift that I'm interested in in getting more of about CRM first to data warehouse first marketing.
>> For those not familiar, what what is it? What does it look like in practice? Tell us about this shift and why you're seeing that.
>> For a very long time, sort of the state-of-the-art of gathering information about your customers um in a way that's actually usable by a customer service team, by a sales team, by a marketing team has been CRM. just collect data in there that's associated with records of people and try to manage their relationship with them CRM using that technology. The problem that became apparent I would say really clearly about 5 years ago, it was sort of there before but really apparent about 5 years ago is that um the kinds of information that we as marketers and businesses wanted to use to interact with our customers was more complicated than CRM were really invented to manage. You know, we we we stopped being just interested in how old is Steve and where does he live. We started being interested in like can I predict based on past purchase behavior what Steve might want to buy next and then use that information to engage with him differently uh wherever you know and then and at one point that was a kind of a science fiction idea maybe 15 years ago but five or so years ago it just became commonplace to want to do that and businesses would turn to the CRM that they'd invested in to attempt to do that and they would find that it was really not a a sufficiently flexible scalable piece of technology to help and what they started to do is They turned to their IT teams and their data teams and they were like, I want to do this thing and my existing technology can't help. What do I do? And so Gardner has actually did a good job of writing about this of late. You're starting to see these IT teams and data teams step into that breach. And the technology they're bringing to bear to help with that is the data warehouse. These these cloud data warehouse environments where essentially you can dump as much information as you want. every little weird scrap of everything your customers have clicked on, looked at, engaged with, every prediction that you might want to do, every every sort of statistical model you want to run.
Um, these warehouses are just these vat incredibly flexible environments for all that stuff in a way that CRM is not. And when you turn to that and make that your source of truth, suddenly these things that don't seem like they should be that hard, um, actually are not that hard.
Um, whereas in CRM you might spend 10 months trying to wrangle your IT team to get you this one little data point so you could customize this one SMS you want to send with data warehouse technologies we talk to our customers and and and it takes them hours days maybe weeks but again you're thinking about collapsing a 10-month process to a twoe process you know it's like night and day difference right
>> and I I lived this experience as a marketer and um where I transitioned from a CRMcentric world to a warehousecentric world and it was so eyepoppingly different in terms flexibility and speed and scale that I literally quit my job and came and joined high touch because I was so excited about it.
>> Wow.
>> It's like you know sometimes you see something you can't unsee it.
>> Yeah.
>> And this that was one of those moments for me. I was like wow the whole way I've been doing my job for 15 years is wrong.
Um, I better go go go do something else. And and that's that's what happened to me to get me to make a big career change.
>> Interesting. It that's fascinating. One last thing before we jump into to the fun section of the of this interview.
>> This is fun for me.
>> I like So, let me let me rephrase that before we jump into the personal fun.
>> Sure.
>> Where my listeners get to know you as a uh the outside of the work. If you're mentoring a young marketer today, maybe you are and you team, is there a mindset shift one or more that you think they need to do the future proof, which I'm asking a lot because we know how volatile and fluid everything is, but I guess what advice would you give to a young marketer today?
I think my current opinion, which I reserve the right to change because I don't the technology is so disruptive right now that it's just hard to say, you know, cheating time.
>> Yeah. At this moment in time. Exactly. I think there's two really big things that I I think about a lot is developing taste and and by that I mean like um being like a a well-rounded holistic marketer that can actually look at something whether it's a strategy proposal from a colleague or an AI or a visual asset that a colleague made or an AI made or whatever and really have the the the the pattern recognition and the sensibility to figure out is this good will it work
>> and give feedback whether it's to an AI or colleague is this good will at work um I think is a skill that's sort of going to be unimpeachably useful and then the other one is is to really develop a sensibility around the sort of outcomes how we're going to work is going to change it's actively changing day by day what we're trying to achieve it will not and so using my uh my my decisioning example like what that CMO of Fundrise had to say to their team was look the goal is the same but instead of logging in and programming all these AB tests instead I want you to really just think about how do you create all these interesting variations of this core idea we have this investment letter and then allow the AI to go run those experiments that you previously would have like very carefully supervised
>> same outcome different methodology
>> so I do think if you have the taste to curate and figure out what you're going to ask the robots to do and then you have that that sense of like what really clear wisdom of like what is the thing we're trying to achieve you actually can switch tools in and out very easily it's like yesterday I built journeys today I asked robots to build journey Who cares? Either way, I'm just trying to get my customers to open these emails, click these things, and then add more money to their investment account, you know?
>> Yeah. The ends justify the means. Of course.
>> Yeah. Maybe not in life, but certainly in this.
>> Certainly. Yes.
>> Okay. Now comes for the fun personal at least for my listeners. And I call it the random five where I just throw and I pull this five very random questions from this massive database and I ask every guess at the end. So, if your game, here we go. Number one, what's one marketing buzzword you'd like to ban forever?
>> Right this moment, it's agentic AI, even though I use it because it just like makes my head hurt a little. Uh, but the truth is it's actually a useful word right now.
>> Uh, so as a B2B marketer, I'll nominate uh accountbased marketing or EBM. As a B2B marketer, the idea that you would not work closely with your sales team to target accounts effectively, like the alternative ABM is just bad marketing.
Um, and I so I find this I find the whole concept to be quite befuddling and frustrating. So,
>> understood. Next, if you had to delete every app except three from your phone, what stays?
>> WhatsApp because it's how I communicate with my family stays. I suppose I'm going to stick to personal apps. Uh, cuz obviously I have work things I have to have. YouTube gets to stay. Yeah,
>> because I really enjoy the weird obscure longtail videos and things it feeds me. and probably my podcast app.
>> Okay.
>> Yeah, I think so.
>> There you go. What do you think your younger self would be most proud of you today?
>> I think probably for, you know, just having a happy, healthy family, you know, living well. I, you know, it's uh all the stuff that we do, it's not really in service of itself,
>> you know, in service of of of building a good life. So, I think that's it.
>> Okay. What's a song that instantly puts you in a good mood no matter what?
>> There's this very strange I can't really recommend it musically, but it but it does put me in a good mood. There's a song that there's this kind of early punk '8s person named Jonathan Richmond who has this song called Ice Cream Man, which it's vocally not particularly good, but uh emotionally it's quite lovely. Uh, it's just this a grown-up man singing an ode to to the ice cream man of his childhood.
>> I love it. Last question. What's your personal definition of a perfect weekend?
>> To me, a perfect weekend if it involves fussing around in my garden, playing with my kids, being near some body of water.
>> I love it. Well, listen, Brian Karth, it is totally my pleasure. I'm so glad we met. This is going to be an amazing episode. I can't wait for my listeners to hear and learn a lot from you. Thank you, my friend.
>> Thank you, Steve, for having me. It was a pleasure.
>> Well, that wraps up another episode of the CMO Whisperer Show. I hope you share this episode with your friends and if you have not already, please subscribe to be kept up to date on all the latest episodes. And if you're so inclined, leave me a review on your favorite podcast platform. Thank you.