Transcription
I think the biggest one pattern that I've seen is is honestly just a failure to measure ROI. The result of that is our friends in research in consumer insights, they have a real PR problem.
You have executives on the one side who are really mandating and pushing their teams to be more customer ccentric, get out and talk to customers, understand our users, and on the other side you have folks in the space who are really struggling to do this work effectively.
Welcome to Insights Unlocked, an original podcast from User Testing, where we bring you candid conversations and stories with the thinkers, doers, and builders behind some of the most successful digital products and experiences in the world. From concept to execution, >> welcome to the Insights Unlock podcast. I'm Nathan Issacs, principal content marketing manager at User Testing, and our guest today is Dave Kerry. Dave is user testing's global value engineering director. His background includes being a professional engineer and a former management consultant with international experience across a number of industries. Welcome back to the show, Dave. >> Thanks for having me, Nathan. Great to see you. >> Dave, uh, as we said, uh, this is your second time you've been on the show and second time we're talking about one of my favorite topics, which is measuring the ROI of customer insights. Now, uh, you've been at the company along as about as long as I have, about five years or so. And so, over that time, you've talked with, you've worked with, you've championed uh, several hundred companies to help them assess their financial impact with customer insights. What's the most surprising pattern you've noticed when it comes to how companies measure or fail to measure ROI?
>> Yeah, I think I was looking this morning knowing that I was coming on here to speak with you. up the 527 customers I've spoken with about this very subject. So, I've got some some hot takes here. I think the biggest one pattern that I've seen is is honestly just a failure to measure ROI. The result of that is our friends in research in consumer insights, they have a real PR problem. Um, what we see and what I see most commonly is you have executives on the one side who are really mandating and pushing their teams to be more customer centric. get out and talk to customers, understand our users. And on the other side, you have folks in the space who are really struggling to do this work effectively because they don't have time to do research. They don't get their recommendations included or embedded and they're often left out of the conversation. So, there's this huge gap. And again, it's somewhat of a PR problem. It's somewhat of a challenge. It's a fun one for me to try and tackle. Um, but it's really where that link of ROI and impact can help kind of bridge the gap there is is is the biggest thing I see
>> the uh and and why I I'm sure we're going to talk more about this uh but I'm interested in why that happens and and I'm and I'm putting that through the filter of my own work, right? And it's not necessarily I'm not a researcher but uh uh I'm a marketer and I'm always trying to you know justify what I'm doing and sometimes and I feel like sometimes we feel like it's we're afraid to the numbers are going to be too complicated or we're not thinking about how do I do that or I need somebody on some other team to help me figure this out. And I think maybe in our conversation we're going to find out that may maybe that's the case and maybe it's not the case. Maybe just how you frame things. And I think this is what you're phenomenal at is uh how do you think about framing these conversations in the work that you're doing? Uh and to that end uh you you've talked about uh four key value drivers. Business growth, faster innovation, reduced operational costs, and risk mitigation. Can you give us an example of how a company translates any one of these into measurable ROI?
>> Yeah, absolutely. So those four what we call value drivers uh are really the basis of our framework for assessing business value. They also if you speak to almost any executive out there are really kind of the priorities and initiatives that they bucket their thoughts into. And so there's a reason for that is because we want to again align to those executive initiatives and be able to measure impact against those. So specifically when we think of an example um I speak to a lot of banks, a lot of credit unions. We were on with one this morning. There's a big push in the industry to drive towards digital towards self-s serve towards improving you know their customer experience. So when we think about that um account signup journey about that customer experience how can we link that to impact an ROI? Well, for every additional prospect or potential customer that we can have sign up through their app or through the website, that's maybe one less person that has to come into a branch or has to call into a contact center. And when you start thinking of it in terms of that, that's directly affecting cost. That's directly affecting experience. So now you're getting more folks in the doors in the digital doors. We're getting on the platform who are making deposits and fewer costs as a result of that. So that's really where we start to see that impact and start to see potentially some ROI from this work. So as you're better able to understand the needs, maybe the friction points and improve that experience again, it starts that flywheel and and starts to uh start to drive impact and hopefully happier customers.
>> Yeah. And you as you're talking about that, I'm just thinking about banks, right? Banks have historically in the last hundred years here in the US and maybe it's true in other countries as well. They were like these big buildings and they had big parking lots. Now in this different world, right, where we can do all these sort of transactions and and the better I'm able to do it using your app, the less I need to come into that space. So from that big huge real estate footprint that you had that you got to keep maintaining, you got to patch that gravel when it when the asphalt has a pothole in it or whatever it might be. All of a sudden, it's not just I need fewer tellers. I need fewer fewer or less real estate costs, right? And operational costs and stuff like that. It's a huge huge savings.
>> Huge thing. And you're really getting into like second and third order kind of implications there. But what some folks, and you mentioned this, really worry about is, well, doesn't that mean there's going to be less tellers or less customer support folks? And what we find more often than not is these folks are now able to spend their time on more better value added activities. So instead of helping somebody troubleshoot their new account, they were able to share with them, hey, here are some other products that might be you might find helpful. Here are some better ways to use the products you already have. And so the types of relationships they're able to build, the type of service they're able to provide really just goes up a level because the nuts and bolts day-to-day, hopefully their customers are now able to self-s serve, which means they're able to do it easier, faster, frictionless, and and hopefully again with a better experience at the end of the day. And I think we're we're talking about banks, but I uh just, you know, grab a sheet of paper and a pencil and whatever industry you're in, kind of map that out, you know, like all right, I'm a you know, I'm not a bank, but I am um a software technology company or uh you know, like I don't you know, whatever it might be and just try and map that stuff out and start thinking about it in these sort of ways. Uh one of the challenges and you kind of talked about you know with the business uh drivers uh is this language of the seauite uh what's what's your advice you know for researchers or for anyone who's doing customer insights to better understand how to talk to the sweet seat seuite.
>> Yeah it's it's a great question and it's one that I'm helping folks with quite often these days. I think for the most part when I speak to researchers and folks in the space, these are super empathetic, highly skilled professionals and my recommendation to them is maybe sounds a bit backwards, but it would be approach their leadership with the same kind of empathy that they do their customers and their users. And what do I mean by that? really trying to understand at the end of the day what their leaders and executives ultimately care about, what they're trying to achieve and then tying that back to the work that they're doing. So, you made a really good point about being able to map that out. There are a number of examples of what's called an enterprise value map and really what they're showing is the exact kind of treasure map to get there. It's taking the work that's being done in consumer insights in customer perspectives and making that link between if we have a better experience here that leads to more account openings that leads to more assets under management that leads to higher revenues and profits for the bank. And so now you're speaking at a language that is absolutely going to resonate at the highest level. you know, if you're in the seat, your recommendations are are really going to carry a lot more weight and a lot more gravity. And you're going to be having that more strategic conversation if you're able to be again somewhat empathetic towards your leadership and really try to understand um what's important to them and and how to make that connection. And it's not just, you know, my chief marketing officer, my chief executive officer, it's also the my, you know, my direct uh manager or my or their vice president of whatever it might be. So, you've talked about in the past this idea of what makes a good day for your boss as a starting point for measuring impact. Can you walk us through how that simple mindset shift helps frame ROI conversations?
>> Yeah, it's a great call out. I'm glad you remembered that one. I think um that question is an old trick from my days in management consulting when we were really trying to uncover KPIs. And it sounds like a bit of a throwaway, but it's actually a really good starting point to understand for somebody in a role. Maybe you don't fully understand it. Well, what makes a great day for them? And you'll often get answers like, "I really, it's great when we have a positive bit of customer feedback, when we're able to ship a new product or ship a new release, or maybe in your case, Nathan, when we're able to host uh a great and handsome guest." So, now we're talking about what's important to them, but we're also kind of setting the stage for potential KPIs. It's it's customers through the doors. It's products we're able to ship. It's guests we're able to host. So you're starting to make that connection and you're also putting it in their language. So again, when you're able to drill down, you can make the connection to the work you're doing, the insights that you're getting, the customer perspectives that you're getting, and really helping them understand how it's going to benefit them in the long run. So um I maybe jumped the gun a bit, but if we turn the tables here a little bit, Nathan, what what makes a great day in your world?
>> Right. Exactly. And and uh is is as I think about that conversation, it's what makes a great day in my boss's world, right? And my my marketing bosses are worried about campaigns that we're running and and the audiences that we're talking to, you know? So, we're we're talking to UX researchers. We're talking to designers. We're talking to marketing professionals. And as I think about that, uh, I could run a podcast and and book, you know, I get probably 20, uh, pitches a day in email from somebody who wants to be on the podcast. And it could be, you know, somebody wants to talk about investments, right? And that's not really one of my audiences that my bosses care about. Uh, and so I try and focus on the people that are going to help support all these other activities. So when I have uh uh Jud Anton on talking about UX research and the future of UX research, his conversation can be as a podcast episode, but I can also then bring that into other conversations that we're having in other the campaigns and make those other campaigns more successful. And then my boss says, "Oh, well, Nathan's annoying, but uh he actually brings value to the company, so we'll keep him around for another quarter." So, that's a great example where you get pitches, you get folks coming to you with what might be good ideas, but they're not in your language and they're not really resonating with you. Now, if somebody came to you and said, "Hey, I would love to come on and speak about how we're using insights in marketing to help improve that buyer journey, the customer experience, even if that's somebody you don't know, all of a sudden you're interested because they're speaking in your language. They're putting it in terms that are going to be important to you and helping you ultimately achieve your own and and maybe your boss's goals as well."
>> Yeah. Oh, it's it's perfect. Those are uh you know one in a one ina million sort of pitches I get but I oh I appreciate them appreciate them when I do get them. I'm wondering also like when we start thinking about these conversations and measuring ROI how does that help us address or does that help us address sort of the in internal politics or even misalignment across teams?
>> Yeah, I think internal politics nobody's favorite for sure. When you look at some of the root causes there quite frequently they can be a result of competing for con for for for resources or capacity constraints or being pulled in a number of directions. Right? One of my least favorite phrases that seems to be gaining traction lately is doing more with less. So how do we ultimately determine and prioritize what we're going to focus on, what we're going to work on? At the end of the day, it's what's going to generate the best returns, the best um revenue or cost savings for the business. And the kind of frame or lens that we look at that through is return on investment ROI. So whenever we're able to use that tool, that framework to tell this story, then we're all essentially playing on a level playing field and we can measure those initiatives, objectives, requests um objectively to really prioritize them. So being able to do this, it is how management is thinking about a business. It is how they're evaluating where to deploy those resources. And yeah, to your question, can hopefully streamline some of those decisions, can give you visibility maybe into some of those decisions and help align organizations and by no means perfect. There's a ton of nuance that goes into them, but it really at the end of the day gives everybody a level playing field to try and look at that this objectively.
>> Yeah. And yeah, in at the end of the day, you might not be able to avoid the politics that happen with any organization uh good, bad or ugly. Uh, and you know, we've we've talked about this on this show and we uh, everyone talks about is like the power of insights, uh, it can cut through a lot of that stuff. It's not just like what somebody's opinion might be about whether the button should be blue or or red, but uh, what your actual customers said about it. And when you can actually then start thinking about what's that mean as far as value and the time it took to switch that color from blue to red. uh and all this stuff. Now you're adding another component to that proof of what's going on. Uh um and we we've talked about banking and you know I kind of alluded to tech companies and stuff like that. I mean and and you've talked to 527 different companies. Is this true really regardless of industry? Is it is it just tech companies? Is just banks can do this or or what?
>> No, I think anybody in any organization and this is agnostic of of research really can look to put their work through the lens of business impact of financial results and ultimately of ROI and it would be um it would be a useful exercise I would say for anyone really wanting to better align themselves to management to get that executive kind of seat at the table and ultimately to forward their own kind of initiatives and resource requests. Right? So um yeah, I would say most if not all kind of industries um rules could benefit from from using this kind of framework.
>> And so then regardless of who you are or where you're working at or what industry you might be working in, uh then the idea is then to go back to something you said earlier, this enterprise value mapping or even in whatever work you do, you you talk about customer journey mapping, right? and what's that journey and start thinking about, you know, reducing friction points. I'm wondering how you can any advice you can give them about how to quantify the value of doing that. Right? We we we know that people are dropping off on our signup page or whatever it might be. How do how can they think about creating a value around that? And then how does that help them sort of prioritize whatever work they're going to be doing or whatever research they need to do?
>> Yeah, it's a great question. I'll I'll I'll use an example um that it picks on the city of the great city of Toronto here, but I for whatever reason seem to manage to collect an awful lot of parking tickets. Now, when I look to pay these, my first stop is the city of Toronto website. Unfortunately, the experience to pay your tickets online here, I will say, is sometimes less than stellar. There are times when it works seamlessly. There are times when I end up having to call in and make that payment over the phone. So, if we understood the friction points there, what's causing some of those errors, what's leading me to bounce from the site or not be able to make that payment, now we can start to put that into these kind of impact and ROI terms. If you think about the cost of that phone call, if you think about the lost payments or the folks who maybe don't ever make that payment, you're looking at now different buckets that you can put that in. So, it's contact center cost, it's lost revenue, it's having to go back at some point, I hope, and redesign that website and experience so that it works better. So these are all the types of costs and revenues that you can look to capture as you make those improvements so that when you're making when you understand those friction points when you can make a recommendation to improve that experience now you're able to link to ultimately those outcomes and then further um really highlight your own work and hopefully get the credit that's due for for improving this experience and collecting all these parking tickets.
>> Yeah. Uh the other side of me, the the the Oscar the Grouch uh side of me says, "Well, they probably designed it that way so that people don't pay their parking tickets on design on time and then they get a bunch of late fees." And I uh maybe not Toronto. Uh we here I'm uh based in Portland, Oregon, but my son uh loves the Blue Jays uh and he's a huge Blue Jays.
>> Join the club.
>> Yes. So, I'm not going to knock on Toronto at all. I'm not allowed to do that uh by household rules. Uh but there are many cities that have designed their systems and and maybe that's, you know, but uh that's another conversation. Um I'm just wondering like uh when we one of the things that we've talked about in the past as an organization user testing is this idea of shifting left and uh this idea of like if you can get insights earlier into the product development cycle uh you how does that affect ROI and can you talk about that?
>> Sure. So again, a lot of the research and insights teams that I speak with feel like they're stuck on the further right of kind of that product development life cycle. It's a lot of validation work. It's a lot of late stage um insights for decisions that you know candidly have already been made. And a lot of that comes down to these companies are trying to ship faster. Development velocity is really important to them and that makes a lot of sense. But if you're not working on the right things and you're not pointed in the right direction, that really means you're only failing faster. So to get ahead of that, exactly to your point, we recommend trying to get further left and further upstream of some of these decisions. But for the same reasons that we've been talking about, this can be really challenging. It can be really difficult to justify resources to do that more strategic and generative work because a lot of companies don't understand the real value of it. And so when we talk about that type of more strategic moving left in practice, it comes back to really understanding what this is costing them maybe not to do that type of work, the risks that it leads to in terms of their product outcomes, in terms of their customer experience, and ultimately getting and aligning those resources so that they can make more informed decisions that they can leverage customer perspectives and make more insightful products that will drive the business. So, um, 100% we we're hopefully helping these teams to shift left in their practice. Um, but candidly, it's it's an uphill battle without really a clearly articulated ROI.
>> Yeah. And I I I'll sorry I I will include the show link um this this link to this in the show notes, but I believe in the tech space that it's like something like a billion dollars or or trillion dollars of costs that just go into having to do rework, right? uh whatever you designed it was like it didn't work or it didn't land with the you know the intention that you had and you had to go back and pay all those people you know to fix it again maybe some of that changes with AI I I don't know but it's still it's still a cost uh and if it's always better to know the right answer you know before you go in to take that test um so anyway I I I'm wondering uh what tools or frameworks and you've talked about the value driver tree. Uh do you how what do you recommend teams do to start building their own ROI models?
>> Yeah, listen appreciate not everybody's a finance dork, right? And spends their days in spreadsheets. My recommendations for folks who are just getting started are are are pretty um straightforward. I think from the onset really important to just track your work, generate data behind the types of work that you're doing over time so that you can show if nothing else volumes are increasing, workload is extending, whatever the case may be, but it will help you start to tell that story. As you get better at that, you can hopefully start to look at the impact the work is having. So collaborating with some of your partners in the organization, those crossf functional teams that your work influences to understand how those recommendations are impacting their work. So now you're really starting to tell a richer story about the volume of work that's going on, the impact that that's having on the organization and you're going to be having again more those more strategic higher level conversations. Now, that will hopefully help you scale your work, help you get further left, like we were talking about, and ultimately see your recommendations go come to fruition more often. So, at a starting point, really just doing what you can to try and track the work, understand the downstream impacts without really focusing on ROI. In the early days,
>> the uh one of the lessons I learned back in when I was in graduate school uh getting my MBA was uh state your assumptions, right? you uh go out there and you know put together your own value driver tree say that I think this is what we're saving here and and when you state that you could be wrong but they will correct you they will give you the right numbers whatever executive you're going to go to it's your direct manager your chief whatever it might be um in the seauite they will say no actually we do it this way and then you can get those numbers but give them a start give them a starting place where they can have that conversation And everyone can say, well, and and I'm working on this personally with our, you know, my measuring my content performance. I'm just going to state the stuff. This is what I have and wait for the feedback to come in. I'm not going to let perfection get in the way of getting started on this project and rolling it out to the team.
>> Absolutely. I would say to exactly to your point, you always want to state your assumptions. You want to make informed decisions for sure. You want to try and be conservative and wherever you can, you want to try and partner with those crossf functional teams so that when you are to the point where you're ready to present, you're able to share where the information comes from, where maybe you're a little less confident. And then, yeah, at the end of the day, you're exactly right, Nathan. If somebody spots something that might be off, hopefully you're close, but they'll help guide you and give you some some direction.
>> And just like you're building a product, this is the beta version of all of this, right? And you kind of keep iterating on that and you will work together and not only do you, you know, start understanding a better ROI of what this investment in customer insights is and that you're getting from it, but you will work together. You'll all be aligned in getting this. I mean, that's, you know, you can't really measure that cost except that it's just going to be a better place to work. You're just going to enjoy going to work and you're not, you know, worried about whether you're going to get laid off that day or not.
>> 100%. This is something we see all the time unfortunately in the industry is that this can be a tricky team um to be a part of for a lot of companies because the impact is so unclear. And just coming back to your point, I always approach these through the lens of a hypothesis. Here's our understanding. Here's what we're trying to prove or disprove. So it's not going to be perfect. Doesn't need to be bulletproof. Um but really with that hypothesis mindset can give some flexibility.
>> Uh you know, for everyone that's still listening and I just want to say thank you for that. uh what what's uh one piece uh once one small step that they can take uh this week to start demonstrating the value of their research?
>> Yeah, I think we've talked a lot about this method or or or or executive empathy. What mean what is how do you know your boss is having a great day? How do you know his boss is having a great day? And again, thinking through how your work helps move the needle for the organization, why it's important to them, why it's important to to folks up the food chain. And again, doesn't mean that it's going to change the way you do your work, but maybe how you communicate that and articulate that so that it's going to resonate and ultimately have have a broader impact, I think, would be one thing to to to try.
>> Yeah. And uh I I mentioned I'll share a link about some of the other research that we've seen in the past, but also uh some episodes have sort of had this conversation uh where and I mentioned Jud Anon. He talked about like focusing on the research that's going to move that needle. There's other stuff that you want to do, but let's you know make make your case by doing the smaller research that's going to move that needle that you can demonstrate that ROI. Uh Tamara Adlin who we had on the show a few years ago talked about this topic of you know speaking the language of the executives uh and understanding understand your work through the lens of a spreadsheet. Um Dave I really appreciate our conversation and uh how does someone learn more about you or this topic?
>> Yeah I think shameless plug if you are a user testing customer reach out to your account team. this kind of framework, this kind of value engineering is part of our partnership. So, we want to make sure you're able to do this, you know, comfortably and confidently. For somebody maybe who's just getting started, my recommendation is always a book called The Pyramid Principle by Barbara Mento. Really, it's this communication framework of starting with the punchline and providing your supporting kind of recommendations. I think it's a bit funny. We all come up through school and we learn how to tell stories and you know there's a setup and a situation and maybe some conflict and a big resolution. But when you're communicating in business, you want to almost flip that. You want to start with the conclusion and then provide those supporting kind of data points and recommendations just like you mentioned. So the pyramid principle is one I always recommend.
>> All right, Dave, I will let you go. Thank you again.
>> Awesome. This was great, man. Appreciate the time.