David Epstein on Mastery, Specialization, and Range 05/27/2019

[Music] Welcome to Econ Talk, part of the Library of Economics and Liberty. I'm your host, Russ Roberts, at Stanford University's Hoover Institution. Our website is econtalk.org, where you can subscribe, comment on this podcast, and find links and other information related to today's conversation. You'll also find our archives, where you can listen to every episode we've ever done, going back to 2006. Our email address is mail@econtalk.org. We'd love to hear from you.

Today is May 1st, 2019, and my guest is author and journalist David Epstein. He is a former reporter for ProPublica and Sports Illustrated. He first appeared here at Econ Talk in September of 2013, talking about his book, *The Sports Gene*. His latest book, and the topic of today's conversation, is *Range: Why Generalists Triumph in a Specialized World*. David, welcome back to Econ Talk.

Thank you for having me again.

Your book opens with a little fable, if you could call it that, of Tiger versus Roger. What's that fable about?

Yeah, what I call the Roger vs. Tiger problem. So, basically, Tiger Woods, I think, is the epitome of early specialization, and sort of his story, where he started, he was able to walk very early, at about six months old. There are pictures of him balancing on his father's palm, and he started swinging a golf club not long after that. And that story of his precocity and early specialization in golf became sort of the core of at least a half-dozen best-selling books, most of which argued that this was just a model that you should think about for anything you want to get good at: this early head start in specialization and technical, what's called, deliberate practice. And so that's sort of the Tiger model.

And what I wanted to do was see whether that is indeed appropriate for extrapolating and whether it's the typical route to success. And I found, you know, I looked at other models and found what actually is what I call the Roger model. So, Roger Federer, whose development story is not nearly as well-known as Tiger Woods', is much more normal. What happened was he played a whole bunch of sports when he was a kid. His mother forced him to continue playing badminton, basketball, and soccer after, instead of specializing in tennis, after his peers were already specializing. In fact, when he got good enough to get bumped up a level to play with older kids, he declined because he liked talking about WWE after practice with his friends. And when he finally got good enough to be interviewed by a local paper and was asked what he would purchase with a theoretical first check if he ever became a tennis pro, he said, "A Mercedes." And his mother was appalled and asked the reporter if she could listen to the recording. And the reporter obliged, and it turned out he had said, "More CDs." In Swiss German, he just wanted more CDs. So she was much more content with that. But essentially, Roger Federer was years behind his peers in focusing only on tennis. And obviously, he turned out okay. And so I sort of conceived this as the Roger vs. Tiger problem, asking which developmental model is more typical and which one is better used to extrapolate to other domains.

And his mother, if I remember correctly from the book, was a tennis coach, which is even crazier.

Oh, yes. She refused to coach him because she said it wouldn't be any fun for me because he never liked to, like, return a ball normally, basically. Which, of course, is actually exactly what you want: that sort of variable movement in development, but wouldn't be fun for an adult as much. And for people who don't know much about Tiger Woods, he was golf 24/7 from a very young age, at least it seems that way, driven by his father to excel. Although that's, that's kind of the story. Although I think we need a correction for the public narrative that he was driven by his father. I sort of, in the course of this book, examined the Tiger and the Mozart narratives because they are so central to so many books that argue for early specialization. And neither one is as they have been portrayed. Not both. In both cases, the fathers were responding to the children, not the reverse. So there's no evidence that you can engineer these performers. And Tiger, in 2000, said himself, "My father has never asked me to play golf. Never." It's that the child's desire to play that matters, not the parent's desire for the child to play. And he said that because he wanted to sort of correct the record. And I found something very similar going through letters from Mozart's childhood, where his, he wanted to play with a group of musicians that came over to play with his father. And, and Mozart's father said, "You've had no lessons. Go away. You can't play." And Mozart said, "You don't need lessons for second violin." And so they finally, he starts crying. And another musician says, "I'll go play with him in the other room." And then they hear the playing coming from the other room, and they're sort of awestruck. And the letter, the disposition left, says, like, you know, "Little young Wolfgang was emboldened by our applause to say that he could play the first violin also," which he then did with totally irregular positioning. So he hadn't learned the actual fingering, so he just played, but he was able to play it with his improvised fingering. So those cases do occur, but the ones that have been portrayed as parent-manufactured, it's, it's not really the case.

Have you read *Open* by Andre Agassi?

I have.

So that's a parent-driven one, and I, at least, it appears to be. I recommend that book. I was, I'm not a big tennis fan. I was somewhat of a tennis fan, and I was really impressed and inspired and touched by, by his story. It gives you an insight into the psyche of a competitor. He's very open about his failures and successes and fears, and just it's like his father at times, for years, a game relentlessly.

Yeah, I agree. It's a great book.

Now, so, so but whether it's the parent or the child, your point with Federer is that his mother and, I assume, his father pushed him to diversify away from one thing. And he was happy to do that. You're implying that it really two things in the, or central to your book: one is head starts are overrated, and specialization is overrated. And secondly, a stronger claim, I think, which is that they're not just overrated, they're less than helpful. The, the diversity that Federer had with soccer and other, other sports made him a better tennis player. And I should say, I think there are as many ways to attain elite performance as there are people. So I think there are sometimes people become elites with suboptimal development, and sometimes people with optimal development don't become elite. So I think there's no perfect path. But that a huge body of evidence across different sports now shows that the typical path is an early sampling period where you do a large variety of sports, you gain a breadth of general skills, you learn about your own abilities, your own interests, and you delay specialization.

Now, whether or not that, because I also make the argument that golf is a particularly bad model of most other things that people want to learn, whether or not that early start trend holds for golf is unclear. I think there is a dearth of research on the best development in golf. So it is possible that early specialization in golf does work. I think the jury is out. But for most of the other sports that are more dynamic, involve anticipatory skills, where you're judging what other people are going to do, the early sampling, delayed specialization is the pattern. And initially, I thought that was going to be purely a selection effect, that it was going to be just the better athletes could play more sports until later, so they did. And then I started, as this question became more important in the sports world, started to see these studies where, say, German researchers would take soccer players matched for skill at a certain age, followed them longitudinally for seven years and see what they did, or for several years and see what they did, and who would improve more later on. And what they would see is that the people who are diversifying their activities would actually end up improving more, even if they were sort of slightly behind at a certain point because of diversification.

And I should make it clear, your book opens with the Tiger-Roger story, but this is not a book about sports. It's a book about, you could say, everything. It's about math, it's about chess, it's about music, art, decision-making, career, generally. So we're opening with some sports conversation, but your book is really, in many ways, it's a, a warning not to generalize from golf, say, or chess. And so tell the story of the, the chess family, and what you learn from that, and the terms wicked and kind.

So, this is the, the Polgár family. Another famous story in, in books that are concerned with the development of expertise, basically from childhood. And in this case, an individual named László Polgár, who had studied a Hungarian man whose family basically, his entire family had been wiped out in the Holocaust, and he was determined to have a remarkable family. And had studied the lives of people who went on to greatness, basically, and decided that he could manufacture greatness, and that he would do so by experimenting with his own children. And he decided at the time, kind of any big, ambiguous moral strategy, but okay.

Yeah, well, I mean, I think that's, that's fair. But that's, that's a different discussion, I guess. The word "experimental" is a little bit loaded, but go ahead.

Yeah, no, that's, that's totally fair. Maybe "experiment" isn't, yeah, I mean, I think his feeling was he was just gonna make them really good at something. So this, you know, was it an experiment, or was he just, I don't know. I mean, I guess everyone can do what they want with their own kids. But I do realize that is kind of a weird way to frame it, so thanks for making that, that interjection. But in any case, he wanted to see if he could make his kids really good at something. And beyond just making his kids good at something, the idea was to prove that you could do this with basically any kid in anything, right? Any kid, and not just in chess. And so in the early '70s, as he had his first, he had his first daughter in 1969. In the early '70s, chess was really, really important, right? It was sort of like a Cold War proxy, more or less. And so for a number of reasons, including that popularity, and also the fact that his first daughter, Susan, might have shown a little bit of interest in a chessboard when she was very little, and also the fact that chess has a very clear rating system that rates a player according to other players in the world, so you can really track someone's progress in a very rigorous way, he decided to base the, well, now I don't want to call it an experiment, but the project on chess. And that he would specialize his daughter in very technical training in chess, very, very early. And so when she was, he started training her hours a day. And, you know, before the first year, like within eight months of training, she, at four years old, she went to a smoky chess club in Budapest, and with her legs dangling from the chair, beat an adult man who stormed off. And from there, she just got better and better and became the greatest female player in the world. I should say she also was way ahead of her peers in other areas, like math and things like that, so she was rather exceptional in a number of areas. But Susan Polgár became the greatest female player in the world, qualified for the, what was then called the Men's World Championships, but wasn't allowed to play. And the rules were eventually changed because of her achievements. And she had two sisters that were part of the project as well, one of whom went on to become ranked, it's at a certain point, eighth in the world, which was the highest ranking a woman had had ever attained. And the other became an International Master, didn't quite make it to Grandmaster status, but the point was, with this early approach of giving a head start in very highly technical training and very focused on chess training, László Polgár showed that he could make his daughters into world-class chess players. And for him, the goal was to show, by extension, that any kid could be made into a champion in anything, essentially.

And you haven't mentioned it, but the claim in the books that you alluded to earlier is that if you just practice enough and specialize and focus, you can do anything at a very high level. And the magical number is ten thousand hours, supposedly. The ten thousand hours of practice can lead to greatness. I assume the Polgárs got more than ten thousand. It would have been a natural experiment. He should have made one of them a swimmer and one of them a chef. But Dolores randomly selected some kids, which was his next plan. He had, there was a wealthy individual who was ready to sort of back him adopting some kids because he's the kind of a brilliant guy, right? And his daughters showed some brilliance beyond even chess. It wasn't exactly a random sample, but that's a separate issue. So, so what's wrong with concluding that? Why, especially with chess, why is chess perhaps not the best way to think about this? And you use the idea of wicked and kind. I really like that.

Yeah. So, so chess is an endeavor where early specialization is very important, right? And even though my book talks about areas where we overvalue specialists and undervalue generalists, it is domain-dependent. And chess is an area where it's important. Like, if you haven't started technical training by the age of 12, your chance of ever reaching, I think, International Master status, which is a step down from Grandmaster status, it drops from like one in four to like one in 55 or something like that. You have to be studying patterns or so-called tactics, which are the short combinations of moves to give you an immediate advantage on the board. It's based on pattern recognition. And the reason why there's been this explosion of young chess masters, there's something like, I don't know, two dozen or something like that, grandmasters ever under the age of like 17, something like that. And the oldest one is like my aging now, because this is a phenomenon that grew out of the availability of computer chess. We're at a much younger age, you can study many, many, many more patterns. So László Polgár gave his daughters a head start on this because he clipped 200,000 different little game reports and essentially allowed them to study patterns. Now, you can do that in the computer. And so that's caused a lot more young grandmasters because there's this instinctive pattern recognition on the chessboard is so incredibly important. As Susan Polgár said, "Tactics, which is which is essentially recognizing patterns, recurring patterns, is 99 percent of of chess, basically." And that pattern recognition. So to go to, you mentioned the kind and wicked environments, the way that chess works makes it what's called a kind learning environment. So these are terms used by psychologists Robin Hogarth. And what a kind learning environment is, is one where patterns recur. Ideally, a situation is constrained. So a chessboard with very rigid rules and a literal board is very constrained. And importantly, every time you do something, you get feedback that is totally obvious. All the information is available. The feedback is quick, and it is 100 percent accurate, right? And this is chess. And this is golf. You do something, all the information is available, you see the consequences, the, the consequences are completely immediate and accurate, and you adjust accordingly. And in these kinds of kind learning environments, if you're cognitively engaged, you get better just by doing the activity.

On the opposite end of the spectrum are wicked learning environments. And this is a spectrum from kind of wicked. Wicked learning environments often some information is hidden. Even when it isn't, feedback may be delayed, it may be infrequent, and may be non-existent, and it may be partly accurate or inaccurate. In many of the cases, so the most wicked learning environments will teach, will reinforce the wrong types of behavior. So one of the examples that Hogarth talks about is a famous physician, a famous diagnostician, who became very prominent because he could accurately tell that someone was going to get typhoid, like, weeks before they had any symptoms whatsoever. And the way he would do that was by palpating their tongue with his hands. And over and over again, he would amazingly say, "This person who's going to get typhoid before they had a single symptom." And as one of his colleagues later said, he was a more prolific carrier of typhoid than Typhoid Mary, because he was in fact giving people typhoid by feeling around their tongues from one typhoid patient to another. And so in that case, the feedback of his successes taught him the wrong lesson. Now, that's a very extreme case. Most learning environments are not that wicked, but most learning environments are not nearly as kind as chess and golf either. And most of the areas that most of us work in do not have just built-in rules and recurring patterns that we can rely on, or built-in feedback that is always immediate, automatically comes, is complete, and fully accurate. So in that sense, things like golf and chess are poor models for extrapolating, and most things people want to learn. And in fact, one of the reasons chess is so easy, you know, relatively speaking, easy to automate, is because it's such a kind learning environment. So there's a huge store of data, very constrained situations, repeating patterns. So the kinder the learning environment, the more amenable it is to both specialization and to being automated.

I'm just going to add a few thoughts outside the scope of your book about that distinction, because I think it's extremely important and powerful in our modern obsession with quantifying everything. You know, in what you're calling a wicked environment, a lot of things that are important can't be quantified, I would argue, and so they get ignored, which is extremely costly. I can't help but think about Hayek's Nobel Prize address, "The Pretence of Knowledge." You understand some of the relationships, perhaps, in a wicked environment, but you can't understand all of them. You can't quantify all of them. You don't understand all the feedback loops. You don't understand the unintended consequences of action, and you're misled about what works and what doesn't work. And when I say that, and I mention my skepticism about science generally applied to social phenomena, people say, you know, first of all, they say, "Well, that's, you know, you're anti-science." I'm not. I'm pro-good science. I'm anti-bad science. But the other, you know, area that I think people tend to, they tend to misjudge the effectiveness of of science, because there are some kind learning environments where we make tremendous progress. So chess is an example. I think about baseball, where people applied statistics and analysis. Bill James was the pioneer of this. And he, when I asked him on Econ Talk if he felt we had pretty much figured everything out, he said, "Oh, my gosh, no. The, there are things we don't understand a thousandfold more than what we do understand." I hope I'm getting that ratio right. It was a figure of speech, it's not a precise measure. But the point is that baseball is a really kind environment. You can get really close to what the value of a walk is. And if you don't pay attention to walks, and then you realize they matter, you, you get a better understanding. I think our understanding of the economy isn't like baseball. And we want it to be.

You brought up a couple of great points there, and you're going to build on James. One of the, sort of, three, three great points I want to glance off really quickly that you brought up with Bill James. I think one of the reasons that it's very clear to people who work in sports analytics that there has been a much greater impact of analytics in baseball than in, say, soccer, is because of the way baseball works with these discrete outcomes and one-on-one interactions and all this stuff you can measure. So it's much kinder than even other sports, which are still kind on the spectrum of the world. And, and what Robin Hogarth, so he called, when you talk about the economy, he said, you know, is I differentiate golf and tennis, where tennis is more dynamic and involves, you know, sometimes teammates, but also human interactions and things like that, anticipating what the other person is doing, but it's still on the kind end of the spectrum compared to most activities. Whereas Hogarth said, what you're doing in the wider world is playing Martian tennis, where you can see that some people are playing a game, but nobody's told you what the rules are. You have to deduce them by yourself, and they can change at any moment without notice. And that's, that's what we're usually faced. And I think that shows up, this, this kind to wicked spectrum shows up in our ideas about things that we can easily automate or apply analytics to. So if you look at chess, the, like the, the chess app on the free chess app on your iPhone can beat Garry Kasparov now, right? It doesn't take a so-called supercomputer anymore. So we've made exponential progress in chess. Absolutely. In a very constrained, but, but not, you know, but slightly less predictable, predictable area of driving, like with, with self-driving cars, made huge progress, but there's still some serious challenges, even though that's an area that's governed by repeating behaviors and regulations and all those things. So that's kind of the middle of the spectrum. Then you go over to something like cancer research, where IBM's Watson has been such a disaster, that AI researchers I talk to were worried that it would taint the reputation of AI in healthcare, because it had so underperformed. And as one of the oncologists I talked to told me, you know, where Watson won Jeopardy, said, "The difference between Jeopardy and cancer research is, we know all the answers to Jeopardy." And so I think I like to think of that spectrum from, from chess to self-driving cars, to the real open-ended questions that don't have these recurring patterns, and that you have to find ways to learn other than just doing the activity and expecting automatic feedback.

So let's get practical for a minute. A lot of listeners to Econ Talk are in their 20s and 30s. Some of you out there in your first career, your first job, your first part of your career. Maybe you like it, maybe you loved it, maybe you don't. And you're worried that if you quit, you're gonna fall behind. And one of the lessons of this book, I hope it's correct, David, but it is, I think, one of the lessons is that quitting is okay. And I think we're often afraid. I know I was, you know, felt this at various times in my career, that, you know, if I step off this treadmill, this escalator, or this, get out of the elevator, get off the elevator at the third floor before I get to the higher floors, I'm gonna have to use the stairs the rest of the way, and I'll never catch up. So what do we know? It's one thing to say Roger Federer got a late start in tennis. Well, I'm not, I'm probably not a lot like Roger Federer. So for an average person who's not extraordinary, what do we know about our ability to, to quote, "catch up" or to make up lost ground, or to just thrive now, you know, we don't need to be at the top, you just throw it, would be great, right?

I think there are a couple of different ways to approach that. So if we look at some of the sort of concrete evidence, let's say, of like head start type programs in, in academics. One of the things that I think is now clear, based on research that's gathered up about, you know, looked at now 70 different programs to try to give kids a jump on, on academics, is there are some good social outcomes, but there's also what's called a ubiquitous fade-out effect of their actual academic skills. And one of the reasons that actually happens is because the easiest way to give someone an apparent head start is to teach them so-called closed skills, like basically just procedures for doing something that worked really well in the problem tests. And, and the problem is, everyone's gonna learn that stuff anyway, and, and you're no good at applying it to new situations. So it's not that they get worse, it's just that everyone else catches up, and you've learned these very narrowly constrained skills. So, so there's that. In fact, I would say, I don't want to get too far off topic here, but I thought one of the coolest studies in the book, and most surprising to me, was the one at the U.S. Air Force Academy. Where the Air Force Academy students come in, they have to take a certain sequence of math courses, and they are randomized to professors, and then they're re-randomized to the next class, and re-randomized again. And they all have to take the same tests, and it's created by multi-professors and standardized and all this stuff. And what the study found was that the professors in Calculus I who teach the most narrow, you'll set, have the students who do the best on their tests. And it systematically undermines them in future classes, so they underperform going forward, because what you actually want to do is teach them how to connect concepts. And it's much sort of broader knowledge. It makes them frustrated, and may make them not do as well in the test, but it sets them up for future learning. That study is just amazing. So it's just one example of where the kids who rate their professors really highly because they did well in their class contemporaneously, because they were given the skills they needed right now for the tests, are systematically undermined and underperform in future classes. So it was interesting to see that the teachers who were rated the best by students were the ones who undermined those students for their future learning. So that's one example of getting what appears to be a head start, but undermines future development. And I think that's sort of a decent analogy for other areas of life.

This isn't to say that people shouldn't specialize at all. I think one of the, if I'd to give a theme of this book, it's not this specialization is bad, it's that society has overvalued specialists and undervalued generalists, and overvalued the early specialization pathway and undervalued the sampling period and delayed specialization pathway. And I think that shows up more broadly in other areas of work. So in one part of the book, I discuss the match quality. No, this is the degree of fit between a person's abilities and their interests and the work that they do. And one of the things that shows up in that kind of research is that people get information signals from trying certain work. And if they, they learn about themselves, they learn how good they are at, they learn how much they like it. And when they use that information and quit, they tend to have faster growth rates in whatever they're doing next, because they've learned something about themselves. And that's maybe they learned that they weren't good at what they were doing before, they're better at something else, or they're more interested in something else, or they're better opportunities elsewhere. But maybe paradoxically, if we think about the received wisdom, the quitters end up as the faster growers, and also often happier.

Right? There's a Freakonomics, well, Freakonomics, the Freakonomics, the Freakonomics homepage now that I discussed in the book, where thousands of people, Steven Levitt, the, the Freakonomics economist, leveraged his readership to get thousands of people to flip a digital coin to make important life decisions. And those could be from getting a tattoo to having a kid to changing jobs, right? Changing jobs was the most common one. And nobody had to change jobs if they flipped, if they, you know, flipped heads or, but they could. And it turned out there was a causal effect of the digital coin flip on the decision people actually made. And the people who followed the coin flip and changed jobs were happier down the line, when he checked back in with them, than those who had gotten the flip that said they should stick with their job and had done that. So there was a causal relationship of changing based on the coin flip and to happiness.

So I think there was, I thought Tyler Cowen talked a little bit about this study and said, basically, his advice was, if we're thinking about quitting, maybe we should. I think that was his exact quote. Yeah. I don't really like that for a bunch of reasons, you know, one of which is, oh boy, the selection issue there. Who chooses to come to the page? Who chooses to flip? But what they were hit, you know, if you really want a lot more information, but it is true, but you should look at his analysis. He tries to establish causality between the coin flip and the subsequent outcome. So there's definitely selection for who comes to the page. But, but I would highly recommend diving into the methodology of that study too, if you're.

Well, but it's more than just to come to the page. It's who thinks it's a good idea to flip a coin to make a life decision. Some people are in, you can imagine somebody who's in desperate anxiety because they can't decide what to do. Someone who finds it amusing. I mean, there's just such an enormous range of emotional things there. But to give the study its due, it does remind a little bit of the placebo effect. It's like, if I can convince myself that I've done this in a certain particular way, okay, I'll be happier. It's like former self-self therapy, maybe? I don't know.

Maybe I do want to add. But, you know, there's other work I discussed in there too, that shows things like teacher turnover, which is this whole, you know, horrible, denigrated thing among school systems, shows that when teachers move, they actually perform better. And so, you know, we don't like teacher turnover because it's an administrative headache, but in fact, I think the, the evidence is that those teachers are responding to match quality information, finding a better fit for themselves. And it's not based on moving to schools with better students, and they actually do a better job of boosting student performance after they move. So I think we should be careful about constraining those kinds of movements. And, you know, you tell me, but I think we, we want low friction in that sort of talent market, basically. And that's not really what we have.

No, I agree. And I also think, and I disagree a little bit with what I said a minute ago. I think change is really powerful. Just change, period. I think, you know, when I moved to came to George Mason, my productivity jumped dramatically for 400 reasons, but part of it was just that I was in a different place. I remember when key people left institutions I worked in, I thought, "Oh my gosh, that's going to be such a blow." And other people stepped up, and other people, new people came in, and were just, you know, gave things to think about that they hadn't thought about before. So when people ask me, "Should I take this job?" One of the things I always ask is, you know, "Do you feel like you're in a rut?" And a rut can be a really comfortable place. You know, it's anything about it more like a hammock. I remember a friend of mine who asked me for us for advice on this, and any his, he was in an incredible job. It was paid well, hit very low expectations, lots of leisure on the job, outside the job, and he was very happy. It was very satisfying. It was a good job, and he did it well. And I had a new opportunity that came into his life that was much harder. It was going to pay a lot more in real terms after cost-of-living changes were taken into account. And I just asked him if he felt, you know, that challenged in his current job, that he felt alive, was it, was it exhilarating? And the answer was no. And he, I didn't tell him what to do, but I, you know, encouraged him to consider the second, the new job. He took it. And I understand it's the data point of one, but sample at one. But, you know, he's, his life changed in all kinds of mostly good ways. And so I think change, I wouldn't tell people every five years should change. Oh, it's a little bit like term limits. We understand why term limits in politics might be good. It just, it seems absurd to say that, you know, after you've stayed in Congress a certain number of years, you should quit, after you've learned so much and you're better props at the job. But we understand that sometimes just leaving is, is, is a good thing. People get into a rut, they get into expectations fall, it gets harder and harder to fire people, and, you know, that, and so you don't work as hard. And so I don't, I think there's a lot to be said for just changing now and then. So I'm a big fan of that. But that's, wait, wait, wait. Some great points. Sorry, can I just, because I think those are those are fantastic points that go to a couple things. The idea that just a change might be useful. And at first, I love that hammock. I'm totally gonna use that. And I'll attribute it to you. That people, we say people get into a rut when really what they're doing more often, like a rut and saying, you know, maybe they can't produce anything. No, it's they're getting into a hammock, which is that they're producing the same stuff, basically. They're not getting off that plateau because it's comfortable, it's pleasant. It's, it's like, it's not like Charlie Chaplin in modern times tightening the same brake that over and over again. That's a rut. It really reminds me of what I used to, you know, for my last book, when I was looking through literature on speed typing, actually. So what turns out, what most people do is we get to a certain speed of typing that's like fast enough, and there's nothing pushing us beyond that, and we settle into it. When in fact, you could get much, much faster. But what you have to do is basically set a metronome a little bit faster than you go. Now, ignore the mistakes, just go at that speed. And you take it up, little by little, and you can like double your typing speed. But that's not our natural orientation, right? It's like, it's to get to a certain place and then sit in the hammock. And, and I, gosh, now I'm mad I didn't use that hammock in my book. But, but it reminds me of one of my favorite, you know, I think this might be related, one of my favorite phrases that stuck in my head in the book was from Herminia Ibarra, professor of organizational behavior, which is, "We learn who we are in practice, not in theory." And I think there's a huge industry of like self-help and personality tests that either explicitly or implicitly want to convince us that we can just take that test or introspect and know what's best for ourselves. When in fact, our insight into ourselves in the world is constrained by our roster of experiences. And so the only way to find out what else is out there and what might fit better is to try some stuff. And while experimentation seems like it might be a waste of time, or it might be scary, that some of the people I think I write about in the book end up sort of, in fact, being generalists just because what they were trying to do is zigzagging till they could kind of triangulate the best spot for themselves. And they end up having a lot of different experiences because that's how they get to know themselves and their skills. It wasn't that they were just trying to be broad, but that is changing things, this experimentation really teaches you about who you are in practice, because we're not as good at introspecting that as we think. And it makes me wonder if some, I was just reading some research by LinkedIn's chief economist that showed one of the main predictors, other than going to a top five MBA program, whether that was because of the school or the student selection, you know, who knows, but of becoming an executive, when they looked at a half-million members, was the number of different job functions that someone had worked across in an industry. And I wonder, you know, maybe that's because, I think the chief economist suggests it's probably because those people get a well-rounded view of the industry, which could be. But, but I also wonder if some of that is they're going through this form of personal experimentation where they learn what's possible, they learn what they're good at, and other interests. And maybe they are better able to find a place where they fit. And I think it's, we have to realize we can't just do that without experiment. And we can't just, you know, introspect everything about ourselves. Like that would, that would seem crazy when we were younger to think that we can just like sit around and introspect and know everything about ourselves without trying things.

That is so deep. I mean, it's, it seems obvious, but one of the things that, you know, we haven't talked about is credible fear of change. And Eric Hoffer, who's wrote a beautiful little book called, you I think it's called *The Ordeal of Change*. I recommend it. It's, I know just how hard it is. And in his case, he talked, he was an amazing stories, a think was blind till he was, I don't know, an adolescent or a teenager. He couldn't read, I know for sure. And at some point, he says, access to books. And it just, he has very little formal education. And he, he just becomes a voracious reader. But a good chunk of his life, he's a, at least part of his life, he's a, he's a farm worker. And he is to move from picking like peas to something else. He talks about how scary it was, because it's pretty, what's gonna be good? It's just a different kind of vegetable. And that's a trivial example. But in the book, so that's an awesome example. A little book about every aspect to change. I'm just changing vegetables. But change is scary. And most of the time, that hammock is attractive, not just because it's fairly pleasant to rock back and forth in it. You don't, you're worried that there's not that other one is gonna be like a really hard chair, can't ever get comfortable. And, and so I think it's very difficult for people to change. And, and one of the themes of your book, which I love and sure emphasizing now, is this idea that it's not just that, oh, you might like this. Mark, you need to try a lot of stuff. You can't figure it out. You need to explore stuff unless you're, you know, some of us are lucky or not lucky, I don't know what the right word is, and find something they love early on. Yeah. And others quote flounder. But it's not, that's, that's a feature, not a bug.

Absolutely. And I think that, that get it. And again, there are a million. There's no single pathway that's right for everyone. Some people will find something early on that's, that's a great fit for them, and that's great. But I think, you know, it reminds me of a psychological finding I mentioned in the book called the end of history illusion. That's this idea that we all recognize that we have changed a lot in the past, but think that we won't change so much in the future. So it leads to some really funny findings, right? Like, if you ask people how much they would pay to see their current and ten years from now, the average answer is $129. But if you ask how much they would pay to see today their favorite band from 10 years ago, the answer is $80. Right? Because we really underestimate how much we change. And that has to, that's including personality traits. Are at the correlation for an individual personality trait from teen years to middle-aged is usually like in the point two, you know, so it's point two, point three. So it's, it's low to moderate. So there are certainly traces of who you were that are still distinguishable, but you're a very different person. And we underestimate that change. And I think between that and our inability to predict the world we're gonna live in, we're basically facing this task of trying to decide how to behave for a future you who you don't yet know, in a world you can't yet conceive. And I think the idea that most of us can do that really well, like a priori, you know, without trying some things, it's just, is, is a very limiting notion, basically. And the idea that you can't better it by sitting in your armchair and pondering it or reading a book that's gonna help you figure it out is probably another delusion.

It's a really an important point. You have a great line in there. You make a more mature one of your people you write about, but about the idea of flirting with your possible self, because you don't know who you're gonna be. And you want to just kind of hang out for a little bit, go on a date, imagine it, try it, do a little of it. And trial and error is just, as a general point, it's, it's grossly underrated.

That's right. And that's because we think of, if, if there is, there's so much lip service to the error part, but who really supports that, right? In practice, say, "Oh, you know, failure is so important." But I don't see anyone's boss being like, "Yeah, this was an important failure for you." Or at least that's never happened to me, right? So we give the lip service to it. But, but what about in practice? And you mentioned dating, which I especially like, because I like to think of careers that way, right? Where we incentivize people to get married to their high school sweetheart, which, you know, if we thought about careers the way we think about dating, nobody would settle down that quickly, or very few people would settle down that quickly, right? It might seem like a great idea to marry your high school sweetheart, was a great idea at the time. But having more experience in the world, in retrospect, it looks like a really bad idea. And, yeah, so I think that's a, it's an important thing to keep in mind. And before we go on, I want to move on to a couple of different things. But when you mention this issue about learning and the importance of what I would, I understand is narrow techniques that work for the, you know, one particular thing, but aren't as generalizable. So that at the time they're frustrating. I want to share my favorite course evaluation when I used to teach in the classroom. So I got a one from this student on a scale of live, okay, five was good, one was bad. And the one, it's really demoralizing. As I look at it, what does the student say? "This course was very unfair. Professor Roberts expected us to apply the material to things we had never seen before." But, but I do think, and I want to use that as a segue to the next topic, which is our ability, and you write about this a lot, and it's, it's complicated, our ability to use analogies and patterns, not in a chess world, not in the kind world chess, but in the wicked world of complex problems. And how powerful that is, but also how limiting it can be. And I want to tell you a story, and then I'll let you use the story as a way to, to riff on your, the ways you write about this in the book. I met a CEO once of a very large company. The company had gone bankrupt. And this was, of course, humiliating to this person. And I don't know why he confided in me. I didn't know him well. I think it's the first time I'd ever met him, most of my office, I was working in the business school. And he is there, almost talking out loud to himself. And he'd, he'd gone to Harvard, one of those top five MBA programs, which leads to a lot of CEOs. And he, he said, "I made it. I made a mistake." When he's talking about why they went bankrupt. I didn't ask him why they went bankrupt. I'm just heard him say, "I made a mistake. He said, I applied the wrong case."

He told me what the cases were. He thought it was gonna be like this case, but it turned out to be like this other one, and it came and bit him in the rear end, and the company went bankrupt. And I thought, what an extraordinary illustration of the challenge of the case study approach. But in a way, that's kind of like life. We see things and we say, "Oh, that's like..." And so I think, "Oh, I'll use that tool to solve this problem." Turns out it's the wrong tool. Right? Right. And I think that is really common among executives, or among all decision-makers, right? And in many cases, I think so.

I write, I have a chapter about analogical thinking. I think what you're referring to. And in many cases, that the case like that executive, where he thinks of, you know, maybe the most traumatic example, or the one that on the surface is the most similar, and that's what he uses as a model of the problem he has to solve. In many cases, that actually does work for us. In a kind world, right? If we are, you know, I fixed my ax drain in one apartment, when, you know, somewhat similar drain gets clogged in another place, like, I'm gonna use the same techniques. And, or because you have a million interactions every day where they're not exactly the same, but in analogy to a pretty similar situation works really, really well. And that's kind of how we get through life, and we don't really have a problem with that.

The tricky part is when we use that same sort of instinct to default to like a single, you know, either most dramatic or just most surface similar analogy, when the problems are much more complex. And I think this can end up leading to a really constrained view. You know, or what, what Daniel Kahneman, my name is first, he called the inside view. Basically, where you have your problem, and you get obsessed with kind of the particular details of your problem. If you use an analogy, it's gonna be like a single analogy, and you're gonna try to like match up details of those problems and similarities. And essentially, you end up having this very narrow view of your own problem. And if you use any analogies at all, it'll probably be a single one. And what you want to do instead is to get out of that, that mindset where you really, you picked the first mentality that came to mind, irrespective of whether it's useful or not, and probably because it has some, what I call surface similarities.

Basically, when you want to do is create a so-called reference class of analogies. You want to generate a whole bunch of analogies, and then think about what usually happens on the broad scale, instead of focusing on the particular details. So Kahneman tells a personal story about this, where he and a group were charged with creating a decision-making curriculum for a school system. And they had a year of meetings. And at the end of that, they, they decided, "Okay, let's have a meeting to talk about how much longer we think it's gonna take for us to finish this curriculum." And they take votes. And everyone votes between a year and two years from now. So the whole entire range is like one to two years of guesses. And then Kahneman realizes there's a guy named Seymour who has seen this process play out with a whole bunch of other teams. And he asks Seymour, who again, Seymour had just predicted no more than two years, like five minutes ago. And now he asks Seymour, "So how did it work with these, with these other teams?" And Seymour says, "Gosh, you know, I never really thought about that, but come to think of it, none of them made it in less than seven years. And a lot of them never finished."

And so the group says, "Wow." Well, and then they discuss their unique personalities and their unique assets and say, "Well, that won't be us." Like, and they stick with their one to two years prediction. Eight years later, they finish. Kind of is diving on the team or living in the country anymore. And the school system doesn't want the curriculum anymore. Right? So instead of focusing on their unique assets and how skilled they were, or even any single analogy, what they should have done is forget about all those little details. You know, forget about their unique, but they think their unique skills are, and gather as many previous cases as they possibly can. Because while those aren't exactly the same, most events aren't unique. And this, this is what's called this reference class forecasting. It forces you, in a way, to think a little bit like a statistician, where you look at what normally happens, instead of getting distracted by all the sort of little details of your particular case.

And one of the interesting findings in this chapter to me was that analogical thinking can be very powerful for problem-solving. But if you use a single analogy, it's not good at all. Basically, you have to use a bunch of analogies. You have to generate this whole reference class of analogies and see what, what usually happens. Whether you're trying to predict what's going to happen with your group, or trying to come up with a novel solution for a problem. You want to come up with a whole bunch of analogies. And if you're, if you're trying to generate ideas, having more analogies causes, you know, leads to generate more ideas for a problem. Or if you're trying to predict how your situation's gonna unfold, you want more analogies and to try to see what usually happens. But our instinct is very strongly to focus on the internal details of a problem. And if anything, to use a single analogy. And, and unfortunately, that basically is the exact wrong way to go about whether you're trying to generate ideas or trying to, to predict what's coming.

The other part of it, which I want to get your thoughts on. I think you talked about this in the book explicitly, although it's, I think, under the surface. It's not just analogies. I like that of thinking as you talked about in the book of tools. And, you know, it's a classic line I've said here before, and I think it's deep, even though it's a cliche. If you only have a hammer, everything looks like a nail. And I think after a while, we forget that we have a hammer. So one of the problems is, is that we like, we're used to having the hammer. It's what we've got. But we really like swinging the hammer too, after a while. And so it's not just it's what you're used to, or you pick this analogy. It's that you get into this habit of using the same analogy over and over because it served you very well. And as you say, it often does. I mean, often a particularly analogy has the right kind of setting, or the right kind of, the right kind of setting is, is extraordinary. And it just, we just take a trivial, stupid example. You know, as an economist, I was like, "Say incentives matter." So the coach of Villanova basketball team, just, I just read this morning, I know if it's true, but allegedly turned out an enormous offer to leave Villanova, go to UCLA. Well, that's a rational, which would say someone who doesn't understand the world, or who only uses the hammer of financial incentives, doesn't understand the role of non-monetary factors, or just other cultural or habits, or fear of the different, or loves Villanova. Who knows what the reason is. But, but an economist's first thought might be, "Oh, he'll take that job because it's so much higher salary." And, and that's just a trivial example where a tool could lead you astray. But this idea that we become attached to our tools in an emotional, almost needy way, is illustrated in your book. And what to me, this is just an unforgettable, extraordinary example of firefighters who die because they get overtaken by a fire. Just talk about that.

So these are wilderness firefighters, particularly so-called hotshots and smokejumpers, who go into forest fires, either hiking or parachute in. And they have to dig trenches, usually in clear fuel, to try to contain forest fires. And an unusual associated team, Karl Weick, made an unusual finding when he looked at, you know, these are incredibly skilled performers. But once in a while, there's a disaster, and a bunch of them die when something unexpected happens. And what he noticed over and over again in those scenarios was that they would die with their tools, with, you know, more than a hundred pounds like axes and things like that, heavy tools, while they were trying to run away from a fire. And on brief occasions, one of them would drop their tools and would survive because they would be able to run effectively away from the fire. And you see in testimony of those survivors, they'll say, "You know, the fire was closing on me, and I just thought I had to put down my axe." And I thought, "Man, I'm crazy. I can't believe I'm letting go of my axe." So they'll like look for a place to to dig a hole really quick and bury the axe to protect it. Meanwhile, they're there, the axe has no use anymore. All they can do is flee from the fire for their lives. And most of them never drop those tools, even when they're ordered to. They die with the tools still on their back, encumbering their running, when they could have gotten away if they had dropped the tools.

And what Weick saw this as an allegory for that kind of, when when all you have is a hammer problem, where the tools become so central to the professional identity of the practitioner, and so bound up with their feeling of competence, that they essentially no longer really realize that they're separate from themselves or their practice. Then their own arms are basically. And so the thought of dropping those tools never even really occurs to them as a way to adapt to an unfamiliar problem, even though it's the one thing that would save their life. So their bodies are found still with, with their tools. And he used that as an allegory to talk about this in different disciplines, where whether it's a real physical tool, or just some common procedure, for example, that people get so attached to that they don't even really realize it's something that they can drop or can change. And that's fine as long as they face the same situation over and over. But when they face an unfamiliar situation, or, you know, rust, like your student said, it's, it's unfair, because you expect them to apply the material to a new situation. Well, that's kind of life, isn't it? Sometimes it's unfair, and you have to apply the material to a new situation. They don't realize that they can use these tools in any different way, or drop them entirely.

And so in all these domains that Weick studied, it's like in air, you know, in commercial airplane accidents, oh, the vast majority of the time, the problem is that when it, when all the signals about a situation show that the crew is in a unique, facing a unique problem, they stick into their familiar procedures anyway, and to their initial plan, until it's way too late. And so there's this, this lack of ability to realize that you can deviate. You're almost like stuck in this pattern. You know, you're in your procedural hammock, basically, to use some of your terminology. And it's really hard to get out of it. It's such a deep thing, actually. I appreciate it enough when I was reading the book, and you're making me realize it now. The emphasis on procedure, which is really important, usually, right? Because it prevents emotional mistakes and prevents spontaneity. That in life-or-death situations is extremely risky. It becomes the thing that kills you. Those tools which save your life, becomes the thing that cost your life.

And, you know, you talk a lot in the book about how many solutions to problems come from the non-specialists. And how often that fresh way of looking at things, the generalist approach, rather than the specialist approach, it's, it's almost, almost doesn't pass the sniff test. Like, how could a non-chemist solve a chemistry problem? That's impossible. And the reason is, the chemistry people just hammering that nail over and over and over again, whatever it is. And somebody comes, "Let's try a screwdriver." You know, that's not a nail. You're doing the wrong, you're doing the wrong thing. But it comes out most vividly in the book with the NASA example. Here, these engineers, we're talking, you talk at length about the challenge or tragedy here. These engineers who, but they're so smart, and they understand so much, but they are paralyzed when it doesn't fit into the procedure.

And that's right. And not just the Challenger. I mean, in the Challenger, a case, they had these incredible procedures that had worked really, really well. I mean, they did an amazing job. And, you know, doing something that's inconceivable, after setting people spacing for catching them when they come. Absolutely. You know, and, and until the Challenger, I think had never lost it, never lost anyone in in space. You know, you know, returning from space. I guess the Challenger didn't. Well, anyway, they had, they had had an accident on the launch pad before. Yeah. And what happened in that case was they had an unfamiliar situation in terms of the temperature. They were gonna have a lot of the Challenger now. Yeah, about the Challenger. And, you know, to make a long story short, there were, there was a small number of engineers who recognized that they were in an unfamiliar situation and raised their voices and and said, "We might have a problem here." But did one engineer in particular, and he was asked to quantify the problem, right? NASA is the, the organization that, that had, they had this mantra hanging on there on the mission room that was, "In God we trust, all others bring data." Right? So I think that sort of set the tone that if you didn't have, and you could see this in, in after the accident, in transcripts of testimony from engineers, they would say things like, "If I didn't have data, you know, I didn't have a right to have an opinion." Basically. And I understand that because you want a rigorous data, generally. It's a good rule. Exactly. At the same time, in this particular case, they did not have the data they needed to make the decision. And so when some of, when this particular engineer argued for a last-second delay of the launch, he was asked to make the quantitative case. You know, "Why does he think the o-rings are gonna fail in this temperature? Show the data points." And the fact was, they didn't have the right data points. His data was primarily based on two photographs that showed at different temperatures that some burning hot gas had gotten past a seal. And at the colder temperature, it looked much worse than at the warmer temperature. But those, one of the temperatures was one of the warmest launches they had ever done, and one of them was the coldest launch they had ever done. And so it spanned almost their entire range of temperatures they'd launched at. And his bosses basically looked at this like, "Well, you have one really warm one and one really cold one, so that's no correlation." And what he was saying was, "I think this qualitative data, these pictures are are telling a story." And that was rejected. It was essentially deemed inadmissible evidence because it wasn't a quantitative story. And their procedure called for strict quantitative criteria. There's an anecdote, actually, the equipment, an anecdote. That's right. It was an anecdote. And it was a hunch. And, you know, later, when NASA managers testified in for the Rogers Commission that was investigating this, Richard Feynman, they made this argument that the engineers, they didn't have a good quantitative case. And Feynman said, "When you don't have data, you have to use reason." And they were giving you reasons. And he gives, he goes on this sort of explanation of the data wasn't there, so you have to find another way to make decisions. And not just stick to like, "Well, our process is, you either have the data or you can't change the decision." It's, you have to recognize that you're outside of the normal bounds and say, "In that case, we need to apply different criteria." And they didn't.

So in this case, like the hotshots, their tools weren't axes and hammers. Their tools were these these procedures that called for very specific types of quantitative data that they were not willing to drop, even though the data that was required to really make the decision didn't exist. And yet, they still had to make a decision. So they just continued with the launch. And we know what happened. And in fact, their next disaster, the Columbia explosion, was so culturally similar that the investigation committee deemed NASA "not a learning organization" because they had not learned from the Challenger launch. So in that case, there were again, a small number of engineers who were concerned that a part of Columbia had been damaged and asked the Department of Defense for high-res photos of a portion of the shuttle they thought were damaged. And this was outside the normal procedure. Again, this is hallowed procedure, that's kind of the tools for NASA decision-making. And their superiors found out, went to the Department of Defense and apologized for contact outside of normal channels and said it wouldn't happen again. And then the shuttle exploded. So in both cases, it was this strict adherence to a procedure that works really well when they're in the bounds of their experience, and that was disastrous when the information they needed to make a decision was no longer available, and the quantitative case couldn't be made.

I can't help but wonder if on virtually every launch, there were people who said, "Wait a minute." And most of the time, when they were ignored, it turned out okay. You know, I've been, after many terrorist attacks, there's always the reports that we had some information, if we'd only, or a better example, be Pearl Harbor. You know, they're there, in an infinite time, infinite resources, Pearl Harbor should have been a surprise. You know, of the zillions of telegrams that were intercepted at the time and, you know, decrypted, there was some suggestion that. And so, of course, afterwards, there are people who said, "We should have been aware of this risk." Do you think that's the case? And this, in this NASA culture story? I mean, it's an incredibly powerful story, even if it's not a hundred percent as straightforward as it sounds. It's still a very useful thing to keep in mind about relentlessly using tools all the time. But do you think there are other times when people raised those issues and just got ignored and it turned out okay?

Yeah, I mean, I think I think you nailed that, right? So one of the guys I spent a lot of time interviewing for that chapter was Alan McDonald, who was the head of the rocket booster program for NASA's contractor Morton Thiokol. So he was on the famous conference call where they decided to go ahead with the launch. And, and one of the things he said was, you know, "If we had effectively delayed the launch without the proper quantitative data, the feeling probably would have been, it would have been fine. We should have gone ahead. The people who stopped it are kicking little to use his language." And it wouldn't have been deemed like, "Oh, that was a great decision." Right? Because we don't know the counterfactuals. And the Challenger, again, that I think that was probably more unique because the temperature was so outside of their normal bounds that that specific instance might not have been the same. But I'm sure at a lesser sort of magnitude, those things were happening constantly, probably. And that's why in that chapter, I had a really, I interviewed a million times. In fact, I also talked about a commander of pair rescue jumpers in the Air Force. And when he has to make decisions in Afghanistan with with very little information. So there's an explosion in a caravan, and his pair rescue men have to go and essentially rescue an unknown number of injured soldiers. They don't know what situation they're getting into. And it turns out that he makes sort of a very difficult decision that turns out really well, and everyone survives. But he was so adamant that, "I better include a quote where he says, 'Maybe it was luck. That decision could have turned out differently. And then even if I used the right procedure, it would be a bad decision because I would have had to go explain it to ten families.'" And so I think that was an important thing to include because, you know, even in something as simple as like blackjack, if you play perfectly, you win, you know, whatever, dozens of more hands in a thousand or something like that. If you play perfectly. And we don't get that many takes. And most of the things we do. So I think we have to be really conscious of the fact that a good outcome doesn't always mean that we used a good process, and vice versa. And, and I think it's really, it's really difficult. I'm really curious what would have happened if they had not launched Challenger. Because I think internally, there would have been a lot of feeling of that that people are being overly cautious. And we're talking about the space program, right? Like, I used this quote where engineer Mary Shafer, a former NASA engineer, said, "Perfect safety is for people who don't have the balls to live in the real world." Right? And that kind of became a famous quote because you can't have perfect safety. You have to take some risk. And so what that chapter, I think, is about is sort of trying to calibrate to minimize two different types of errors. The, the errors of of mindless conformity, where you follow the procedure and use the tools no matter what, and balance that with errors of reckless deviance, where you're our following procedures and always sort of ad-libbing and improvising. And, and what I try to get at in that chapter was how do we try to diversify the tools of an organization so that we, we strike a balance between those errors of deviation and errors of conformity.

I'm just thinking of Nassim Taleb, who's taught me that expected value is the wrong way to think about rational decision-making. And often, and you don't want to just look at the odds of a bad outcome, you want to look at what, what would be the consequence of that bad outcome. And that's really hard for us to do. We often just say, "Well, it's unlikely, but yeah, so I'd have to worry about it." But if it's unlikely and it means ruin, you want to say really far away from that. You want to be, as he says, anti-fragile. And I, it's a fascinating, I think, part of human experience that we're not really, we struggle to deal with uncertainty. You know, when I talk with people about this, they'll say things like, "You know, that was a bad decision." And their reason for thinking is, "Because it didn't turn out well." Right? That's not a very good, that's a bad way to think about the feedback between your choices and your outcomes. And, you know, I don't want to overstate. I said earlier, the importance of change. And the parallel of change. And, you know, someone listening might say, "Oh, great, I'm gonna quit my job tomorrow." And you're not really badly. And, and you might decide that I gave really bad advice. And in a way, you have to quit your job ten times. Yeah. And we don't live long enough. We don't live long enough to really have to quit your job. But like, it's like blackjack. You have to quit your job a thousand times so that the 12 extra times that you win outweigh the losses to make it a valuable thing. And that, I just think these kind of, that story, I'm not going to go into the details, but it's so powerful in the book of that Afghanistan commander. It's, you realize how well, one thing you realize is that you have a really easy life in the decisions that you make that are, you're worried about, or trivial, come to what he had to deal with. It's just, it's hard to figure it out.

But when I was interviewing him, he, you know, he's a very stoic guy. And he, like, at that, when he talked about delivering this decision, with essentially a major part of his decision was that he was not going to accompany his men on this rescue mission because they didn't know how much space they needed. They were space constrained. And he was guessing how many patients they would have to deal with. And some of his men sort of rebelled at that, or even suggested that he was afraid. And he broke into tears when I was interviewing him about this, which was totally unexpected to me. You know, saying that the peer leadership is hard. And the way I used that in the book is to suggest that this incredibly strong cohesion culture made sure that he would not deviate recklessly from normal procedures. But at the same time, he had enough autonomy and and pure outcome accountability that he was willing to deviate and ad-lib if he thought it was tremendously important. But that the bar was really, really high. And so I think that's kind of the best we can do is try to set up these forces that cause people not to conform excessively mindlessly, and not to deviate all the time and ignore standard procedures. And, and then hope that over a large number of people, that that gives us a little bit of that blackjack advantage, even when individual decisions go wrong.

Well, I assume that commander, he was afraid. And I mean, that he knew he was afraid. And he probably hated the idea that by making that call, he was being selfish. And that's an unbelievable dilemma, right? Where your, your brain is telling you, your brain is telling you, "Go, because that's the procedure, that's what, that's the right thing to do." For some reason, maybe it was fear, he imagined that it might be a good idea to stay home. And it turned out great at that time, that one time. They should point out. But being aware that he may have come up with that solution partly out of fear, especially since it was a particularly unknown set of unknowns, probably haunts him. Terror. I don't think personally, I don't think he was afraid of dying, because he had gone on many of those category alphas, so-called, those very dangerous, you know, situations with lots of injuries before. I think, yeah, I think he was afraid of having to make a second trip back there, basically, if they didn't have enough room for patients. But I think, as he said, was the worst outcome for him, then dying would have been. He would have had to watch if something went really wrong, he would have watched his whole team die, and then have to explain that. And I think for those guys, that's a fate worse than dying. And I don't, of course, I've done nothing about this particular individual. I'm really working on the fictional art version of the story and playing it for for educational purposes.

Can we shift gears? I want to talk about something in the book that, you know, wait, I guess it precedes what we're talking about, which is about problem-solving generally. I'd like you to talk about the Flynn effect and the testing that a man named Alexander Luria did of pre-modern IQ. And because it eliminated a lot of things for me that relate to past episodes of EconTalk and questions of how to think about about the world. Tonight, I'd love for you to share that.

So in short, the Flynn effect is the name for this, the the rising scores on IQ tests around the world at us in the 20th century at a steady rate of about three points per decade. And basically, to help the whole curve just shifting over. It's not particularly concentrated in a in a particular part. It's not concentrated in a particular area of the world. It is, however, most extreme in the more abstract sections of tests or on the more abstract tests. So there's a test called Raven's Progressive Matrices that was created to sort of be the, you know, I don't know if you want to say, like the end-all of a cognitive tests, where if it required like, no, it wasn't based on anything that you had learned in school or studied in the world. So it was just, you get, you get these abstract patterns, and one is missing, and you just have to deduce the rules from the patterns and fill in the missing pattern. And so this was supposed to be the test, like, should Martians alight on Earth, that would be able to determine how clever they were, because this test wouldn't, wouldn't require any sort of cultural background. And what James Flynn found was that not only was that not the case, but in fact, the biggest gains in scores over time were specifically on this Raven's Progressive Matrices. So each generation did better than the last to the point where, you know, our great-grandparents would look as if they were mentally handicapped, because they would score these tests are always normed so that the the mean score is a hundred. But in terms of the actual number of questions they got right, they would look like they were impaired compared to us today. And, and my impression is that my great-grandparents are no smarter or no stupider than I am, in terms of reliability. Right? But you are much more equipped for that, that kind of like, for that pattern. You feel it, right? That's sort of those abstract. And, and if you look at improve, some improvements in scores on material that's more related of what people learn in school, have like barely budged, if they budged at all. And in cases where they've budged on on vocabulary, it's all, it's largely come on abstract words. So things like law or pledge or citizen, as opposed to, you know, much more concrete nouns. And so the Flynn effect is the name broadly for this increase in IQ scores. But an interesting facet of it is that it's, it's more apparent in the more abstract tests.

And tonight, sister Alexander Luria, you mentioned. So Alexander Luria was a brilliant young Russian psychologist who in 1931 decided that he wanted to use essentially, when this is a time when the Soviet government was forcing agricultural land to become large collective farms and for industrial development, took over socializing agricultural land. And Luria saw a natural experiment possibility here, where he said, "Okay, I'm gonna go out to these areas of these very remote areas of what is now Uzbekistan, and see if going through this shift from subsistence farming and herding to collective agricultural work and vocational training and, and in some other sorts of school opportunities, will that change the way that people think? Like, will it change their habits of mind?" And when he went out there, he learned the local language and everything. He brought a team of psychologists. There were some areas that were so remote, they were still untouched. In some areas that had gone through various degrees of transformation to collective farming from subsistence farming and herding. And so he started studying those people in both conditions. And what he found was that, you know, the so-called pre-modern people who were subsistence farmers or herders were very constrained. And I don't mean that in a way to denigrate them, but they were their habits of mind were very constrained to their exact experiences. So he would ask them questions, and they could only answer for things that they had directly experienced. Whereas the greater a dose of modernity they had had, whether that was some exposure to school, or to vocational training, or even just collective farming, the more they could start to abstract and make generalizations and, and use formal logic and sort of answer questions about things that were they had never experienced.

So, and this work, even for really basic things, like the people in the more pre-modern condition, if you gave them, you know, a circle and a dotted circle and asked them to make groups of shapes together, they wouldn't put the circle and the dotted circle together because they would say, "Well, one of these is a coin, and one of them is a watch, and you obviously can't put those two things together." Whereas the people who had some sort of dose of collective work or some school, even if they didn't know the names of the shapes, they would be able to see that they had sort of abstract qualities in common and would group circles together. And so that was some of the most basic examples, but it went all the way up to much more important abstractions, where the people who had a dose of modernity were much more able to transfer their knowledge to unusual situations. And that's not to say that that one way is better than the other. It's just that one is much more adapted to the kind of need for transfer of knowledge that we experience on a daily basis, basically.

The one I found so striking was the three adults and a child are shown. And the question is, "Which one's different? What doesn't belong here?" And supposedly the person couldn't answer it. And I said, "Well, don't you see that the child doesn't belong?" No, it says, "The adults are working. They need the child to help them get stuff when they don't have it and to run errands." So you take the child out. And, you know, there's something childlike about that way of thinking. Almost. They had the same example. Similar examples, like you tell the story of, there's a hammer, an axe, a saw, and a log. Which one doesn't belong? They think, "I can't figure it out." Because he says, "Well, if, why would you throw out the log? Then what's the use of the saw?" Right? Right. It has no use. So all they can think of is like, right, like three are tools and one's a log. So, well, you could, I guess, you know, the hatchet works because you can use that to cut the log. The knife isn't as useful, but you could hammer it with the hammer into the log. So it's, it's all this very practical kind of thinking. And again, it's not, it's not worse. It's just more adapted to a different kind of situation. And that, that was this repeated pattern that Luria kept seeing. Like where he could ask sort of formal logic. Like he had this, this one question where he would say, it was sort of a logic puzzle. He'd say, "Cotton grows well when it's hot and dry. England is cold and damp. Can cotton grow there or not?" And sometimes, if you really pushed the farmers, because they had, they had direct experience growing cotton, they would resist answering this question. They would say, "I've never been to England. I can't tell you." And, you know, the psychologists would say, "But I just told you it's cold and damp, and as you know, cotton grows well where it's hot and dry." And they say, "Well, I've never been to England, so I can't tell you." And if he said, like, "Well, you know, but what if words imply, like, if a place is cold and damp, that grow there?" And they would finally maybe say, like, "Okay, it's not gonna grow there." Well, if it's cold and damp. But then he would ask a separate, very similar logic puzzle, with different details, which was something like, "In the far north, where there's snow, all bears are white. Novaya Zemlya is that, is that example he used, is in the far north, and there's always snow. What color are the bears there?" And they would absolutely refuse to answer. No amount of pushing could get them to answer. They'd say, "Your words could only be answered by someone who's been there." Even though they had previously, sort of, with pushing, answered the question about England, because they had experience growing cotton. Whereas with the bears, they would absolutely refuse to take knowledge and transfer it to another domain. It would say, "How could anyone know? You'd have to ask someone who's seen it." And we take for granted the fact that we do this kind of knowledge transfer all the time. So we're able to use knowledge that relates to things which we have never directly experienced.

So in McGilchrist, in his book, "The Master and His Emissary," so on EconTalk, we talked a while back. And he talks about, I mean, this just all kinds of bells went off when I read those stories, because of the McGilchrist book. So McGilchrist talks about the right side of the brain, unless the right hemisphere and the left hemisphere. The left hemisphere is is analytical, its precise, it it tells itself stories all the time. If it can't fill in the blanks, it's really good at it. Patterns. And it's a little bit reckless because of that. It oversamples, it overestimates its ability to make the world conform to the to the things that it sees. The right side of the brain is holistic, connected, etc., etc. McGilchrist has a lot to say about that. And it's, it's a fascinating book. And it was, I hope, a good conversation. But I couldn't help thinking that you're the Luria examples are perfect for this distinction. The inability and unwillingness, both. It's two things. The inability and unwillingness of the of the Uzbekistan farmer to weigh in on cotton growing in England strikes me as incredibly wise. It's like, you know, the 800 SAT student nails it. Oh, yeah, no cotton in England. And polar bears are white in Novaya Zemlya. But those farmers had it, had a rich, as I don't think of it as they'd only adapted to their experience. I see them taking a much more connected view of of the universe, of what we encounter, of what we perceive. And that the three adults and the child is just such a perfect example of it's like, it's a silly question. And it reminds me a little bit of the trolley problem, you know, that these sort of abstract moral dilemmas. You can save one person if you, one person will die if you switch the track the train is on, but otherwise five people die. This is something that no human being, almost no human being, has ever actually had to do. And, you know, there's different versions of it. There's a, there's a horrible version. Is a fat guy on a bridge, you can push him over the bridge and stop the train. Would you do that to save the five lives versus the one? Would you be that active? I remember telling my adolescent son, he said, "Well, what if the guy is bigger than me? What if he pushes back?" And you're there's a temptation to say, "Well, that's a stupid answer. You'd understand that's not the point of the problem." But that is the point. That's what life is like. Life is complicated. It's never that simple, right? Oh, yeah. We're just trying to abstract from that. But those farmers understand that that abstraction is risky. And it just struck me that that this nuance between analogy, case study, transferring insights from one field into another, is one of the most powerful things that human beings can do. And it's unbelievably important to bring us to the modern world. And at the same time, it's dangerous. And you have to understand its limitations. And those farmers, they're on they're out there in the wicked world 99% of the time. And the SAT kid is in the kind world all the time and thinks that, you know, everything's straightforward. It's all connected. It's all, you know, linear and mathematical that I can solve for X and Y and just a simultaneous set of equations. And that's not the way life works. Unless it does.

I think that's a, I think that's a great point. And I think it gets again to this, that you're one-star review where the student said, "This isn't fair. You're asking us to apply the knowledge of things we've never seen." Right? And that's kind of what some of the pre-modern farmers were saying, right? "I'm not gonna answer this question. You're asking me to apply knowledge of things we've never seen." And it's an essential thing for us to do. But it can also be a dangerous thing for us to do. And I think it's important to recognize when we are doing it. Right? Flynn himself, I remember, this is a little bit of an aside, but he, he told me this story where I think during a Montgomery bus boycott, where he was, his father, who he said was very much a, you know, a man, he kind of pre-Flynn effect. He didn't basically, so he would not have been in Flynn's estimation as, as far along on the Flynn effect and the rising curve. He said, a man very much like grounded in the, in the literal, I think was how he put it. Said that he, his dad made some derogatory comment about the bus boycott. And Flynn said, "Well, how would you feel if you woke up tomorrow and you were black?" And Flynn told me that his father said, "That's the most ridiculous thing I've ever heard. Who do you know who ever woke up black?" Right? And so it's like, well, laughing. But it's, it's a tragic, it's an unbelievably powerful counterexample to my story of my son and pushing the guy over the ledge. But it's not, but it's not. But I think that's important because these aren't, and I think you're identifying this, these aren't zero-sum things. Right? It's a power, and it's a danger. Yeah. And I think that's the really important thing to recognize. And I think it goes along with that issue of tools, which is this is a kind of knowledge transfer, an abstraction we use without even knowing it on a daily basis. And I think we would be better off to sort of recognize it. Is we're gonna wield its power one way or the other, you know, but where sometimes blind to to what's really going on. And it would be better off we sort of thought about our own thinking in order to to kind of limit our errors a little bit. Yeah.

I'm interested in mindfulness and meditation. And one of the things I'm thinking about right now is how to be mindful about our mindfulness. It's not, it's not easy to do. You need, it's a very high level of self-awareness to realize you're applying a tool that you've used a thousand times, and maybe the thousand and first time is not the time to use it. It's very, that's it, just it's very powerful.

Well, I want to close with two things. First, I want to say something about Adam Smith. Because we're talking about specialization and the range of things we bring, skills we bring, and the range of tools we bring to a problem. And, you know, Smith saw specialization correctly, I believe, in "The Wealth of Nations" in 1776. He was able to understand, despite the relatively small amount of growth he was experiencing in the world at that time, that specialization combined with trade, and I don't mean just international trade, but exchange generally between people in market settings, that that is, that the great engine of of growth and the great engine of the transformation of the standard of living of the world. And continues to work that way. And it's the idea that you don't have to do everything for yourself. The idea that you can rely on others is one of the deepest ideas in economics. And I want to let you talk a little bit about about that. I often point out that in a world of, say, in a world of a thousand, take the thousand most talented people and put them on a place that's rich with resources, on an island, they're gonna be very poor. I don't care how talented they are, how smart they are. You can pick them, you can decide who they are, you can pick it for a range of skills. There's just not enough scope for specialization among a thousand people to have a modern standard of living. And the reason you and I can have a conversation across Skype and have something that exists called EconTalk is because we live in a world of seven billion people. And we're in a with hundreds of millions of them indirectly through exchange and trade. And that's allows me to specialize as a podcaster and you as a writer. And that's just not imaginable 500 years ago. It's not imaginable really 200 years ago. And I often use the example of a pediatric oncologist. And I'm sure there are specialties within pediatric oncology. And most of the time, that's a really good thing. And you can't be a great pediatric oncologist as a hobby, is my guess. So talk about that. Balance. Specialization is in some dimension necessary for a modern era living. But I'd say the theme of your book is it can go too far, as you've said earlier. So talk about that. And also, yeah, I mean, right here, you are the podcaster, economist, interviewing the former sportswriter with the geology masters. And Adam Smith, of course, I learned how much his personal range from your own writing. I had no idea he wrote about happiness and such things like that. I think there's an issue of semantics for one, right? Because, and this is, and this is a difficult, this is one I'm gonna have to try to harp on as I try to discuss my book, which is that what it means to be a journalist in one era is not.

The same as what it means to be a generalist in another era. All right, so there's been like a flint effect of specialization, right? The background itself has changed. And so it's very much one of the reasons I tried to make sure to include a number of scientists in the book and doctors was that not just as quoting him on a topic, but actually talking about their own careers. Is that I think for most people from the outside, they may look like the epitome of specialization, a scientist, right? And I sort of thought about that because I was a science grad student and and I wanted to think, well, okay, what, you know, if these from the outside to most people, this is the epitome of specialization, so in that sense, what does it mean to be broader than they have to be or to have range, you know, to use my own terminology? So I think some of what's in the book is I even get at what that even means to expand your breath when you don't really have to. And so, so many practitioners today would be compared, are, you know, are more who I would think of as being broad, are still more specialized than someone was hundreds of years ago, for sure. So I think it's very much context dependent. Into today, of what it means to have more breath today. And and I think there's some evidence, and I go through some of this pattern research in the book, that actually, at least within the sort of 20th century and beyond, there's an increasing importance or opportunities for generalists. Where we see like, and the outer Kirk in the book, this this inventor who then who won R&D Magazine's Innovator of the Year and then decided to study inventors, finds that the relative importance of deep specialists and comparative generalists, and the way he characterizes this by looking at millions of patents, and you see people who kind of drill down into a certain area more and more and more, and others who work across like a large range of technological classes. And he sees the importance of these different types of individuals to breakthroughs changes over time. And that in around World War II, the the importance of the specialist, their contributions were sort of peaked. And it ebbs and flows. And then right now, it's declining. And he doesn't know for sure why, but he thinks some of the reason is that there's so much knowledge out there, and communication technology allows it to be so effectively transmitted that there's way more opportunities to, and and more likely successful opportunities to recombine well-characterized knowledge it's already out there in new ways, than to actually push the cutting edge just a little bit. And so I think even within these very technical domains, I tried to take them on and examine what it means to be a generalist within that given context, as opposed to just being like a dilettante, to someone who's not particularly interested or good at anything, which is what I really want to differentiate from a generalist.

You know, I think this gets to something that I did, I i've heard, I don't want to be wrong here, that I've heard you talk about this for, you know, maybe not, sorry. But when we think of technological transformation, you can think of Robert Gordon, and he says these, well, all the biggest, we're actually slowing down, right? Like we've, we've made enormous strides, and now technological progress is slowing down. And I think maybe that discounts some of the more serious applications of communication technology, which communicate the results of specialists, of who we still, we still desperately need hyper specialists, but their contributions can be more broadly and quickly disseminated, which provides a lot more opportunities for people who are broader than specialists. And I think that's why some of those things like InnoCentive, these, you know, which is set up by to solve for like random people to solve the problems that have stumped pharmaceutical companies, work for large amounts of money, right?

Describe that site quickly. It's so started by a VP of research at at Lilly, where first Lily would post problems that had stumped their chemists. And so many of them were solved by just like random people outside, coming from other disciplines and bringing some totally separate knowledge. That this VP turned it into its own separate company that helps other companies post problems that have that they've gotten stuck on for just outside solvers. And so I think you mentioned earlier a problem that stuck NASA for 30 years, got solved in six months by a guy from a totally like a retired cellphone engineer who was like living on a farm in New Hampshire, and just brought a totally different approach to it and was like, you know, I kind of can't believe you guys didn't think of this. And so as I think the disciplinary boxes get more and more narrow, right? We don't have disciplines, we don't divide up study into disciplines because that's how the world is. We do it because it's easy for us to categorize. And then we try to like afterward, put the world back together to understand it. More complicated, more complexity. And I think as disciplinary boxes get smaller and smaller, more often the knowledge that people need for their problem is outside of that box. And so it's important to have those specialists, but it's also more important to engage people from outside and people who are broader. And I think we've really seen that in medicine, where specialization has been inevitable and fruitful, and also incredibly problematic in ways where a cardiologist used to be highly specialized. Now a specialized cardiologist is some who might only study cardiac valves, like the little flaps that let blood in. Now the electricity, the rest of the heart muscle is totally out of their purview. And what happens in that case is everyone works on what's called surrogate markers, where someone might have a problem. And so that cardiologist might fix the problem with the valve, but what you really care about is if that person is gonna have a heart attack, stroke, or die. And what we find in many cases is that a specialist affects the surrogate marker. And so everything's great. And then the person just dies with a better heart valve, if it has a heart attack and stroke at the same rate. Or we regulate blood pressure, and you what you get is people dying at the exact same rate with great blood pressure, because everyone's working with surrogate markers. And so I think that's inevitable and useful and also very problematic. And that we need to recognize both sides of that.

So normally at this point, I would say thank you for being part of Econ Talk. But I'm gonna add a personal note here. And we're way over normal time. I don't care. I hope listeners are enjoying this. I am I want to let listeners know that yesterday, you and I, David, tried to record this episode. And you live in the DC area. I live in the DC area. And we thought, I thought, you know, at mostly as I do over Skype, but you live fairly nearby, we'll do it face to face. And we had technical problems in making that recording. And it didn't happen. I dragged you down to my office in in downtown DC and wasted your time. And I was embarrassed that didn't work out well. And I offered to take you to lunch, partly because I thought we'd have a nice conversation, well, partly just because I felt bad. So we went to lunch. And we had a great conversation. I thought at lunch, I was a little uneasy about it because I thought, you know, we may end up talking about stuff we're gonna talk about in the actual interview. And it'll sometimes it'll sound stale if we've already talked about it. But I said, oh, well, whatever. And so we had that conversation. And I am confident, although I have some confirmation bias here, so I have to be careful. I'm confident this is a dramatically better conversation then we would have had yesterday face to face, where we'd never met before. I should that's important. We'd never met each other face to face. We'd only had our conversation over the Skype when in 2013 with your first book. And because we met, I think this conversation today, a day later, is much better. So I think it's a small example of failure. It's really good to something really, really excellent. So I want to thank you for your patience yesterday and your patience today. But there was a benefit from that that trial and error. I think.

Oh, that's great to hear. And I didn't know you were embarrassed. You absolutely shouldn't have been because it was out of your control. So not your fault. And then you bought me lunch. And I ended up with like a half dozen new things on my reading list, which is since I have no idea what I'm gonna do next, actually tremendously important for me to get suggestions to read things that I, you know, wouldn't otherwise come into contact with. And it's one of the perks of my job to get to to scrutinize my own ideas and other ideas with people like you. So there is absolutely nothing to feel embarrassed about. It was, you know, you can't control the ventilation system at that building. And it was a pleasure. So I didn't expect it, but I really enjoyed the conversation.

So let's close with a tougher personal question. How does this experience to write this book, which took you a while, it's a lot of digging and and interviewing and thinking and reading and then writing, has it changed your own perception of yourself and how you see your own career?

For one, I don't have no idea what I'm gonna do next. Like when I recently went to MIT, just alone, support, satellite, not nearly as much as I would have been. So I have career changed a lot. And constantly been told that it's a bad idea. I will get behind and again, and this is my for sure, some my own confirmation bias, right? But I now think that these things, this changing, this experimenting, which in many cases did put me temporarily behind, but then my growth rates were very quick. Ultimately, what I did, I think, is a crude sort of group of skills where I may not be the very best at any one of them, but I kept sort of zigzagging from one area to another where I could be pretty good at a whole bunch of different things, which ended up with me being able to compete on my own ground. So I'm not in zero-sum competition with anybody else for whatever beat I'm writing about. And that was the same thing at Sports Illustrated. I ended up writing, you know, a book that found an audience because my science background was the most useful thing there. And it meant I wasn't waiting in line to be the next NFL beat reporter. So I think traveling this zigzagging journey has led me to have a toolbox where I might not have the absolute sharpest tool in any one of those things, but I end up competing in a place where I'm the only one. And so I'm just competing against myself. And if I can do something interesting, then then I can have some good outcomes. And I'm now confident that, you know, I've read quotes like this, just like this, from Christopher Nolan, the director, and Eric Larsen, the writer, where they say, between projects, I just have to read with no apparent purpose to find my next project. And I used to criticize myself for that, thinking it was inefficient. And now I think it's actually what gives me this expansive personal search function where I might come up with projects that others don't. So I feel more comfortable and emboldened in the fact that I am proactively not going to look for another job quite yet. And and just go back to letting my mind roam and hope to alight on that next project. Where I used to always think every time I did a project, I'll never find another good one. Now I feel much more confident that my meandering actually will continually lead to new projects.

I guess that has been David Epstein. His book is Range. David, thanks for being part of Econ Talk.

It's a pleasure. And thanks for challenging and challenging all my ideas. It really helps me sharpen my own thinking. And and a lot of my ideas should be critiqued. And I appreciate that the way you do it.

[Music]

This is Econ Talk, part of the Library of Economics and Liberty. Former Econ Talk, de Contort, where you can also comment on today's podcasts and find links and readings for allegiances. Today's conversation. The sound engineer for Econ Talk is Rich Koster. I'm your host, Russ Roberts. Thanks for listening. Talk to you on Monday.

[Music]