An Evening with DAVID EPSTEIN, Author of Range: Why Generalists Triumph in a Specialized World

At this time, I'd like to introduce the awesome human beings we have on stage tonight. To my left here is Molly Sullivan French. She has 14 years of experience in television, 11 of those years in sports. Molly currently works for the film. What happened in those other three years? As a host and reporter. And before that, she was the Philadelphia 76ers sideline reporter for seven seasons. A graduate of the University of North Carolina, Molly currently lives in Philly.

Further to my left is the man of the hour, David Epstein. He is the author of The New York Times bestseller, *The Sports Gene*, and the book we're all here for tonight, *Range: Why Generalists Triumph in a Specialized World*. He has master's degrees in environmental science and journalism and has worked as an investigative reporter for ProPublica and a senior writer for Sports Illustrated. He currently lives in DC.

Now, if you haven't heard the big news, David's new book, *Range*, has just been listed as an instant New York Times bestseller in its first week. We give it up for David! As a bookseller, this is actually a pretty big accomplishment. It's not an insignificant feat. So, we're here tonight to keep the momentum, momentum building for David and the book.

Now, I could list all the praise this book has received from *The New York Times*, Malcolm Gladwell, Daniel Pink, NPR, Susan Cain, Adam Grant, *The Atlantic*, and many other more outlets and people. But I'll leave you here with my favorite blurb about the book from author Amanda Ripley: "I want to give *Range* to any kid who is being forced to take violin lessons but really wants to learn the drums too. To any programmer who secretly dreams of becoming a psychologist. To everyone who wants humans to thrive in an age of robots. *Range* is full of surprises and hope. It's a 21st-century survival kit."

Without further ado, please join me in giving a warm Harrisburg welcome to Molly. Indeed, Alex, thank you so much. I'm gonna put you on the spotlight here for a second because I think we all specialize in one way or another, right? I mean, think about that for a moment. Alex, your Twitter profile: Bookseller with #TTP. That's Trust the Process. That's kind of a Philly thing. So, just a little highlight there. But I, you know, I'm so excited to be here in part because my big takeaway from reading the book, and I think it's such a vital read for everybody, right? And that's what we all share being under this roof, that we're all gonna take different lessons. But I think that the big takeaway for me is that there, there's not one path to success, but rather there's the most effective and efficient path to excellence. And so that's kind of what I took from it.

How many experts do we have in the house? How many experts? Come on, don't be shy. How many experts? Perhaps all right, fair enough. I like the moxie over there. I like the moxie. Perhaps a better question is, how many of us really want to improve at performance, right? Dave? Okay. All right. So, case in point, on the cover we have here, at the very top, I loved *Range*. Now, big words come in from Malcolm, right? That we talked about a little. But why was that significant to have him at the top here? Well, one, I think he's one of the storytellers, you know, of a generation. But also, the rest of the words on that blurb are, I think, something like, "For reasons I can't explain, David Epstein makes me thoroughly enjoy the experience of being told everything I thought about something was wrong." Which, as a reader, I just think is kind of a cool blurb, first of all. But the, the book part of the idea for the book came out of a debate I had with Malcolm previously, where after my first book, *The Sports Gene*, as he would say, I devoted several pages to attacking his work. That's how he puts it. And we were devoted to the, the MIT Sloan Sports Analytics Conference was founded by the general manager of the Houston Rockets to have a debate that's on YouTube, 10,000 hours versus *The Sports Gene*. I was criticizing the science that underlies the 10,000-hour rule for four reasons I, I can explain. But he's very, this was the first time we were gonna meet was for this debate, and he's very clever, and I didn't want to get embarrassed. So I tried to anticipate his arguments. And I knew we were talking about athletic development, particularly. And I knew he would have to argue for early specialization as a primary advantage. So I just went and looked at, I was the science writer at Sports Illustrated, and I said, "I'm gonna go, if that's the hypothesis, let's go look at what the science of athletic development says." And it turns out that pretty much everywhere you look, athletes who go on to become elite, to actually have what scientists call a sampling period, where they play a range of sports, often in an unstructured manner. They, they gain this broad variety of skills that scaffold later learning. They learn about their interests, they learn about their abilities, and they systematically delay specializing until later than their peers, with golf as a possible exception that we can talk about. So I brought that up on stage, and afterward, when he came off, he sort of said, "You know what, you got me on was like, I didn't have anything for that." Because it was incompatible with his hypothesis. So he sort of became running buddies after that and started talking on her own time. And we were invited back in March, and in this one is also on YouTube. And at the end, he says, "I now think I conflated two ideas: the idea that it's important to have a lot of practice to become good at something, with the idea that in order to become good at X, you should start as early as possible doing X and only X." And so I thought the way he nuanced his view is very representative of some of the thinkers later in the book who developed good judgment about the world. So that's a long story, but so it was important to me because I think it was, instead of seeing our debate as zero-sum, I think we both went into it trying to learn and nuancing our views, and it became like a very productive intellectual partnership. For me, it was very much a "real recognize real" moment because you guys both went toe to toe in a healthy debate. I invite you guys to look that up on YouTube. But research, like we're talking before we stepped on stage here, doesn't always say what we think it says, right? So I'm gonna stick with the cover. I can hear Mary Poppins singing from my daughter's room because the cover is not a book, right? But I'm gonna stay with these keys, and I think there's 12 of them. Count 'em up. But the significance of that, there's not one master key. So, will you break that down for us? That's right. So there was, there was a quote I loved when I was reading Arnold Toynbee, this famous historian who sort of studied, you know, one of the themes of what he studied was just social adjustment in a changing world and in response to technology. And I came across this quote that I loved in his, I think it was Volume 12 of his *Study of History*, that says, "No tool is omni-competent. There is no such thing as a master key that unlocks all doors." And so I put that as one of the epigraphs in the front of the book because to me, it represented this, he was in some ways, he was talking about other historians who had these single theories of everything that clearly were not appropriate to the complexity of the things they were analyzing. And I thought it represented this having many, instead of having one master key, you have many different keys or many different ways that you can approach a single problem. And that's one of the strengths of the kind of people I write about.

So, in Philadelphia, we're all about the process, right? So let's, let's kind of start here. When you first set out to examine this project, what was your motivation? What was your source of inspiration for this? Well, so there was, there was first that issue of, so the reason the introduction is called "Roger versus Tiger" is because the Tiger Woods story is probably the most famous story of development of anything, I would say. You know, and, and I think there are six best-selling books just in the U.S. that were centered on extrapolating the Tiger Woods model to anything else that you would like to be good at. And the, where's Roger Federer? We, we all know him as a pro athlete. We know him as well as Tiger Woods, but nobody knows his development story, which is that he played a ton of different sports. One of which was tennis. His, his hip, let's see, skiing, wrestling, soccer, basketball, rugby, badminton, skateboarding. I'm sure I'm missing a couple. And his mother was a tennis coach. He actually refused to coach him because he wouldn't return balls normally. He, when his coaches wanted to bump him up a level, he declined because he just wanted to talk about pro wrestling with his friends after practice. And one of my favorite stories, when he became good enough because he was a good youth athlete, and the reporter asked him, "If you ever become a pro, what will you buy with your first paycheck?" And he says, "A Mercedes." And and his mother's appalled at this and asks the reporter if she can hear the interview tape. And the reporter obliges. And it turns out Roger just said, "More CDs," in Swiss-German. He just wanted more CDs, not a Mercedes. And so it was a very different, as as Roger Federer noted in 2006, "My story is completely different from his." And so partly my question was, which one of these is the norm? We hear one, we don't hear the other. Which is the norm? And it turned out that it was the Roger path. And I kind of filed that in the back of my head. And then when I got involved with the Pat Tillman Foundation, of the foundation named for the late NFL player, I was invited to talk about, well, I was just invited to talk because one of my former track and field training partners was a Tillman Scholar. They give scholarships to aid veterans, soldiers, and military spouses in career changes. And I decided to talk about late specialization in sports and research. And since they weren't athletes, they research it in a couple of other areas and tacked that on the last five minutes of the talk. And it was like cathartic for them. Like, this all started coming up, saying how they felt behind and being told that they didn't have like the normal resume for whatever they're applying for. One of the guys was a former, he was on SEAL Team 5. His undergrad degrees were geophysics and history, and he was in Dartmouth and Harvard grad school at the time of the talk. And he sent me this note saying, "I'm so relieved. You know, even told him so behind." And I'm just like, "If this guy is getting the message that he's behind, this is kind of crazy." And so those, it brought the sports stuff back to my head, and I decided, "I think there's something to look into here."

Yeah, and to be candid, Alex, when you first reached out, I don't know where Alex went, he's very busy. But I originally thought this was essentially just, you know, part two of *The Sports Gene*. The New York Times bestseller, because, you know, David, you've been all over all the, the media outlets that I, that I tune into, all the biggies, right? You guys have seen him left and right. But really, this is so much bigger than sport. I think that we've all, we've all had a bad boss, or too bad is such a loaded word, right? But for the suits, for the executives who perhaps are reading this book, and from a management leadership level, what do you hope they take away? Yeah, I mean, a number of things. But but one of those is, I think, and this does have an analogy in sports, actually, is to diversify their pipelines. Because I think things like LinkedIn are actually making it much easier for them to find square pegs for their square holes. Whereas some of the research I highlight by a woman named Abby Griffin, who studies so-called serial innovators. These are people who make creative contributions not just once, but over and over and over. And when I was reading through her research, it's like this very staid, you know, psychological surveys of these serial innovators and so on. And then, and then eventually, in in one of her works, she sort of steps outside that and says, "Okay, now I want to give advice to HR people. You're all screening these people out because you're making, you're defining your job too narrowly." These people have usually zigzagged. They've often come from another domain. They have a need to be in contact with people outside of their domain. They tend to have more hobbies. They read more and more widely than their colleagues. They appear to flit among ideas, all these things, and on and on and on and on. They use analogies from other domains in order to problem-solve. And one of the chapters about analogical, probably using analogies to solve problems. And so you're screening them out because you're, you're making this too, too narrow description. And it turns out that's, that's actually in some sports programs that have reformed. Like when the UK was not doing very well in the Olympics for many years, and basically their revelation was to diversify their entry pipeline to allow these people who didn't have the perfect sort of resume for coming in to try to get in. And so I would think for people who do personnel selection, they, they should be wary of that. And, and they themselves are often that, right? LinkedIn did research on a half-million members. They have these great databases, right? Because previous research is like 50 people, and LinkedIn has a half a million to see what is the best predictor of who would become an executive. And one of the best predictors was the number of different job functions that someone had worked across in their industry because they get this sort of holistic view. But that's, each additional job function saved them three years in terms of experience in getting to the, to the C-suite. But we don't hear that stuff. We just hear precocity. I was just, sorry, one other thing. I know a very digressive brain that I attempt to organize on the page, but it takes time. It's a reporter's dream when they just go, "No, I know." Right? Just used to be like, "Score more points than the other team." And so I was just at this event for *The Motley Fool*, which is like this investment publication. Ever. And before we went up, because I write about this in the book, they put a poll up for the audience where they could vote with their cell phone, which was, "What do you think the average age of a founder of like a blockbuster startup is on the day of founding? Now, when it becomes a blockbuster." And that the options were 25, 35, 45, 55. And 25 was the overwhelming favorite, right? So we all know, like, when Mark Zuckerberg was 22 and said, "Young people are just smarter." Like, that sticks. But research by Northwestern and MIT and the Census Bureau, just out, shows that it's actually 45 and a half on the day of founding. That these people have often zigzagged quite a bit before they can even identify that sort of turf that they want to compete on uniquely. But just like the Tiger story, that's the only one we hear. And then Mark Zuckerberg story, that's the one we hear. And so we extrapolate from these stories that are not really helpful or real or representative. All the science in this area. So I sort of wanted to rebut that.

Which leads right into a chapter where the words jumped out for me: "The Cult of the Head Start." You're a new parent of a four-month-old son. Props to that. What, you know, parents, we've got a three-year-old, my husband's here, you know, a lot of parents in the mix. Break that down. "The Cult of the Head Start" in terms of sampling. And we've got summer season on deck, right? What, what do you hope parents take away from from this? Yeah, so when I was, when I was living in Brooklyn, until now, I live in Washington D.C., but there was a U-7 travel soccer team that met across the street from me. And I don't think right that anybody thinks six-year-olds can't, I'm good enough competition in a city of nine million people that they have to travel, right? This is, they are customers for someone who is trying to keep them from doing all these other sports sampling. And so this doesn't happen in some places like France, where they have a more holistic development pipeline. And they're not, and it's not a zero-sum between the youth coaches and me and the adult coaches. So for me, I think, well, so "The Cult of the Head Start" is, is what I refer to as sort of the, the headstart industrial complex, that that tries to make parents feel that they will let their kid get behind if they don't do certain things. And one of the other stories, so some of the stories that are essential to this are the Tiger Woods one, which shows up in a ton of places, the Mozart story. And, and we're telling both of those stories wrong, by the way. So as Tiger, both of those fathers, there are a number of books that talk about how the fathers manufactured those kids, which is not the case. Tiger's father was responding to his display of unusual interest and prowess in golf. Mozart, the same. I was looking at at letters and found these letters where a musician who visited recounts Mozart when some other musicians come to play with his father, he recounts little Wolfgang coming in and saying, "I want to play a place that can violin." And his father's like, "You haven't any lessons. Like, go away." And he starts crying. And so one of the musicians says, "I'll go play with him in another room so he'll stop crying." And then suddenly they hear the second violin part coming from the other room. And his father starts crying. And they come in, and the letter writer says, "Little Wolfgang was emboldened by our applause to insist he could also play the first violin, which he then does with with his own made-up fingering." So I think, first of all, parents should not be worried about missing Tiger Woods or Mozart. In fact, if they want to increase the chances of that incredibly rare phenomenon, they should expose them to more things and see if they take to it like that. But the story I focus on is this one of the Polgar family, that is as famous in like the performance literature, but not as famous as Tiger Woods in popular literature. And this gentleman, Laszlo Polgar, whose family was basically wiped out in the Holocaust, decided that he wanted to have a large and very special family. And he studied education and decided that he could turn, he would turn his kids into geniuses. He said, "Normal education produces the gray average mass." And that if he gave his kids an early specialized start, he could turn them into geniuses. And really, this would be to show that you could do this to anyone. You could turn anyone into a genius by early specialization. And he picks chess too, because this was at the time when, you know, the, the chess was like a Cold War proxy for the U.S. and Russia at the time. And he starts training his, his first daughter, Susan, at age three. And she becomes really good. By four, she's going to like smoky chess clubs in Budapest and beating grown men. And she goes on to become the best women's chess player in the world. And in fact, she's the first woman to qualify for what was then called the Men's World Championship. And I think because of the things she did, it's now just the World Championship, not the Men's World Championship. And her two other sisters became part of the sort of project or experiment. One of them became an International Master, which is a step down from Grandmaster. And the other, Judit, became the best female player ever and up to that point. And this is another story. So like the book *Talent is Overrated* uses this story and says, "This is the key to get good at anything that you care about." The problem is, chess is what the psychologist Robin Hogarth calls a "kind learning environment." Which means it is based on repetitive patterns. It has a huge store of previous knowledge. People preferably people take turns for a kind learning environment. The rules never change. The next steps are very clear. Feedback is automatic and fully accurate after everything you do. Like golf. And these kind learning environments turned out to be the incredible rarity in the world. So they are, as opposed to "wicked learning environments," where next steps are not often clear, rules can change, feedback can be inaccurate or delayed. That's kind of more of the work that most of us do. And one of the problems with extrapolating from kind learning environments, other than that most of the things we care about aren't, is that they are extremely easy to automate. Which is why it's one of the reasons why chess is one of the first things that was that was automated. So as one of the, and then those even the companies have extrapolated from those kind learning environments and said, "Well, we can apply our AI to more complicated things." So I don't know if anybody followed Google Flu Trends. But Google AI got good at gaming. And they said, "Well, now we're gonna use it to predict the flu." And there's this big paper in the journal *Science* saying, "Google's using search query information predicted the spread of flu in the U.S. as accurately and it, and more quickly than the CDC." And but then it started getting worse because the rules don't stay the same for human behavior. And about three, four years out, it missed by a hundred percent. And if you go now, Google has a holding page that says, "It's early days for this kind of prediction, so we're gonna put it on pause for now." Right? Or you look at Watson and healthcare, which was this huge, "Who's gonna transform healthcare?" Right? I'm sure you've all experienced how Watson has transformed your healthcare. But so one of the researchers I, researchers I talked to were worried it performed so poorly that they were worried it would taint the reputation of AI in healthcare going forward. As one of the researchers I talked to said, "The reason Watson destroys at Jeopardy and does horrible in medical research is because we know the answers to Jeopardy." And I think that's sort of the, the, the definition of the kind learning environment. So we've, we've extrapolated this headstart stuff that works in these, in very particular types of environments. You have to specialize early if you want to reach EHS Grandmaster. Your chances of becoming reaching International Master status or higher drops from one in four to one in 55 if you haven't started pattern study by age 12. So works in chess, but chess is not representative of almost anything else that you would like to get good at. See what I mean?

David makes us think and rethink, and you're so smooth about it, and that's what your writing is too. When we, we talk about grit and how you measure grit, what's the trouble with too much grit? Whether it's draft day or the boardroom or the grocery store, Wegmans on a honest Sunday, what's the trouble with too much grit? Yeah, that's a title of one of my chapters. People probably heard of grit, the psychological construct of grit. And it's based on a 12-question survey, most associated with Angela Duckworth. But, and the survey rewards half the points for perseverance, essentially your resilience, and the other half for consistency of interests. So if you sometimes don't finish a project or your interests change, you lose points. The most famous study of grit was done on incoming cadets to West Point, the U.S. Military Academy. And it, it turned out that grit was a better predictor than traditional measures of who would get through what's called Beast Barracks. It's like the orientation at West Point that's physically and emotionally rigorous. And most people get through it, but grit was a better predictor than were this thing called Whole Candidate's Score. And so that sort of lit this grit fire where school systems started testing for grit, companies test for grit, and things like that. But there are a number of problems with some of, if using grit that way. And some of the critique of grit that I have in the book comes straight out of the researchers' papers. Like they were pretty honest about some of these limitations, they've just sort of been lost in translation. So for example, those, those gritty cadets, if you then zoom out and watch, you know, look at their career progression, about half of them drop out of the army on the day that they are allowed, right? So have they lost their grit over the course, you know, of that progression? No, it turns out that they've developed other interests. So, well, I guess in that sense, maybe they have lost some of their grit because they would score differently on the grit survey. And the army at first thought that this was indeed a great problem, but they couldn't fix it. Then they thought it was a money problem. So they started throwing money at their most talented officers to try to get them to stay. The ones who were gonna stay anyway stayed, and the ones who want to do other stuff and took the money, and the ones who want to other stuff left. Here's your career path: go up or out. They started what they call talent-based branching. This phenomenon of West Point cadets quitting was only since the knowledge economy. It's like since the '90s, where you can get certain skills and transfer them laterally in a way that you couldn't when people were doing more repetitive tasks. So I think, I think the rise of the generalist is in some ways rising with the with the knowledge economy. But, and so this talent-based branching program, they started where they take an officer, they pair them with a coach, and they say, "Here, a bunch of career tracks. Start sampling one at a time. The coach will help you reflect on how this fits your interests or abilities." And then you'll start a zigzag until we get you better, what it kind of is called "match quality." The degree of fit between your interests, your abilities, and the work that you do, which turns out to be incredibly important for your motivation, for your performance level. And what these researchers who study that, their basic conclusion was, when you get fit, it looks like grit. So when you put someone in the right spot, they start displaying all the habits that you associate with grit. And that this is not actually a stable characteristic that runs across everything you do. And I mean, right, I was a college athlete. Some of the, I would say greediest people I ever saw on the 200-meter runner. Not just any college athlete, that's the most grueling race. Walk side note for you. I think the mile is the most grueling, but it's probably one up from whatever you train for. But the, right, some some of those athletes were the super gritty on the track and total chickens in the classroom, and vice versa. And it's like demonstrably true in psychological research that grit is not the stable characteristic anyway. And so I think it made the focus on more testing for something that is purported to be a stable personality trait, and that is, is clearly not. It's, it's like psychologists who studied to say it's a state, not a trait. It's, it's a function of the situation you're in, as opposed to something that's inherent and unchanging in your personality.

Another phrase that comes up in your book: "Lateral Thinking versus Wither Technology." And you say that this is the Golden Age of opportunity here, this golden age of opportunity for generalists. Why so? That, I love that phrase. So, can I explain that for you? Please. Yeah, "Lateral Thinking with Wither Technology." So that phrase comes from this guy named Gunpei Yokoi, Japanese man who didn't score well on his electronics exams. And so he had to take a job in Kyoto at a low-level job as a machine maintenance worker at a playing card company in Kyoto, where all his better-scoring colleagues went off to big firms in Tokyo. And the playing card company was in trouble. And again, he's just a machine maintenance worker. But they were, they realized the president realized they had to diversify from playing cards because that wasn't cutting it anymore. And Yokoi realized that he was not equipped to work on the cutting edge, but that there was so much information becoming available that he could just combine stuff that was already well understood and cheaper in ways that specialists could no longer see because they were so narrow. So lateral thinking meant taking information that was ordinary in one area and taking it somewhere where it becomes extraordinary. And wither technology meant technology was already well understood. So we didn't have to be at the cutting edge. So he started just doing that. And that company, he started a toy and game operation at that company was called Nintendo, which was a 19th-century playing card company prior to him starting a toy and game operation. And all he did is try to come look for technology that had been left behind, what people were racing to the cutting edge, and combine it in new ways. So one of his great breakthroughs was the Game Boy, which used a processor that was a decade outdated, a screen that looks like, you know, like rotting salad for grayscale shades of graphics that smeared across the screen when it moved quickly. And came out right when Sega and Atari's color handhelds came out. And Yokoi, he recounts actually, this was a cool thing because there's a little lucky for me, none of his work had appeared in English. So I hired some translators to translate his work. And it's like, "Great, I've got some new stuff that's new for Americans." So it's lucky research find. Very savvy. Yeah. But so, so the, the color ones are coming out. Yellow. He recounts his colleague coming to him and say, like, "Bad news, Sega and Atari are hitting the market right before us with color." Oh, hitting the market for us. And Yokoi says, "Are they color?" And his colleague says, "Yes, they are." And Yokoi says, "Then we're fine. Don't worry about it." Because he realized the stuff that was important to the consumers was the gameplay, the number of games they would have, durability, the battery life. And so using this older technology, he was able to make it cheaper, sturdier, could go for weeks on batteries. And it was sort of like how iPhone app developers are now. Because the technology was understood, they started pumping out tons of games where the game development was much slower for the newer technology. In fact, in the research of this, I went to my parents' house in Chicago, in the basement. Dad's here. Yeah, yeah. And I picked up my Game Boy, which was my old gamer, which was like covered in some black stuff I couldn't figure out what it was. Flipped it open, the batteries had expired in 2007 and 2013. And I flipped it on and played Tetris for a couple minutes before it burned out. So that became the best-selling video game console of the 20th century. And, and set the philosophy for Nintendo from there forth. Like, so with the Wii, they realized that that cutting-edge graphics was not the problem, it was the complexity of gameplay. So they just made the controllers easier to use. And then Queen Elizabeth, you know, got video playing Wii bowling or whatever. Nepal. So that became the philosophy for Nintendo. And, and if you look at patent research, so I try to, I use these stories in the book, but I try to ground them in, in what the science says about this more broadly. Because I don't want to do the thing I'm criticizing people of, of saying, you know, "Here's this one person, use them as the example." If that's not what the science says. So patent research showed that from about the middle of the 20th century to about the late '80s, people who focused their work in one technological class, so the Patent Office has about 450 technological classes, were making bigger contributions. But starting in about the 1990s, that changed. And it became people who are spreading their work across a larger number of technological classes and merging them for their projects. So I think this is, this Yokoi success sort of was at the front edge of this, this phenomenon that developed more broadly. And, you know, it's clear why you're one of the best science writers in the game. But you were recently on Bill Simmons' podcast, which if I, typically if I mention this Boston guy in Philadelphia, I get 86. But the Sixers and ethics aren't playing. So I digress. But you guys did mention the, what was it, the "Dark Horse Project" and short-term opportunities, which I found fascinating. Can you break that that down a little bit for us? Yeah, the Dark Horse Project was this study at Harvard, essentially, of how people find match quality work that fits them. And in the, the criteria was actually, they were actually looking at fulfillment. Not a lot of the people were successful in the ways that people like to measure money or whatever, but, but fulfillment was really their, their dependent variable. And so they started, as they were doing the research and talked to their subjects, what they realized was these people who were fulfilled would often come in, not all of them, but the large majority, and would say, "Don't tell people to do what I did. Like, I started in this other thing, I thought that was my life's work, then I got off, and it took me a while to figure out what I want to do. And I did this and that, and then I combined something to do my own thing." And they'd say, like, "So I'm a total outlier." And then when, like, 45 out of their first 50 subjects were like, "I'm a total, don't tell people to do what I did." You know, they started to realize that there was something going on here. So they actually named it the dark, they renamed the project to be the Dark Horse Project because most of these people viewed themselves as dark horses, having come out of nowhere to do what they did. And they're basically their common trait was sort of like the anti-grit, in a way. Now, that they weren't resilient, but the part of the survey had asked about consistency of interests, where instead of setting these long-term plans, you know, like the commencement speech advice to "picture who you're gonna be in 10 or 20 years and march confidently toward it," which is the investor Paul Graham, noted computer scientist, calls that "premature optimization" because you don't know where you should be going yet. The Dark Horse is all had this common strategy of short-term planning, where they wouldn't look around and say, "Here's someone who's younger than me and has more than me." They would say, "Here's what I'm right now. Here are my skills. Here are my interests. Here are the opportunities in front of me. Here's a hypothesis I have about something I want to learn or try." So I'm gonna try this one. And maybe, you know, three years from now, I'll change because I will have learned something about myself. And they do this, like, lots of zigzagging until they get to a spot that fits them really well. Is just doing this short-term planning and spending a lot of time reflecting on how it fit them. It's called self-regulatory learning. And doing that kind of reflection. And so that really is not, I mean, even they realized that wasn't the advice. And this, that resonated with me a lot personally, one because I was living in a tent in the Arctic when I was for sure decided to become a writer. But also, even then, when I got to and entered SI as a fact-checker, this amazing story in itself. And they, so once I became a senior writer there, you know, you get contacted by aspiring sports writers saying, "Well, what should I do if I want to work at SI?" And the question was usually, "Should I major in English or journalism?" And my first instinct was to say journalism. And my second instinct was to say English. And then my third instinct was, "They have no idea. I majored in geology, astronomy." But even for me, it was such a strong compulsion to say, like, "Well, obviously you should get a head start." So only now I started become comfortable like not giving that advice. It's, it's hard to internalize. It's okay to zigzag.

My final question here, 'cause I see Alex with the mic, he's gonna pass it off to all of you now. You can take the stage and ask him. But what study, what study surprised you most when writing this book? Is there something that you can single out for us? Yeah, there was one, both because it was, he was surprised, counterintuitive to me, and because it was such a cool study. So there's this one done at the U.S. Air Force Academy. And they wanted to study, there's a chapter on on learning techniques, basically. And in this study, they wanted to, they wanted to study the the impact of teaching quality, essentially. And the Air Force Academy provided this incredible experiment that you could never recreate in another way, because they bring in their freshman class every year, and those students all have to take a sequence of three math courses: calculus one, calculus two, and a follow-up course. And they are randomized to professors for calculus one, then they re-randomized for calculus two, and then they re-randomized for the next course. And they all have to take the exact same test, and it's graded by committee, so there's no subjectivity to it. And so you have this incredible experiment where you're randomizing and re-randomizing people, so you can really see the impact of the teacher over the thousands of students. So he had a hundred professors that were in the study over a decade. And what they found was that the better a professor did in calculus one at causing their students to overperform based on those students' characteristics they came in with on the calculus one exam, the worse those students then did in the follow-on courses. So, for example, the professor whose students did the fifth-best on the calculus one exam, that practice one professor, the students rated him the sixth best, was dead last out of a hundred in how those students then went on to do in future courses. They then underperformed in future courses. So why was this? Well, the researchers found that this was because the way to get the quickest short-term improvement was to teach a very narrow curriculum and what's called using procedural knowledge, which is essentially how to execute procedures over and over and over until they become automatic. The professors whose students struggled a little more in calculus one, but then went on to overperform in their their future classes, they rated their professors lower because they felt more frustrated. Learned making connections knowledge, where they have to tie together concepts instead of learning how to execute procedures. They learn how to match strategies to types of problems. And when you have to do what's called transfer, which is taking your knowledge and applying it to a problem you haven't seen before, which is what modern work basically requires, that learning in a way that causes you to figure out how to match strategies to problems instead of execute procedures is crucial. And the scary thing, I mean, it was, and it was just so deeply counterintuitive to me that, and this became one of the themes of the book, that the things that you can do to cause the most rapid improvement, and it causes the learners to rate their own learning the best, can undermine your long-term development. Even though I sort of knew that in sports, where we know that the way to develop the best 10-year-old is not the same as the way to develop the best 20-year-old, it was just deeply counterintuitive for me in an incredible study. So that was a real surprise. You guys have questions. Alex has the mic. Marvelous. I listen to you being interviewed on WITF too, so now I really have a good background. It's, yeah, thank you. I appreciate you coming. For more of me, yeah, I want it to come. I'm tired. You're my own voice. So, well, I was wondering how well the people that you found, the good generalists, the generalists who, I don't want to say our success, I don't like the word success. I can deal with uncertainty and even prefer it, like improvisation. Yeah. Oh, sorry. Okay, that's an interesting point. One of the, one of Abby Griffin, who studies the serial innovators, phrases that I left out was "high tolerance for ambiguity." And, and I think the chapter that bears the best on this is chapter 10, which is about some work that people may, that's the work that is probably the most familiar to people in the book. And it's about judgment, essentially, testing people's judgment and ability to predict political and economic trends. And the study that went into this, actually, this was, I guess this one was, let this one was as surprising to me when I first learned about it. I just learned about it a while ago. And this is, this is a professor at Penn in Philly named Philip Tetlock, who in the '80s started realizing that experts at that point on American-Soviet relations would make predictions that were totally authoritative, immune to counter-argument, and mutually exclusive. And so he wanted to decide, you know, who would be right. And he didn't want to hear them saying the things we hear pundits say on TV times, which is, "There's a strong possibility that..." Right? So he would study what percent people think that means, and it's incredibly variable. But so he started a 20-year study, required 82,000 predictions of things that would happen in the future, political and economic trends. And they had definite deadlines for each prediction, and you had to give percent chances of different outcomes. And it was scored in this kind of elaborate way. And the conclusion was basically that the more narrow the specialist, what he called the "hedgehog's" - so this from this from this philosophy essay by Isaiah Berlin, the hedgehog knows one big thing, and the fox knows many little things - the hedgehog, you know, someone was the worst. They were basically because they would bend every every possibility into their lens of their single specialty. And that dovetails with something that Daniel Kahneman, you know, won the Nobel Prize for illuminating, cognitive bias, cause the "inside view." They had this one lens, and they knew so much information inside of it that they could fit any story to create it. Whereas the foxes, sometimes had near expert expertise, sometimes didn't, but either way, they roamed way outside of it, aggregating perspectives. So they needed specialists. I don't mean to denigrate specialists, maybe we need both. But they would use them for information instead of opinions. They had very high tolerance for ambiguity. They updated their beliefs a lot, right? They flip-flopped like crazy. Like we punish politicians for flip-flopping. I think so. Yeah, yeah, absolutely. It is improvisation. So they would see their own ideas as hypotheses in need of testing, and they would constantly update their mental models. Tetlock likened it to dragonflies. So dragonflies have thousands of lenses on their eyes that each take a different picture, and then it integrates in the brain. And so he would identify them as, as like these mental model collectors. And that, that the follow-on study to that was sponsored by IARPA, which commissions research on the U.S. intelligence community's like most pressing challenges. And to their credit, they decided to have a prediction tournament and say, you know, "Can university-led teams beat our intelligence analysts who have access to classified info, and you guys don't have classified info?" And so Tetlock and Barbara Mellers, his wife and collaborator, decided to look for people in the general public who had those wide-ranging interests that they had looked for. And they beat a market of intelligence analysts with access to classified information by 30%, right? With no access to classified information. Wide-ranging reading habits, not only that, huge tolerance for for ambiguity. But when they were put on teams of 12 together, their their individual predictions became 50% more accurate because of the way they would politely antagonize each other, basically, and update their mental models. And if you watch their conversations, one of which I exercised improvisation, they're like constantly changing their ideas and sort of triangulating like this. And no grand theory, like jazz music. So, yeah, there's a chapter 3 is on music. So this, this parallel, the parallel is not implicit in the book. Question in the back. Hi, one, I want to first thank both of you for coming to Harrisburg and the Midtown Scholar to speak to us today. So I'm a Sports Illustrated subscriber. I've been getting the magazine since the 1970s when I

was a kid and just a few days ago in the current issue there's an article about a 13-year-old girl in California who's a soccer phenom named Olivia Moultrie. So I, you know, read the article about how obsessed she is and she's training like crazy to be the best player in the world. Then I went online, Googled her name, you know, watched a lot of her videos. I mean, she's got amazing skills for a 13-year-old girl. So I put up, you know, after listening to your talk, I'm wondering if she's doing the right thing, just putting so much focus into soccer. Like, if you could speak to her and her parents, what advice would be?

The soccer community has read his book. I mean, the soccer community, very right. Yeah, yeah, yeah. When I started writing about this specialization stuff, I get these messages from Europe saying, like, maybe you're dumb, American sports, but like, not in our sport. And so I started looking and I'm like, no, actually, a lot of the research has come from soccer. So right after Germany won the World Cup, they had a study come out that followed the development of a bunch of their different leagues, but also the players who went on to the World Cup. And it found more unstructured play, more other sports. Didn't matter if it's formal sports, you know, not more organized practice and amateur league players until age 22. Right. And then they did another study where they matched kids for ability at age 11 and 12, tracked them over several years, and see who got better. And it was the kids who did more unstructured stuff. Because at first, I thought maybe it was just gonna be a talent selection issue. But then there's studies starting up.

Hearing like that. So first of all, I think Chris Ballard wrote that, right? Because I think he was calling me for something and I didn't call him back. So, oh well, I meant to. So new book and new born, he'll let me off the hook. And my first answer is, I don't know what that family is doing overall. So I don't, I don't think it's a problem to practice a lot or practice hard. I think it's a question of what else is going on. And, you know, if you go to like the French soccer development pipeline or in Brazil, the kids are playing a ton, but they're playing futsal. They're not been playing soccer. Those kids who go on to become the pros in futsal has small ball and you'll play one day, they're on sand and the next day they're on cobblestones. And it's, you know, this big space, the sizes stage or a basketball court. And, and I think the thing about playing different sports is really just a proxy for diversity of movement and diversity of problem-solving. I don't think it actually matters that you put on the jersey of another sport. And so I think futsal is a great developmental sport in that way. And so if she's getting that stuff in, then I think it might be okay.

Like Cirque du Soleil, for example. And it makes them less fragile. Started having their performers learn the basics of three other performers disciplines, not because they were gonna perform them, but because it dropped their injury rates by 30%. It makes you less fragile. For some reason, we have theories, but now I don't know that any is right, but it just does. And so I think it depends what else she's doing, really. But I think what you want to be be worried about is there's a famous study of Swedish tennis players from youth to the pros. Some of whom went on to become top 10 in the world, some of whom went on to become top 100 in the world. And one of the real patterns that emerged there was when a girl who got identified as talented really young, she would get taken away from what she was doing that had been working and put in what the researchers called a more restrictive environment. Or someone would say, like, oh, you did your thing, but now we can make you really good. And then they put them in and they lose all that self-directed play. And now they have start drilling. And most of them were gone by age of 17. Right. So, so I think there's a devil in the details. I don't think it's bad to play a lot, but I hope she's getting an improvisational play, a lot of diversity of movement, and that she doesn't get put in that kind of restrictive environment that will. And and that was way more with girls that were good than with boys, you know. So I think there's a lot of devils in details. So I wouldn't just automatically say because she is good young that they're doing stuff wrong. I'd have to know like more about the specifics because I don't think it just I don't think there's like I wouldn't tell people like make sure you're bad when you're young, you know.

Question the back. Yeah, there was a lot of, you know, memorable and compelling stuff in your book. And one of the most, one of the most memorable and compelling was the drop of your tools. Oh, yeah. Do you say, could I say something about it? Yeah, this was, he said he felt the most memorable chapter, which was also the hardest, like chunk of writing I have ever. Yeah, I went just about killed me. Um, this, this chapter is called learning to drop your familiar tools. It's about how specialists can often get like so attached to certain procedures or tools that they sort of cease to realize that they are work in a certain situation when the situation changes, they don't anymore. And the dropping your tools is from this sociologist named Karl Weick, who noticed that these very elite firefighters, smokejumpers, and hotshot firefighters who go into wilderness fires, when they would die, they would die with their tools still next to them. And when they were close to safety, even if they had dropped their tools, they could run and survive. And most of them still wouldn't. And that kept happening. And so they would go into a fire, something unusual would happen, and they'd be told to drop their tools, and they wouldn't do it. And even so, that reports would say that they'd find the bodies and they'd still have their tools. And in some cases, when one would drop it and survive, they would do weird things like look for a safe place to put their axe or dig a hole and bury it, right? Cuz and and they'd report saying, I couldn't believe I was dropping my axe and these things. And so for him, that was sort of this allegory. We then started looking in other disciplines and seeing that certain types of training could sometimes cause specialists to be very effective when a situation was repetitive, but that those tools would become, you know, they would no more realize those were external than their own arms. And they would, they would cease being able to improvise, essentially. And so in, he looked through airplane accidents, and the primarily, the most common cause of human decision error in commercial airplane problems was sticking to like the same plan when to any outside observer, it obviously the situation had changed and this wasn't a good thing anymore, right? Like, like people would stick to a plan when like running to the ground because they were going through familiar procedures when anyone on the outside could see that wasn't the thing to do. And so I get into this in in other fields and particularly with with NASA and some of these things and how you can diversify. So this part gets to late in the book. I talk about I move away from individuals and talk about organizations and and systems. And this chapter is more about how organizations can essentially diversify their cultures in a way that knocks people out of that sense of just automatically using certain types of procedures and tools.

We have time for maybe one or two more questions. Hi, thank you. It's a song. Yeah, there it is. Okay, thank you so much. I actually look forward to reading. I haven't gotten a chance to, but what you were talking about, and maybe you were just, maybe you've touched on it, is human-centered design. Do you know about that with organizational development? And I was reflecting that a lot of what you were saying sort of is the philosophy but behind human-centered design, because one step is to then look outside of the organization and find an analogous situation in a totally different industry. Yeah, yeah. I mean, so chapter 5 is all about analogical problem-solving. And how the best problem solvers, instead of taking that inside view where they look at all the details of what's right in front of them, they will look, take, take what Kahneman and Tversky called the outside view, where they look for problems that have an analogous deep structure, maybe different surface features, but an analogous deep, deep structure. And they look across a lot of them. And and that's sort of what they use for problem-solving. And so in that chapter, it looks at scientific labs and and how the use of analogies is like predicts how whether they'll make breakthroughs and not analogies from outside their domain. And in one study I loved, where this woman in Northwestern named Dietrich Enter, is probably the world's expert in analogical problem-solving, like using, you know, partly what related to what you're talking about. And she did this study where she basically, long story short, students these problems and asked them to identify that the structure of the problems. And this is sort of explained in more detail in the book. But and the students were good at doing it within their major. But the only students who were good at doing it outside of their major were these students in what was called the ISP program, the Integrated Science Program. Those students had no major, they had a minor in like six different things. They just take classes here and here and here and here. And they learn how to identify, you know, so there's this classical research finding that can be summarized as breadth of training predicts breadth of transfer. What that means is the broader your training is, the more you are able to transfer that knowledge to things you have never seen before. So if you're gonna see the same thing over and over, a narrow training fine. But if you have to see problems you haven't encountered before, then that broad training builds those conceptual models that you can that making connections knowledge. The thing was, when I went around Northwestern and asked other faculty about that ISP program, they were like, not good, those students get behind. And so that spoke to me because here you have the world's expert in analogical problem-solving on your faculty saying these of our students are the best problem solvers, and her colleagues saying like, behind, which I'm just like, that's just like gives me a headache, you know.

So this will be our last question to the left. I just have kind of going along with what you were saying about like the breadth of training, breadth of transfer, everything. It seems to me, and this is just how it stuck out to me, but was focused on, hey, let's give diversity of play at a really, really young age. Or and now here once we're in college, yeah, I don't know if you should major in English or journalism because I didn't do either of those things and here I am a writer. But I I teach high school. And so I kind of wonder then too, what does that sort of look like either being pushed up from the younger ages or pushed down from the higher ages? Like, do you have an opinion or thoughts on how we make something that is very, very structured, kind of at the heart of it, like four years of English, four years of science, four years of this, into a more generalized field to kind of better equip kids moving forward? Yeah, I mean, that's a, that's a huge structural question. And and let me start a little older than that, which is one of the studies I liked of an economist who looked at the higher ed in Scotland and England. And England, you have to specialize mid-teen years because you have to decide what you're gonna apply to. And University in Scotland, basically the same education system, except you, you don't specialize as soon and you can actually continue sampling if you want to quite a bit. And he said, who wins this trade-off? And it turned out that the early specializes do jump out to an income lead, they have more skill-specific, I mean, domain-specific skills. But by the later specialized errs pick better matches. And so their growth rates are higher. So by year six, they catch and pass the early specialized errs. And you're the specializes start quitting their careers and much higher numbers. They're basically made to choose too soon, so they make worse choices. Like, I like to think of it as like, if we thought of careers like we do dating, we wouldn't tell people to settle down so quickly, right? Because you learn things about yourself and you can make better matches. And by the way, the period from your late teens to your late 20s is the fastest time personality change over your whole life. So you're in the position of choosing something for a person you don't really know yet. And so I think, oh, and the other thing, so the reason I brought this up, listed the kids in England, very often picked things that were related to things they had done in high school because that was what they knew, right? So it's sort of limits their match, their ability to make match quality. So I would love to see maybe a little more of that kind of talent-based branching where you make more things available. And one of a teacher or mentor's role is to say, well, how did this fit you? You know, and how did this, let's reflect on how this fit. But there are also things that that kids have to learn, right? And and that's difficult. And so what I think, I just saw a math study, this was 13-year-olds. I think what age are your your students? So it's close. And so this is a tip that's only semi-related, but this is in chapter 4 on learning strategies. So this study, these 13-year-olds in math classrooms were randomly assigned to different types of training for math. Some of them got what was called blocked practice. They get a problem type AAA, type BBB, type CCC. The other got what's called interleaved practice. And this is in chapter 4, which means no, you never get the same problem type twice, or if you do, it's randomly, it's all mixed up. But they all studied the same problems. Come test time, the interleaved group, the interleaved group, more frustrated, right? Says they're learning less. Come test time, when they have to transfer, destroys the block practice group. The effect size was on the order of taking a kid from the 50th percentile to the 80th percentile, right? Obviously, I picked 50th percentile because at the top of the curve, a standard deviation doesn't move it as far. So I just picked the most, you know, impressive example there. But, but so I think some of those techniques, without changing structural things about education, we could be doing a lot better. The problem is they make the kids frustrated, mate, right? Their teachers worse. If the test is like short term, you know, that that could be a problem because it, it they have to, it takes them a little while to get that learning. And so I think the, the testing, you know, I think it's like the ten-year-old sports coaches. If you incentivize calculus one professors in the Air Force or teachers in high school to make the best eight-year-old team, then that's what they're gonna do and not set the person up to be the best 20-year-old. So I would start with trying to have a little bit more sampling, trying to make maybe connect types of problem-solving across disciplines, but without changing anything, just mixing up the sort of learning styles. So in Chapter 4 is about those those learning techniques.

Well, yeah, but I mean, I think about this a lot because there's so many forces at play in the education system and teachers are asked to or held accountable for so many things that are not in their control, really. And and so, so I actually think one of the structural things is that we need to build some bridges from teachers to other domains. They can understand that challenge better. And then we can all support it better, rather than just like testing and now they're doing a crappy job, you know. So, so I think the structural change needs to come also from outside in the rest of society to support this, this kind of learning and and developing experience, you know, and bringing people from other domains into schools or creating opportunities for kids because I just don't think like teachers are already asked to do a ton and and I think held accountable for things that they have a little influence over, in my opinion.

We go to poor David. Thank you. That's a just one other thing I would say is I love independent bookstores. And I think like we all want bookstores in our neighborhoods. And there's a very easy way to support them, which is buying the book here instead of on Amazon. So you don't buy my book, but obviously this is a beautiful bookstore. So maybe buy elbow. So thank you very much for having me here. Thank you guys. [Applause]