Transcription
You're one of the greatest teachers of machine learning AI ever, from CS231n to today. What advice would you give to beginners interested in getting into machine learning?
Beginners are often focused on, like, what to do. And I think the focus should be more like, how much you do. So, I, I'm kind of like a believer on a high level in this 10,000 hours kind of concept, where you just kind of have to just pick the things where you can spend time and you you care about and you're interested in. You literally have to put in 10,000 hours of work. Um, it doesn't even like matter as much like where you put it, and you'll iterate and you'll improve and you'll waste some time. I don't know if there's a better way. You need to put in 10,000 hours.
But I think it's actually really nice because I feel like there's some sense of determinism about being an expert at a thing. If you spend 10,000 hours, you can literally pick an arbitrary thing, and I think if you spend 10,000 hours of deliberate effort and work, you actually will become an expert at it. And so I think it's kind of like a nice thought. Um, and so, uh, basically I would focus more on like, are you spending 10,000 hours? That's what I'm focus on. So, and then thinking about what kind of mechanisms maximize your likelihood of getting to 10,000 hours. Exactly, which for us silly humans means probably forming a daily habit of like, every single day, actually doing the thing, whatever helps you. So I do think to a large extent is a psychological problem for yourself.
Uh, one other thing that I hope that I think is helpful for the psychology of it is many times people compare themselves to others in the area. I think this is very harmful. Only compare yourself to you from some time ago, like say a year ago. Are you better than you a year ago? This is the only way to think. Um, and I think this then you can see your progress and it's very motivating.
That's so interesting, that focus on the quantity of hours, because I think a lot of people, uh, in the beginner stage, but actually throughout, get paralyzed, uh, by, uh, the choice. Like, which one do I pick? This path or this path? Yeah, like they'll literally get paralyzed by like, which IDE to use. Well, they're worried. Yeah, they're worried about all these things. But the thing is, some of the, you, you will waste time doing something wrong. Yes, you will eventually figure out it's not right. You will accumulate scar tissue, and next time you'll grow stronger because next time you'll have the scar tissue, and next time you'll learn from it. And now next time you come into a similar situation, you'll be like, all right, I messed up. I've spent a lot of time working on things that never materialized into anything, and I have all that scar tissue, and I have some intuitions about what was useful, what wasn't useful, how things turned out. So all those mistakes were, uh, were not dead work, you know? So I just think you should just focus on working. What have you done? What have you done last week?
That's a good question, actually, to ask for, for a lot of things, not just machine learning. Um, it's a good way to cut the, the, I forgot what the term will use, but the fluff, the blubber, whatever the, uh, the inefficiencies in life.
What do you love about teaching? You seem to find yourself often in the, like, drawn to teaching. You're very good at it, but you're also drawn to it.
I mean, I don't think I love teaching. I love happy humans. And happy humans, like, when I teach. Yes, I, I wouldn't say I hate teaching. I tolerate teaching, but it's not like the act of teaching that I like. It's, it's that, um, you know, I, I have some, I have something I'm actually okay at it. Yes, I'm okay at teaching, and people appreciate it a lot. Yeah. And, uh, so I'm just happy to try to be helpful. And, uh, teaching itself is not like the most, I mean, it's really no, it can be really annoying, frustrating. I was working on a bunch of lectures just now, I was reminded back to my days of 231 and just how much work it is to create some of these materials and make them good. The amount of iteration and thought, and you go down blind alleys, and just how much you change it. So creating something good, um, in terms of like educational value, is really hard. And, uh, it's not fun. It's difficult.
So for people, should definitely go watch your new stuff you put out. There are lectures where you're actually building the thing, like from, like you said, the code is truth. So discussing, uh, back propagation by building it, by looking through, and just the whole thing. So how difficult is that to prepare for? I think that's a really powerful way to teach. How did you have to prepare for that, or are you just live thinking through it?
I will typically do like, say, three takes, and then I take like the, the better take. Uh, so I do multiple takes and I take some of the better takes, and then I just build out a lecture that way. Uh, sometimes I have to delete 30 minutes of content because it just went down an alley that I didn't like too much. There's about a bunch of iteration, and it probably takes me, you know, somewhere around 10 hours to create one hour of content to give one hour.
It's interesting. I mean, is it difficult to go back to the, like, the basics? Do you draw a lot of, like, wisdom from going back to the basics? Yeah, going back to back propagation, loss functions, where they come from. And one thing I like about teaching a lot, honestly, is it definitely strengthens your understanding. Uh, so it's not a purely altruistic activity. It's a way to learn. If you have to explain something to someone, uh, you realize you have gaps in knowledge. Uh, and so I even surprised myself in those lectures, like, also the result will obviously look at this, and then the result doesn't look like it, and I'm like, okay, I thought I understood. Yeah. But that's why it's really cool to literally code. You run it in a notebook, and it gives you a result, and you're like, oh, wow. Yes. And like actual numbers, actual input X, you know, actual code. Yeah. It's not mathematical symbols, etc. The source of truth is the code. It's not slides. It's just like, let's build it. It's beautiful. You're a rare human in that sense.