Solveit lesson 10

Hey there, Eric. Nice to see you again.

Hey there, Jono. Good to see you. Always a pleasure.

Hey, everybody. Um, well, people loved the last lesson, Eric. Oh, good. And as you've seen from the Discord, like I did, loving your book and this way of reading it. Um, so I am keen to explore, as are many of our students, two things this week. One is, how, how did you, um, write such a good book? [laughter] Uh, and, uh, in particular, how did some of the ideas we've been talking about this class help you? But not just that, how did you write it in general? And, uh, also, um, tell us everything about everything, please. So, those were the whole two things I was hoping to cover.

No problem. We should have plenty of time for that.

Great. Um, but Jono, I know there was a bunch of things that blew both of us away this week, and your list of things heavily overlapped with mine. So, should we start with you?

Yeah, let's start with that. Just the, um, I guess student showcase slash, um, pieces that have shown up in the share-it channel. Um, so I'll start with, not to be biased or anything, but one of my favorite projects from the, uh, cohort. This is Jack's personal website. We're going to be talking, we're going to be talking about, um, writing today, and lots of encouragement that you should be maybe doing some writing here and there. So, Jack has a fantastic personal site. It's got an about page. It's got what he's working on now. It's got a blog. Um, and he's shared the code as well. Um,

Just that was so fast. Can you click on those buttons again?

Like, that's crazy. Look at that. Wow. Oh,

I love it.

Stylish. It's got dark mode. Um, you know, there's all the different, uh, posts. And if you work this way nowadays, it's so nice.

Yeah, it's, um, it's staggering. And this is the entire source code is 298 lines of

Very nice, very tidy, very understandable. It's very idiomatic.

HTML. Yep.

Well, actually, some of it I can see rooms to even make it easier. That is looking great.

Jeremy is like, "Wait, I can get this down to 200 lines."

Yeah, definitely. [laughter]

Very, very, very, um, putting yourself out there on the internet and sharing some ideas. So, um, huge thumbs up.

And that's available for everybody on the Discord to, to learn from. You know, maybe fork it since it's a, you know, it's a GitHub thing, and then you can edit it and,

put your own posts and posts and edit about.md to change the about page,

contribute back if you add anything, remove one line of code, for example. [laughter]

And to do that, by the way, after you click fork in GitHub, you'll find there's a button you press that says, basically, contribute back my changes. Uh, which is really cool. So, it makes it easy to, when you appreciate somebody else's work, if you improve on it, makes it easy to give back.

Yeah. Um, cool. Okay. So, that's one of four. Um, the second one was, um, more a extremely long and thorough dialogue of exactly the kind of thing we encourage people to do when engaging with the course.

So, this is someone actually going through the fast.ai deep learning course, and there's a section of building up matrix multiply yourself. Um, but, you know, it's not just like, oh, I reran the code that Jeremy gave me. It's like, at every point, wait, what does that do? Explain this chunks thing. Let me try it out. What does it look like? You can see every step here is like a little piece of code, some output of the code, and then like checking that they understand. Um, back and forth with the LLM, back and forthing with more examples. Um, so, yeah, just really great to skim through and read through somebody deeply engaging with the material. And when we say like, make it your own and go try the notebook, this is what we mean. We don't mean like run all and then now you've done matrix multiplication. We mean go and engage, and when there's pieces that you don't understand, dig in. Um, so, props.

You'll remember when we made this together, like Eric's book, we intentionally made it full of partially investigated rabbit holes.

Um, and as you get later in the course, there's actually quite a few, uh, still unanswered research questions that we discovered together and literally decided like, okay, we won't dig into it. We leave it as an opportunity. So, people who are interested in going super deep, there's various opportunities to literally write a paper, um, based on following up some of the stuff in this course, and using this approach would certainly allow you to go to that level of depth.

Yeah. Um, okay, number three is one from, uh, one of our own. This is Karum shared the dialogue for something that Jeremy had kind of left as homework for the excited reader. So, if you'd like to see a reference, this is, um, I'm in Turkey. I want a, uh, WireGuard VPN so that I can do the kinds of things one needs a VPN for. Um, and accessing Discord in Turkey.

Yep. Yep. Exactly. Getting on to Discord.

Um, so he goes through building on the Hets lesson, um, setting up the machine, setting up, you know, using this remote to execute commands on it. Um, all the way up to like getting a QR code that he can scan to get everything set up. So,

And like, just by way of background here,

you know, it's,

it's great that Karum did this, right? And like, Karum is the best student I had who came through our Masters of Data Science program at the University of San Francisco. Um, and like, this is just, this is why. He was like, "Oh, Jeremy gave us some homework. Okay, I'm going to do the hell out of it." Um, this is the kind of attitude it takes to be good, you know, it's like, yeah, it's great effort. Mhm.

Um, okay. And then the final one was a smaller one, but, um, Marius shared, um, he's shared multiple things that we've, um, enjoyed and featured. Um, but this one was just taking, there's a new set of models released from Mistral. Um, and so doing a little bit of benchmarking on one particular set of multiple choice questions, in this case using a framework called DSPI. Um, but yeah, working through like looking at the data, um, picking one row, checking at the different answers, setting up the SPI, you know, just moving through the process of, can I feed a bunch of question-answer pairs through and get what the LLM thinks is the right answer, and then score that and make a pretty plot to say, oh, look, these are the different models in the same sort of size category. Um, is Mistral standing up to the, um, reigning open source champion? Um, and in this case, no. But that's fine.

[laughter]

At one point.

And it's worth noting the branding open source champion being GPT-O-S-S-2-0 and 1-2-0. [snorts] I feel is a horrendously underappreciated pair of models at the moment. Um, they, they're currently winning the Kaggle math competition fine-tuning competition. Um, uh, so definitely well worth studying those, and even just for using them, uh, particularly on Grok with a Q, they are so fast and cheap.

Who did the, um, just a question, uh, from the chat, who, who do you know who did the, uh, matrix multiplication deep dive offhand?

Uh, yes, I, so I shared the four dialogues in the, um, if you go to the lesson channel, um, and so that was, um, Rubanza Silva, if I'm guessing the first name, surname, spit. Um, so, yeah, great work, Rubanza. Um, and I've put the links to all of the shared dialogues in the lesson chat, just for easy access. They're also in the share-it channel as well.

Before we dive into writing, um, I had some more reading stuff that I wanted to show. Um, and I will say, uh, lots of people on the Discord have been like, "Hey, Jeremy and Jono, share your reading dialogues so we can use them." And both Jonno and I independently decided like, "No, we're not going to do that. Make your own." And nobody quite has yet. So, there's still an open opportunity here for people to really contribute a lot in terms of like, because we want to see what people create and how it goes, and not just do whatever Jono and Jeremy did, because, as I said last time, this is all brand new. [sighs]

Um, so we don't want to just have you repeat what we're doing. So, please, folks, try things out and share them. And even if you do just repeat what Jon and I did yourself, then share how you did that, too, you know. Um, we'll talk more about that, I think, at the end of this [snorts] lesson, Eric, about what it means to be on the cutting edge, as fast.ai students were in 2016 when they did our first lesson. And so many people have come back to me in the years then and said like, "H, now that I'm a professor, you know, I wish somebody had told me back in 2016, did you know you're actually on the cutting edge right now? Like, you should really do these things." Um, so that's where a lot of folks, I think, are going to be in the future.

For sure.

Um, so, um, Ren's added a nice thing to solve-it this week, which is, some of you might already know, we have a, a function called URL to note, which just takes a URL and turns it into a note. Um, but he updated it to use, you know, this new reading style. Um, which is to say, by, it's now got an additional parameter, um, split-re, which is a regular expression that it will use to split the URL contents, and we'll create a separate note for each one. And this is going to be a separate one for each, uh, heading level and, uh, each paragraph. Um, so this particular [clears throat] URL is the "Attention Is All You Need" paper. Anybody who follows deep learning will recognize. Um, it's a great paper, if, although I will say, did not invent attention. That was probably better known at, at, or even earlier, um, Alec Graves. Um, but showed a new way of using it, um, deeper than we maybe thought it could be done. I'd also say that Yan LeCun's group author did a memory networks paper that was on the similar path. Um, so this is quite cool. I can, um, run URL to note. And so this is the end of the D. So I run this. Pop. There we go. And you can see it's now added, uh, a note per section. Um, so actually, it's doing it on headings here. It's a little bit different to what we did for Eric's book, which was to do it on paragraphs. So, it'd be easy to adjust that re to also look for, um, pairs of new lines if you wanted to go a little bit more short, but these were quite short sections. It actually works pretty well here. Um, he's also made sure that the math works correctly, um, based on what your math mode is, if you have use LaTeX enabled. Um, that's so cool. Images come in. And something we have in, um, solve-it is that, um, if your images end in hash AI, then that says, um, please send these to the, um, uh, to the LLM in a prompt, and it doesn't change the URL. And so there's, there's now an AI equals true or false, which lets you turn on or off whether those get added automatically. Um, so then, um, you can now run a prompt with dialogue helper in, which does the other thing that we showed for Eric's book, which is to provide it the kind of the, the oracle view, which is to say, read the whole dialogue, because normally it can only see up to here, right? Um, wonder if I haven't, I probably need to run this, but I'll just make sure. I'm guessing that little thing actually said it had an error. Let's just try it again. Um, so this is going to let it see the whole dialogue reading ahead. Um, which can be quite useful. There we go. And I'm in learning mode. Um, and then it's going through, showing me, you know, how I could think about reading it and helping me think through what I should tell it to do a better job. So, this is a, a very fast way. Um, and you'll note that for archive papers, there are HTML versions nowadays, which work pretty well in, um, in our markdown converter. Um, so, yeah, very handy thing from from Ren's. Um, so if I headed down here, please explain this image. Um, so because I had hash AI on it, because the default now is to add the hash AI, um, it can see the paper. That's a picture. Now, why would you say no sometimes? Well, if something's got lots of pictures, and most of them aren't really necessary for the AI to see, it's going to really blow up the context length. So, you can decide which ones to include or not include. And it's nice. It's showing us the different colors as well, which is super cool.

Now, um, there is a key piece, um, of, um, close reading, incremental reading, etc., that we didn't discuss last time. Um, which is, if you want to remember things that you have read, um, it is actually possible to be guaranteed that you will remember anything you wish for the rest of your life. And this is a thing I told my daughter Claire when she was, I don't know, five or six, and she responded the same way pretty much any would one would when I said that, which was, first of all, I don't believe you. [snorts] How is that possible? I forget lots of things. And then I said, explained how. And then she was like, "Wow, how? I assume this must," she said, kind of said like, "Oh, wow. This must be like the first thing everybody learns at school." It's like, "No, nobody learns this at school."

So, for those of you that don't know, uh, Hermann Ebbinghaus in the 19th century discovered that by, by can't believe he went through this process. He wrote down random lists of random letters and studied how well he could remember those lists using different revision techniques, where the revision is basically have question and answer flashcards to see, did you remember what's next after this random string? And, um, he discovered that, um, obviously, as soon as he's first looked at a list, 2 minutes later is, you know, or a minute later is when he has the best recall of it. Um, and then the next day, his recall's way worse, and the next day, you know, then a week after, it's way worse, and you kind of have this drop-off curve. [snorts] Um, depending on when you first review it. Um, he discovered that there is basically a, a fairly predictable optimal schedule to restudy it. Uh, in other words, to to ask, hey, what's the next one? Hey, what's the next one? So, I kind of use the flashcards. [snorts] And the optimal schedule is very interesting. Just before you were probably about to forget it. Um, and so we now actually know quite a lot about why this is. You know, this is this idea of desirable difficulty, which crops up a lot, and we've discussed previously, which is that, um, if your brain has to work hard to figure something out, it strengthens the connections. And interestingly, this is very different to how something like math is taught, where you drill something again and again and again and again until it's like ritual. That's a terrible way to remember how to do it, because those times that you, it's like ritual, you're learning nothing. Literally no new connections are being created.

Um, so, uh, Anki is a piece of software which automates those algorithms. Um, and so, um, Eric, I know you've used Anki. Um, what kind of stuff have, what do you like? Languages or?

Yeah, I've used it. I use it for language learning, but I also use it for skill-building stuff. Like, I was, um, use it to improve my Go and chess skills quite a bit.

All kinds of things where you, you need to remember, um, and, uh, and I used it for, um, interval training in, uh, ear training with, uh, with music. It's really good. So, I like it. I, I really went through a phase of really liking, um, using it for stuff like that.

A lot of people drop out of that phase, Eric. And I'm rather suggesting you're hinting that that might have happened to you, because there is this important question of like, what do you put in Anki, right? Because now you've got this super skill where you can choose to remember anything for the rest of your life. But it adds overhead. Now, when you add something to Anki, it, it, the schedule, the optimal review schedule is like exponentially increasing,

length of time. So you can actually add 50 cards a day. So when my wife Rachel was studying for her immunology masters and PhD, she added many thousands of cards, you know, which of things like, what is a CD48? What is a HP75? It's not that much more straightforward than Ebbinghaus. Um, and she, she did it right. But if you add everything, then your life is overcome by nothing but Anki, right? And is that what kind of [clears throat] happened to you? Did you kind of fall off it a bit because it's too much, or realized you didn't want to learn it for the rest of your life, or?

Yeah, I didn't. I made two mistakes. One was I really liked, um, I put, I, I like putting too much information in there, and I especially, a lot of the things I was doing, I found ways to create either to to generate decks, like whole decks of information, or or use pre-made decks.

Oh, yeah.

Which is definitely a bad idea.

Yeah.

But I didn't know that at the time. And, um, and then I would, you know, then I have to have this like titrating situation where I have to like gradually add the cards from the deck. And if you, if you just get the frequency of which you're adding new cards into your thing slightly wrong, you can wind up with far too many reviews. And the second problem is, yeah, I didn't only include stuff I wanted. Like, I wasn't really, I was more playing around with it to see what it was good for. So I wound up including a lot of stuff that after a while I just thought,

do I really want to keep learning this, you know, given the amount of free time that I had? And I also went through a phase where my amount of free time copiously and dramatically dropped. And so it was like, went from a fun hobby to a survival thing. I was like, there's just [laughter] there's just even the small amount of time it would take to maintain these hobbies.

I guess what I was saying is, um, it's not so much that I ran out of time for Anki, as I ran out of time for having a life. And so that Anki was one, one of the small casualties of that.

Yeah. I don't know about you, Jonno, but I find like people often ask me, do I use Anki for remembering computer science programming stuff? And the answer is no. I just practice using it. As soon as I know something interesting, I just try to use it. And so you don't need it for stuff that kind of you

Yeah.

Yeah. My uses

Yeah.

Exclusively stuff that I'm not doing in my day-to-day, but that like once every few years, I really want to be able to have that conversation with someone who's deep on it. So, like I have a brother-in-law into philosophy, and I've got, I don't know, say, 50 philosopher names so that I can remember who was the one who said, "What it's like to be a bat?" All right. It's Thomas Nagel, you know. So, it's like, this is for like, one Christmas conversation every few years. Um, but they only come up, you know, they they stay happy for months, so it takes almost no time.

That's actually a great idea, because I studied philosophy as my major, and I still don't remember. I remember which one Nagel is, but I don't remember which one Hegel is. [laughter] So,

It's challenging.

Yeah. So, um, the guy who rediscovered, um, repetitive spaced learning, uh, is a guy called Peter Wnjak, who I mentioned last, um, last lesson. [snorts] Um, and, uh, he created his own software called SuperMemo, um, which greatly predates Anki. Um, and he has 20 rules of formulating knowledge. You should read them, um, because, you know, if you're downloading stuff from the internet to memorize, you've broken rule one.

Yep. Tell me about that.

And rule two, and quite possibly rule three. [laughter] Um, and then also, Eric, it looks like you didn't do this one. Minimum information principle. So,

I didn't. I didn't do any of these. I, I didn't know anything about this when I first started playing around with this.

Yeah. Um, so, um, great to use with solve-it, right? Because with solve-it, you can just add cards for things that you've just learned, and that you've already demonstrated some level of mastery of, and you can check with solve-it, have I demonstrated mastery? You know, and so,

This is so, uh, I just created this thing called fast-Anki, um, yesterday, because I wanted to start, as I said, this is also new, right? But I really want to start bringing this in so that my daughter, for example, doesn't have to manually create cards, because that level of friction is high enough, she tends not to do. But I'm a bit the same. Um, fast-Anki. This is what it looks like to use. So, um, this is the first pre-calculus lesson. Um, and this is where we got to. Um, and it mentioned trinomials. So, it's like, well, what is that? So, this next bit actually was not clear. This is me, this, you know, literally an hour ago, just wanting to show something of how this works. Um, so it's like, can you suggest some Anki cards? Now, this is actually quite interesting, because it actually didn't do a very good job of it, even though I'm sure Opus knows perfectly well what a term is. Um, but this is just not right. A term is not a product of a number and a variable. Um, and if you're going to do Anki cards, you want to be remembering the correct definition.

Um, so I, I pushed back. It's like, your example is not exactly that at [laughter] all, right? And then it's like, it's just not an expression made of terms. I think the problem was it's like, oh, CLA's 10, so I'll just give like hand-wavy explanations. But actually, even a 10-year-old needs the correct definitions. Um, so it's like, all right, do it correct, um, please. And now I can just say, please add cards, um, because we have got these tools now, and so it can go ahead and call add front and back card with a and a back. Um, so then, uh, I can, uh, I clicked sync. I, I ran the sync function, and then I looked in Anki web, and the math was showing up with dollar signs. So I was like, "No, that didn't work. Can you delete them and do them again? Please read the docs for me." This is so cool. Like, when you've got a canvas with everything in it, you just never have to flick around and copy and paste and explain things. It's just happens straight away. It's like, okay, I deleted the old cards. Here's what I found out about math. Needs to use these different symbols. And so then this is nice, right? I could just paste in exactly what I saw in Anki web. And I can say, no, that didn't quite work either. There's these stars on terms. Like, oh, okay, we need, um, HTML. So, it's doing web searches to get the correct information.

[snorts]

Um, uh, and so again, here, um, I just, we just said like, great, delete, delete them and try again. So, it's gone ahead and deleted the cards, um, and recreated them with HTML. Um, and so now, um, if I refresh, I created a new deck, so I was worried about destroying everything. Uh, so I've only got four cards in it. But if I click it, there we go. What is the term in algebra? Nice. All right. Um, and if I click edit, that's looking good. So, um, then I said, cool, can, and this is a really helpful thing. Again, you'll see all the same patterns that you saw from me reading Eric's book, which is, okay, write detailed notes for the next, please. And it's done similar stuff. It's like, this is what who CLA is, this is how we're doing things, this is how we create cards. Um, so I am,

Nice.

Super excited about this, um, because it's the big missing piece. Guys like Andy Matuschak, um, with his, um, um, Quantum Country. Have you guys seen this?

Mhm.

So, if you're interested in quantum computing and quantum mechanics, uh, Andy and Michael, friends of mine, wrote this book where, um, repetitive spaced learning is actually built into, uh, the book, um, as you see.

[snorts]

Um, and the, "didn't remember, remember" thing is all built in. Um, so this is a key part of of reading. Just like, uh, what do we call it? Fast Daisy? FH Daisy? I can't remember. This was entirely written end-to-end in solve-it. [snorts] And it's quite interesting because, um, Anki doesn't exactly have an API per se. Um, so what is, I, what I did is I popped open the terminal and I get cloned the Anki repo into here. And it turns out Anki is actually written in a combination of Rust and Python. So, there's a kind of an implicit Python API. [snorts] So I then installed Codex, which is, uh, one of the terminal-based CLIs, and I ran, um, let's close that. So I ran, um, I opened up Codex and I typed in this prompt in the root directory of it. This is the source repo for Anki. Write a detailed API.md document, uh, showing how to do Python, use Python to do all these things. And so it spent 10 minutes or so looking through the source code and ended up writing this document that I then pasted into solve-it. Um, so specialized tools like Codex and Claude Code are are generally going to be better at doing really deep dives and long agentic things than solve-it does. So, like, use the right tool for the job. So I thought this was a good thing to use that for. Um, so then I've got that here, and then I say, great, in learning mode, right? Show me how to do things. I want to get everything up and running. Um, and it's like, oh, well, what does this impact Anki desktop? Like, there's no Anki desktop here, so can help explain where we are. Let's not go too far too fast. Um, and so it's like, starts taking me through the process of like figuring out where your data is. Um, now, this is a really cool trick. You guys, I bet you guys have both done this before. I was just like, pretend we had the coolest API. Imagine. Normally, I would do this by hand, right? But this is, but LLMs are very good at this. Just say like, write code as if we had a great, um,

Totally.

API, and it's like, cool, that looks [laughter] fine. So, um, great. So I didn't care that much about learning the ins and outs of it. So I tried to see if it could kind of write it, most of it for me. Um, and the answer was, it couldn't. Um, but that was okay. And here I did some deep dives of like, what the hell's going on, you know, 'cause none of this is really documented before. So, you can see it did a lot of using, uh, the RG and all that kind of thing to find out why I was getting weird errors. Um, and so kept on doing things like that. Um, and yeah, gradually got things working. Um, and so you can see I'm doing a bunch of, some of my own code, and quite a lot of its code. Um, gradually changing this sync thing to try to get the damn thing to work. Um, and even though I thought it was working, sometimes it would like crash again later. Um, anyways, got that all working. B back and forth about different possible, um, APIs. Um, but I realized this thing of like, it's all SQL like database behind the scenes, and if you kind of leave it open, it becomes a really big problem. So you have to remember to close it when you're done. And the whole thing was just a bit of a nightmare. And so I got to this point, I was like, okay, let's start again. This API is not very good. Um, so I was like, hm, I was like, maybe we shouldn't do any of that. Uh, what if we just patched the, you know, this collection object class they already have, you know, talked about that a little bit, and it's like, okay, I think I'd rather do that. So, I literally threw everything away and started again. And [snorts] this is the kind of thing that you're not going to get if you just vibe code things, right? You know, this kind of thoughtful, careful exploration of different options, and then you start using it, and then you start putting things together. [snorts] So, this one turned out to work really nicely. Um, and I was able, you know, I was able to be like, okay, here's the next few steps. I could kind of again start going through it quite slowly. And then I started adding the little things, um, you know, like adding a markdown representation, um, so that I could kind of see things better. So like now I can say models and get back the nice markdown list. Um, and so for things like, like, okay, add the rest of the markdown representations for me. Away it goes. So that kind of stuff works out very nicely. Um, and then I kind of got to the end of this process and again, and you know, quite a lot of this, and it doesn't tend to write very good code, right? So I'm often kind of saying like, wait, you wrote all that code when you could have just said this, and it's like, oh, yeah, I could have just said that. Um, okay. So I got to the end of this section. Um, I don't have a header for it. But what I then did was realized, um, that actually I, whole, that's right. So what I then did is at the end of that, I had something basically working, and I was able to paste in my thing. I was like, oh, it's working. Um, and I've got added something new to dialogue helper, which I'm really excited about, because I don't want to share this whole dialogue with my team. I mean, I can make it available, but I don't want this to be the source code because there's so many bad attempts. So, check this out. Um, Eric, I don't think you've seen this before. Um, I told it, "Show me a list of message IDs that would export to a library." And it said, "Okay, here it is." Right? So, here's my list of message IDs. And then I grabbed all those messages, and then I did this, um, really cool thing. I said, dialogue helper settings, dialogue name is. Then I created a new dialogue called Anki Tools. And then I went through each message and added it to my new dialogue. And what that does is it actually allows you to have cross-dialogue communication. Does that make sense, Eric? So, so instead of copying the cells manually from one dialogue to another, this is just like,

Programmatically.

Creating a new duplicate with only the bits that you wanted to keep.

Mhm. Yeah.

By writing it to the IPF vibe file.

No, by using at message to write to my open dialogue.

That's already open.

Yeah. So, this one's Anki Tools.

Ah, that's cool.

So, this now goes at message, which you've seen a lot of times.

Mhm.

Placement at end. And it,

You're not passing the dialogue to it, you're just set changing.

I'm not. I'm setting it once. Exactly. Uh, I was like, cool. I set the dialogue helper to another dialogue I have open. Uh, so press find, use find messages to see the messages you just added. And now add a note after each one with documentation for it.

That's so clever.

Uh, thank you. So now we have those functions. Um,

This is passing over the websocket.

Uh, yep, exactly. And so there it is. Um, and then at the end of that, I was like, okay, I don't actually, that none of that's very useful for tools or for normal people. So then I added a functional API to it with just things like add card, and you could do, you don't have to do any elbow stuff. You just say add card frontiho back hello, um, and that's it. No opening, no closing. Um, it, it, you know, we added a context manager to it, and then you can just say with, and then it closes itself at the end, and then it all just works.

Nice. Nice.

And then these all have, um, annotations, and therefore they're usable as,

As tools.

Tools. So then I, um, used nbdev new to create a new folder. I did, I did gh repo create to create a new, uh, repo and folder, and then I said nbdev new to create a new, um, nbdev project. I copied and pasted that into ooore, and then I also duplicated it to create index. I then went through the index and deleted, uh, all of the source itself, and ended up with just, and I reordered it. So, got the functional API first, because that's what most people use, and then I also added a whole tool use section showing examples. Um, and then the other APIs are out at the end. And then I pushed that to GitHub, and that automatically gave me my documentation. And I did nbdev pi, and that gave me a pip installable thing. So, uh, that is all fully built in solve-it. Um, and we now have an Anki API.

And Jeremy, just roughly how long do you think that took you, that process?

I guess it was a day and a half, you know, with regular interruptions to hang out with Claire and stuff.

Mhm. Cool.

It would normally take me less time, but it was, uh, it was kind of like, it was a day of like trying to wrestle with the API to figure out how to stop it from having weird Rust crashes, because when Rust crashes, it just says panic error. It's like,

Yeah. [laughter]

But it was, um,

Yeah, you know, that amount of time to create something I can imagine me and my family using for the rest of our lives seems like a very reasonable amount.

Yeah. Yeah. And fun. Um, and very, Jeremy, I feel like you much more than most people I know are happy throwing out version one and rewriting version two and then say, "No, I still don't like that API. Let's do version three." You know, I feel like that's highly underrated, especially as code gets easier to make. But even before LLMs, you'd write a whole dialogue, a whole Jupyter notebook, and then say, "Nope,

Scratch that.

You know, like I've got a better way."

Yeah. Yeah.

And the second and third times are easier for sure. Um, yeah.

Um, now, um, okay, so talking of that, um, Eric's writing. Eric, um, I was going to say, I think I want to show, um, yeah, just trying to find it. So, uh, Scott Holley, uh, who's a professor, um, who's in physics, has been reading your book with, um, uh, some of the tricks we've been talking about. Um, and see if I can find it. Do you guys have a link? Because he had a dialogue, which I thought I had open, but apparently I didn't. Um, let's see. Here it is. Um, um, and so, uh, Scott, um, as well as being a physics professor, is also a musician and composer and lyricist. Um, and, uh, he was reading your book, Eric. Um, and it's interesting to read his discussion about the book, which is kind of like, what's, what's Eric talking about? Why did he write it this way? I'm confused. And then quite quickly started to become like, wait, is he doing this? And is he doing this? It's like, wait, why am I enjoying this so much? And as a writer, it's like, oh my god, look at this metaphor.

Oh my god, got this reuse of metaphors.

Um, and this irony. He's like, so interesting. And so he was actually started to be like, oh, as a lyricist, I want to understand how Eric writes, um, and how does he create these strange, the status quo can throw quite a punch when the status quo is ritually referring to the entirety of the, I don't know what you call it, the, the capitalist industrial investor,

Shirt, yeah. Complex.

Uh, I was like, "Oh, I see. Oh my god." The entire thing. Um, and I, I thought it was very interesting to read this reaction to like seeing the word choices you were making, and even things like, especially things like, oh, the choice of the word corruption. And I remember I asked you, Eric, what the [snorts] book's going to be called? And you're like, "Oh, for now, it's just called Incorruptible, you know, but I don't know. I'll come up with a better title later." And I was like, "Eric, that's such a good title. You know, you had this the whole time, you know, it's so good."

No, that's good.

Um,

It only took me two years to come up with it. Yeah.

Yeah. Um, so I think it's well worth for anybody interested in reading, um, it's well worth taking something that you think is is great writing. And I also love this circular firing squad. It's just a such a delightful, horrible idea, this absurdity of the self-disruptive group of people.

Are you describing the whole system?

Um, yeah. I just, um,

Oh, was I not sharing my, uh, screen the whole time?

I thought you, yeah, we're not finding it. We're just recalling, recalling the,

I thought, yeah, we thought you were just describing [clears throat] from memory the parts that you really like.

Yeah. No, [laughter] sorry. Okay. Circular firing squad.

Um,

Oh, yeah, this, this dialogue, this, this was so great. Yeah, I was so grateful that somebody took the time to do this. It was really,

Staccato beat, um, which is talking about this kind of like, uh, choice of like paragraph sizes, variation, um, yeah, this idea of corruption, um, sorry, I'm now going backwards to all the things I thought I'd just been showing you. [laughter] The bigger moment of realizing what status quo was actually referring to.

Oh, yeah, no, that, and that's so great because some of these are actually like, and it's funny, solve-it points this out in this dialogue, that when you write in this style, you're taking a risk,

Right?

You know, for the sake of the more sophisticated reader.

Well, this is what happened right at the start. This was, this was his first question. It's like,

Yeah. Did I miss something?

Why is he, what's I? This is all I know. He's like,

"Yeah, yeah, yeah. This is Eric's, just bear with him." [laughter]

Yeah.

Yeah. So, so you first of all, it's great to watch somebody actually go through, but also, yeah, you know, if for the close reader, by not, I think most people think they're getting the gist of a book when they read it, or any any prose, and if you, if you miss something, you have this like cascade of misunderstandings that can cause you to really like, take away the wrong thing from what the, what the author intended. So, to be able to stop, pause, make sure you understand, and then like, stay on the train of thought that the author intended, is not to say that that's the best train of thought, but at least if you're going to take the time to read a book written by a person, may as well find out what that person was trying to say.

So Eric, to do this, I guess you just prompted,

GPT-5 and said, "Hey, GPT-5, please write me a book called Incorruptible about the problems of governance and how to fix them." And you changed a few bits of grammar here and there, or is that not quite how it worked?

I wish I wish. No, it's not that process of it.

Yeah. No, I'm, h, I'm happy to. Yeah, like I was saying, this took me, this book took me several years to write and, um, was was very difficult because this is a problem that touches on so many different aspects of modern life and draws on so much research and so many case studies. And so, you know, there was like a lot of information to synthesize, and then of course, trying to weave that in with my own personal experiences over the past 10 years grappling with this thing that nobody can quite put the put a name on or a finger on. Um, it was was really challenging even for myself to figure out what things should be called and what, you know, vocabulary to to use. So, you know, I have, I have the luxury of of being able to work with a research team. So, you know, had had people to support me through this, through this journey, but yeah, it was very much like a wrestling match. And, um, what's interesting about it is like, solve-it, I remember you had actually finished the first draft, right? Like, you, you had a version of the book,

A complete first draft.

Um, which was terrible, but it, I did have it. Um, and I was really struggling, like profoundly struggling to get it to the next level. Because editing this, writing it was really hard, but editing it was much harder. The book, the draft at that time was, um, you know, two or even three times larger, longer than the draft of the book that people have access to here. So, I've, I've cut out in the process of editing, I cut out a giant book-sized amount of material to get the get things down to this manageable level. And so, anyway, so I was, I was struggling with it a lot. So, this book, you know, this book was kind of born along the same timeline as Answer AI, interestingly. And it was like this convergence of things in my life where like, we're doing all this research at Answer AI and making all this progress and doing all this stuff and working on on solve-it, and that I was working on, uh, on this book, and then just at the right, as things do sometimes, everything just worked out very beautifully, and I was able to, um, pioneer some of the techniques that we've, you know, we've shown in the course of like using the solve-it meta-programming tools and, and stuff like that to to pull pieces of the book into solve-it to come together at the right time.

[snorts]

Um, people say this about me, complain this about me all the time. Just like, Jeremy, how come everything seems to always happen for you at exactly the right time?

And I've noticed that seems to happen to you a lot as well. But like, actually,

You were the person at the company who did did that thing you said last week, which is like, I'm just going to pretend that solve-it is perfect, does everything exactly what I want. Imagine what the world would look like with that, and now proceed on that basis. And like, I, I think you were the, the one person right at the start who really was our biggest power user, and you were the one who just decided that this was the world that you now lived in, and you were going to make sure it happened that way. Like, there's definitely something there I experienced, you know, as with you as my customer,

That you were the one who was just felt like such a believer. And so it wasn't just a coincidence. It's like, I think there's something about how you operate which,

Yeah. Yeah. I appreciate things saying that.

My experience, people use the word serendipity, which I think a lot of people think is a synonym for coincidence,

Um, or or random random synchronicity, you know, that there's these words for these ideas, but like, I really think serendipity, synchronicity, these are these are not random occurrences. Uh, and to me, it's like, in my life, yeah, like, you don't.

It's like the difference between allowing the right thing to emerge. Like, like we talk a lot, obviously, in this book and in this course about emergent intelligence, like things emerge from a process that is in motion. That's different than being able to, to intentionally plan.

"You end up being somehow both one of the most optimistic people I know and also one of the most cynical bastards I know. Do you know what I mean? Like, somehow you have to be both of those things to be ready to engage. You have to be able, you have to be able to see the world the way that it is."

Um, and I think where a lot of cynics miss out is that the optimistic perspective and the cynical perspective are both correct because the world is more complicated than we can comprehend intellectually. So, there's a lot, you know, it's like it's a crystalline thing. You look at it from multiple facets, from multiple angles, you see multiple things. So, I, I found, you know, you could easily look at this story and say, "Well, you founded a company to make sure you would have the things you needed when you needed them." You know, which I, which I, which gives me like far more, way too much foresight than I deserve. I did not have in mind that it would solve this specific problem. But it is true that by staying very close to the details of what we're making at Enray, I do think that, you know, and Jeremy's being very modest here, like he and I together figured out, "Hey, okay, we're we're on the cusp of being ready to attempt this. Like, let's give it a shot."

"Well, I did the same thing, Eric. I was like, okay, like, I know success in the world is not accidental. Eric's been successful at lots of things. He's probably going to be successful at this thing. I want the world, I want this to exist in the world. I want to be a part of it. I assumed it's going to be brilliant and I want to be part of that brilliance." You know, I was definitely part of that. But sorry, I, we should, we should, can we dive in? Can we see some of your process of?"

"Yeah. Yeah. Yeah. No, I'd be, I'd be, I'd be happy to. Um, um, so, yeah. So, so I'll, let me just pull up, uh, should I share a screen and pull something up?"

"Okay."

"Let's see."

"Thank you."

"Um, nobody wants this course to end today, according to the chat. So..."

"Okay. Well, I have all, I have all day. We can take this potentially." [laughter]

"So in my solve it, I have just hundreds and hundreds and hundreds of dialogues that all look exactly like this where, um, I just, I was work, this is the one I was working on this morning. So, um, from December 3rd, this actually, ironically, is the to-do list I made myself of feedback from watching Jeremy read chapter 1 that you all saw in the course, the course on Monday. So I was like, 'Oh, there's some things in there that I'd like to consider adding to.' Um, so I just duplicated my previous version of this dialogue and customized it a little bit, and I'll, should I just walk people through kind of how it, how it worked? This was basically my, now my..."

"If those collapsed bits are bits you're allowed to see, I'd love to see all of it."

"Yeah. No. Yeah. I'll just do them, do them a section at a time. So like, I, a lot of this is boilerplate, but sorry, the zoom thing is in my way. So I, you know, started like I've been, I've been kind of by trial and error getting the instructions of what kind of context does Solvent need about what's going on to do a good job. Um, of course, a lot of this was tuned. This I've been doing this through three or four, uh, Anthropic model releases. So I bet you a lot of this is not needed in, now that we have Opus 4.5. But, um, anyway, I've been by trial and error trying to figure this out. So like, this is some things that it needs to know. Um, I spent a lot of time trying to get it to stop doing, um, AI-isms, which is really difficult. So, you'll see a bunch of that is about just trying to help it to train it on on things I don't want it to do. Um, I wrote this really elaborate, uh, style guide. Sorry, the Zoom window is getting in the way of my controls here. How to make that go away. Maybe like this. Can you still hear me?"

"Yep."

"Mhm."

"Great." [snorts]

"Um, so here you can see I, I actually worked with, um, uh, this is, this I developed, I think actually with Gemini Pro a long time ago, um, a guide to my writing. I have the advantage of my, my writing has all been illegally, uh, train stolen for the corpus of the training of all these models. So, one of the benefits of that is that, um, it knows your writing pretty well. So I did, we worked on this and kind of developed like, what is my style? What, when, when it's evaluating something that I wrote or when it's trying to help me, I [snorts] want to make sure it knows the, the style of writing that I like to do. So you can see there's like a lot of this stuff, uh, to it, kind of a long one. Then I had always gave it a briefing on the book. This is a much worse summary than the one that..."

"Did you skip the AI-isms to avoid one?"

"Oh, sorry, I didn't mean to. Yes. Let's do that first. This I took from, um..."

"The Wikipedia AI-isms list relatively recently."

"And this helped a lot because it has all these examples of what we don't want."

"Yeah. So I shared this on our Discord. I took the, we might, I think we shared this on our Solvent Discord as well, but we can do it again. I, I took all the stuff from Wikipedia's list of things to avoid and kind of cleaned it up and shared it with our team, and, uh, yeah, it works great, doesn't it? Yeah, this is, is extremely good except for M-dashes. Now, what's interesting about it is I like to write with the M-dash. I'm actually very annoyed that this has become an AI-ism marker."

"Yeah."

"Um, but sorry. Um, I had to have a special instruction to be like, 'No, really, specifically because I have so many test readers. If test readers see an M-dash, they get really stressed about the idea that it was written by AI.' Yeah."

"So, I tried to be like, get it really to avoid the M-dash like the plague so that I could choose to use it intentionally when I really wanted to for stylistic reasons and not overuse it. So, that was a battle I went through a while ago."

"And interestingly, your Gemini-based one is good because it's explicitly saying what you do want to see. If you only have what you don't want to see, it can be difficult because it can cause AI-isms to kind of glom onto those things. You see this often on the internet. [snorts] People will be like, 'Never use emojis.' And it'll be like, 'Sure, I won't use emojis.' Smiley face."

"Right. Right. Exactly. It gets, definitely that's one of the biggest things that that I learned from Jeremy early on here is that like the more you tell it not to do something, it's just, it's, it's actually like, you know, with a human being, like if you tell people like, 'Don't think of an elephant,' like the first thing you do is think of an elephant. Like, it's actually very difficult to instruct neg through negatives. So, as much as possible, I'll try to give it examples of what, um, what I want it to do rather than, um, telling it what not to do. So, anyway, here is a, um, I would very often rewrite this briefing with the help of, uh, of an AI just to give it some basic idea of what the different chapters are. And I found that if this is too long, it gets, starts to get confused that this is the chapter I want to work on. So this is an extremely concise, um..."

"...um, summary just so that it, like, you know, can can resolve chapter references and and when I ask it, you know, to look up a case study or something, it has, it has an idea what I'm talking about."

"And like feedback for others. Bit late for you now, Eric, but I would, I, I would say that kind of summary is best written in the kind of prose you want to write your book in, rather than as bullet-pointed lists because again, it's follow stylistically. So when I was..."

"doing summaries and stuff of your book, coding was encouraging it to write it in the style that I will want to be using myself [clears throat] later."

"Yeah. Yeah, that would have been, um, that would have been really helpful, um, to do it that way. So then I have some code here. Um, I'm super lazy and so, um, I want it to use the name of the dialogue to figure out what chapter we're working on so I don't have to go look it up, which file it is. So there's a bunch of stuff like that that's just pulling in from dialogue helper. Um, so meta, metaroming. I don't think these are super interesting to look at the code for."

"So, just people might not know about underscore add message unsafe. That's, uh, a..."

"Oh, don't do that. Yeah. Well, no, no, it's, I just want to tell people about it. So it does exist and if you've got to lesson 10, then we trust you. It's not included in the default list. It is a tool you can use. It's exactly the same as un add message, but it's got one extra parameter, which is run. It's a boolean. Uh, so it lets, um, it lets solver use the entire power of Python without you watching. It can, it can write itself messages to do its own follow-up prompts. It can, so it can do full emergent loops. It can write code. It can read the code. It can read the results. So that's something that, uh, maybe create a new instance for it with nothing that you worry about losing. But, yeah, be real, be real careful with this. Yeah, you can. I, I've shot myself in the foot many, many times with this."

"Can I just mention, just like how real this is when I was doing this, how the hell do I get the SQL light database of Anki to actually close properly thing? Solve it on three occasions said, 'Oh, I see the problem. It's locked. I'm just going to delete the .w file.' And then I, I didn't have tools on, um, and I'd immediately say..."

"No, solve it, the .w file contains the uncommitted transaction containing the actual data of the database. If you delete that, you'll lose all of the data. Be like, 'Oh, yes, of course, I can't actually do.' And then like three messages later, 'Oh, I see the problem. It's locked. I'm just going to delete the .w file.' It's, yeah. These machines, you know, you never know."

"You never know what kind of insane things it's been trained on."

"Yeah. And as soon as it's like out of the happy path, which in this case was like, 'I don't know why this bug is there,' it it suddenly becomes stupid because you're, by definition, out of distribution."

"Um, so it just does wild and crazy things."

"Yeah. So, anyway, I was just showing a few of the meta, some of the tools that I wrote that you will come, I'll show you how they, they're easier to understand them when you see, um, what they do. But just, um, I, I, um, very quickly realized that my chapters are kind of long. So like, if I was doing a blog post or something shorter, um, I wouldn't bother with all this CR to-dos and metaroming and whatever. But because my chapters are long, I have to work on them in discrete portions and context length just becomes a huge problem. If there's just too much, you know, all my scratch work is sitting there in the context, then it gets confused. So, what I found is very effective is to, um, divide up the work into what I call the to-do list, and then we would work on a to-do, finish it, and then hide the details of that from, um, from solve it so that it didn't get, that didn't get cluttered up and I didn't run out of context. Now, context windows have gotten a lot longer since I started doing this, so this isn't as necessary as it used to be, but you'll still find if you work on extremely long dialogues as I do. I'm definitely the longest dialogue person in the answer team."

"Sure."

"Um, you start to get a sense that like the LLM just gets stupid after a while. It's like, it's, it's really hard to describe, but like its intelligence starts to break down and just, I started to think of it as like the amount of attention it has is just, it's like a finite quantity. It's like you're spreading the peanut butter too thin over too much surface area. And so oftentimes I would just start in dialogue and and keep it more focused on something more narrow. Anyway, these are a bunch of things that, um, I pulled out of Jeremy's, um, test reading of chapter 1 that you saw last time. Um, and so, you know, some of these are quotes of things that he sent me. Here's some screenshots that I pasted in of things that he had pulled out of his dialogue. So it's just kind of a heterogeneous. Excuse me. Sorry. Someone's trying to call me. Um, if you would just give me one second. Can you, uh, pause for a second?"

"Absolutely. Sorry about this. Chad, have you done any diances with, um, AI-assisted writing or editing or anything of prose or, um, not sure I've ever talked to you about that or if you've just done stuff with coding?"

"You know, in the, in the last course, we talked about AI for writing and, uh, I was kind of playing the skeptic because I am a bit of a skeptic. Yeah."

"If people send me articles that I sense or AI-written, I, I lose interest pretty fast."

"I find the same thing. There was something on the front page I had yesterday that was AI-written, and I, I literally couldn't read it. It's something bothered me about it."

"Yeah."

"Okay, I'm back. If you want to, uh, continue ahead, John, I, finish your thought and we'll go back."

"Well, I was, I was just going to say, um, I, I've used it to review like, 'Here's my post on this. Could you see errors or things I could do better or typos?' Especially like, I, we had a paper that I, um, did with a friend, and that was really helpful to say, 'Like, are there pieces we should spruce up? Details we missed? Things we can take out?' Especially like cutting down to the submission limit. But for my writing, my own personal writing, you'll see my personal site has lots of typos because I actually have just decided to go back to, I don't even have Grammarly on." [laughter]

"So, um, yeah, I just edit in, um, markdown."

"Uh, and, and I want to be super clear for everybody. This is the important thing that I learned during this process. We call this AI power, AI-assisted writing, but it is not AI writing. The AI just is not the thing that should be doing the writing. It is here to assist the human in the loop of doing good writing. Uh, and anytime I would get, especially in the early days of doing this, I would kind of get that soporific effect that that vibe coding has on you, right? Where you just, you start to be like, 'Oh, yeah, good. That sounds fine.' Like, I would often have to delete huge swaths of writing that I thought I had done, but actually the AI had done, and, you know, when you look at it in the cold light of day the next day, you can't believe that you ever thought it was good. So, it's a, a really important, um, really important skill to master."

"I feel like everybody actually has to go through that personally, cuz..."

"Yeah, everyone thinks it will affect them."

"We all do. I know it's terrible. Anyway, uh, you know, this is, so, so if you look at this to-do list, it's very heterogeneous. Like, some of it is like images, some of his blocks of text. There's all kinds of stuff that, um, that I thought was interesting about, um, Jeremy's test read. And I tell it that they were going to blend this with, um, other reader comments. So, one of the things I learned, um, over the course of doing this as I sorted to have more test readers is that whenever I'm looking at a piece of text from the book, it's just really helpful to have comments from test readers with you at that moment. And it's really nice, like solving out just the comments that are needed. I'll show you the code for how that works, uh, in a second. I wrote this dialogue called reader comments, which is just manages this CSV from the test reading platform. Um, and it can pull out the, the individual comments."

"And like, just to give a sense, that took us like..."

"Yeah."

"15 minutes, right? You and I wrote that together."

"Totally. Yeah. In fact, if you, people want to see, if I just come over here and I just do Nope. This is just a CSV file with the raw, um, comments in it. So you can see like, every single one is just like, chapter 15."

"And maybe just scroll down to the code. Just give people a sense. So we kind of got this global comment numbum. It look, it couldn't be simpler, you know."

"It's extremely, yeah, it's extremely boring. Um..."

"I had to, um, the, the numbering scheme of the chapters changes whenever I change the structure of the book. So we had a little bit of this, um, this nonsense going on. Uh, and then I wrote this, a little bit more complicated one that just pulls all the comments for a given chapter. And this is the tool I actually use, um, to give solid a whole chapter's worth of comments, which you'll see over here. Um, I'm just telling it some things. I got so lazy at writing certain things over and over again that I made myself a little macro list of things that, um, I type all the time. This isn't, this I don't think you need this anymore, but there was a time when ultra think, evaluate honestly, really, like, with with older Anthropic models, just seemed like it gave me way more rigorous assessment. And so, yeah, ultra think analyze is more like, be open-minded and give me different possibilities, and evaluate honestly is like, give me the cruel logic of what is your assessment. Anyway, so then here are the actual instructions that I had to teach it how to use these tools to, um, commit changes. I won't go through all this. It's kind of boring."

"But it's a funny kind of programming, isn't it? You know, it's like, this, it's kind of reminds me of like, kind of, Singaporean English, you know? It's like, you know, you're kind of mixing bits of code and bits of prose, a bit of pseudo-code and a bit of this and that."

"Yeah."

"And this, this was written by sitting down and thinking, 'What's the ideal book-writing API?' This is written by frantically, kind of..."

"And I feel bad showing the current version. You know, if I showed you the first version of this from six months ago, it would be extremely bare. You know, this is like, this is accumulated all the stuff that I carry with me in my for my template. And like, you know, there's just these little things in here like I had to really hammer on it to understand that like, once I introduce reader comments, the readers are not editorial experts. So I don't want you to treat a reader comment as like a command to do what the reader says. I want you to treat it as, you know, as as one input. But when my editor says, 'You need to fix this thing,' then I want you to take that seriously. So some of these are just like little quirky things I picked up along the way. So, this is my like, the main task, um, instruction that I have learned to give it where I'm going to go through each to-do. We're going to find the relevant comments for that to-do. We're going to build a briefing note as a note in solve it, and that will be the the start of a task. Um, again, I won't go through all the, the coding here. It's all, you just like the specific things that I want. Make sure I always ask it to acknowledge that it understands. And then like here's the, um, code to grab some comments. So we're gonna, I have 8,42 reader comments, you know, across all chapters, all versions."

"Wow."

"This chapter has 625 comments because people are much more likely to comment on chapter one than later chapters, let me tell you."

"So like, as a, as a human, I've read every one of these comments myself over the, you know, weeks and months that they've come in."

"But if you just were to say me like, 'Real quick, what are readers' favorite and least favorite parts of this chapter from the comments?' M..."

"There's no way I can answer that question. Honestly, I'd be telling you my favorite parts."

"So, instead, let's have Solvent answer that for me. So, here you can see, um, the tool call. Grab all the comments for chapter 1 and please summarize what's going on. Here are the favorite things people like, things that people find confusing, specific things that they didn't like about, um, uh, the most recent version. Shout out to Janice Fraser, who got a thorough reader, uh, uh, appreciation from Solvent, um, which I thought was really, she is really great. Um, I was very grateful to her. You know, here are some typos that were found. So, just like, I like to start with this, like, 'How are we doing on this chapter?' And I found that this is better than just asking solve it to evaluate the chapter because, um, it will always, you can do this forever. It will find problems."

"You can, you can even if it wrote the chapter itself, it will then find problems with it. You can never make it happy. So you have to be real careful with that. Anyway, then we begin. So I always have a section to mark whether we do the beginning, and we just start with to-do 1, and you can see it using all these tools to produce this really nice briefing note, which I'll expand so you can see all the work related to it. Um, this is the passage that we're going to be working on. Here's the specific, um, thing that I wrote, which was just, 'Maybe I should add a footnote about this YouTube video that Jeremy had mentioned while he was reading through.' Here are some reader comments related to this section. And here's some analysis of what it thinks might be going on."

"Um, I forgot that Solvent cannot read YouTube, um, URLs natively. So, I just, I'm so used to just sticking a URL in here and having it read the URL and find me the context. Can't do that on YouTube. I have to use the YouTube transcript API, which I found on Pi. Um, so I just very quickly, this took me like two seconds, like, 'Write me some code to go get the transcript.' I messed it up. What I had to do wrong. I was just, um, vibe coding my way through this process, and pretty soon..."

"Just scroll up a little bit to that. I think there was a D or something there. Stop. Yep. So scroll down to just that where that D, all those underscores, I think one more page down."

"Yeah."

"There. Stop. So you can see here this, uh, thing of like where he's got a dur, print dur. Um, this ability to reflect upon the capabilities of an object in real time based on actually loading it in is really important and it's massively underappreciated. If you just give that, um, if you just give solve it a tool which gets passed a string and returns dur globals, the string, you can then just say to solve it, 'Figure out how to do this,' and it can just go ahead and run that for you and figure out everything based on what you actually have in your namespace. It's just super powerful. Or just do the way that, um, Eric did here, which is just to let it give you bits of code that you can hit W to include."

"Um, so people often say like, 'Oh, you know, how do you do solve it when you don't get to like click around in your IDE?' And to me, I'm like, 'I don't need to do that anymore because...'"

"Yeah. Why?"

"That was the before AI. Now it can click around through the object."

"Exactly. And this, I felt, I felt really gross with this one because I was like, 'I've, I, I've done this before and I, like, I was like, I vaguely remember how to do this module, but I was like, I was like, I'm, I'm in a rush. I just need this quick, like, summary of what this video is about.' Like, so I was just like, 'Whatever, whatever, whatever.' And so, you know, it just, and I was like, 'Great. Now give me a note with a summary of this video. What's it about?'"

"And I got this nice, nice video summary."

"This is the Dylan Williams traffic cups."

"Yeah. Which is a great video if you haven't seen it. It's super cool."

"Um, and yeah, then I'm like, 'Okay. But anyway, what I really wanted wanted to know is like, should I put a footnote in about this or not?'"

"And it was like, 'Here are the pros, here are the cons. No, not a good idea.' I was like, 'Great.' Like, this was just so helpful because like, when you're in your head as a writer being like, 'Ah, should I do it? Should I not do it?' It's like, just very useful to have analysis like this. Now, this was a really clear..."

"Person who brought this up. I think that's right. It's probably not a footnote. If it was to be anything, it would be like, you know, diving in more deeply. It turns out this is something that impressed today. And if, like, well, it's not what I, what I want to say in this chapter, in which case..."

"It's a delightful aside."

"Yeah. Yeah. Exactly. It was like a very cool, very cool, uh, connection and like, I found this, like, normally I would, I would go back and forth with it quite a bit to make a decision like this, but this one was really easy. I was like, 'Yeah, you know what? This is, this is interesting to me, but it's not going to make the chat.'"

"Although I will say that con number one, I, I actually disagree with it. I love it when people write books where there are footnotes that just have random aides. I actually find that..."

"Yeah, I, I like that too. I like that a lot too. Yeah. So, it was like, yeah, I could, I could feel like I could have, um, talked myself into this one, but I was like, I was already on the fence about it and was like, 'Okay, never mind.'"

"So, you type skip to-do, and that's some kind of macro you've..."

"Yeah, I have a macro for, by macro, you didn't write the macro per se. You just told it. When I say skip to-do, you should do this."

"Exactly. Yeah, macro is actually like, very misleading. I just literally just, um, was tired of having to say like, 'Okay, let's skip this to-do.' And what I found was that I've done this so many times, um, if I just used plain prose to describe what I wanted, like 95% of the time it would do the correct thing, but 5% of the time it would get confused about what I meant."

"I found this is much better."

"Can you just do a to-do completed? Question mark, question mark, to see like what, just remind us what that function..."

"This function? Sure."

"Yeah. So, it's kind of a funny mixture of prose and, yeah. Yeah."

"So, it's gone through and it's using, and this, if we scroll, yeah. So, you got kind of ad messages too."

"Yeah. This just, you know, and if I probably was doing it over again, I probably wouldn't even bother with this because now Solv can just do ad message so successfully by itself. But this is from an ear, this is from quite a while ago. Um, but it's just like, I'm very lazy and so every time I complete a to-do, I was typing in all the same things and I was like, 'This is getting tiresome. I like to have it do it for me. So when I type skip to-do, it creates a note to summarize what we did. This is going to be helpful for solvages get hidden. Then I have this block of things, um, that it does. So it skips, this is, um, hiding all the all the work of this to-do. It hides the tool usage prompts, um, and then it rereads the chapter into its, um, current context. So you can see here it turned 14 messages off. If you've noticed while we were going through, all these messages are marked as hidden from."

"So you went down that rabbit hole, but you're not distracting the main flow. It's just while you're working on that to-do, you're loading up this transcript and writing all this code and everything. And that's..."

"Yeah, exactly. It's like when I'm working on to-do 2, it doesn't [clears throat] need to know that we, you know, looked at the transcript. It has this very simple summary right here."

"Right?"

"That, wow, we did this, did this thing."

"And then it actually, I'm so lazy that I have it write this message for me. This is an ad message. It writes the next, 'Let's do the todo two myself.' Um, so I just have to run that prompt, um, myself, and then it makes the next one. So here's the briefing thing for the next one."

"And it's just interesting for me to hear how you describe yourself as [snorts] lazy in this way. You know, I'm I'm similar, right? People will be like, at the start of this dialogue, we think, 'Dude, you're the opposite of lazy. Look how much you've got here.' Um, but it's true, right? This is the Larry Wall, you know, three print, you know, science program of like, 'We shouldn't have to do work. Computers can do for us.' So you set it all up so now the computer can do it all for you, right? [snorts]"

"Yeah."

"When I find myself doing the same thing over and over again, it's like, why, why not automate?"

"Is what I would try to do."

"Um, and this is another thing, another rabbit hole that, um, Jeremy had encountered, which is like, so crazy for those who've read chapter one. Jeremy Bentham is one of the investors. The great Jeremy Bentham, the utilitarian philosopher, was one of the investors in Robert Owen's, um, uh, project. So like, that's such a cool historical detail, but I had decided when I wrote the book not to include it in the manuscript, but then when Jeremy went through the rabbit hole, I was like, 'Gosh, it really is very cool. Let's consider if this is something that we should, um, do.' So here you can see this is actually a bit of his own dialogue of stuff that he found when he was web searching, and here are some comments about this section, and it's, you know, solve it has decided here's what we need to decide. So I ask it for an ultra-think analysis. You know, what makes sense here? So it's, um, you know, giving me this, its analysis of the situation. I asked it to web search for additional information about it because like, and I had forgotten this actually completely, but it's not just that he was a random investor, but one of the times that Robert Owen's investors tried to force him out, the the mechanism they did to do it is they forced him to auction off the assets of the company. This was meant to be a liquidation, and only because Owen found new investors who would allow him to be the highest bidder in this auction could he create a brand new company to reacquire the same company he had formed in the first place, thwarting the first set of investors, and that's when he brought, um, the famous Quaker William Allen and and Jeremy Bentham into the situation. And of course, what's totally crazy is that that then they, they also wound up ousting him again. So this happened to him three times, um, in case you were wondering if this is a consistent pattern. Anyway, so like, so solv, more more interesting than I thought. So I feel like solving, like this is actually maybe something worth including. But as, as I'm reading through it now, I'm like, 'I think this is too much.'"

"Yeah, I would agree."

"Ultimately, at the end of the day, Jeremy Bentham didn't do anything. So it's not a very interesting bit of the story. And this is, I know because we have these reader comments. This is already a part of the of the chapter people find a little bit slow. It's history from the 19th century. It has archaic language. It's already some bits in it. So, um, I was, this is what I was doing right before I, um, came to be, um, on this course with all of you. So I'll do it live. I'm just going to skip this to-do right now so you can see exactly how this works."

"Um, and I think, you know, I definitely agree with the result there, right? Um, because a lot of it also is just this whole thing of like, well, religion's a big part of, you know, you know, they disagree about his atheism, agnosticism, or whatever it is, and so like, it's very interesting historical stuff, but it doesn't really pertain to the point you're making, right?"

"Yeah, exactly. So here you can see it's created this summary about why this is intellectually interesting but tangential. Um, I'm going to run this block of code, which is now going to mark all those things as hidden. So you can see they're all now hidden, and we'll do to-do 3. So anyway, so this is like the level of flow that I was able to get into as a writer working this way, um, compared to sitting at at your blank terminal being like, 'Gosh, what should I work on now?'"

"Um, was really, really remarkable. And then this..."

"And it also has a lot of patterns, reflections of stuff we've seen before. This idea of get the outline at the start and then let it say like, 'Where are we up to?' Or like, when Cla was reading it, it's like, 'Okay, go back to this question.' Or it's so helpful to have something to keep you in the flow, right?"

"Yeah, it's exceptional. And I, you know, I have a little bit of ADHD brain, so like, even just to be able to keep track of what are we working on, or, you know, I get, I'm busy, so I get interrupted. I don't, I'm not the kind of writer that gets to have, you know, 12 hours straight in my mountain retreat every day, you know, me and the blank page. Like, I get interrupted. My kids want to talk to me. I have work things I have to do. So, like, you know, to be able to work for 15 minutes, get interrupted, and then be like, 'What were we working on? Where were we? What's going on?' You know, just to be able to like, always be right back at the place where I left off."

"Um, actually, can we just stop on that one for a moment, cuz..."

"Yeah, sure."

"That was one of the questions in the kind of AMA that we'll get to later, but now's a good time to talk about it, which was, actually, it's a question to me, which is like, Jeremy, how do you get so much stuff done in general? How should we go about getting stuff? You know, I've got a family, I've got stuff to do. It's like, how do you get so much stuff done personally? I've got like 15 minutes here, 20 minutes there. But actually, what you're describing is kind of what I would be saying as well, which is you, you're building processes to allow you to use these smaller pieces of time effectively and dive, dive back in, right?"

"Yeah. Yeah. Exactly. You, you just, you have to. And you have to always make forward progress with whatever time you do have available, which, you know, which true, obviously, this relates to one of the ideas in the book, which is like, if you, if you have a larger scale vision and ethos and like set of principles, like one of the many advantages of that, I don't talk about in the book, is just personally, it's always much clearer what to do because you, you have everything that happens, you can slot into an existing mental framework. And I found that to be, um, very helpful. So..."

"It took her quite a while to do all this, didn't it? It worked quite hard to get all this."

"Yeah, this one, it worked really hard. I think because this was an image, I, I'd screenshotted your your dialogue instead of copying it because you sent it to me as a screenshot. Um, but I think it's also like, kind of a big question about whether the thesis of the chapter is completely crisp and clear. Um, and so..."

"Oh, yeah. And it pulled in some stuff from a different to-do, which that's actually unusual. It doesn't usually do that. Um, pulled in a lot of reader comments. So, yeah, had a lot, lot to do. The other thing I noticed is when I'm whenever I'm multitasking or like thinking about what to do, I'll be like, 'Well, I'm just going to have Solvent do some thinking while I'm thinking, just so it's not idle, you know?' So, here I'm like, I'm like, 'I don't know. Do I really want an analysis of this? I'm not sure.' But while we're talking about it, let's have it do it. Sometimes I would write three. I would sometimes type..."

"Brains going."

"Yeah. Like, I remember multiple prompts in a row here just to be like, 'Keep it busy while I'm thinking. Don't bother me right now.'"

"Even during the first cohort when we had like live students. Okay. It was smaller than this one. But I looked at the like, some breakdown of token usage per user or something. And there's like, basically Eric and then everybody else, um, in terms of like, total tokens consumed and processed on a daily basis. Um, [laughter]"

"Yes. Well, I, I mean, I was really, there were, there were, there were months where I was really like spending a significant percentage of my waking hours on solve it and then, yeah, and working on these extremely large context, uh, things. I'm definitely a high token consumer."

"Um, um, Eric, um, can we switch directions a little bit?"

"Yeah."

"Um, to something else that you do, and I actually showed something a little bit like this, just now. So with the Anki thing, I did, I showed an example of how I, uh, got Codex to study the source code repo of Anki and write an API.md document for me, which I then pasted in. Um, you've done something a bit similar with, cuz I said like, 'Oh, you know, Codex is good at that specific task,' and, um, you've recognized that OpenAI Deep Research is very good at the task of doing web research, and that you often paste that in. Um, do you think it might be worth talking a bit about how you take advantage of that, uh, combination of tools or kind of fact-checking research that's, um, that is..."

"Be happy to do that."

"Thank you."

"Um, let me see if I have one. Oh, but yeah, so, so first of all, Deep Research is exceptional. Um, you know, so, and so is Gemini's Deep Research, and so is Claude's Research mode. Um, I, I've, I got used to using OpenAI as Deep Research because it was first. It was the first one that I used and..."

"Um, and it's really good. And let's see, like, um, see what this is. No, that's not, that's not too, too interesting. Let me just see if I happen to have one up. But I, the zoom toolbar is now blocking my viewing."

"It [laughter] does it stop it."

"Um, and like for this kind of research, the, the Codex research or the Deep Research, it, it doesn't bother me at all in the same way in terms of that friction of multiple tools because it's something where I'm literally saying, 'I want something to go and spend half an hour studying this,' so I'll go and make tea. Um, so like for most things, I don't like to, you know, I'd be like, 'Oh crap, I really want to get this built into solve it.' Um, but, and, and indeed, solving better at doing much better at doing this kind of research, but it's still nowhere near as good as Deep Research for web stuff or Codex for, um, source code stuff."

"Yeah. Yeah. So I don't, I don't, these, these are actually Google, um, these are ChatGPT Pro, um, analysis, which is now almost as good as Deep Research, especially for fact-checking and stuff like that where you don't need too much, there's an, too many shorter answers, but well-researched."

"Yeah, depending, depends on, depends on what you want. In my, in my downloads directory. So what's funny about Deep Research on ChatGPT, let me see if I can pull one up. Who knows what I have in here. Oh, you can't even tell which ones are Deep Research anymore. Oh, all right. Never mind."

"Um, but anyway, if you look at my, they changed the UI of ChatGPT to, um, if you do a Deep Research, you can no longer copy it as markdown. So, that's what I used to do is I would just take something like this. I'll just show here. Copy it as markdown. Take it right into solve it. Um, you know, give it a heading of whatever. And boom. Now I have, now I have the Deep Research into solve it, and I would ask solvent to summarize it for me. Um, they changed it so that you can only download Deep Research as a PDF or a DOCX file, um, for whatever reason. But that's actually turned out to be really great. So now my downloads directory on my laptop is just full of zillions of DOCX files of, um, Deep Research. And I wrote myself a little, um, pandoc, um, alias in my terminal where I can just, it automatically converts them to markdown and copies them into the clipboard."

"And so, um, I can just pull them straight into, um..."

"Uh, so let me, you say a thing about what Pandoc is briefly, because I think that's worth."

"Oh, sorry. Yeah, Pandoc. I mean, Pandoc is just a, a one of the all-time great, um, command line tools for doing document conversions of all, many, many, many different kinds. And it's one of these tools that's just exceptionally well-written and error-free. Um, people can look up the history of it. It's a, it's very cool how it came to be. But anyway, um..."

"It's built into, it's built in to solve it, to be clear. So you all have it."

"Yeah, you all have it. And, and it's easy, you know, it's easy to get on other platforms, too. It's, it's very, very widely available and it has exceptionally good DOCX, um, rereading and writing abilities, which is a notoriously difficult format for people to read and write. It can even, um, by the way, it can, it can consume track changes and other metadata stuff from a word document, which I've used my publishing team, my editors, and stuff, they give me back information in that format. Um, and so I [laughter] can just, it's like, it's terrible. It's just an extremely bad format for machine reading. And so Pandoc has turned that to be no big deal. I can turn it into, um, GitHub flavored markdown, and all of the word annotations and track changes and everything show up as XML blocks that solve it can easily read natively. So I just paste it in. So now that we have AI, one thing I have found to solve it..."

Is uh Pandock is extensible with short little Lure scripts, which is the kind of thing I would never bother to learn before. But now that I've got AI to help me, I'm suddenly creating Pandock [snorts] extensions, and it's great. I can convert between all kinds of crazy formats easily.

So here's an actual asking about, like, do we, do we track changes and how do we store the books? Um, a lot of the ground truth file representations are Markdown, right? Both for the book and then also even for, like, our legal work. A lot of contract drafting and things, and Pandock is a translation between, uh, Markdown and whatever final, you know, PDF or whatever that someone wants to consume.

I mean, Answer AI runs on Markdown. Yeah. Anything else we have to deal with PDFs in legal, Word, with publishing, we, we do it at the last minute translation, basically, and try to avoid it until we have to.

Yeah, now I have also, um, and I don't think we'll have time to demonstrate this today, but, but Jeremy's shown Solve It looking at, at, at screenshots, you know, live screen reading of something in, in, um, on the document. So I also will sometimes, if I have to be in Microsoft Word for something related to my publisher, I can also have Solve It help me with that by just looking at the screen, kind of looking over my shoulder, what's going on.

Um, but anyway, but so like, just here's an example where I just pulled a random, um, hey, notable cases where founder control was overridden or lost. That's a pretty common thing for chapter one. So I pull it into a note. It's already in beautiful Markdown.

What's that?

Alibaba. That was quite a fun.

That was quite. I mean, it's just some of these other things. That's cool about this is like, some of these stories are things you know, but you don't actively remember in the key moment, you know. So here it's like, oh yeah, right, that was crazy what happened that time. Or then some of these things, you know, you know, but I had never heard of Magna International.

May know that.

Um, but that was pretty. Anyway, so this is kind of like the ways that dual-class founder control mechanisms get defeated. People, people often think these mechanisms are inviable, but actually they are pierceable if you don't know what you're doing. Anyway, uh, and then I would very often just be like, summarize this research note as it relates to this chapter. Um, even like, again, even while I'm thinking about whether I want to use this research or not, or while I'm reading it, I'll have Solve It be also summarizing it because, why not? So, we'll see what it, it comes up with here.

Yeah, the Blue Apron story, that was crazy that he was ousted like within months of going public. Eric, um, is this a good time to have a five-minute break and then come back to talk about startups and the future and all that kind of thing?

That would be great. Yeah, I would be, uh, I'd be absolutely delighted. I guess a break would be great.

All right, see you in five minutes. Hello.

Hello.

Break time's over. Having trouble getting my phone to work for me once. Hello. Hello.

Hi. To show my commitment to appreciating this part of the conversation, I have moved to my comfy recliner. [laughter] Eric, I

Very important.

have just sent you links to two things. The very interesting list of questions from Aditia, uh, who has the most thorough list of interesting things to ask about startups I think I've ever seen, and the kind of AMA thread from the Discord. Um, but, um, maybe if we could start by, if you could help us to kind of put this in perspective here. Is this, this is, um, uh, sadly the end of this course. Um, it is a course which we found almost impossible to tell anybody what the course was going to be about. Uh, some people demanded to know such things before they decided to spend money on a course. Uh, uh, but now that we've done it, I still don't know what the course was about. But, um, it, I think it's because like, it's about so much. It's about a certain time in history and a certain group of history people in that time in history that are trying to, uh, do things in a different way. We're now all part of this community together, which is obviously a very strong one. Yeah. I guess, I guess maybe we could start with like, your sense of like, you know, what is this moment and where do we, as the Solver community, sit in it? And, you know, as people go back to their families over the, you know, the, the season break, they're going to be talking about stuff. Yeah. Where, where do you kind of feel like we're all at and what might we be thinking about towards the next year, you know, and,

Yeah.

how to get the most out of this and, yeah, just your reflection on to before we, you know, where we're at as a starting point.

Boy, yeah. Um, you know, it's funny, you started with the, the feeling people had being in the Fast AI course when it first started, that they were kind of, uh, they didn't even appreciate that they were on in at the beginning of something exceptional.

And I can remember, I felt that way so many times in my life that I missed, and that I'd missed the boat.

Really.

I can actually remember in 1999 feeling like I missed the internet, all the great companies that already been created. I, I missed my chance to be part of something really important and foundational, uh, on the internet. Yeah. Like so many times I felt that way.

And when you look back on it, it's so silly. You're like, "Oh man, you, you, you know, you're so lucky. You're so blessed to have even been aware of the internet at that time, let alone you missed it." And the vast majority of people on the planet had never heard of it even at that time. You know, it was still very, very early.

Um, so I think that's really true of this. Like, we're living in extremely tumultuous times, you know, from a technology point of view, but also from, um, you know, geopolitical point of view. Like, there's a lot, a lot of big problems that need to be solved in this world. And the people who are going to do the solving are not the most prestigious people of the past who have the old systems credentials and knowledge and know-how. A lot of the people that are going to make the most impactful difference in the years to come are going to be people who harness these new abilities to make transformational change. Uh, and of course, that's part of why I wrote the book.

Um, but it's also part of why I'm excited about Answer AI. It's part of why I wanted to be part of this course. That whoever's listening to this, you have powers you can't even possibly imagine. And just to say also, like, a lovely thing about this just popping up in the chat right now is a lot of the folks in listening to this were actually, uh, there. So 2016 is when we did the Fast, first Fast AI in person, and it was released on YouTube in 2017. I don't know who Simpleman 8556 on YouTube is, but they said, "Here, I joined the first deep learning course in 2017, and I am now leading a team to build deep learning models for cyber attacks detection."

Um, yeah, it's been,

Awesome.

eight years. [laughter] So maybe you can help us, help us find a path to start our next eight years. Um, but I think something I want to make clear, or, you know, I'd love to hear your thoughts about, particularly is like, we're not asking anybody to predict, and we're not asking you to predict what the next eight years holds, right? That's not, that's not how we decide what to do tomorrow.

No.

No. And I think it's actually great. Like the fact this is happening right before the holidays is really great because for some people who are listening to this are going to go on to create startups that are going to be really important companies. That's like definitely one thing that you can do with this new technology. And I'm not saying you have to or you even should do that, but some people I know are already thinking about this. And the most important thing you need for for that to be successful now that you have this incredible tool is to find a problem worth solving.

And being out of your normal environment, like being home for the holidays or being on vacation or any anything that gets you into the real world is like, now that you have this, um, this idea. I have to tell it to stop doing this thing with the camera making me crazy. I don't know what it's doing. Um, it can, you, you never know where you're going to find the problem where you're like, wait a minute, why is the world that way? That seems strange. And what's cool about the truth is, this is, you know, my own personal view, is that every part of it is connected to every other part of it. So if you're working on something that's true, you never know where it will take you. Could take you to the most fundamental truth of the universe, uh, if you, if you pull on the thread, uh, far enough. But you don't have to know. So sometimes, like, what I would recommend people who are really not sure what to do is like, find problems and solve them. So you have this incredible tool, you know, in your hands. Maybe it will turn out to be a trivial problem with a trivial solution, and you'll completely solve it, and it'll be completely done, and you'll move on the next day. You know, uh, Jeremy did that for Anki, Python influencing of Anki yesterday, and now it's done. And, you know, not to say that there's not more to be done there, but he might say that's, that's as much as I want to do. Or he might be like, wait a minute, this is a way to create a whole new kind of space repetition company, and I'm going to go, like, you never know how far you can go with anything that you find. And the more you practice solving problems, the more you can fix one of the biggest psychological problems that we have that prevents us from finding breakthroughs, which is that we fall in love with our own creations. So if you've only ever solved one problem in your life, you'll tend to be like, this is my one chance to make a company out of this one thing that I finally had an insight into my. But like, if you do it all the time, you start to get develop taste for which things are big and which things are small, which things are important and which things are not, and which things really move you in an inner way that makes you say, I want to dedicate years of my life to this, and which ones you're like, okay, yeah, that was fun, good, problem solved.

I mean, it's the, it's the, it's the PhD's curse, right? Most PhD students I know who go on to want to build startups basically try to commercialize their thesis. Have you seen that too?

Yeah, of course. Well, because what, what else, what else is it going to be? And I think what is hard to explain, and it's just something you have to experience, is a startup is, is in some ways a reflection of your whole life. You put everything into it. And, um, so yeah. So if you're like, I need to commercialize my thesis. Well, that, that presumes that every part of your thesis is equally useful to the commercializing solving of the problem. But often times it'll be just be one element of it, or there'll be some thing that you learned along the way that is actually the really key insight, or the really key thing that allows you to create something exceptional. So the key is to really be very open-minded and to use, you know, these sort of techniques like minimum viable product, pivots, experimentation, like all these techniques are there to help you feel your way into, um, the right solution, the right thing for you to work on, um, by helping prevent some of these psychological bugs from getting in your way.

And Jonno, you don't have to start your own thing. Like, something very interesting about you is that you are employee number one at Eric and my startup, and that's an extraordinary thing to do to be like, no one else has made the decision [laughter] to jump on what these guys are doing, uh, but I'm going to dedicate some part of my life to doing it. Like, that's a very kind of founder-like decision of its own. I'd be interested to kind of hear, you know, your thoughts about how, either in general, how to do that, or why you did it in that particular case. Like, what, what's the thing running in your head to be like,

Okay, I will, I don't need to, you know, I wait until 50 people have started doing this thing to decide that I want to do it, too.

Yeah. I mean, as a general rule, I tend to, the more people doing something, the less interested in it I am. So that's just like a personality trait right off the the bat. But I also, for me, there's a certain, um, mindset and set of like mental skills and ideas that one needs, I think, to be a founder. And I don't know that I have those, at least not developed enough that I want to start my own company. It's kind of against a lot of the parts of my personality that I'm aware of. Um, so I've been talking about start, maybe starting an AI company with a friend. We had some ideas that were possibly going to get off the ground, and we'd been looking at it. And the more I looked at it, the more I felt like I'd almost rather be employee number one, where I can still steer where things go, and have that vision, but I'm less responsible for everyone else joining. Um, and maybe that's just because I'm still relatively early in career. I don't have the confidence to say, "Hey, join my thing. It's going to be good." Um, but I do have the confidence to say, "Well, I can contribute to something that I hope is going to be good." Like Jeremy's got this thing. We've been talking about this idea. Um, I think I can help try and make it good, but I don't know that I could necessarily, like, convince other people to jump from their comfortable jobs to come join me. I don't have the confidence there, or necessarily want that responsibility on my shoulders. Um, so, yeah, for me, the, like, the choice was very obvious as soon as I just finished writing out a document. This is, here's why I'm not planning on taking on any full-time work. Like, I literally just finished the, like, the Markdown draft one to go on my personal site. Um, and then we'd been talking for a while. Oh, wouldn't it be great if we could do something? And so there was always this little mental asterisk. It was like, well, unless Jeremy decides to do that thing he's been talking about.

And that's interesting, right? Because I think that's, that's an important point for me too, is like, Jen, and for investors too, it's like,

But most things, it's not really about like, what's the specific idea,

but actually, it's who you're working with. So in this case, you were like, a definitely consider doing something that Jeremy's doing rather than, I would definitely consider doing something if it was in the space of X topic.

Right. Right. Yeah. I know. Exactly. If, if someone else had pitched similar ideas to Answer AI, I would almost certainly have not been interested. Um, it was only the alignment of worldviews and ideals and ways of working and everything that I already knew worked well. Um, yeah.

Um, yeah. In the, this is a, one of the ra, one of the rabbit holes in the book that you can easily skip over if you don't, if you don't pay attention. I have this concept I call the torchbearer,

which is like, there's these people in every organization that kind of like, they embody the company's ethos personally in a really profound way.

Yeah.

And they're not always the founder. They're often, um, you know, be it'll be a first employee, like Jonno, or it'll be a 10th employee. But if you talk to people who've built really world-scale companies, you know, have hundreds of thousands of employees, they'll often be like, it was the hundredth employee or the thousandth employee, or sometimes, you know, I, I know people who joined an organization, I was just talking to somebody who joined a very famous organization that's been around for 40, 50 years. They, they joined like five or 10 years ago, and now they're running a very important part of the organization, and they just, you know, when it's right, it's right. You, you find these, you find these connections. If you talk to founders who've built these big companies, they'll, they'll often remember people who joined it a lot of different times. Which is why again, that feeling that it's too late, you missed your chance, or whatever, is so misguided. Um, you could found a company, but you could go lend your gravitational weight to a company, or by taking a job there, by being an early adopter of their products as a customer, by in the networks you create,

you know, so much value.

Yeah. The and the networks you create are great, right? Like, if I look at, like Answer AI, I think four of the first five people were still here. The other one, Benjamin, went off to lead AI research at another friend's company, Tanish's. And then one of the next people to join was Vic, who then decided he wanted to start his own company, and we're now a customer of his company and recommending Data Lab to other people. And, you know, it all becomes this, uh, interweb, which is also where the, uh, kind of ethics and ethos comes in, right, Eric? Like, if

Yeah, exactly. If you know, we didn't end up operating as decent human beings and or they didn't, or whatever, none of that happens, right? You don't end up with this connection of people who want to keep doing stuff together. So, you know, ethos is more than just what makes a company successful. There's something kind of personal about it too, maybe.

Definitely. And and and it's, it's bigger than a company or a single organization. And it's, it's, you know, when people talk about something as a movement, you're like, what does that mean?

How can something be a movement? You know, that we have these, we have these kind of fuzzy words in social science about that. But that's really how I think about it. Like, that, yeah, when you have an idea that's really profound, it, it's something that can really take off, take, takes on a life of its own. It's actually its own living thing, and, and it can spin off companies and partnerships and friendships and all, all kinds of stuff and parties and conferences and you name it. So if you think about any community you've been part of, you know, an open source community, a physical community, a, a corporate community, an organization, you know, in the book, we talk a lot about just apply these ideas to an organization that you care about, whoever that is. You, people have a hard time seeing themselves in these roles. But if you look else, look outside, key, get your ego out of it, you'll see there's all these key people who play key moments, you know, like the person who ran the cafe that the famous revolutionaries would meet in, you know, also played a really important part. That, like the person who writes the documentation, you know, and popularized a certain open source project, or the person who just decided to use it for a famous project. There's so many roles and opportunities for someone to make a really big difference. And so, um, yeah, restricting yourself ahead of time to say, well, I only want to do it if I can be the, this or that, um, you miss out on a lot of opportunities, um, uh, for creating a lot of value and having a lot of personal satisfaction. But also, if you say, well, because I, if I can't be the leader, I don't want to do it at all. A lot of those people wind up miserable in crap jobs. So nobody watching this should be in a job they hate.

No.

Okay. You now have like an, an incredibly lucrative skill set. Please, if you know, at least go make a lot of money with it. But maybe even better yet, use it to, to see your personal values amplified.

Eric, somehow almost everybody who's been in a role for a while, uh, deeply misunderstands how easy it will be for them to get another really great role, and that the next role will probably be better than the last one. Like, have you noticed this? Like, somehow companies have this psychological ability to cause people to feel like they're lucky to be working there. They'll never get another job as good as this again. But like, it's the opposite. I feel like most people I know who are in some kind of environment that they don't love, and then they leave, they do end up somewhere way better.

Oh yeah. Yeah. And and I mean, listen, I want to be super clear. We're talking about extremely privileged people with, like I said, a very lucrative skill set. That's obviously not generally true in the world. A lot of people out there are genuinely struggling, finding how, I find it hard to find a job and all that. Okay, that's true. But that's,

But I think our audience here is,

Yeah. That's not who, that's not who we're talking about. And in fact, I, I write about this thought experiment. If you get to the end of the book, you'll, you'll see I, I write a thought experiment that for people who, who are in this privilege position, we should start to think about, um, what is an acceptable way to make money. And if you're creating more value than you capture, then by definition, any dollar you make is making the world a better place more than that dollar you are receiving for yourself. So in that case, it's like not only ethical, but I would say even required that you go try to make a lot of money because then you're, you're actually fulfilling the commandment to make the world a better place, to leave it better than you found you. But if not, then it's actually really, like, I think very morally questionable to be on that on that treadmill. And therefore, like, people say, like, okay, therefore I have to quit my job and go join a monastery. There's a lot of other things in between joining a monastery, taking a vow of poverty, and, you know, creating a brand, creating a brand new company that is truly, truly, truly profoundly, um, uh, value creating. There's a lot of things in between that that you could do. And so I think one of the things that's really important, you have a skill like this, is I'll just, I'll tell you a funny story. Like, often times people require the world to intervene to find out how much better things could be. And I once worked at a company with a really talented technical team that, uh, made a bunch of startup, startup scale mistakes. They weren't technology mistakes, tech, like startup mistakes, business mistakes. And as a result, had went through a successive round of layoffs. Almost everybody who was laid off by this company went to go work at pre-IPO Google, which means that the earlier you were laid off, the more money you made in the end. So it was like a very funny situation. And the people who didn't, who never got laid off, missed their chance to go work at Google. Um, so that's like an extreme example, but it's pretty common. Like, sometimes, you know, you just, you need something to jolt you out of your, um, out of your feeling of complacency, or out of, um, that feeling that Jeremy was saying, where like, oh, I, I, the, the devil I know, right? The certainty of what I have, it seems so risky to give it up. But, um, you know,

And also with the Christmas period coming up, it's a good time to reflect, right? You know, we around,

Reflection. Yeah.

Am I doing what I want to do? You know, and it, and,

It feels like an important time to be thinking about whether you're doing what you're doing. Like, it feels like there's more opportunities in the world than there was two years ago.

Is that, do you think that's true, Eric? Is there,

Oh, the best time for, for from an entrepreneurship point of view, the best time ever.

Why is that? And well, so entrepreneurship is a function of, um, stasis or change in the world, and then like capabilities or stasis in terms of individual power. So whenever we see a quantum leap in people's individual agency, that's really exciting. But we also see the same kind of opportunities when we see like a, a paradigm shift or some kind of collapse of the dominant order. And I don't mean to talk about the geopolitical crisis that we're all living through in positive terms. I know that feels very weird, but one of the side effects of all that turmoil is most of the institutions that govern our civic life are going to have to be rebuilt because they're, they're failing in a catastrophic way. And that means companies and governments and agencies and all kinds of like things are going to become thinkable that were previously unthinkable.

And with AI, there are things we can now do that before, or at a different economic level. Bo, both like, so, so like, what is the, you know, think about the, the capability of an individual person armed with a tool like Solve It compared to someone who's, uh, you know, armed with a slide rule. It doesn't mean that you're smarter, it's just you have better tools, so you can do more things, so you're capable of doing more.

Actually, can I ask about that? Because we've got a question here from from Naveen Judah, who said, what, what, what's the difference? Like, what are the thoughts on starting a one to two-person lifestyle business versus creating a big company? And a couple of other people said, "Yes, please, please talk about that." Because,

Yeah. Yeah. Yeah.

It's like, or, or a slight wrinkle on that is a one to two-person lifestyle business versus a one to two-person business that you're thinking maybe could become huge without growing big in headcount versus the traditional VC hyperscaling. Most, most of the stories you hear from VCs are all about how to create an Uber, drive everybody hard, go big or go home. Answer is trying to do something quite different, which is we want to create a multi-billion dollar business with 12 people. But then there's also the one to two-person business that ticks along very nicely, thank you very much. These all feel like things that particularly the latter two feel like things that are maybe much more available now, maybe than they were,

Certainly. Yeah.

before.

If, um, I'll give you, I'll give you my true, true answer to this question. It's a question that Answer I've given a lot of times, and everyone always finds it unsatisfactory.

Oh, great.

You probably will too. [laughter] Yeah. Nobody, nobody likes this answer, which is like, the truth is fundraising strategy should be an output, like a downstream effect of the business strategy, which is itself a downstream effect of the vision of what the person is trying to accomplish. So already, as soon as you say, I want to create a venture-back company, or I want to create a lifestyle business, we've already made a mistake. How can you know if it's a lifestyle business or a venture-back business until you know how big the opportunity is? Well, how can you know how big the opportunity is until you've actually given it a try? So, it's one of these funny things where I actually, if you, people tell me, I want to start a lifestyle business. I start a venture. But like, start wherever I, it makes no difference to me. The first step is the same, no matter what you have in mind, which is like, let's go figure out if there is an opportunity here. And then you may find, like, so many lifestyle businesses wind up being venture-backed, and so many venture-backed companies wind up being lifestyle. It turns out should have been a lifestyle business. And so, like, the idea that it's like pejorative or bad, that's only a lifestyle business. I'm always like, you want to have a good lifestyle. Nobody, the VCs who denigrate this thing, they have a pretty sweet lifestyle business themselves.

So I don't think they should be talking smack about anybody else's lifestyle.

But also, it's not just about the size of the opportunity anymore, right? Like, you can create a business that has huge opportunity and has huge results without necessarily being venture-back or without necessarily scaling,

headcount.

And it, and it also depends on how you go about it. So that's the other crazy thing is like, even two different people pursuing the same opportunity might pursue that in very different ways. And so, yeah, like, I wouldn't get into the question of raising venture capital until you have a specific situation. Here's the, like, the thing you're looking for, if you're like, you want to know, okay, do I have to raise venture capital? Um, if there is an irreducible obstacle that you cannot solve except with money, then of course you need money. But that's pretty rare, actually. Like, if you're like, I want to create a semiconductor fab, or something. Okay, that happens sometimes. You know, we need years of regulatory approval for something. Okay, there are those times. But if you're like, well, I just want to hire a bunch of people, that's, that's a mistake. That's not this category. Irreducible, right? So, it cannot be done any other way. Now, it used to be that if you wanted to write software, you had to have headcount for software. But obviously, with with LLMs, the, the economic calculations very has changed. Um, so that's one reason for venture capital. The only other reason really is if you can figure out a way to get the engine of growth of a business turning. We can talk about these terms if people aren't familiar with them, but the engine is turning. You have a product that is like gaining usage or making money in some kind of sustainable way, and there's an irreducible or artificial barrier that is preventing the loop, the engine from turning. You can raise money for that. So, you know, very famously, you know, be, think about the early days of Facebook. Once Facebook got the viral loop of Facebook going, um, they weren't making any money yet, but they were accumulating something of tremendous value, namely people's attention and engagement. So, the thing that they were making was a highly valuable, but illiquid asset. And they needed to buy servers. It wasn't even about headcount at that point. It was actually like, they needed to have buy the physical servers to keep the thing keeping up with demand. That was like a classic good use of venture capital because the engine would turn a lot faster if you don't have to wait to get the money to buy the servers. So there are these kind of critical moments. Anyway, I give you these examples not because we want to make this into a class on on venture capital, when to raise it, but just like notice how detailed that situation is, how t how it's like tied in with the like very specific details of Facebook and what they're dealing with. And of course, the thing that created the need for venture capital was that the product was really good and was growing really fast. There were lot, they had lots of competitors that didn't. It could easily have been lifestyle businesses because nobody liked the product. So it didn't have the need for rapid scaling. Therefore, it didn't have these other needs of,

My two companies were,

lifestyle businesses, and or at least not venture-backed businesses, Fastmail and Optimal Decisions. And in both of those cases, I think particularly in the case of Fastmail, I could have chosen to seek investors to scale it faster. But actually, both of those two companies were big enough and successful enough that I would never have to work again. Um, partly because I got, well, they were both with co-founders, so I got 50% of the upside from the sale. No dilution, no, like, lifestyle business can be a bit of a misnomer, you know, with, um, with two people.

I hate that term. I actually don't.

Um,

And I didn't want to have investors, and I was perfectly happy with how big Fastmail is, and Fastmail still exists today, and still employs people, and still has happy customers, and still has a role in the world. And that's fine. I don't need it to be as big as Gmail, or whatever. I don't need to have everybody in the world using it, you know.

Yeah.

Um, I've got a, I'll just share my screen here. You can see, uh, I've, I've got, uh, so this is Aditia's list of questions.

So good.

Um, and I thought the first three were kind of connected. Uh, I suspect I know how you're going to answer this, but it's like, how do I find a business idea worth building? Should I focus on solving my problems or other people's problems? How do I avoid committing to bad ideas? How long should I persevere for? Um, yeah, that's kind of a one big meta question, I guess. I, I feel,

Yeah. Yeah. Yeah. Yeah. You know, um, there's so many great essays on this on the topic of where you get the good idea from. Like Paul Graham has one. There's, there's, there's a whole bunch. So, so there's a lot you can read on this topic, but generally speaking, it comes from life experience. So, like, go have some experiences. Like, honestly, like, that it's that simple. Like, you, you have to be, you have to be on the lookout for something. But like,

Airbnb couldn't be Airbnb if those guys weren't going to conferences and hanging out on people's couches.

Yeah. Exactly. Exactly. Like, they have to have, you have to have some kind of pro. So, so that's that's the thing to look out for. Either you're really frustrated about something, you see something in the world. I like the question of like, why is it the way that it is? People are like, "Well, that's just how it is." Those are great, those are often great businesses where it's just like, that that doesn't make sense. Um, but it has to be something that you personally feel a level of passion about. It does not have to be yourself. You don't have to be the customer, although that's obviously very helpful. Um, if you have like a friend, um, or a co-worker or colleague that is suffering immensely, and you just think that's wrong, you're like, why, why should it be this way? You know, I, I, um, I once helped start a social media, um, anti-bullying company, just watching someone I really cared about get bullied on social media and be like, why? Like, this is 100% solvable. What are we doing? Um, and it was, it was really that simple. It was like that I was like, like I only had to see it one time. I obviously known it was happening, like I didn't know, but to see it viscerally, look at it, and be like, this is, these messages shouldn't be allowed. Like, I understand that the writing perfect filtering is hard, but I could filter these messages out for sure. If these platforms wanted to do this, they could. So it was like from that to starting a company to do it was like, took me only a couple days to be like, "Oh yeah, this is a good idea and I'm going to find somebody who wants to work on this." So I had that happen to me many, many, many times in my life. And so if you keep your eyes open and you go do interesting things, you'll, you'll find something, um, or, you know, and talk to people and go to conferences. And that's also why learning is so valuable because if you're learning, any anytime you're learning, you're potentially seeing a good idea that it's like, "Oh, I now know something that most people don't know." So the reason they do it this dumb way because they don't know what I know. So hey, what if we did it the way that we now know? And there's like the academic literature. Every form of knowledge gathering is just loaded with gold mines of stuff that the specialist person has figured out, but the rest of the world doesn't know. And the work of an entrepreneur in many cases is just to bring this idea into the mainstream, whatever the idea is. So yeah, go, go, go live your life and keep your eyes open.

And yeah, and as, as one lives life, yeah, you, you gain experiences and you also gain connections, you know, because Aditia is also asking about how do you grow teams and stuff like that. And actually,

Most early startup teams are bunches of people that know each other, you know, so like Answer AI,

they worked together on something before.

Jonno and I have taught together before, and we're part of the Fast AI community together. Um, Rens and Tommy are both from the Solver community. Um, Alexis is part of the Fast AI community, and we helped, he helped build the Fast AI library with me. Just, I think, I think probably everybody in our company is,

people we already knew.

Uh, Eric was an early NB dev enthusiast. Eric, not Eric Ries, Eric G. Um, head of Eric Grace and I know each other from lots of different things. So, yeah. So you can make more.

Yeah, go on.

I, yeah. I made a few suggestions in the book too, because this can sometimes sound like people are saying, like, go volunteer.

But I would say like, and that's of course a perfectly fine way to get experience, but like, go build something. If you have building skills, build something. Go build an open source. Yeah. Bu be in a community with other people. Building open source software has like so many people their entrepreneur journey started that way. Even, even like try to build an open source project and have people, realize how hard it is to get people to use it is like,

incredibly educational experience if you've never had that experience. It's humiliating in its own way, but, um, super, super helpful. Um, yeah, and like hang out. If there's someone who you really admire, go hang out in their community.

Yeah. Yeah. You don't have to create something new from scratch, right? Like,

it's, it's incredibly hard to overstate how much you stand out if you're in, in any, in any community. Like, say you're in the Hugging Face Discord for the Hugging Face library, right? Or you're like a user of some software tool like Weights and Biases, or some cloud service. If you are like, there participating, sharing a tutorial that you wrote, writing a blog post, answering people's questions, like, as soon as you do that, you raise your head above like 99.95% of,

Totally.

people there, and, and the people running that community notice. Um,

Well, like we are doing right now, like we're, we're showing Aditia's questions because in the community is somebody who's created, yeah,

he's created a project, you know, awesome project list, he's created a great list of startup questions, and if, I mean, he's never, I think he's never contacted directly, but if he ever did and was like, Jeremy, can you talk to me about something? I'm think I'd be like, absolutely. I'll do anything.

Of course. It's a, did you, exactly, just a random person. I'll tell this story, um, one of the first engineers I ever hired at one of my startups, I was using a certain open source project, and therefore I was on the mailing list for that project. And I noticed like a successful open source project often has somebody whose job it is to just whenever someone fires off some idea onto the mailing list, it's like, really stupid to, like, if it's a good community, if this project's going to work, there's somebody who like gently and calmly like helps that person understand why that's not the way it's done in this community. Here's how we do do it instead. Like, here's the technical reason, and helps them like redirect them into some constructive form of engagement. And there was just this person who, that's, they had no formal role in this project, they were just a user, you know, of this open source project, but they just, they knew it really well, and they would do this thing. And I eventually occurred to me, I was like, why should I hire that guy? He was living in Iowa at the time, you know, I was like, hi, you don't know me, but I'm from Silicon Valley, and I'd like to change your life, um, you know, are you, are you interested?

Like, there's so many ways to contribute.

Yeah. I mean, every one of these interactions matters. It reminds me just yesterday, actually, Tommy reached out to me. He's like, "Oh, this guy, the executive assistant reached out to our support. They wanted to know, can we still sign up and is it too late?" And we had a little bit of chat about, will it still suit them? And I was like, "Cool." It's like, who was it by, you know, anybody I've heard of, said the name. It's like, "Oh, dude." You know, that guy's a super famous venture capitalist. You know, it's like, it's nice because we had made the effort to make sure everything that goes to support, we answer it carefully. We have an internal culture where we try to care about if one of our customers. So Tommy specifically reached out to me to ask whether his answer is okay. And he didn't, [clears throat] know that he's actually helping the EA of a super famous venture capitalist. And, you know, but every one of these interactions, you're, you know, they all, they do all matter. Um,

Yeah.

I got to, I'll just jump on. We've actually been answering quite a few of these questions as we go here, but I just, uh, want to just skip over to these ones. Uh, sales, marketing, talking to customers. Um, yeah, you know, um, this is something you talk about a lot in The Lean Startup book, Eric.

Yeah. Yeah, go read The Lean Startup.

Talking to potential customers. Um, but it is a bit scary, right? Um,

It's scary. It's scary. There are, so this is a skill, again, people have to realize, like public speaking is a skill. Talking to customers, each of these things is a discrete skill. And you, and, and, um, for me, I always think about it like, think of something you find very scary, like downhill skiing, um, or like, if you ever watch the Olympics, one of my favorite funny things to watch Olympics is they do these, like, you know, human, you know, human interest interviews with people.

They always do one which is like, um, you know, what do you think of other sports? And you'll have the guy who does the like bobsled being like, the luge, now that's crazy. Those guys are doing something really dangerous and difficult. You know, you're like, okay, your thing seems pretty dangerous, difficult too. So like, you got to remember that if, you know, if you've ever overcome a fear in your life to develop some skill, um, this is like that. So, you know, if I said, if you said to me, I'm, I'm nervous, how do I learn how to do downhill skiing? You know, I'd be able, I could tell you, well, you, you're going to have to at some point, you're going to have to put on some skis and get on the mountain. You're going to have to do the thing that you fear. And, you know, you can do it with a teacher, you could do it with a coach, you do it on your own. But like, so talking to customers is something you, you have to, um, you have to learn how to do. And there's a, I can't remember who, who did this, who taught me this exercise. There's an exercise where you just, um, you practice before you even talk to customers or anything. You just practice getting turned down, like 10 times a day. And it's just in your daily life, you ask people for things that they're going to say no. And most people find this exceptionally difficult. Like, you go into Starbucks, it's like, can I have my coffee for free? No, that would be inconceivable to me to do.

This exercise is for that. But like, yet, but that. So you're like, "Okay, I can't do that." But can I be like, "Okay, you know?" And you might start with something, you know, the answer is no to. So you're like braced for it. Like, "Do you have, you know, macadamia nut milk for my coffee?" But anyway, so you start with like the smallest, trivial thing you can, and you get used to people saying no to you until it becomes not a big deal anymore emotionally. You're not scarred and traumatized for life. You've done it a million times. And then you can consider like going to a customer like, "Hey, would you really want to talk to me about my product?" knowing that if they say no, it will not be the end of your life. It will be one of 20 things you said no to you about that day. Anyway, there are exercises like that that can be very helpful.

Okay, I got some more here. Um, I'm actually going to be really interested to hear what you say about this 'cause I don't know your answer. A lot of these, I know what you're going to say. This one, I don't think I do.

Yeah.

Um, does my physical location matter? I'm guessing a ditch here by his name is in the aforementioned small city in India. Um, and then how do I sell when I'm geographically distant? Uh, so I'm mainly interested in the first one, I guess, but in as much as the second one on it.

Yeah, it does. It does significantly impact your chance of success, but I can't tell you in advance whether it will be positive or negative until [clears throat] you tell me, until you tell me what it is you want to do. Yes. So, so here's the the general trade-off. So, for startups, it's like, imagine you told me you want to be an actor, and you know, you're like, "It would be better or worse for me to be in Hollywood." I had to be like, "Well, first of all, do you want to be a blockbuster movie actor, or you want to be at your local community theater?" You know, like, if you go to, if you go to Hollywood to be an actor, the plus is you're surrounded by people that make movies. The con is you're surrounded by other actors. Every actor in the world who wants to be a Hollywood wants to be a Hollywood actor goes to Hollywood. So startups are like that, too. Like the bigger cities, the bigger startup hubs like Silicon Valley, they're like the Hollywood of startups. So if you want to come to Silicon Valley, of course, there's a reason.

How long have you been in the Bay Area for? Are you?

Me personally?

Yeah.

I came out in 2001, I think. So 25 years coming out.

Where were you before that?

Um, I was in school. I, I was, this was the first place I moved out of out of school. Um.

Right.

So after I got my degree, I came, I came out for my first like, real proper job, um, to Silicon Valley.

You actually had your whole working life there already.

Yeah. My whole, my whole working career was here. And I, but I had done a startup in, um, in college. I had been an intern at Microsoft. Like, I had done a bunch of stuff that made me, um, stand out, just like a little bit, enough to get getting jobs at startups in Silicon Valley. And if someone wants to do that, I mean, that's been an incredible career for me. I love being here. I think, you know, Silicon Valley is very different now than it was 25 years ago, but still, it is a magical place. And part of the reason I wanted to come here is not just, you know, all these iconic tech companies were here, but and I didn't even know this at the time, but I had done a failed startup in college during the dot-com bubble. And I was like, in certain cities when I would go interview for jobs, I was a pariah. I never forget this. Um.

Because you were a failure.

Because I was a failure. M.

Um, uh, I remember going to, to, um, to an interview in Boston and sitting down with a company and they were like, "Why is this, why did this, why do you have this failure on your resume?"

And I was like, "Oh, 'cause I did this startup and it didn't work out." And they were like, "Okay, well, at the level of strategy, what did you do wrong?" And I was like, "Uh, we built this product and nobody wanted it." And they're like, "That is not strategy. What at the level of strategy did you do wrong?" And I was like, I realized having this interview like, I had no idea what the word strategy meant.

So I could not answer this question. That was like, good sign that maybe I wasn't qualified to run a startup. But anyway, but I had done it anyway. It's just like, it was humili-, it was humiliating and like, and when I look back on it now, I'm like, "What the hell is this? This guy's never done a startup. What does he know?" You know, whatever. But like, it was like seen as this big like question.

I think there's lots of successful startup founders that would not know the word strategy. I'm not sure it shows that. I, I know tons. I know tons of them who couldn't, who still now having made billions of dollars, still couldn't answer that question.

Um, when I came to Silicon Valley, I would do these interviews and people were like, "Now, tell me about this startup failure."

Mhm.

"You know, what was it like?"

And I was like, "It was humiliating and terrible. We screwed everything up." And they're like, "Great. Like, great. Like, yes, that means that whatever you learned, someone else paid for you to learn those mistakes. Someone else paid for your education. We don't have to pay for you to make those mistakes. You've already made them." They didn't have to. They didn't even ask me what the, what the lessons were, because a big part of this is understanding that failure is its own teacher, and you often learn things you can't put into words. So anyway, there are really wonderful things about it, but there's also problems. We have these manias and bubbles and and fads and stuff, and it's, it's vicious and cutthroat. So.

It's not always necessarily best for every kind of.

I want this ties in.

Oh, go ahead, please.

I was say, I want to hear Jono's view as well, but I also want to put mine in because we all have such different backgrounds, right? So, uh, my background is, I did not find my way to the US and San Francisco until, was it, 40 something? Um, and my, my perspective, if somebody in Australia says to me, "Should I go to San Francisco?" I say, "Yes." I just say, "Yes, definitely." Because from, just from my personal background, growing up in Melbourne, the perspective I got was just so different. It was like, "Jeremy is weird. He's interested in weird things. He's kind of a lunatic." Like, and, and I had to hide everything about my interests, you know, because I was otherwise incurably weird. Um, and kind of a wanker, and kind of like somebody who thinks he could do things better than other people. And it colored my whole experience of schooling, the first 20 years of my career until I went to San Francisco and suddenly I had the same experience, but in a different way, which is like, "What are you trying to do, Jeremy?" Like, "Well, what to like, make this thing that's better than other people have made?" It's like, "Waiting for the kind of like, 'Oh, who do you think you are? Didn't Microsoft already try?'" But no, it was like, "Wow, how interesting. Tell me about that. That's great. I know four people who are kind of in that field. Do you want introductions to them?" Yeah. It's like, just, oh my god. And so that's that's kind of A. And then piece B is like, "Oh my god, there's all these other people here. Peter Norvig, Hal Varian, Drew Houston, like giant names. They're so much better than me at everything. God, it's going to be so intimidating if I ever meet." And then it's like, I meet all of these people and discover like, "Oh, they're all nice, thoughtful, perfectly reasonable people, but they're not gods, you know?" And like, not only is Peter Norvig not a god, but he was actually genuinely what I had to think about things. It's like, "Why would that be?" It's like, "Oh, 'cause it turns out people who get good at stuff actually care about what other people think." And we had a nice long chat and he asked me my comments about things. And, um, you know, I'll never forget, like, I was in Boston and Michael Stonebraker, you know, the database guru, reached out and was like, "Would you like to have dinner?" I was like, "Sure." And I was waiting all dinner to find out why he wanted to have dinner. He kept asking me feedback about his new startup and his new business. And like, after a while, it's like, "Michael, this is all really interesting, but what, what did you want? You know, why did you want to have dinner?" Like, "I, I just, I just genuinely want to hear your thoughts about my business idea." It's like [laughter] "But you're Michael Stonebraker." It's like, "Well, yeah, but I'm like, I like talking about my businesses with people I think are interesting, and I wanted to hear your feedback." And yeah, it's just such a different perspective from anything I got in Melbourne, you know. Um, it's just such a different atmosphere and one that filled me with much more confidence that now I don't need to be in San Francisco anymore.

I don't know.

It's really the perspective is what's is what's valued because like you're in Zimbabwe and like I feel like somehow you blazed your way in really interesting ways without, like, I mean, yes, you're in the US now, but it doesn't seem to have changed you in the way it changed me. I don't know if you feel differently.

Yeah. It was, it was definitely interesting. I, I feel like there were definitely hurdles where if if you said, "Look, I'm interested in starting a business on the internet," then it makes very little sense if you've got any choice in the matter to be in a place where it's so hard to transact on the internet.

Yeah.

Right. When, when, originally the electricity goes out, has the little star that says, "Look, we, we ship worldwide," asterisk, "except, you know, Yemen, North Korea," um.

I, I got so tired of like missing those asterisks. So there's, there's certain hurdles there. There are also like, I knew, I knew people. I know people who find that you can be the best in that small pond, right? A lot more easily, potentially, than being the best in the world. Um, and so for a lot of local businesses, a lot of physical products, that kind of thing, um, it's a very successful model. That's just, they could probably not compete in the US on the grand stage against the big retailers, but nobody else in Zimbabwe imports those products, right? Or does that thing or makes that piece. Um, and also everyone around you has got the same like limitations and hurdles. And so within that completely different environment, you can still like find interesting things to do and find interesting workarounds and work with everyone else who's also finding the interesting workarounds. Um, yeah, so for a lot of things, it feels like moving to the States has opened way more doors. It's way easier to incorporate and do all these things, but I didn't feel like you couldn't do interesting things anywhere else, you know, that it was a requirement. It's, it just makes it easier to do the sort of more established path.

And perhaps also a difference to me, 30 years ago is now we have the solver community, the fast AI community. We have Discord. We like, you can engage. Like, all I had back then was Usenet, and that was pretty limited. But now you really can become a deep, like Rens and Tommy are both at Answer AI because they became a great part of the solver community, you know? So you can kind of do it online, I guess.

Yeah, that's a good point. I never, I never had people around me that were interested in technology or computers other than like one or two friends who like taught each other. Um, but I grew up on IRC channels with the Linux nerds, you know, and, um, and yeah, these like, later on, different like forums and communities. Um, I, we had this thing called Zindi, which is a lot of like machine learning people getting together.

Kind of the African, right?

It was. Yeah. Um, but it meant that even though like Zimbabwe had very little data science, but I knew, you know, the handful of people who were there, and then I also had a much larger community of people in like South Africa and Nigeria and abroad. Um, so I had my, my people there, you know, sharing. Lastly, I started notebooks and, um, tips for for deep learning. So, yeah, that, that online community aspect meant that the physical approval rating didn't even come into the picture, really.

One that also, I'm interested in your answer about Eric. So I'm not sure I know it. Um, we had a bit of a discussion of this about online as I was reading your book, actually. So there's this question I did here: "How do I stay focused on building instead of becoming a glorified manager?" And then so one of my bits of kind of questions about your book was like, uh, I don't know, Eric doesn't seem to be talking yet about technical versus non-technical founders, about when you become a founder, like, um, should you kind of keep coding? You know, if you're there because you're a domain expert. I've talked to Chris Lattner about this a lot as well. He's still the biggest contributor to their GitHub repos whilst being a CEO. Uh, I know, um, JJ Allaire at, um, what was RStudio, now Posit, you know, is always the guy out there on the GitHub answering issues, fixing my bugs that I complain about in Quarto. Yeah, what, you know, like most people, I guess, who start a company as a domain expert end up not actually writing their code. That certainly happened to me at Analytic.

Yeah.

Because I spent all my time with like building up healthcare company partnerships, talking to investors, stuff like that. But yeah, I'm curious about your thoughts on what founders should do.

There's a lot of ways it can work. Actually, I don't, I don't think there's like a one best thing. Now, I, I challenge the premise of this question because builder versus glorified manager is pejorative, and you know, all of us would be dead without the millions of managers that work extremely hard every day to do a good job at what they do. So management is not something to be denigrated. And in fact, the challenge is that, um, I, for, I forget what book this is from. This is a framework I always found very helpful, which is that rather than seeing the job of the CEO as an individual job, it's better to see the office of the chief executive as a corporate function that has the responsibility for all the top-level decision-making of a company. So it has responsibility for the highest-level architectural decisions. You know, think about like, you know, when Apple decided to go, you know, get off PowerPC and go to x86, right? Like, yes, that's a tech, purely technical decision, but at the end of the day, somebody's got to make the call. Yes, we're going to bet the company on this architecture versus that architecture. It is the place where the most important product design decisions have to get made. Um, at the end of the day, like, are we releasing this product or not? Is ultimately got to be the chief executive function's decision. But it also is where the organizational design questions are made. Partnerships, promotions, what kind of org? Be big, small, flat, you know, whatever. So anyway, so you see that as as the, as a, um, responsibility. Now, when a startup is at the very beginning, that's easy. There's not that much to do. So, one person can easily do it all. But as the organization grows, the size of this responsibility grows. And so, some roles have to be shed. Think of it like it's a split, like mitosis. Split this office up into multiple roles. And so, it's actually not a problem if you want to be a technical CEO and continue to code and be the number one contributor. You just have to understand that that means the other aspects of this job have to be delegated, and you have to build an infrastructure and a corporate design that is compatible with your desire. Now, it's not always possible because there's some businesses where the people you're doing, for example, let's say you're doing business development partnerships, and the people doing the partnerships with need to do the partnership with the CEO, and it takes months to do these partnerships. Well, then if you want to be the CEO, and you have to, if those things are all true, then you must spend your time on the road doing this business dev thing, and therefore technical stuff is going to have to be delegated to somebody else. But that's very rare. Most companies, you have a lot of discretion about how this is, um, how this is designed. The reason why people mess this up in my experience is they just don't acknowledge all the parts of the job as real. So you're like, "The organizational design stuff is just some HR. You just like, whatever, it'll take care of itself." Maybe I'll delegate it to head of HR, but I know a lot of people are like, "We don't need a head of HR, which is everyone just do a good job." Or like, "Yeah, we're in a very personnel-intensive business." Like Jeremy, you've made the, I think, very intelligent decision with Answer AI to solve many of these problems by simply keeping the headcount small and accept the trade-offs that come with that. Um, that's also like, it's, it's a coherent, it's a rational approach. Other businesses that wouldn't be positive. If you're doing a business where, like, the provision of labor is the business, and you have to have a huge headcount, well, then the organizational design, HR stuff is going to be a huge part of your job, and you have to hire someone really strong to do that if you want to delegate it. So I think the most important thing is to treat each corporate function with the reverence and respect that it deserves, and understand all of them enough that you can then delegate effectively the parts that you want somebody else to be the one who does it.

Thank you, Eric. Yeah. Um, I think the last question is kind of the most interesting one in some ways.

Yeah. Just like, how does, does AI change everything? You know, or, or, you know, can it, um, like, do you feel like your answers, or or people other people's answers can be different because we have AI? I mean, clearly the fact that Answer is doing what we're doing means to some degree your answer is yes, it can be different because we're everything differently.

Um, you know, it reminds me, it's funny, this happened to me just today. Um, for, for the longest time, they used to be much more common. It made me laugh today because it was, um, it hasn't happened to me in a little while. There was a really long time when people would just pitch you everything using the word blockchain.

Like, "We could solve that problem with blockchain."

And a huge percentage of the time, the thing that they're describing is just a database. And you're like, "Yes, we could solve that with blockchain, but we could solve it today. Like, we can solve that problem right now with a database. We don't really need like the decentralization, the tokenization, like the other attributes of blockchain are not really essential to this thing."

Um, and that, you know, I think there's some of the things that we're doing in Sri that are that are non-conventional, that that's true for, that we probably could have done this before, but there's some things I think were no, it wouldn't, it would not have worked. So it's really, it's difficult to tease out. You know, I, I feel like the right answer to this question is like, well, if AGI, then just ask the AGI the answer to all these questions. And if they turn out to be different than mine, then it turns out to have been different or not.

But, um, if we have to reason.

There's no point assuming AGI later, let's think.

Well, exactly. I'm just like, that's what's so funny about so much of the AI discourse. It's like, "Well, if it's so transformational that we can't even think for ourselves anymore, then what are we doing? Why are we even talking about it?"

Yeah. So assuming that we're in, you know, one of these, uh, variations of the world where AI is both a transformational technology and a massive financial bubble at the same time.

Um, it's really difficult. Like, this is actually a very hard question to answer honestly because we have to sort through a lot of second and third order effects. For example, right now we have easy access to lots of intelligence on demand, super cheap. The cheapness is being heavily subsidized by billions of dollars and losses at the foundation, you know, all these foundational labs.

So, is that going to continue forever?

Probably not. So like, we don't even. So a simple question like, well, what is the actual cost of this technology to use? The variable cost of it is unknowable, both because we don't know how the scaling laws are going to continue on where we are on the S-curve, and also because we don't really understand these financial arrangements and how sustainable they are. So it's, it's pretty hard to say. But I, if I had to answer, like, give you one answer, it would be to look to the second and third order effects. So for example, take the question of whether you need to be in San Francisco or not, you know, directly AI doesn't have that much effect on it, but it has some direct effects. Like a lot of the secret knowledge of Silicon Valley can now be learned from Solvers. You could just ask it, "How would someone in Silicon Valley think about this differently than someone who lives in my rural village?" And I bet you would do a pretty good job being like, "Oh, well, probably they think about this way." And it wouldn't be wrong. So there's a certain amount of knowledge diffusion. But the bigger question is like, will our current economic arrangements where, um, there's tremendous returns to density for industrial specialization by region or so-called regional or industrial clusters, is that still going to be true? Depends a lot on, well, well, how, how is this going to change how knowledge diffuses, how companies are built, how big companies are, how education works? Like, there's all these second and third order effects that are going to be really transformational, and probably will look back and say, "Wow, it did completely upend the answers to all these questions." But it won't be because AI specifically makes it easier to do the thing we're talking about. So, like, for example, will AI make it easier for venture capitalists to allocate money more efficiently? Maybe. But we've already had the technology to do that more efficiently for a long time, and they don't use it. So maybe not. But will AI allow so many people to create solo companies that, you know, need a whole different kind of funding mechanism than venture capital can provide? Probably. So probably a huge business opportunity there. So yeah, maybe because of the third order effect, the monopoly of venture capital on the funding of startups is broken, and that distributes opportunity much more widely. Like, I would be looking for those kinds of effects more than trying to literally say, "Okay, this specific thing, you know, does AI make it easier or harder?"

Yeah. And like, on the, some things won't change. Like,

Something that's hard to explain to people is the degree to which everybody in San Francisco knows each other, and the degree to which that matters. You know, like, that's just something that's just hard to break into because, you know, humans being humans, most humans treat other humans that they have personally interacted with extremely differently to then those that they don't. You know? So, like, a lot of people I saw this week on Twitter were like surprised to hear that like Sridhar Douglas and Dylan Patel and Leo Ashbrer and, um, that other podcast guy, I don't know his name, like are roommates. I was like, "Of course they are. You, that's how good you know, like, that's how San Francisco works. Everybody knows each other. They see each other every day or two." Uh, same with VCs. Like, literally Sand Hill Road. Everybody knows that name because that's literally where the VCs all are. There is a [snorts] hotel at one end of Sand Hill Road. What was it called? Rose. Rosewood.

Rosewood.

Where they all go on a Friday night and they all have drinks and they all know each other. And, yeah, it does make a difference. Like, it's, in the end, it's, uh, there are some things that AI can't stop, which is like the fact that human relationships are human relationships.

Um, um, I wanted to, yeah. So also, I wanted to ask a specific one about this, and this one's from, um, Tommy from our team, Eric.

Great.

Where, where do you think value will be created in this post-AI world? Or, you know, how will that change? I think, I think there's always going to be returns to creativity. Like, human creativity is like the still like the most valuable ingredient in all this stuff. So like, I, I even noticed that, you know, in the lesson we were showing earlier, like, even when vibe coding, or even trying to get the AI to do something for you or analyze something, if you give it even the tiniest, slightest little hint of what you're thinking, it's amazing how much more intelligent it gets. And it's like, I don't know, I don't, I don't even know how to quantify this exactly, but there's something about like the possibility space that it's navigating. You know, it's just, there's, there's so many like seemingly equivalent branches that it can go down, and somehow being given like some slight hint of what the answer could be from an actual person seems to give it, um, tremendous, um, tremendous intelligence. So, yeah, so I think writing, coding, creating, building, art, think like anything that requires actual flow state thinking by a human being, um, is going to be, is going to be rare. And I think it's going to, the scarcity of that, as as AI generates slop becomes pervasive, the the premium that you can command for doing actual some original thing is going to be, um, is going to be really high. Um, so like that's kind of like my, my general go-to answer. Um, and then the other thing I think people are misunderstanding about AI is people are like, "Oh, AI is a massive productivity enhancer," which is true, but that doesn't mean that in aggregate it will create more human productivity. And I've just been using this example a lot, is like, everyone, pretty much at this point, who's done any kind of pitching or cold emailing or or, you know, trying to convince their boss of anything, everyone's had this experience of like taking three bullet points into ChatGPT and having it produce a beautiful artifact, which you then send off to the client based on the three bullet points. And most people, not as many people, but most people have had the experience or at least can imagine the experience of taking a proposal you got from the outside and asking ChatGPT to help you render it down to the three key bullet points you need to know. So like, what's happening is like, three bullet, if you, if you connect these two things together, three bullet points are being rendered as a 40-page beautifully crafted document with images and formatting and all this other stuff, which is being like rendered back down into three bullet points. Like, think how many millions of tokens of, you know, reasoning models of these massive, how much electricity and water and everything is being spent to turn three bullet points back into the same three bullet points. Like, it's insanely wasteful. But most business processes, that's how it is. So the technology, the fact that technology makes, the productivity was higher there, necessarily, because the productivity of each of those ends was higher, but the productivity of the process is not.

Yeah. Yeah. Probably, probably was, and it was probably far more expensive, you know, um, economically speaking, than it was before. So, we actually made the situation in some ways worse. So, therefore, I think that's really important because we have to look for places where AI can, um, not just make a faster horse, but to create the automobile, right? Like, try to find some kind of breakthrough that actually leaves people better off. And this is a little bit the esoteric interpretation of the book that I wrote, which is like, very much about trying to use, you know, use organizational technologies to maximize human flourishing.

Yes. Um, and I wrote the book as if, you know, any value system is kind of equally good, and you can be, you can have a good ethos or a bad ethos, or a coherent or like, just like, you know, because I want everyone who reads the book to be able to say, like, "Oh, I can apply my own," um, but of course, my actual belief is that there are value systems that are better than others. And that the real true power of this system only works, it actually, like, it is not correct that you can have an evil ethos. I don't think it's possible. I think the thing collapses because, um, unless you have that deeper connection to like the hearts of the people that you touch, it doesn't, it doesn't cohere. It's much more, it's much more difficult to, uh, to make it work. Now, you can still make a lot of money that way, but, um, you can't get these kind of, kind of superpowers. And the same thing I think is true here. Like, if you really have the commitment that I'm going to use this technology to leave actual human beings better off as a result, I think that is a really useful guide to things that are going to be value-creating in a way that, um, still allows us to retain an economy that exists to serve people rather than people serve it. And that, to me, is the crux issue that we're grappling with as our society is like, what is the purpose of all this stuff?

I suspect we're in a like a, um, somewhat temporary transformation bit right now where like, it's kind of what we discussed earlier. We keep, we we ourselves go through and we see others go through this change in relationship to AI. Like, we just mentioned Dorash, for example, who has this, uh, interview podcast series, and he was driving me a bit crazy like a year ago because he was bringing on every AI doomer and every AI hyper, and they're all like, "Oh, in six months time, there's going to be no room for humanity anymore, and then the world's going to end, and we're going to turn into nanobots." And he's, you know, it's like, "Oh, and then like yesterday, I saw his tweet earlier today, he's saying like, 'I don't know, guys. Every time I try to do AI for anything that actually matters to me, like find a list of people to bring to my podcast, or write a, you know, a letter to help me introduce myself to somebody, or like, every single thing in any way related to anything I do on the podcast, it's actually net negative. It's not helpful at all.'" [laughter] You know? Yeah. Uh, it, it's, um, and I saw Paul Graham like a few months ago saying like, "Oh, I just met a founder who said he's writing 10,000 lines of code a day with AI." And, yeah, I think people touch it, they get this like inflated sense of what it can do, and then they use it for a while, and then they start to become more grounded. And maybe in a year's time, we'll be much closer to more of people being at that grounded level and kind of like.

I think very optimistic to think it will take only one year to for that to happen.

Yeah, probably.

I love, I love your optimism. Yeah. Think about how long, I mean, just think about how long it took with the internet. Like, I, I remember walking around, you know, walking around campus with a cell phone in like 1997, and having people just be like, "What the f is that? Why would you, like, why would you even want that? What is the point of it?" And like, I remember being like, "And also, it can access the internet." They're like, "Why is that? Like, why? Oh, how on earth could that be important?" And so, you know, like, how many years between that and like, now everybody's got an iPhone. Everyone considers it as like, a basically a human right. So, so yeah, like, it took a little while, but, you know, people get there eventually.

Yeah. I had my, I had my Zaurus with a modem in my pocket in 1990. And then, yeah, then I had my Treo. What would that be? Like, mid-'90s. And I was just like, "Wow, it's so weird that not everybody uses this. It's really helpful." And then suddenly, once other people just started discovering it, they were like, "Oh my god, nothing else matters in the whole world." It's like, feels like exactly like happened with AI. It went from like, nobody cares to like, everybody cares way too much.

Yeah. It'll settle down into some kind of reasonable medium. But again, like, the people who have the superpower right now are the ones that are the lucky ones. You can, like, so many, the vast majority of AI companies that are being started right now, like, not only are not going to work out, they have no chance of working because they're just like, they're just the enthusiasm. They're like the overenthusiasm of someone who's just like, "Oh, this technology is good for everything, so it's going to be good for, therefore, it's good for this." It's like, "No, actually, like, think about how many like AI agent startups. Like, it just, the thing can't possibly work. There's no way to make it work. It doesn't, it's conceptually flawed from the start." So if you actually have a, a skill that is grounded in the technology and how it actually works and what it's actually good for, you have such an advantage over people who are just like flying into the hype cycle, you know, based on very little knowledge or information.

Eric, um, I'm just looking at the analytics and like, almost everybody from the peak still on after two hours and three quarters. But, um, nonetheless, despite the data demonstrating enthusiasm, I think we should try to wrap up. And I thought maybe we should wrap up with talking about this community. Um, I, I just know from hindsight and experience that after a course wraps up, um, most people leave, and a year later they're like, "Oh my god, this has been happening this whole time. I wasn't a part of it. I wish I'd known." Like, um, so I'd encourage people to like, not leave just because this is the last video for now. But, um, yeah, I was just wondering if you, you know, for both of you, Jono and Eric, who are both great at taking advantage of online communities, you know, what's your advice for, um, what to do with, you know, over the next couple of months? Um, how can they keep practicing? How can they take advantage of the community we have? I mean, I think we should keep, maybe we've got a bonus lessons channel, we could keep popping stuff there from time to time, but shouldn't just be about us, right? Like, what, what do you guys think, uh, would be good for folks to take away and keep doing over the coming months until the next course?

For me, I think the number one thing is to try and actually keep doing the things. And, and the things are probably different for everyone, but I would guess that like, if you've just watched through the videos to this point, or or you've been engaging, you've got between 10 and a thousand things, you know, like avenues that you could go, like, "Oh, I really want to build a blog like that one we saw at the start of this lesson." Or, "I really want to commit to actually doing all 12 Advent of Code this year rather than just the first four." Or, "I really want to go deep into web apps or AI LLM agent stuff." Or, "I've got this really great idea for." So whatever it is, you've got that card of ideas, it's very easy to go, "Oh, okay, but let me just take a break after the course."

Yeah, my life.

You know, yeah. And it's like, "Oh, I really, I did that solver thing, that was kind of fun." If you can like somehow trick your brain or or engage into like, "Let me just keep actually doing that." It's, it's one of those like compounding things of, "Oh, now I've been, now I've been doing my coding practice every day, and I've been getting better, and I've been learning more." Um, and it's two months later, and I'm sharing the stuff that I'm doing. The stuff that you'll be sharing and doing after two months of that continued work is going to be so far above the current position later, it's going to be higher, right? Cuz at some point life will intrude. That's fine. Just come back, you know?

Just come back. Exactly. Exactly.

Don't let the sunk cost.

Kill you.

Yeah. Don't let the sunk cost kill you. Don't let the fact that other people are way ahead kill you, right? Like, "Oh, look at these people. They've already done five projects, right? The course hasn't even finished yet, and like, they're sharing these amazing things." [laughter] Like, "No, no, no. Like, you just got to do your your."

Yeah. Sorry, Eric, to interrupt.

Oh, I was just going to say, one of the tips I have, I've noticed with online communities, especially that just, we talked about talking to customers as a skill. Um, being a member of the community is a skill, and most people are lurkers. They only consume and they don't post, and because they're nervous about posting. But communities that are all lurkers die. So if you're just lurking in a community, like you're actually not contributing to it in the way that you could. And the solution to this problem is just, and it just, given that the course is ending, just do this for a week, and you'll be shocked how different your relationship with the community is. And the, and the homework is simply just post something every day. You have to make a post in the, in a channel on the Discord. Just do it once, do it once a day for a week, and see how different life is. And you're like, "I don't have anything to say." That's okay. Think about how you feel when you, like, imagine you did some post something. "I have a new project out." Think how it feels if people reply to that message and say, "How cool," versus if they don't.

Or, "Hi guys, I've promised myself I'm going to post every day."

That would also.

"I don't really know what to post. What do you guys?"

Anyone just like, "Hey guys, did anyone see this link?" Or, "Is what you guys working on now?" But even just as simple as, "Oh yeah, I saw your demo. I think that's really cool. Keep at it." That is so additive to a community's life, you can't even imagine. And of course, once you do it a couple times, it starts to become more natural to do it. People start to know you, they know your name, they reply to you, you give you, it just like, it becomes a more natural thing. And next thing you know, you feel good about being on the community. And especially if you're going to waste your time doom scrolling on social media anyway, like, be be doom scrolling over here in the community where you actually like, where the quality of the content is going to be really high, and you have opportunities to really turn it into something useful instead of just doing the, you know, the typical thing. So I would be, yeah, that would be my be part of it.

This, this comment from from Kelly on the chat, um, I was going to read it. "I'm deeply afraid to lose my job. I seriously believe I could not get another high-paying job." Just wanted to kind of tackle this as we're finishing, because it's such a common thing for our brains to tell us. And like, my, my, my response to to you, Kelly, is, wow, how are you such a freaking genius at knowing exactly what's going to happen? How are you exactly so aware of your exact level of competence and position in your company that you can with such confidence tell us, "I could definitely not get like another high-paying job?" You must be amazing to be that clever to say that. Like, either that, in which case surely you could get another high-paying job because you're a genius at a level unknown before, or your brain is bullshitting you. You can't possibly know that, you know, like, like what's the data you have to know that? It's like, no, that's, you're almost certainly wrong. Your brain is trying to hurt you, you know, to tell you this for some reason. And I think you should not focus on like, "Wow, I guess I shouldn't try to get another job because I couldn't possibly get one, right?" But but why is your brain trying to hurt you like this? Because it, like all of our brains, does it to us all the damn time. Before every single lesson, my brain tells me, "This lesson's going to be, everybody's going to finally figure out that you're a total fraud," [laughter] "and that you've somehow sucked them in all this time." And I'm always shaking, and I always have to go to Rachel and say like, "Rachel, is this going to be okay?" And she's like, "Yes, Jeremy, it's going to be okay." Okay. But like, I don't know, like, what, what do you do, Eric, when your brain tells you you're going to fail? "Everybody's going to realize that you're, you know, that you're a fraud." Well, who are you to be writing a book, Eric? You think I never had that thought? You think I've never felt that way? Oh, man. I spent this, I'll just tell a funny story just to to admit to this. When I was running The Lean Startup, I spent just a crazy amount of time being absolutely convinced that somebody had already written the book and answered all the questions that I was going to answer, and therefore it was going to be a waste of time, and it was only my own ignorance that meant that I hadn't read such a book. And I spent months reading everything, I mean everything I could get my hands on, like obsessively being like, "Ah, finally." And I remember it's just my, this is how my brain works. One day I found out that the great Peter Drucker had written a book about entrepreneurship. It was now out of print because it's not very good. But I didn't know at the time. I was like, "Ah, it went out of print, but surely when I find, when I track down this out-of-print book, which I will do, and find a used copy of it somewhere, and I will be able to read the thoughts of the great Peter Drucker about it, then I'll finally realize that there's no need for me to write this book." And of course, I did in fact track it down. I did in fact read it. It is in fact not very good, because it turns out that entrepreneurship is a totally different field than his field. He didn't know anything about it. So he was, you know, didn't have that much to say. But like, until I actually got my hands on it, I wasn't convinced. And so to answer to your question, like, what do you do? There's only, there's only two different ways you can tackle this problem. You can address it practically, or you can address it spiritually. Practically, you could just be like, "Is it really true that you can't get another job?" That's actually really easy to find out. You know, even if you have a job, you're allowed to interview for other jobs. M. It's, in fact.

One person on the chat says, "I took a coding test with a recruiter. Turns out the last job kept me around 'cuz I actually knew what I was doing."

"Yeah."

"Just like, why do you have your current job? Is your boss so stupid they haven't realized that you suck and you shouldn't be there?" [laughter]

"Yes. And and we we're laughing about this because it's helpful to see this in ourselves, not as like a character defect or some kind of serious flaw, but just like a quirk of the of the."

"It's our psychological protection mechanism."

"Incredibly ridiculous."

"It's trying to it's trying to keep us safe, isn't it?"

"Yeah. It's it's doing your favorite favor. Now, so that's one way. So, you can try to tackle like, is it really true? And it's actually very easy to find this out. So, go find it out. Just go find it out. Um, it's not that hard to to find out. Um, but also of course the the other way you can tackle it is to be like, the fact that I have this voice in my head telling me things doesn't mean I have to do whatever the voice says."

"Yep."

"Because I am not the voice. So there is a whole different way you can tackle these kinds of feelings which is to to make yourself able to do the thing you want to do anyway in spite of the doubts or the feelings of being an impostor or whatever else. And these two techniques are not actually independent. Doing one will probably help you with the other. But it's just really important. Jeremy already already explained like you you can't see these judgments as true."

"Just because they're being whispered in your ear."

"No."

"And I actually I find LLMs have been really helpful for me in realizing this."

"Because like it just seems to me like the human brain like people always like these things aren't intelligent. They're just stochastic parrots unlike humans. And I'm like, I don't know. You don't seem to know the same set of humans I know. I feel like my brain is constantly just like whatever random things come into its head and as far as I can tell the most people I interact with, not most, many of the people I interact with on a daily basis just seems like they're just doing that probability generation thing and just letting it flow right out their mouth. So um the more you deep make it less precious like, oh, this eternal wisdom is being, it's just like, no, this is just a psychological defense mechanism. Often times it's just a knee-jerk reaction to to stimulus response and you you are something far more precious and far more amazing than just the blah blah blah that you hear in your."

"And also like huge thank you to Kelly right."

"For sharing a a a genuine."

"Totally. It takes a lot of courage."

"John Lee, you know, which actually tells me Kelly is, I don't know if Kelly is a boy Kelly name or a girl Kelly name, so I'll say they. I I think you know, the fact that Kelly was able to express this vulnerability shows me that Kelly is an incredibly strong person. So my guess would be they are."

"Highly highly employable. Yeah. Well well courageous as well as um as well as self-aware. So those are those are highly likely to be highly employable skills."

"Well Eric Jono, this has been a great pleasure. At one level I'm very sorry this is finishing. At another level I'm very glad I can."

"We have to we have to let people go."

"Yeah."

"Relax. Yeah. So."

"Yeah, I will. Um, there's been so I feel bad that there are just so very many questions on the Discord that we haven't gotten a chance to answer. Um, so I will try."

"We're going to do a bonus lesson with you, right?"

"Yeah, we'll do a bonus lesson. Yeah, we'll we'll stay tuned on the Discord for that. We will find a time to do a bonus lesson. I'm happy to answer some more questions and especially want to hear from people who have taken the time to do the um solve it powered reading uh of the new book. Really just delighted to hear your thoughts, questions, and spend some time on that. But if anyone who's interested, we will post where are we going to post information about that in."

"Uh, we'll post it um somewhere else."

"We'll probably post it in announcements. We'll certainly post in bonus lessons."

"So remember, you can right click on the bonus lessons channel and say notifications and choose to be notified of all messages there. That way you won't miss anything."

"Awesome. It's been a pleasure, gentlemen. Uh thanks for me to come on these last two lessons and and thanks everybody for for being part of the community. It's been awesome."

"Yeah."

"Bye-bye."

"Bye."