All right, we're back for part two of "Let's Build with Crew AI," and this time we have a very special guest, Joe, the founder and CEO of Crew AI. Welcome, Joe!

Hey there! Thank you so much for having me. I'm so excited to be here.

Yeah, so if you've watched my channel at all, you've seen Joe. He's been on a number of different videos and live streams. I'm an investor in Crew, and I absolutely love Crew AI. What do I always say? Bullish on agents! Crew is my favorite framework by far. We've been building something really cool, and in this video, Joe's going to help us extend it and build it out better than what I probably could by myself.

So first, let me just explain what we've built so far to you and to Joe as well, and then we'll start building again. What I have built is I had this need where I'm thinking about creating an educational content portal all about artificial intelligence. It's all going to be written content: articles, tutorials—very thorough on a plethora of different topics. I thought, great! Let me put a crew together to help do the research and at least create that first draft of the article, the piece of content, or the tutorial.

And it worked incredibly well, Joe! Not only was it easy to install—although, of course, I struggled a little bit with Python environments, which I always do—but I know you have some tips for me, so we'll get to that in a moment. Overall, it just works so, so well.

Once we get into it, I'll tell you I've tested a bunch of different models, and I can see which ones worked better and which ones worked worse. You probably already know which ones worked best; we'll see. By the way, I'm going to be pushing all of the code that we're writing today to GitHub. It'll be a public repo, and I'll drop that link in the description below.

So here, first is an example report. The research topic was "Basics of Fine-Tuning a Large Language Model." I'll show you the definitions of the agents and tasks in a moment, but it did pretty well. This is using GPT-4, I believe currently—maybe mini; I can't remember. I'll show you in a minute. But like, decent, decent—not very long, not very thorough. I've tried many different ways to try to get it to be more verbose and more detailed, more comprehensive, but I can't get it to do that.

Now, I have tested the 0.1 models, and those crush! I mean, they just come up with the most comprehensive, detailed, well-written report possible. I suspect there's a way to coax the non-1 models to do this because the 0.1 models are exponentially more expensive. So I'll note all of these things. I'm going to be asking you all these questions, Joe.

Let me show you now. I have my agents: I have my researcher and my educational content creator. Now, I was thinking about adding a reviewer agent, so that's maybe something we could do together today.

Basically, the senior researcher gives it a topic, does thorough research on the topic using the Serper search tool. That's something that I added. I'm using Serper, signed up, grabbed the API in the previous video, and it is doing the research well. It seems to be working very well.

Then I have the educational content creator basically create the content—create a detailed, thorough, and compelling educational content based on the topic research and research findings provided to you. Make sure you explain everything in great detail. As you could see, pretty much on every line, I'm saying comprehensive, extensive, detailed, and I just can't seem to get the non-1 models to do that. But we'll come back to that.

So those are the agents. Then for the task, I have the research task, and then I have the reporting task. So the research task conducts thorough research, and then for the reporting task, review the content you got and expand each topic into a full section for the educational content piece. I even said right here, each section should be a minimum of five sentences long. Actually, that should be different—five sentences long.

Then I also started just putting a placeholder for the reviewer tasks. So I'm going to stop talking for a moment, Joe. I want to get your thoughts on all of this.

One, great use case! I love it. Two, so many ideas! Yes, 0.1 is definitely way more expensive, and that's the reason why a lot of people decide not to use it. Just because it takes a long time as well. What I found is, honestly, it hallucinates more often than regular models on their GTIC behavior. The reason why it does that is the normal agents don't have much option other than to use the tools that you provide them. But the 0.1 models, because they have chain of thought, they tend to come to the conclusion before wanting to use any tools. They're like, "Well, you know, I think I can infer if I do this, that, and this other thing."

So I think we can very much replicate this behavior. We have seen use cases like this where you want to produce more long-format content in the past. There's usually a concept that works very well: you separate your crew into a planning stage and then a writing stage.

It's almost like you do the research, then with the research, you have an agent to think through, "All right, given that I understand this topic now, what would be the ideal way to present this? What would be the topics, the titles, the subtitles? What are the things that we're going to zoom in on?" So you get a plan out of that, and then after you got that plan, then you go into actually writing it. That usually works way better, and you can have a reviewer at the end as well.

Ah, that makes a lot of sense! In fact, that would probably enable the agents to work more collaboratively because right now, it's just one agent doing the research, passing the research off to the reporting agent—or the content writing agent, I should say—and then it just writes it. Instead, it could be like, "Well, here are all the different sections. Okay, go get that section. Let's write it. Go get the next section. Let's write it," and so on. Is that right?

That is right! For that use case, what I was referring to was even one step earlier, where we just have one extra agent kind of like doing the planning. But while you said is where I would get next, and that is where you have two crews. You have one crew that is focused on coming up with a plan, and you have another crew that is focused only on writing each paragraph or page. We have seen this for people that are trying to produce 50-page long reports and things like that.

So you can definitely do it that way, and we can use one of the CI new features for that. That is actually flows, and this would work pretty well with that.

Let's do it!

And so, Joe, one last thing. I'm using Lang Trace AI to track the traces, and they have a nice Crew AI plugin. I know you know this already. They are not sponsoring this video; just an awesome partner I've worked with in the past. I really like being able to see all of the information, including the cost and the time taken, right here really easily. So I usually reference this as I'm going through it.

I love it! Honestly, observability is a big thing on the Bing and building AI agents, so I think you're doing right.

Awesome! Thanks to the sponsor of this video, Vulture. Reap the benefits of the world's largest independent cloud provider when you bring your GPU workloads over to Vulture. They have the latest Nvidia GPUs spanning 32 locations across six continents. Vulture delivers industry-leading price to performance and serious accessibility and reliability.

Vulture's global, fully composable cloud infrastructure moves your applications closer to your users and frees you from vendor lock-in, allowing you to bring your own networking and database solutions. If you need to scale beyond just a single cluster, Vulture's Kubernetes engine allows you to take full control over your deployment, offering up a 100% free control plane.

So if you're tired of waiting for GPUs from other providers, make sure to use Vulture. You can deploy at any scale immediately, and they have H100s, L4s, and more available right now. You can get a fraction of a card or fully dedicated bare metal systems, which gives you full control over your hardware and your throughput. They also have one-click installation of all the applications you might need for advanced machine learning workloads, allowing you to get up and running in minutes, not hours.

So experience the Vulture difference! Don't get bogged down by severe wait times or limited locations. Try it free today with a $300 credit for your first 30 days when you visit getvulture.com/Burman and make sure to use code Burman300 at checkout to get that credit. Thanks again to Vulture for sponsoring this video.

So, Joe, where do you think we should start?

I would say that probably just set the crew aside for a second. We're going to bring it up. I would open a terminal and go into a folder for us to use a new CLI from CI for flows, and we're going to bring this crew into a flow in a second.

Okay, so this is functionality that I have never used before. I'm very excited about it! Let's give it a try.

So, new terminal. Let me get back into the right folder. Ah, I think I'm already there. Let's see... VM base edu. Yep, all right, I'm in the right place. What do I do?

All right, so you want to do `crei create flow` and give it a name.

Is there any standard that I should think about when I'm creating the name?

I would say for a case like that, you would probably say something like "educational content" or something like that, but there's no right or wrong here.

Okay, can I name it the same as my crew, or should I try to name it something different?

Something different, yes.

All right, I'll call it—since my crew is named "edu," I'll call this "EDU flow." Anything else that I need to do?

No, you should be good to go.

All right, let's hit enter. Maybe, Joe, you can, as we're doing this, explain what a flow is and how we should think about using it.

Yes, so flows is something that we came up with because we realized there's a lot of use cases where you might want to have crews collaborate with each other, or you want to have some regular Python code—like you want to pull emails from somewhere and then pass that to a crew. We needed to provide people a way for them to easily have this data move from one to another in a way that made sense.

Got it! And this is how we came up with flows. But it's a very thin layer, very simple and easy to use, that allows you to do some of that.

Awesome! So just by running this command, `crew ai create flow`, and then I gave it the name, it created a folder within my main edu folder for this crew source.

Let's see what's within the source. We got crews, tools, so it feels very familiar. What do we do now?

I would say let's start by moving your existing crew inside that crews folder to make sure that we can use it.

Okay, and which files or folders do I move in to do that?

The entire source, assuming this is the base for it.

Can you close that just to see what is in there?

No, I think you want everything else. So you probably want the main folder with everything within it—the main folder.

Oh, so I see on the left there, on your terminal, you were inside of the edu folder. That edu folder is the crew, right?

Yes, that is the crew.

Yes, so what I would say is you probably want to move out the flow folder to somewhere else because you want everything inside the edu flow.

Ah, okay, okay, good to know.

Okay, so let's do that. I'm going to move the edu flow folder to my desktop. Then I'm going to take the edu folder—the entire thing—and then move that into edu flow. Is that right?

Exactly! Into the source crews of edu flow.

Okay, so now let me reopen it because I don't think it looks right in cursor.

All right, so we have edu flow with the edu crew within it.

Yes, and you can move that to SRC in crew right here.

So, Theo, the whole edu folder inside the crews folder.

Okay, so I have in source edu flow crews. So the P crew, which is a sample, I assume. So I'm going to move the edu crew into there.

And let's just make sure it gets reflected over here. Crew edu crew, and there's—where is it? Source config. There's our definitions. Okay, great!

Yes, okay, so everything's in there now. So you have a flow, and you have a crew on it. One thing for us to do real quick for admin is you probably want to copy your environment variables from .env into the main crew flow .env.

Absolutely! Okay, let's do that.

So, yeah, we got it. So you got a crew. Now let's step back and check how the flow works. If you're going to the flow main file, and that's towards the bottom.

Okay, FL main file right here.

Yeah, so you can see the flow is very simple. As I said, it's a very thin layer in here. It has a poem flow, and you can see there's a function there on line 19 that is annotated with a star. The star thing that annotates that function indicates that that's the initial function of your flow.

And then what happens is you have an event-based system, so anything that is listening to generate sentence count is going to be executed afterward.

So on line 23, you see that you have another function that is listening to that one. So once that one is done, then it's going to generate, it's going to call the generate poem, and then once generate poem is done, you're going to call save poem right there.

This is a very simple flow, but the way that we can use this is we can have a start function that basically triggers one crew, and this crew does all the research on the topic, gets pretty deep into this, and creates a plan. Then you're going to have another crew that's going to be listening to that once that's done that actually goes in and in the loop writes the content for every single chapter or paragraph, depending on how big you want this to be.

Okay, so we are going to have, if I understand correctly, one crew doing the research and the prep, and then another crew actually creating the content?

Yes, and this is the most advanced setup that we can get. We could try to do everything on one crew, but this is kind of like how you want to get them for real.

Okay, let's do it! I like advanced! It's so funny; I would have probably just been trying to do it with one crew, but it's nice to know that flows are available now.

Yes! So let's change a few things here real quick. Let's remove that state from the poem. You can just drop that whole thing—like this whole thing.

I would say let's delete from line 10 to 13 at first.

Okay, yes, and then we can rename some of the classes instead of poem flow to be your actual flow, edu flow, and also import our right crew in there.

So, yeah, it says poem crew is the crew part of it, or I guess what I'm asking is it should just be edu crew, right?

Yeah, I think it's edu in your case.

Yeah, it might not even be crew; it might just be edu, right?

Yeah, I think for your case, yes, that's it. I think cursor has got that right.

So if you go back to that line and you—I think the last one is e—you if you delete one character real quick there, yes, try to just—yeah, there you go! That works! If you just tab, then this should be edu.

So the cool thing is one of the cursors does is kind of like the auto-complete thing. So if you go back to the edu first and you just start typing something like a dot, you can see that it suggested to you. So if you press tab, it auto-completes the line for you.

I don't know if that is necessarily—oh, that seems to be right! You can see that they color change, so it was able to improve `cw.edu source.ed crew`. So it's really trying! It's getting it all the way from the nested directory here.

Yeah, we can double-check in the end if it's working, but yeah, we're GNA. There's a way to import it for sure. You don't need to pass an attribute in there; you can basically delete the PO state. You can—you still, I think you still need the flow in there; we just don't—yeah, you can drop the brackets.

Yeah, and then I'm going to rename this to edu flow.

Okay, and then same thing here. Let's just get that fixed.

Yeah, so I would say on line 13, let's rename that generate sentence to research content or research topic.

Okay, yeah, so now let's just drop lines 14 and 15 for now and just add a comment so that we know that that's where our first crew is going to leave.

Nice! And now we can do a similar thing with the second function, but now it's going to be listening to the generate researched content, right? And then we can rename that as well.

Yeah, do the whole thing, and this should be renamed to that.

Yep, and we can drop the content and replace it with a comment for now just so that we know, like, "All right, this is where the second crew is going to come."

Sorry, just leave it like this. I know, I mean, it's up to you how you name the function. I meant that we can delete the content of the function from 18 to 26 to just have it as a comment for now.

Okay, can this be the same name as this function up here?

Not the same, but in the way that they're doing there, like, it's pretty good. They are different names.

Oh, I see, I see. Yes, yes, it is. Oh my God, I didn't even notice!

Okay, and then the problem that you're seeing in there is because a function with only a comment with it doesn't work. So if you want to just get rid of this for now, you can type `pass` in the next line within the function.

Yeah, then you can do the same thing for the first one as well.

Yeah, there you go! Lovely! We can drop lines 22 to 27; we don't need that, and just rename some of the later ones to edu flow right there.

We'll also do edu flow right there. Obviously, let's rename all of these to edu flow.

All right, so now before we actually do the crews and anything, if you want to, in your terminal, you should be able to type `crew ai flow plot`.

Okay, and what is that going to do?

I'm going to show you in a second. We notice a new—you know, maybe you can help me with this. I see this new environment has been created, and I never know whether I should be selecting yes or no.

I think I always get it wrong. Right now, it's on base, which doesn't seem—oh, well, it disappeared. Great!

Do you usually select that?

I don't get a lot of that because I prefixed a specific Python version, but I got to say that the way that I set things up is a little unusual. I have a natural Python installed on the computer. I don't use much of virtual environments myself.

Okay, okay. All right, so I think it's done.

Yeah, so open up on Finder real quick, and then you should have created a flow HTML that you can render in there.

Hey, look at that! Oh, this is awesome! I'm going to move that zoom in on it. You click and expand, and again, this is a very simple—but by some of the labels, like below, you can see that it can get quite complex with routers, triggers, NS, and everything.

Wow, this is so cool!

Okay, obviously, it's still a very simple use case, but this is awesome.

Yeah, but when is an example here? Let's say that you didn't want to push this into a CMS automatically, right? You don't need to get out of this to do that. You can basically create another function that listens to generate educational content, and then you can write regular Python code that does an API call and pushes this into a CMS. Now you have a more complex flow.

That's exactly what I'm going to need to do eventually! I want to push this to a CMS, so I'm definitely going to take you up on that when we get there.

Okay, very cool! All right, so now let's add our crew, and we're going to probably reuse your—let's start creating a new one for the research.

So you can type, and let me see if I remember this from top of mind: `crew ai flow create -crew`. I think that's it.

Oh, by the way, `crew add crew` is one single thing, and then you can give a crew name. You can see "edu research," for example.

All right, all right, let's see.

So `create ai—oh, sorry, C—what?`

Yeah, try to do the best help. I think I remembered that wrong, so create flow -D help. Let's see what the command is.

ADD crew, there we are!

All right, so what we're doing is we're creating a new crew within the edu flow, our current flow, and the command will be `crew ai flow add -crew` and then the name, let's say "edu research." Let's hit enter and see what we get.

Yes! Okay, it went ahead and created a crew for you inside the folder that you have in there.

So if you—yeah, if you close the edu folder, there you go! Now I have edu research, and you can drop the poem crew by the way.

Yeah, let's get rid of that right now because we're not using that anymore. Great! Now we have edu research.

Beautiful! And just like normal, we have the agents definition in a YAML file, the task definition in a YAML file.

Yeah, very nice! You can see, though, that these are way more compact setup, especially if you compare with your old one because that other one was a crew in isolation. It has its own environment, YAML, its own tools folder, a bunch of different things. This one, because it's leveraging the whole setup that is going on in the flow, it's just kind of thinner and zimmer version of the crew.

Got it! Okay, cool! All right, so we have a research crew. Should I also create a—basically, I think the edu crew, the old one, is there just for reference. I'm going to guess, like, going forward, we're going to create a research crew, and then we also need the actual content writing crew, is that right?

Yeah, I would say go ahead and create the new one as well.

All right, so edu content writer.

And so same thing: `crew ai flow add -crew`, then the name. Let's hit enter.

So we should see—yep, look at that! Beautiful, beautiful!

So we have these two crews: the edu content writer and the edu research. We also have the old edu, which we're just going to use for reference.

Yeah, so now we should be able to just add agents and tasks to our YAML files as we use normally and set them up as a crew on that crew file the same way that we would.

We can keep it simple for now; we can just like get some going, and then we really improve some of those agents.

Okay, so I'm going to open up the edu folder for now. Let's grab the old definition so I don't have to rewrite some of this stuff—agents. So this is what I used before.

Whoops! We don't need both of them; we just really need the researcher for now. So I'm going to grab that.

By the way, can I give you some cursor tips?

Oh, please! Please! I'm so rusty, by the way, with coding. I haven't really been coding much.

Please, no worries! So going to your agents YAML, the old one—yep, that one.

Yeah, you copy the whole thing.

Uh-huh.

So the both agents, just select everything, and you're good to go.

Okay, yes! Now going to your new one for research, for example—yep, let's see agents.yaml.

Now you can select all this, and what you want to do is you want to type command K, and now you can say, "I want to create an agent for doing research on the topic. Describe your use case." And after that, you say, "Here's an example," and you paste the content that you just copied from the other one.

Interesting! Okay, let me see. So do you usually use Llama 3.1 versatile for this, or should I be using like Cloud 3.5 Sonet?

I'm a huge fan of Sonet for CSER, so I usually side with course both Sonet. Which one do you use, by the way?

I think the new—I think the other one is a cut. I'm using the regular 3.5, the one in the top.

All right, let's use that one.

Okay, so create agents for creating educational content based on this information from my previous agent definition.

Okay, and by the way, before you press enter, I would add a few things. So I would say, "What exactly do we want to take out from this?" So you can say, "The final result from the final agent should be a complete plan on how to write long-form content with titles, subtitles, and why each topic matters, what is important by it, and maybe even include sources where information came from."

Okay, so the final goal for the researcher agent should be—let's see, how do I want to do this? The final goal for the researcher agent should be to collect all of the information necessary, including doing a web search with the Serper tool, to find out the relevant information about a given topic.

And then I would definitely add at the end of the thing that you run, you want a plan out of this, and this plan should have a few things.

And just be very descriptive about what are the things that you want to have the resulting—how do you say that, though? Because it's part of the goal, right?

Yes, it's going to be part. Oh, you're right! These are the agents, right? So maybe we save some of this to the tasks. Maybe this should be good enough, so you should be able to press enter now.

And this is the part where cursor really shines. Now, when you're ready, you can just press command and enter, and it's going to accept the suggestions.

Did it finish?

Yeah, it finished!

Okay, senior researcher, accept!

Okay, but it had both of the agents, which we don't need both of them in this crew, right? We only need the researcher.

I would have both because what I would say is we can have the researcher really focus on finding out. So the way I'm thinking is the first agent in the research phase is going to find all the different information. It's going to go wide and broad. The second one, I want them to create a plan. I want a very structured thing out of this. I want to say, "All right, it's going to have five sections. These are the sections from everything that the researcher found. These are the main things that I want to have in each section." So I would go that route with two agents.

Okay, so in my previous version, I had a searcher, and then the educational content creator was the agent that was actually writing the content. But that's not quite what we want here. It should be educational content planner, maybe?

Yeah, I would say that's good.

Oh, and by the way, whenever you change one thing, do you notice how the tab thing is showing up?

Yeah, geez! You can just keep pressing tab, and it's going to complete the thing!

Oh my gosh! I started paying for cursor today. Totally worth it!

It's so good! So good!

All right, so how does that look?

Let me check it out.

All right, you keep your ability to turn complex research into clear—I think for the agents, this looks great. I think on the tasks, though, we're going to be more descriptive of the counts of the plan that get—should I be a little bit more descriptive with the researcher's name, like educational content? I guess it's really just for internal use; it doesn't really affect how—

Okay, so let me—let me—I'm going to make that more simple, then. I'll say that's the planner.

Yeah, all right, cool! Now let's open up the task. Oops! No, this is the old one, though, so I don't need that. We need task here. Here we go.

Uh-huh! Okay, so resource task, reporting task. So let me grab the old information now and copy-paste. Nope, don't paste!

Hit command K!

Yep! Ah, cool! Okay, so edit these agent tasks to include information from the following.

Okay, so we got that, and then from the following snippet of older code, then maybe we'll do something like, "The—you know what? I just want to see what it does, and I want to edit it inline, so let's just see."

Sure! And by the way, you can keep doing like command K, and there's one feature that I didn't tell you about that is the command L, and that is for more like a longer check. Like command K is more like ad hoc, change this. The command L is going to open a chat menu on the right for you to constantly chat with it.

Okay, so we have the research task that seems right. I think this, instead of being reporting, it should be planning task.

Yeah, okay!

So let's look at the research task first. So conduct a thorough research—let's just clean that up. Conduct thorough research about the topic. Make sure you find any interesting and relevant information given—put that on the same line—given the current year is 2024.

Should I explicitly tell it to use the Serper tool here?

You don't need to. What I would do, though, is I would add some flavor to the research, right? For example, if I'm trying to produce material that is talking very much about the future, I would say we're looking for the leading-edge content, the most relevant information. Or if I'm trying to do something for executives, I would say I'm trying to find things that are meant for executive decision-makers to understand the market or understand the topic.

Okay, so what I'm looking for is content—yeah, I'm looking for content that is beginner-friendly but also comprehensive. I think it also will depend on how I define the topic. So if I say "Intro to Fine-Tuning Large Language Models" versus "Advanced Fine-Tuning of Large Language Models," I can kind of adjust it in the topic right there, right?

Yeah, that right there I think is good. But if one thing, like, it seems that what you're trying to get is you want to get—you're going to get information from basic level to kind of like an intermediary level because the idea is to kind of like build up the content, right? So I would say it might be worth adding something like this, but again, we can always come back and change it. We don't have to get it right on the first one.

So good point! If you feel like this is good enough, maybe—yeah, let's see! Let's see what happens! I like testing it along the way.

So expected output: a thorough research report. Is it a report, though? Really, what I wanted to do is gather information, so I guess it's a report. Is that a good word for it, do you think? Would you use a report?

I think a report's good. And then the second, like, the key for the second task is a plan more than anything.

Okay, so a thorough research report on the topic with relevant information that—oops! That it can be turned into—that can be turned into an educational piece of content afterwards. The research report should also include references to the sources you use to create the content.

Perfect! Yeah, one thing that I would add as well is instead of saying that it can be turned, I could say that it will inspire educational content.

And what's the reasoning for that change?

It's just that I don't want it to kind of stray away and start to actually write any pieces. I wanted to give as much raw information as we can into our next planning agent because the next agent is the one that's going to try to find out those, "All right, how do I make this interesting or edgy or explainable in a different way?"

And I think the rawest information that we get for the second agent, the better it will do.

Okay, awesome! Awesome! All right, so then for the planning task, review the content you received and expand each topic into a full section for an educational content piece.

Yeah, we want to change out that. I would say review the content you received and get inspired—and get—not get inspired. Let's say, "and come up with a plan for an amazing educational content that is aimed at..." and then you can use another variable, persona, in there.

Yes! I'm going to call it audience level.

Very audience level! Yeah, that's so cool! Maybe I should put that up here as well.

Yeah, for sure!

Okay, so, but I guess it doesn't matter. Like for the researcher, grab everything. I mean, honestly, it might matter because it might change some of the searches that we will do.

Okay, an ex—a thorough audience-level research report on the topic.

Yes! I mean, maybe because if you use beginner data, it might do a beginner research. So maybe you want to put that later in the sentence, like a research on our topic that would inspire educational content afterwards aimed at audience level.

Okay, aimed at audience level. Great!

And then I would say lines 13 to 15 there, we want to replace them with like what is the actual plan. So in here is where you can even interpolate more variables or like you can say, "Oh, I want this to be edgy. I want this to be insightful."

And then you go into, "What exactly is the plan that you went?"

And I think that—sorry, sorry, go for it. The plan should be highly detailed and comprehensive.

So let's see, the plan should be highly detailed and comprehensive. Maybe just that.

Okay, and then I would say you return a list of—and then you can do a series of bullet points, and you can say titles, subtitles, and that would be in the description, not in the expected output.

I think it would be better in there, yes.

Okay, so formatted as markdown. Make sure to include any references or sources you use to create the content—the plan. We don't need—I guess we don't need that for the plan, huh?

Yeah, no markdown whatsoever because we're actually going to be using a PTIC object there.

Got it! Okay, fully fledged educational—no, that's not it. So this whole thing needs to go.

Okay, so a detailed plan for content level, or audience level, I should say. A detailed plan—how else should I define this? My take is if you want to one single page, what do you want is a plan for each section.

So I would say the plan should be a series of sections, each section with a subtitle, a high-level goal, each section with a title, subtitle, I level goal, and I would say one thing that worked for me in the past is why this section matters, why it's important, and a detailed breakdown of the section might be worth it.

It might be worth it to add the sources where the comp could be sources referenced.

And I guess do we need the detailed breakdown of the section? No, I would save that.

Yeah, and I would remove title because remember you're going to have one title. This is only about like each section of your article.

Okay, so with a subtitle and sources referenced.

Okay, and then educational content planner. Let me just make sure—no, it's called planner now, so I got to change that.

Okay, how does that look?

Looks pretty good!

So can we just try this and see what comes from it?

Yeah, go back into your main.py file, and let's import this crew.

Okay, edu flow, this is it?

Yep!

Okay, yes! So you can go online 9 there and try to import—doing from point crews.

There you go! Research crew!

Geez! All right! Oh no, that didn't work. Let's see. Try to delete line number eight for a second. I think that's the cursor off.

And I think what you went in there is there—let me get—what would be the right path? One second.

Oh, there you go! No, that didn't work. So crews, edu, edu research.

Yeah, that's—I don't think you need the crew thing. Maybe that—oh, you don't?

Okay, research. Maybe that is—oh, and then you got to do edu research again.

Edu search crew! Sorry, edu search crew!

There you go!

All right, yeah! So now on the first—and by the way, this is all what we just did is to find this file right here.

Exactly!

Okay, so now online 14th, you can actually call edu research crew kick off.

And for now, because we're testing it, we can just pass the inputs in there on the kick off, and we're going to pass audience level and topic.

Okay, audience level, beginner, topic—let's say fine-tuning LMS.

The one thing, though, is that the audience level and topic should be a dictionary with a parent key called inputs.

So there we go! Oh my God! I wouldn't have known how to do that! That's so cool!

I'm sorry! I'm so blown away by all this!

All right, except that looks right! That looks right!

Yes! So we should be able to—you can delete that pass line on 19. That is not required anymore, and you should be able to do `crei flow kickoff`.

Okay, so `cre ai flow kickoff`. Let's see! Cross fingers!

Running the flow!

Okay, all right! Has no attribute kick off.

Oh, I think what it is—oh, there's a couple things. So the first one is I think what is missing is you're calling kick off as if it was a class function. You need to instantiate, so we need to do the two parentheses on the crew before calling kick off.

Okay, that's it! I think that's it! Let's give it a try!

Right, so that's not a class method. Now it's an instance method, yeah, or you're calling it on an instance.

Yep!

Okay, there you go! So if you're going to edu research crew, that file that we created within the crew.

All right, so can I show you another interesting thing from CSER?

Yes, please!

And look, it already knew! It already knew the change before it does that! You can do command L now.

I want to say you can do @ and type tasks. It finds the right tasks YAML for this, I think it's the second one.

Yep! You can press enter, and now you can type, "Change this file to use the right agents and tasks."

So what is it doing? It's pooling from your agents and tasks files, and it's updating.

It did something wrong in there. It didn't work because it's updating the YAML file, so maybe what you do is go into the edu research file again. Click on that one.

Now try to do command L for a second. Let's see if it closes it right now.

Yes! Now it's right! Now you can add agents and tasks with the @, and you can say, "Update this crew file to use the right agents and tasks definitions."

Okay, so, Ed research, update this file to use the right agents and task definitions.

And by the way, it might be worth just mentioning the crew Python fire or something, so the right one.

Yeah, you see it's updating the YAML, but you can update it. So if you click back on where you wrote on your prompt on the chat side, on the upper right, you can update it.

So you can say, "Make sure that you're updating the research crew file Python file."

And now you can do it!

And then there we go!

All right, so once it finishes, you can click on apply in there.

Okay, we can see the diff. Interesting! It changed—it removed that.

Okay, that's fine, I guess.

And we don't need this. We do need this. We do need this.

We do? Yep!

I see! I see!

I mean, do you know one thing? Like, I'm noticing that there's like a few things that are hurrying out, so maybe let's try to reject the whole thing.

All right, and what you can do is on your chat there, you can reference back the other file.

So on your chat, you can see, "Make sure that we are working—we're doing it similar to edu research crew."

Okay, let's see if that works now.

Same thing—no, let's see. No need to load YAML, right? What is it trying to do here? I'm confused.

I think it's trying to manually work around a lot of the things that crew already does.

Okay, let's just fix it manually. Sometimes you got to do that.

So, IDU content writer crew, so that's not right.

Now I'm so dependent on cursor, I'm like, "Wait, how do I use cursor to do this?"

So I think the only thing that I want to do is I want to get the agents' names from agents.yaml and replace them in there correctly.

And then the same thing for the tasks.

Okay, so agent—yeah, agents.yaml. So it's content writer, editor, and quality reviewer.

Okay, so I'll try one more time. Adjust the names of the agents and tasks to match at agents.edu content writer and at tasks.

Okay, there it is! Editor, writing task, Ed.

Yeah, so that looks right!

Okay, apply! Close that! Accept!

Let's see, does this look right now?

Yeah, I think it's only missing one agent. I think we have a reviewer agent in there as well.

Yeah, there to add the reviewer agent.

Yep, looks good! And also the reviewing task, I would guess.

Yeah, the reviewing task as well.

Okay, assuming the names are right, this should work.

Okay, let's see!

Running the flow!

So we actually used this back in the day to simulate a test of having a crew to write a whole book, and it's basically there and steroids where you would have the crew PL chapters, and then you could have a crew write a whole chapter for you.

And yeah, you can produce way longer-form content this way, so that's the trick: the plan up front and then have the agent or the crew kind of go through each section and build out the section rather than trying to have it do it all at once.

Yeah, we found that that usually produces way higher quality content if it's kind of like a chain of thought on steroids.

We're going even one step higher!

Yeah, so cool!

Okay, so while that's—yeah, so that looks like it's going. I'm going to switch back to Lang Trace, and let's just see how we're doing. If we go to traces, let's see how much these are costing, by the way, per run. I'm always curious.

So three cents! I mean, so good! So cheap, right?

So cheap! I think it's using Far Mei, maybe? I hope so; it's Far—it is! That's the default!

Yeah, actually, we can tell. Let's take a look.

No, it is using GPT-4!

Oh, and so we could use GPT-4 mini!

I think I actually manually specified 4.0 mini somewhere.

I don't think—let's see. It would be on the end—it would be either on the agents or the end files.

It's one of those, and agents or the YAML files, right?

The crew, right? I don't specify it here.

No, it would be on the agents.yaml or on the—oh, you put it in the YAML! You can choose to, right? You can put it in the YAML if you want to set per agent different LLMs, or if you want to set overall, you set the YAML in that broadcast.

The way that I was doing it previously was like right here, I would just say, "comma llm equals this."

Also works, but I found the YAML to be a little easier.

Yeah, it's cleaner to have all the definitions in one place, definitely.

So I'm not actually saying it anywhere.

GPT-4, there it is!

Okay, oh, but this was in your old one, was it?

Oh, no, I don't think so!

Yeah, here! So, but yeah, this is your old crew.

Yeah, this is the old crew! By the way, we should just—we can get rid of that now, right?

Yeah, okay, so let's get rid of the old crew. Where is that? Edu, right? This whole thing can be deleted.

Yep!

And by the way, one thing that I'm seeing that I heard that we got there is probably because if you go on the content writer crew tasks YAML file, we can see that the name of the task is quality review task.

So we should update the edu content writer crew to use that one instead. I think it's using—if you see, it's using a—yeah, review task.

Okay, and also line 52, line 52 as well.

Okay, let me—before I run it again, make sure all definitions match naming in tasks and agents.

Is it saying to change anything? Quality reviewer?

Yeah, so it did find another one that was wrong. I think—yeah, got it right! Quality reviewer!

Amazing!

Okay, okay, let's save that! I think we're all saved!

So should we try again?

Before we do, let's just double-check the tasks file and make sure that's referencing the right agent as well.

So you see that the tasks don't have an agent within them, and we should probably—

Yeah, should probably add this one.

Does this one do?

This one does!

So, okay, so let's do this!

Add agents to each of these tasks based on agents.yaml.

There you go! You're a pro in cursor already!

You got this!

I'm addicted! It's so good! It's so good!

All right, apply!

Yeah, that looks good!

All right, save! Good to go!

Yeah, let's do it!

I gotta say, the one thing that is bad about cursor is that you got very used to it. Like, I gotta say, coding on planes became way harder now!

Yeah, although you can get it all running locally.

So I tried to do it, but I struggled with it. Maybe I need to make a video about it.

I'm pretty sure you could do it with cursor. I know that there are plugins with VS Code that you can get it working. It's probably not going to be as clean, but yeah.

Yeah, you don't—if you don't have access to the internet, it's going to be tough.

Okay, so crew AI flow kickoff!

Let's see! Hopefully, we got this one working!

Okay, so fine-tuning LMs, educational content planner, reviewing—oh, look at this!

Yes! I'm going to make this a little bigger so we can see more of it!

All right, second crew is working!

Second crew, but it's—wait, so this was—was the right? So the writer crew is going to write—it's going to essentially kick off multiple times for each section, is that how I should think about it?

Exactly! It's going to run once for every section, and by the way, there, when we created the agents and tasks for the writer crew, we didn't spend too much time on it. One thing that might be worth doing is just kind of like tuning in because now it's going to probably write too long sections, and we don't—that was like, "Hey, write like a shorter section."

Like, we want one or two paragraphs because in the end, everything's going to come together, right?

So I would say that now that's probably what I want to do when spending some time later on the content writer crew just to kind of like make those prompts in a way that it works great for writing sections of a greater article.

All right, that sounds good! We may save that for another video because this one's getting long already, but that sounds like a good next step!

Yeah, but they're ready! I think this is the third or fourth thing that it's writing already, so we should have a final one within like the next few seconds.

I think we forgot something super important in here, so all right, stop this real quick. There's one thing that we missed.

Okay, go into our flow, the main.py file.

So if you see the second crew, when we exit, you can see that we are passing those input variables, right?

So we are passing the topic and the audience level, but what we're not passing is the section.

So we need to append the section into that before we pass it, and we need to—I think you can probably do that in another—I don't know if that works. I think it might.

No, I think that won't work.

Okay, yeah, I think what you want is on the line above, you want to create writer inputs above the four.

Or within the four.

Within the four!

Okay, writer inputs!

Yes, that's going to do a copy. Then you can tab that, and then the next line you want to add the section exactly as it suggested.

Yeah, and then the only one thing that is missing is I don't think that our agents in tasks, especially the tasks, I don't think our tasks in the content writer crew expect the section variable.

So we want to make sure that we add it to there.

And do I need to pass in the input variables as well here?

Oh no, it did it! It did it already! Never mind!

Yep!

Yeah, got it!

So on the tasks for the writer crew, you want to make sure that you add a spot for the section right here.

Yeah, on probably on the three, I would add them.

So you can go at the final, like for example, on the line five, on the description, and you can break maybe a couple lines, and you can do section, and then there you go! Something like that!

Yes!

And you can add that probably to all the tasks. Eventually, once we get to spend more time on this, on this agents and task definitions, we can probably make it better.

But that way, now we're actually telling what is the section that it's writing about. Before, we were not doing it.

Got it! Got it!

Okay, cool, cool, cool! That makes sense!

All right, so let's try again! Ready to go?

Yes, I think so!

All right!

All right! No, so it's not two Jon. Maybe we ask cursor to do it for us.

Like, what is the method that converts a penic to Jon?

Oh, I don't even remember where we were! It's right there on line 27, that other file.

Yeah, there you go!

What's the right way to convert a penic object to Jon?

Yeah, ah, model dump Jon!

Yes, classic!

All right, so dot two—let's just make sure we don't have it anywhere else because I thought I remember we do.

No, I guess not!

Okay, okay! Let's run it again!

All right! Nice!

And by the way, notes for the future for maybe the follow-up video that you do is I think there's space for some of this agent for the second crew to also have certain tools in case they want to expand on things or fact-check things because they're going to have the references.

So you could add Serper to them as well. That would be another thing.

Co, other thing that might be helpful is you can set up a custom tool that allows them to search for previous documents and content that you wrote that you like, so they can check examples.

That's something that we also have seen work out in the open, so there's a few different things that you can definitely do.

I have something advanced that I really want to do, and I'll throw it out there. We're not going to do it today, but maybe just get your thoughts on it.

I want to be able to create images, graphics, diagrams based on this content. That seems incredibly difficult, but maybe it's possible with Crew.

It is possible! Yes! We did that recently for a use case where basically we were analyzing support data consistently, and what we want is basically for it to draw insights out of it and kind of like plot charts, do the math, plot lead.

So we created coding agents that would actually write the code to do it.

Oh my God! This is so cool!

This is so cool!

Okay, so it did finish!

Yes! I don't know why we got none in there, but if you look at the final ones, we should have all the sections in there.

So if you start, for example, from the bottom, the last one that you're seeing is the final section.

Oh, oh, oh! Right, right! Just one section!

So how do I see the whole thing?

So I thought we were printing it out, but for some reason, it didn't—I think I see!

Do you see that online 28? We call point.pent.

Yeah, we don't need that! We can drop that point.pent, and maybe what we can do is we can do raw, and I think that should work if I recall correctly.

Yeah, okay! It might not be necessary. You might not even need that, but again, you know what?

Can we just put it to a PDF for now?

How would I do that?

Oh yes! Let's do this! Create—oh, go ahead!

Create one function after this!

Yeah, just break the line and create one function similar to the generate thing.

Yes!

And you can say save to PDF or save to text file to keep it simple.

You want to make sure that you bring that function one level deeper, so it's inside the flow.

Ah, yeah, of course!

Okay, you want to annotate that with the listening annotation, so listen annotation similar to what we have there.

You go!

And then you can basically save it to a text file.

And if you start, like, if you do command K and say save content to a text file, what going to do for—

Oh yeah, to text file!

Yeah, I'm going to change it to actually being a text file. I don't really need a PDF to be created every time to text.

Okay, accept!

And then one thing I would say on line 39, what you want to do is break the line and do a return final content.

Got it!

So now data return is going into the content variable on line 33 and doing the whole thing.

Okay, let's go!

All right, let's do it!

Do we want to use GPT-4 mini?

Let's use GPT-4 mini!

All right, where would that be set, or why would it be using—actually, let's go back to Lang Trace and—

Yeah, so it is using GPT-4! Multiple versions!

Why would it not be using GPT-4 mini?

I don't—the only reason I can see for that is if you have an environment variable set on your system level that would be pointing out for that.

That said, let me get you what is the env var that you could use to set it for me.

One second! Just because it'll also be a lot faster too!

All right, so this is what you want to do. You want to set on your .env, and I'm going to send you here in the chat that environment variable with that setup.

Got it!

Okay, so we're going to switch over to the environment variable. Alex is going to cut this part.

All right, so here we go! Now let's run it one more time!

Crew flow kickoff!

Go!

All right! Now it's going to be fun to see!

Like, uh, a link tracing works, but it should—the one thing that we're not doing as well, and we should do, because it honestly has no cost associated with it, is this agent can search the web, but they're not scraping the pages.

So you can add the two on the researcher ones that you scrape web pages, and they were allowed to now go deep into those pages and learn more because now they only have the research results to work with.

Okay, I need to take notes on what we're going to do next! This is all really cool!

Let's remember to do that!

Okay, so we got our sections!

Good! Yes, there you go!

So now they're basically writing the content for the first section, and you can see the section information in there.

It says Jon, but it should be fine!

So it knows that an introduction—you know, the high-level goal, you know, everything that we said, including the references.

So basically now it knows what it needs to write!

So it wrote an initial section, "Why LLMs are Important." You can see that right in there, and now it's going to the quality assurance specialist.

Beautiful! This is so cool!

It is going! I wonder, like, what is the section that it's working on now?

It's techniques for effective fine-tuning, so it's going deep!

Yes!

Yeah, yep, yep! There it is!

Overall, the content effectively aligns—so that is the reviewer essentially saying what the feedback is.

Yeah, but you see that's already the thing! Like, one thing that I'm noticing is because the final agent is a QA agent, and the expected output is the review, what is happening is the review is being considered the actual final section.

Yeah, so it's something for us to look into. I think we're still going to be good enough, but yeah, it's something for us to tune in on the agents and tasks from the writer crew.

Got it! Just because we didn't spend much time on it.

Yeah, because if you look into that, for example, let's expand the terminal there real quick.

So you see—let's scroll for the one above, like if you see there—go just a little more above that.

All right, so this was the first one, right?

No, this was the QA doing performance quality assessment.

So the one above that, maybe—yes, this one is the output from one of the sections.

Like, it wrote a bunch of things!

Yep!

And then it passed that on to the next one that is the ad to change it.

So what we should have said to the answer is like, "Hey, whatever you change in the end of your execution, we still want the final section—the whole section."

But if you scroll, it's going over and kind of like giving you suggestions and everything that is going to get interpolated.

So we have the educational content text file in the left if you want to open that up.

Yeah, so you see it's adding the quality assessment report as the actual content, and that's the problem!

Ah, yeah! If you don't mind, can we fix that real quick?

Yes, let's do that!

So let's go into the tasks real quick for the content writer—edu content writer tasks, edu research—no, edu content writer tasks, right there!

Yes!

Okay, so let's spend some time on this.

So the first thing is we want to make sure that on the editor, we say on the expected output from the editor, we say a polished version of educational content.

We've improved CLA, and then we can add to that we only want the section content back—nothing else!

Exactly! Do not include the feedback!

I'll just be extra explicit!

Perfect!

And then for the quality review task, we probably want to do something similar where on the expected output, it's saying like, "I want to detail quality assessment."

Like, "No, no, no! I want you to update! I want an updated version of the section that incorporates all your feedback!"

Yes!

There we go! Incorporates the feedback! Do not include the feedback itself!

Yeah, just the improved content!

Okay, so let's see how that works!

All right, so let's run it again!

Crew AI flow kickoff!

Let's see!

Okay, so we should be good!

Yes!

So now we should see the final output!

And if we see the educational content text file, we should see the updated version of the section!

Yes!

So let's see!

Okay, so we got the educational content text file!

And let's see!

Yes!