Transcription
So, I've been experimenting building an AI agent for my daily planning app, Ellie. And I wasn't going to make a video, but check this out. So, I have this agent I just built, and I can ask it things like, "What time should Siccilia and I go to the gym on Monday?" And it's able to look through both of our calendars and find the best slot. And I can even ask it to create the task and time box it for me. And the craziest part of this is it was built with a lot less code than you might think. I did this in about 2 days and I'm going to break down exactly how I did it in this video.
If you're a developer and you're interested in building AI agents for your own app, this is the video for you. And if you're not a developer and you're just interested to see what this stuff looks like under the hood, I hope that this video demystifies it for you. If you're new to the channel, welcome to the video. My name is Chris. I build productivity apps. I usually focus on one productivity app per video. And today, we're focusing on Ellie. Quick context. Ellie is a daily planning app. It's basically your to-do list and calendar combined. So, that's the app that we're focusing on today.
So, back to the agent. It's a chat interface where you can ask it simple things like, "What's my schedule?" But it didn't stop there. So, as a bonus, I actually found something pretty game-changing. Turns out I could hook this thing up to a thing called Zapier MCP. So now I can even ask it things like, "Brian sent me a Slack message asking me to take a look at our server bills. Can you find it and make a task for today with details?" And now the agent's going to search through Slack, find the message from Brian, and then it creates the task in Ellie with the details in the notes. So, even though I never built this integration directly with Slack, by giving my AI agent Zapier access, it already has access to this and thousands of other tools like Gmail, Notion, To-Doist, and a ton of different tools. The agent was already good, but then hooking it up to Zapier made this thing 100 times more powerful than I anticipated, and I'm very excited to show you guys what that looks like. So, in addition to how we built the agent, we're going to be covering the Zapier stuff, too. So, definitely stick around for the rest of the video.
Okay, so let's jump into it. What are agents and how do they work? The simplest way to describe agents are they're basically AI's like ChatGPT, but it's given a set of tools that it's allowed to use. And you can give it any tools, but since this is an agent specifically designed for Ellie, we're going to give it tools specific to Ellie, like the ability to create tasks, search through users' calendars, update tasks, all the stuff you can do in Ellie. So, at the core, the agent is an LLM just like Claude or GPT. And we give it a set of tools. It's going to run through a loop and keep calling tools until it feels confident it has exactly what it needs to answer the user's request.
Let me show you a practical example of what this loop might look like. So, if I ask the LLM, "When can I go to the gym this week?" The agent's first going to look at this and say, "I'm going to need to call a tool for this. There is a get calendar events tool available. Let me call this first." Once it calls this tool, it's going to look at the results and say, "Cool, I have everything I need to answer this user's question." It's going to go ahead find an empty slot through the day and then answer the question for the user. And then the loop basically ends there. So, that took one loop to do. That's basically it. This is the foundation of how all agents work. They go through this looping and tool-calling process.
So now let's go see how I actually implemented this in Ellie. So the first thing I did was I actually created a very simple chat UI. It's just a chat. There's nothing magical here. Cursor did this in like three prompts. So I had this chat. I can ask questions, but it doesn't really do anything. So the first thing I did was define a simple endpoint that just hits up an LLM and then just spits a response back from the LLM. And in this case, I'm hitting up a service called Open Router, which basically hosts a bunch of different LLMs, and I can switch between them with one line of code. So, the reason I do this is so I can test out a bunch of different ones really quickly. And we'll see more in the video why I had to do that. But just know I'm calling this service instead of hitting something like OpenAI GPT directly. So, I have the power to switch between Claude, OpenAI, Gemini, and all these things really easily. But the process works whether you're hitting Claude or OpenAI directly or going through Open Router. It's all the same. It's really simple. It takes in any messages that I'm inputting. So, could be one message like, "What is the largest ocean in the world?" and it takes in the model I want to use, which in this case, it's GPT-4o Mini from OpenAI, and it sends the request directly to Open Router to be processed by this LLM model that I chose. The GPT-4o Mini model is going to receive this, process it, and try to figure out, "Okay, what's the largest ocean?" It's going to send this response back, and I'm going to send it directly to that chat interface. So when I ask it, "What's the largest ocean in the world?" the response then shows up in the chat. It's not really an agent because it doesn't have tools or anything, but this is the starting point.
Let's actually make this into an agent. We're going to add our first simple tool to make this thing functional. The way it works is we call Open Router exactly the same way as we did without the tools, and we just pass in this new parameter called `tools`. And we have to define what tools we're going to give it here. We're going to define the tool here. And this is what it looks like to define a tool. Might seem a little complicated, but I promise it's not. So first, you give the tool a name. And in this case, we're going to call this `create_task` because it's a tool to create tasks. We give it a description, which is "to create a new task." And the description is actually really important because the LLM really looks at this description to figure out when should I use this tool, how should I use this tool. Very important. Do not overlook it. And then we give it the parameters. So these are the things that the tool requires to run. And in terms of parameters, these are all just things that you can put in an Ellie task. But the only thing that's really required, so you can see it's noted here, is the description. So at minimum, this is saying to the LLM, "To use this tool, you need to pass in all of these parameters, but the only one required is this description or title." So this is the first step in defining a tool. This is almost like the blueprint of the tool. This tells the LLM what this tool is for and what I need to pass in to use it.
Okay, so that makes sense. But we haven't actually defined the tool, like how does it actually run? And here's where we actually define the tool. I have a separate function called `execute_tool`, and it takes in two parameters. It takes in the name of the tool, which in this case is `create_task`, and then the arguments or those parameters that we defined for the tool. It's going to go through the code, look for a match in the name. So in this case, it's going to say, "Cool, `create_task` matches here. Okay, this is the code that we need to actually run for this tool." And then here, we're able to extract those arguments or those parameters that we defined, like the task description. And then we're actually able to run code. And here's the actual code to create the task, which in this case is really simple. It's hitting up a pre-made function that I already have to create a task in my backend. This part is going to look really different depending on what the tool is, but just note this is the actual code to create a task in my case. Again, it's going to look completely different. This is just how I defined it, but this is where you'd actually put the logic of the tool. And then, really important, you return the results of the call so that the LLM knows, "Cool, here were the results." So in this case, the result could be, "I created the task, you're good to go." And now the LLM can see that and then tell the user, "I've successfully created the task for you." Or maybe if it fails and something went wrong, that could be in the result, and it can then properly tell the user, "Hey, I tried, but it didn't work. Do you want me to try again or something like that?"
So, we defined the tools and we fed the tools into our call to the LLM to Open Router. Now, we have to do one more thing, which is actually handle that loop. So, I'm going to try my best to explain this, but the way it works is first, it's going to call the LLM with the tools, with the user request. Once Open Router is actually going to respond, and in the response, they're going to tell us, "Does this require a tool call or does it not?" Like, "Are we good? Do we have we already answered the user's question?" If it does require the tool call, now we're actually going to continue and start this loop. And what's important here is that I've actually made it so that we're counting how many loops there are, and I've defined a maximum of three loop calls that it can do. The reason I did this was for safety because in theory, this thing can actually just loop infinitely, which would then rack up a huge bill. I would go bankrupt and then no more videos. So I decided to set a hard limit of three loops just for safety purposes.
It's probably a little bit easier to visualize this with an example. Let's say that we're asking it to create a task to go to the gym today. So first, what's going to happen is we're going to send the request to Open Router to create a task to go to the gym today. We're going to send it all the tools that it has available, which is the ability to create a task. So the LLM is going to send a response back, which is actually going to contain that it cannot answer the question. It needs to call the tools, and specifically, it wants to use the `create_task` tool. So what's going to happen is we're going to go through here, and it's going to trigger this. It's basically going to enter this loop because it's going to tell us that it needs to call the `create_task` tool, and it can actually call multiple tools in one go. But in this case, it's just going to call one. It's going to loop through all the tools that it wants to call, which in this case is just `create_task`. It's going to execute that tool call, which again, is that function that we defined previously that takes in the tool name, the arguments, and then actually goes and does the tool call. It's going to call the tool. We're going to get the results of that tool call. So in this case, we're sending back to the LLM that, "Hey, I successfully ran the tool call, and the new 'go to the gym' task was created today." And then that follow-up response is actually also going to contain a, "Does this follow-up require tools?" and "What tools do we need?" The follow-up response is going to be something like, "We created the task. I have everything I need to tell the user. No more follow-up." So, "I don't need you to call any more tools." So, I'm going to actually just send back an empty `tools` array. And when it loops around and gets back to the top, it's going to see that empty tool array and know, "Oh, great. Okay, we don't have to call any more tools. Let's go exit the loop. We have everything we need to answer the user now." And it's going to exit the loop, and then it's going to return the final response message back to the user or back to the chat UI.
So hopefully you guys followed all that, but it does work. So now when we ask it, "Can you create a task to buy groceries today?" The agent is going to recognize it needs to create a task. It needs to use a tool, which is the `create_task` tool. So it enters the loop, calls the `create_task` tool with the right parameters. And then that tool sends a response back that says, "Yes, cool. We created the task." And then when the agent sees that that's successful, it says, "Great, I have everything I need to answer the user. Let's exit the loop." And then it sends this final response back to the user that, "I've gone ahead and created the task in Ellie." And now we've created our first real agent. This is an AI that has access to a real tool that can actually impact and create tasks in Ellie. And now the fun actually begins. We can then start creating even more tools to make this more powerful.
And so that's exactly what I did next. The next thing I wanted to do was see if it can handle more complex scenarios. So a great next test was the ability for it to update tasks, which actually required two steps. I needed it to first search for the specific task and then call a tool to update it. So two tools would need to be called for this. An example of this would be if I asked it, "Can you move my dentist appointment up 1 hour?" It should be able to, in theory, search for the dentist appointment and then update the task by moving it 1 hour. And in terms of the tool, really simple. So I defined a `search_task` tool and here's the description: "It's able to search for a task based on description, label, or date range." And then here are the parameters we feed in. The main one that's required is this `query` parameter. And in terms of the tool definition, super easy. It's basically calling this pre-existing search endpoint I already had in my backend that powers the existing Ellie search. So it was very few lines of code to implement this. `update_task` was the exact same thing. I created the `update_task`. Honestly, very similar to the `create_task`. The definition also leveraged the existing `update_task` endpoint that I already have in my backend. So again, just a few lines of code here too. But when you put it all together and you add those two tools to the `tools` array alongside the `create_task` tool, it's very powerful and just immediately worked. So when I ask it, "Move my dentist appointment up 1 hour," it first calls this `search_task` tool and it feeds in "dentist appointment." Once it gets the results of that, which is the matching task, it then has enough information to call the `update_task` tool and then it changes the time of that task to move it up 1 hour. Then when both tools are run and we got the results from both of those, it now has enough information to answer the user and it sends a message confirming, "I have updated your task for you." Since we already have the base layer, super simple adding those next two tools.
So let's dial up the complexity again. The next thing I wanted to do was see, can it handle calendar stuff? Because Ellie actually has access to your Google, Outlook, and Apple calendars. These endpoints already existed in my backend. So creating tools was super easy. I just repeated the same process. I defined them in my `tools` array and then I created the implementation, which was very easy. Leverage the existing backend code. So I gave it two more tools, which are "get the calendars the user has access to" and then a tool to "fetch the calendar events from those calendars." So now if I ask it, "When am I free tomorrow afternoon?" What the agent does in this case is it checks, "Okay, I need to get the list of available calendars. I need to figure out which one is the primary calendar and then return that. And then I need to then call the `get_calendar_events` tool that we just defined." And then it fetches the events for that primary calendar. It looks through the events and then it finds the next availability in the calendar and then returns it to the user in the chat. And it worked surprisingly well. So when I asked this question, it's able to correctly return when I'm free.
So now here's where things got a little bit more tricky because then the next thing I wanted to do was ask it, can it coordinate between two users? So if I ask it, "When should Siccilia and I go to the gym on Monday?" In theory, if I asked at this, it should be able to correctly pull my calendar, pull Cecilia's calendar, and then pull the events for both those calendars with the second tool, and then check where are the free gaps. However, for some reason, it was kind of struggling with this. I think it just kind of got confused because it didn't really know, okay, which ones are Cecilia's calendars, which ones are my calendars. I was hoping it would pick it up because technically the ID of the calendar has her name and my name in those IDs. I was hoping it would pick it up, but it didn't. So, I actually had to modify the system prompt a little bit, and I had to give it a little bit more guidance on how to handle that specific request. And so, here's the prompt when I said, "When a user asks you to find a good time or to check mutual availability, and here's some specific examples, I want you to follow this workflow." And I defined the specific workflow on what order to call the tools, how to distinguish between two calendars if possible. And it did take a lot of trial and error, but with this prompt, it was able to consistently start fulfilling this request, which is cool because this is actually an action that I do every single day. Instead of having to toggle both our calendars on and then look for a slot myself, I can just ask it. That genuinely does save a little bit of time each day. And then obviously, I can just follow up and tell it, "Yeah, can you go ahead and create that task for me?" And since we already have the `create_task` tool, just by doing that follow-up, it's able to create the task and correctly put it at the right slot in the calendar.
And the last thing I had it handled was the ability to time box your day. I actually thought this one would be easy because in theory, it has all the tools that it needs access to, but it was actually really challenging for it to understand. I think the problem was I was using GPT-4o Mini to do all this stuff, and it's a pretty small model, still very powerful, but I guess it's still not that good at handling things like creating an agenda based off multiple calendar events. So I decided to create a specific tool that calls a more powerful LLM that is able to actually handle that. And so now that's one of the tools. So that's another cool thing you can do, which is define tools that are actually just other LLMs that you can call, and you can chain them, and it just becomes really powerful to be able to orchestrate these complex workflows.
So as I promised before in the intro, this is where things actually got pretty interesting. So I could have stopped there, but I'd actually heard about something called Zapier MCP. If you're not familiar with Zapier, they are an automation platform that lets you hook up basically any app into any other app to create automations. So Zapier is really powerful. They're already hooked into thousands of apps. And the premise of their new service, Zapier MCP, is that it'll allow AI agents to hook into the thousands of apps that Zapier supports, which sounded really interesting to me as I was working on this agent. So, I wanted to see how easy that would be. And first, a huge thank you to Zapier for actually sponsoring this video. I'm a big power user of Zapier. I use it to automate a ton of workflows for my businesses. I've already created an Ellie integration for Zapier. But, if you're interested in automating workflows or, as you'll see, integrating Zapier with your AI agents, definitely go check them out below. I'll leave a link in the description. But thank you to them for sponsoring this video.
So going back to Zapier MCP, the premise is it allows your agent to connect with thousands of apps with very minimal integration code. Let me show you what the end result is and what this enables. So I've hooked up Zapier to my agent and I've authenticated with Zapier, and it now has access to any tools that I give it through Zapier. So in this case, I've given it two tools: the ability to search in Slack and the ability to send messages. So now what I can do is say, "Brian sent me a Slack message asking me to look over our server bills. Can you find it and make a task today with the details?" So, what it's going to do is build on the existing agent we made with those existing tools, but it's going to add these two Slack tools from Zapier automatically to that list of tools we have available with literally no work on my end. So now when I send this message, the agent runs through the loop and says, "Okay, I can't answer this question. What tools do I have?" "Oh, great. I have a `search_slack` tool." It's going to call this. It's then going to search for the Slack message from Brian. It's going to find this, and now it has everything it needs to create that task for me, and then it creates the task, and you can see it appears in Ellie, which is amazing because it was such minimal code to do this.
Here's another example of something cool. "You summarize all the tasks that I worked on today and send a summary in the Slack channel." I have a tool to get all the tasks in, but I also have this tool from Zapier to then send messages to a Slack channel. Let me use both of these tools. And now when I run this, it's able to get the tasks and it calls Zapier and sends the message to the Slack channel. You get the picture here. The implications are now, without having to build direct integrations into these tools, this agent can basically access anything. Now it has access to GitHub, to Linear, to To-Doist, to Slack, all of these things. So if I tell it something like, "Get all my Linear tasks for today and create them as tasks in Ellie," it'll automatically go ahead and do that. So I've heard about MCPs in the past. I'd tried them in the past, but this is probably one of the first genuinely useful cases I have seen, which is to supercharge existing agents by allowing them to connect to other tools. And again, with very minimal code, that is an incredible use case. And it just made this agent 10 times more powerful already.
But let me show you how I set this up in case you wanted to do this yourself. So, what I had to do to connect to the Zapier MCP server was to make my chat and agent into an MCP client, just like Claude and Cursor are. There's really good documentation online from Anthropic. So, I basically followed the JavaScript version of this, installed everything on my server, and now the chat is an MCP client, and it can call these MCP servers like Zapier directly. And here's the code where we're initializing this MCP client, and we're loading up the tools from the Zapier MCP server, like the Slack tools that we gave it. We inject it into the existing array of Ellie tools that we defined. And so now it just lives alongside those tools and can be called anytime. And when a tool call is made from Zapier, it sends the request to Zapier. They handle it, and they send the response back, just like we do with those other tools we defined in Ellie. It might seem overwhelming and daunting, but overall, I think this took about 30 minutes, probably under 100 lines of code, to be honest. And I was able to just feed the documentation into Cursor, and it was able to get it in a couple prompts. So don't feel intimidated by this. If you're building an AI agent, definitely go check it out. Try to implement it. See if it makes your agent more powerful. But it's something I'm going to keep exploring here. And I'm really curious to see what services users are using when I ship this to production.
To be honest, I think this feature is kind of done and ready to be tested. I'm going to open up a closed beta for a couple of Ellie users so they can test it and I can see how useful this is, find some edge cases. I would love to port this thing to the iPhone version of Ellie so I can just open it up. Probably add some dictation capability so I can just ask it, "Can you tell me when I should go to the gym with Cecilia tomorrow?" and then just have it go do it for me. I think that's the dream. So, I will be releasing this on the iPhone version as well, which I'm really excited about. But I am extremely impressed by how useful this thing is just after working on it for 2 days. I've tried to build an Ellie agent a bunch of times in the past, but every single time the LLM was not good enough to execute the tasks consistently, or it was just way too expensive to do this at scale. But now I think they're smart enough and way more affordable to the point where I can finally release this to users.
If you're a developer building an app and you've been considering building an AI agent, I hope that this video helps and pushes you to do it. I think this is a really cool interface that I hope comes to more applications. And if you're not a developer, I hope this video demystifies agents for you. If you like this content, check out my Instagram and TikTok. I post almost every other day about building productivity apps. And obviously, if you like this content, don't forget to subscribe. But thank you guys so much for watching, and I'll see you guys in the next video.
[Music]