Transcription
Did you know that Airtable now allows you to generate structured data using AI in your automations? This means that AI is getting smarter than ever before, and can be used directly in your ongoing processes to help you work smarter.
In this video, I'm going to show you exactly how you can incorporate the structured data element of Airtable's AI automations. So if learning more about this amazing feature is of interest to you, stick around and let's get into it.
Welcome back to the channel. If you're new here, my name is Gareth, and this is Gap Consulting. It's my mission here to help you unlock the full potential of Airtable and the related software and tools in the no-code space, so that you can bring new efficiency to your business and to your job.
As I said, in this video, we're going to be taking a look at Airtable's structured data output. This is specifically available in your automation section, and it's going to use some AI credits. But it is really powerful, as you're going to see here in this video.
But before I get to the heart of things, I first want to invite you to join me for some free training. We've built an Airtable crash course to teach you the basic building blocks of Airtable so that you can start really understanding the key components of Airtable and how they all work together. To grab that crash course, sign up at www.gapconsulting.io/airtable-crash-course. I'll include that link wherever you found this video.
But without further ado, let's go ahead and hop on into my screen. I'm going to start with just a brief background on what we're working with here. This is recipe tracking, and we care mostly about two different tables in this example that I'm working on.
The first table that we really care about here is the recipes table. You can see that we have the recipe name, the source URL (like where we found this), the PDF file. So this is the actual recipe file that's uploaded to Airtable. And then we are producing a lot of information about the recipe, like how many people does it serve, what's the prep time, what's the number of calories, what's the cook time, what's the cuisine type.
Then the next part is we're linking it to the different ingredients, or more particularly in our example, we link to a recipe plus ingredients join table. Now, this is a complex topic and Airtable that really deserves its own video, so I can't go into all the detail here. But rest assured that what this table does, recipe plus ingredients, it joins our recipe and our ingredient tables. It allows us to link to an individual recipe and an individual ingredient in every single record instance.
In the name of reducing the complexity here, let's just focus on the recipes and the ingredients. Ingredients, as you would assume, of course, are the ingredients that you use to cook with, whether they are produce or poultry or canned goods or whatever they are. So we're tracking information like the ingredient name, what's the category of the ingredient (whether dairy or something else), what's the default unit. So we could list out a bunch of default units like kilogram, gram, milliliter, liter, you know, all of the different unit types, come up with typical package size and also package unit. And of course, just as recipes linked to the join table, so too does the ingredients table. But again, for our example, to keep things simple, we're going to look at recipes and ingredients.
A new recipe comes in and we want to parse out all of the ingredient data so that we can work with it further in an automation. This used to be all but impossible, frankly. I mean, my goal here would be to say, hey, I just uploaded a brand new recipe. Here it is. And I want you to look through this recipe and find all of these different ingredients. And if they don't already exist in my ingredients table, then I want you to create those ingredients. That's a tall ask, right? Especially without AI. But now that we have this amazing new feature, the ability to structure our AI output within an automation, this is all possible. And it's much easier than you might even imagine.
So let me break down exactly how we turn on or activate this new feature in Airtable. First, we're going to roll over to our automations in our automation panel, right? By clicking on automation, we create a new automation and set up whatever rules you have for what you want to trigger your automation.
Now, in my example, what I'm imagining is we have a new recipe record that comes into existence, and when it's created, it will always have a PDF for that recipe in it. So if I can make this assumption, I can use the information in that PDF, have it analyzed by AI, and parse out that data as we've described.
What do I care about parsing out? Well, for my example, if we were to flip back to the recipe here, we look at our ingredients and we see that we need: number one, what is the actual ingredient. So here, for example, we might say lamb leg. Number two, what is the quantity or volume required? So in this particular example, we're looking at 1.5kg. And I kind of already wrapped in number three. Number three is what is the unit of measurement. So two and three really don't mean anything without both of them being present. We need to know what's the number, and also what's the unit of measurement. So makes sense, right? These are the components that we need. But I want to point out that we need to do this for every ingredient that we find in our recipe here.
So how do we do this? This is all done so easily with the new option to generate structured data in your automation. And note that this uses AI and it requires that you have AI turned on for your Airtable workspace. This can only be done by the admin or the owners of your workspace, and it will use AI credits. So bear in mind that you have a set number of credits. Depending on your plan, make sure that you're keeping that number of credits under your plan's threshold. I personally have not seen many people going over on their credits, so I don't think it's a huge issue, but it's something I do want to call out to make sure that you are understanding that you're paying for an amount of credits every month.
So how do we even get this structured data element here? Well, we can click here and go down to "Generate with AI." And it used to be that we would just generate an AI element within the automation, generate text, for example. We also now have the AI Labs option of generating an image, which is nice, but not what we're talking about here. I'm looking at this "Generate Structured Data," the new feature that is released within just the last few weeks. So make that selection, "Generate Structured Data," and then I'll show you exactly how I've set mine up.
Now, for my example, this is really important. I want multiple ingredients. Every ingredient in the recipe to be extracted in the AI analysis here. So the first thing I need to do is make sure to identify my output schema here on the right-hand side of the screen. And I need to use an object first. This allows us to communicate that we are looking for multiple things, and for each one of those things that we're going to identify, we are going to break down elements within.
So for our example, we're looking at pulling back a bunch of different ingredients. And for each ingredient, we need to know the name. We need to know the quantity. And we need to know the unit of measurement. Here is how we break that down beautifully. First, we say the thing we're looking for is called "ingredient." It's an object. Inside of each one of these objects, inside of the ingredient, we have properties. We're looking for the "ingredient name," and the data type. Here we have to select "is string." We want it to have a text name. Next, we are looking for the "quantity," and what's the data type? Of course, it's a number. And then lastly, we're looking for "unit." Note that we're putting in the element that we want on the left-hand side. We are responsible for naming this, and then we choose the type of data that will be produced. And here I've chosen "enum." This allows me to put out a list of different options, and I can set up rules inside of the automation so that it only uses these options. And I can even tell it in the prompt, "Hey, if you're not sure or if it doesn't match, just go with 'as needed' as kind of the default."
So first, what I've talked about here is putting together your output schema. And this is the important part, of course, the highlight of this feature. But we also have to write a very strong prompt. And this, of course, is non-negotiable. We need this every time we are using AI, whether that's in Airtable or elsewhere. It's all about the prompts.
So you'll see what we've done here is refined a prompt to say, "You're extracting ingredients from a recipe PDF. Here is that PDF file." We can reference anything that we know is going to have happened earlier in our automation. And for us, we already have assumed or stated that every new record as it comes into existence will have that recipe PDF. And if we can require that, make sure that it always happens, for example, use a form to create every new recipe and make sure that that PDF is a required field. That way we can say, well, we know we're going to have this data and we can reference it in our automation. And in order to bring in that information, just access any step that already happened in your automation and find the information you're looking for and drop it right in. As you'll see here, I've already brought in the PDF file, and this is where we're uploading that particular attachment. Then set up your prompt. And again, for me, I'm giving it rules. I'm telling it how it's supposed to work, what the allowed units are, what it's supposed to be pulling back. A lot of this I probably overdid because it already knows what it's looking for with the output schema, but I would rather give an AI prompt too much information than not enough.
Once you have this all built out, of course, choose your model and your randomness, and then be sure to go ahead and give it a test. So I've already tested this for the recipe that I already put in our table, but let's actually put a new recipe in our table, and I'll show you some other agents that are already going to be working in the background to extract this information.
So I've spun up a new one from BBC Food here. I found a new slow cooker Chipotle pulled pork, and I'm going to go to print this and save it as a PDF. And I'll just save it to my desktop. Now I'll flip back into Airtable, drop in here and add that new PDF. We'll find it from where I just added it and upload the file. It's going to take a moment to do that, and you'll start to see that other AI agents are already running. These agents I built to look at that particular PDF. They're not part of the automation because I only need them to run at the recipe level. I only need them to occur that one time for the recipe, and I don't worry, or I'm not worried about iterating multiple times in the way that I am over numerous ingredients.
So you'll see that we now have default servings that came out of here, also the prep time of 30 minutes, a cook time of 480, which is eight hours. Let's go ahead and see if all of that lines up. So we see here, cook over two hours. That's not really lining up now. But we do have prep of 30 minutes. That did line up. And it serves four to six. So I'm checking here. We see that this says it's producing eight servings. So I don't feel great about that. That didn't seem to come in right. Let's actually see if I can extract the recipe text. I'm sure that will be able to. Yeah, this is an easy one. Slow cooker Chipotle pulled pork. Great. So all of that got brought in. Some of these could be overridden. And this is actually a great demo because frankly, when we use AI, we do want to keep humans in the loop. It doesn't always get it right. The cook time here doesn't really seem to match what the recipe says. Let's take a look again. I mean, it says over two hours. I wonder if later in the... Oh, actually, look at this. It says, "on for on low for 8 to 10 hours." So, in fact, I feel like this part is a little misleading, perhaps. And the automation did a good job of finding that. So I'm curious to know how it came about using a serving of eight, because this says serving four to six. Our AI agent scraped eight from that. So we see that, you know, again, not everything is exact, but this is more accurate than I originally thought. We also nailed the cuisine type. I mean, a Chipotle pulled pork sounds like Mexican food to me. So this is coming together nicely, but let's look at what we're here for. It's all about that automation.
So we're going to use that trigger, but I'm going to do this manually because you'll note this automation is not turned on. I'll choose to actually use that record, and we're going to now be able to go into the next step and test it. Always do this with your automations. Scroll on down here, test the action. Make sure you get the output that you would expect. If you don't, you might need to go back in, tweak your prompt and make changes, especially when using AI.
So you can see the spinning wheel here. It usually takes, you know, five seconds or so to go through. And we have to click into the AI generated response. And what we're going to see is a list of objects. Each one of these is the next ingredient on the list. So we say, "Okay, ingredient one: pork shoulder, 1.5kg." Let's match that up to our data and make sure that we are on point. So 1.5 up to three, you know, three pound five ounce pork shoulder, okay. Next, "two tablespoons chipotle sauce." So I'm actually impressed by the fact that this AI did not get confused by the pound and ounces. So you can see that the recipe, you know, as it's written, says it's either 1.5kg or three pound five ounce of pork shoulder. And given that we have stipulated that we want to use kilograms, it brought it in that way. So really helpful. Next, let's check our next ingredient, which is "two tablespoons chipotle sauce." Did it extract it correctly? Ingredient two: chipotle sauce, two tablespoons. Perfect. So on and on it goes. And, you know, all the way down to flour tortillas. Of course, it doesn't have a quantity or unit. And that's probably because it has a range, as you can see here. And it's like, "Hey, throw these on when you're serving depending on what you need." So really helpful here.
But this is just the first step. Now that we've been able to extract this data in different objects like this, as each one its own ingredient with multiple data points, now I task you with building the next component of the automation for yourself. So what I would challenge you to do next is take this information and actually move forward in your automation with it.
Now, I had hoped that we would be able to take this data as a structured list and to move forward in the next step. So I did check to see, can we add this as a repeating group? Let's say I want to take an action for every ingredient. Maybe I want to look at every ingredient that I identified from the PDF from the recipe and say, "Does this already exist in my ingredients table?" If yes, let's connect it to the recipe. If no, let's create it and then connect it to the recipe. Well, I can't quite do that. If you notice, here I go into a repeating group and I choose an input list, and I don't quite have access to the generated structured data from that previous step, at least not when I treat it as an object as I did in my output example here.
But I leave this to you to experiment with and explore how you might get to that next step. For me, I think it's probably easier to take this well-documented structure and to write a script to go through whatever steps I'm looking for and iterate over those objects. But if you're not very comfortable writing scripts and or using ChatGPT or Claude to help you write your scripts, then this might feel a little advanced. But of course, don't let that stop you. The whole goal here is really to showcase what is now possible, and to open up new opportunities for you that you didn't have before Airtable launched this great new feature.
So that's it for this video. I hope you got a ton of value from this. If you have questions that we didn't quite answer, swing by our website. We do offer complimentary strategy sessions on our website. You can ask questions and we'll help you focus on your goals so that you can build your next Airtable project. Of course, if you liked the video, we'd love a thumbs up and subscribe to the channel. Really helps the YouTube algorithm. And of course, most importantly, keep on building.