Apify FULL GUIDE 2025 (Scrape Literally Anything)

Hey there, this is a complete beginner's guide to Appify. I, and many other people, have made hundreds of thousands of dollars using this tool. So, in this video, I wanted to run through Appify from start to finish to give people a zero to one roadmap to understanding and mastering the software platform.

Over the course of the next hour, we are going to cover actors, tasks, data sets, storage, the Appify library, all the bells and whistles on the dashboard, how to make an account, how to scrape. I'll cover webhooks, how to integrate Appify with no-code tools like Zapier, Make.com, and N8N. I'll even cover how to make your own Appify scraper using their IDE. And then we'll cover schedules, storage, proxies, and at the very end, even how to add your own custom actor to the Appify store and start making money with it.

As I'm sure you guys can imagine, this is going to be one of, if not the most exhaustive resource out there on Appify. So, if you want to become a professional using this tool and then take it and make real money with it, this is the video for you. Stay tuned and let's get into it.

So, as I'm sure you guys can imagine, this is going to be a very comprehensive video. As a result, we are going to cover a ton of ground. If you guys need to access some particular information, feel free to use the chapter subheadings below. Click on the spot in the video that corresponds most closely to what you're looking for and have at it. Otherwise, if you are with me for the long haul, let's get started with the beginning, which is the introduction to Appify.

All right, so why Appify? Well, I think if you guys have watched any sort of tool tutorial before, you'll know they always start with a section like this. I'm going to try and keep it as simple as humanly possible because I don't really think it's that relevant. You probably already know why the hell you're using Appify if you're watching a video on it. But essentially, Appify just solves a simple problem. It's in the name: Appify, API, FY. That first half is all about APIs, and Appify's whole promise is that it can help convert any resource out there on the internet into an API, even if it wasn't really meant to have one.

Think about social media platforms. If you think about it, what is Facebook or Instagram? Well, it's just a giant ass database. That may be a reductive way to say it, but it's a giant ass database with a ton of data points on every person basically on planet Earth, right? Facebook doesn't want you to be able to call an API and get data on any person on planet Earth. That's kind of the whole business model. Appify lets you make an API anyway and get that data, just through sort of a roundabout, unconventional means, which is, you know, scraping. Scraping Facebook pages. So that's kind of the use case of Appify. It just allows you to turn anything into an into an API, even if the thing wasn't necessarily meant to be.

So, I'm sure you guys can imagine that gives you a ton of leverage if you're in like the sales, marketing, growth hacking space. If you're doing any sort of like business use case where you need access to a structured data source, but they don't want you to have one. That's a great alternative to paying large sums, especially for structured data sources, especially those gatekept ones that I was mentioning earlier. Some people would charge you like astronomical, several thousands of dollars per month to access a resource. Why? Why do that when you can just make an Appify scraper in five minutes for 30 cents a day that does it instead, right?

So, another reason is because of how easy it is to get up and running with and how much value it affords to people at different levels of experience. I know how to build my own scrapers at this point. My own scrapers can do basically whatever the hell I want, but I find myself returning to Appify over and over and over again because it just makes my life a lot easier. As opposed to me having to start from complete scratch every single time I want to do this, I can just jump on Appify, check out their store, get something that works 80, 90% of the way, and then either massage that data or add some additional input parameters or something, and then get what I was looking for anyway.

The last reason is, um, the pricing is very, very scalable. When I say very scalable, I mean, you know, we're at the point now where we can get several thousand records. In my case, I usually use Appify for lead scraping purposes, right? Because I'm selling automation services and so on and so forth. So you can get several thousand records for like a dollar. It's crazy how high leverage using Appify is to get data. If you think about it, back in the day, you might have to pay somebody, you know, $5 to $20 an hour to go and scrape 50 records for you. Now you can just pay Appify, you know, a dollar, get a thousand, and have them perfectly structured and basically 100% accurate every time.

So, the purpose of this tutorial is to sort of take a slightly different look at Appify. You know, I care more about Appify for the purposes of making money, not necessarily for academic purposes. So these are the key features that make Appify worth using for me.

All right, so let's actually get started and dive into Appify. Just before we do, we need to understand three core principles. The first is we need to understand what an actor is. Then we need to understand what a task is. And then finally, we need to understand what a data set is. This tutorial is meant to be an active one. So I'm actually going to go and I'm actually going to scrape real resources. I don't want this just to be dry, academic theory. I want to show you guys how to actually use this stuff to make money. But in order to really get there faster, we just need to understand these three. So bear with me for a moment.

An actor is basically just Appify's branded term for a serverless program that runs in the cloud. Because we're using Appify, the scraping platform, um, these are almost always scrapers. I mean, there are different terms for this: a scraper, crawler, there's some browser automation things that like click on links and do things for you. But I'm just going to bundle all of this under the term scraping because that's probably how you arrived here. So that's what an actor is. An actor is just a program, okay?

A task is just one specific run of an actor. It's like an instantiation of that actor. It's the program running top to bottom and actually executing and delivering some output. What is that output? Well, that output is usually stored in a data set. And a data set is just a place where the data that is scraped from a resource, say we're scraping Facebook, it's a place where we go, we get that data from Facebook, and then we put it somewhere, and it's where you pull your information from later on. Um, and the cool part about data sets is Appify lets you do this in a variety of formats. You can store data as JSON (JavaScript Object Notation), CSV (Comma Separated Value), HTML. You actually have a lot of freedom to design this if you create your own scraper, which I'll talk about in a moment. But okay, that is enough of the dry, academic three.

Let's actually jump into creating an Appify account, then navigating the dashboard. Let's talk about a couple of things on there. Let's actually just dive right in with a real practical example, which is scraping a social media platform.

Okay, we are now on the Appify homepage. This is what it looks like as of the time of the recording of this video. It may look different when you are looking at it. As you see, one of the very first things that we have in front of us is a bunch of different scrapers and data extractors for different platforms. Obviously, they're doing this for marketing purposes, as is their landing page. They just want to show you that you can do basically whatever the hell you want with Appify, which you can. So you can scrape TikTok, Google Maps, Instagram, websites, Amazon. And then you can also build your own actor over here, which is sweet.

They have, as I've mentioned before, a marketplace of scrapers, which are these. A lot of them are extraordinarily low-cost and/or free. You can build your own, and then they also have some professional services at the beginning of the offer where you can basically like get connected to a network of developers, which is pretty sweet. This stuff isn't super relevant to us. How do you actually like get up and running and use the platform? Let me show you by making an account.

So, I'm just going to click "Get Started" over here and then I'm going to sign up with Google. Just going to use this email address over here. I'm just going to use that to create a new account. Okay, let's just enter our phone number. Let's click "Continue." This phone number is an open phone number. This is a service that I use to manage my business number. So let's jump back in here and then let's paste in the code. Confirm. Awesome. I'm going to set up my own profile now. I used to go by Nick Wells, which is why everything is Wells. I just thought that nobody would know how to pronounce my last name. I was correct. Nobody does. But it turns out it's way easier to compete on Google when you don't have the same name as a famous baseball player. So I went back to SAR. Awesome.

And now we're in Appify. We have the store in front of us, which is pretty cool. Now, first things first, I just want to walk us through a couple of things in the top left-hand corner. I want to walk you through this dashboard, at least just the buttons. We'll actually dive into every one of these later. And then after I'm done with that, and after, you know, like the 80/20, we can actually jump into like making a scraper and doing something cool with it.

So, first things first is this "Store." The store lists every single actor currently available publicly on the platform. Now, as you'll see, a lot of these, if you look at the URL, say Appify/something. So Appify website content crawler, Appify/Instagram scraper, Appify/Facebook post scraper, right? Appify Cheerio scraper. You'll also notice that some of these say different words: Compass/Google Maps Extractor, Clockwork Free TikTok Scraper. What does any of this mean? Well, Appify develops a number of these themselves to make it easier for people to get up and running with their program. So a lot of these are actually developed by the Appify platform. But a bunch of these are also developed by people that are regular human beings like you or me, or companies that hire developers and use them to build these sorts of flows to make money off of them later. And I'll cover all the economics momentarily. But essentially, this is Compass. This company is Clockworks. This company is Streamers. This one is Epictax. And as you guys can imagine, this is just this is just an open marketplace. These are all scrapers that have been developed by other people that you can then spend a little bit of money on, so you can compensate these people to use.

The good news is, when you make a new Appify profile, you'll see down here in the bottom left-hand corner, it says "Free Usage: Zero out of $5." Well, they actually provide you a whole, like, every month, they give you $5 of free usage with their server. Meaning, a lot of the time, you can use these completely free. $5 is a lot of money on Appify, and it's money to like actually go out there and do like tons of real-world things. You know, scrape hundreds, if not thousands, of listings on one or more of these actors.

So, I'm not going to cover every single one of these, but I just want to show you guys the magnitude available, just the sheer diversity of them. LinkedIn Jobs, Facebook Reviews, Indeed, Google Trends, Zillow, Zip Code, Amazon, TikTok, Trip Advisor, Facebook, Airbnb, Vinted. Like, you really can scrape virtually anything else. And so, as just a rule of thumb, anytime that I'm starting a new scraping project, before I actually go through all the rigmarole and BS of building the scraper, I will always jump on Appify first, just search it, and then see if somebody else has done the legwork for me. If so, I can usually just use that, and it saves me hours upon hours of actually having to go through the rigmarole of putting one together myself.

Okay, so that's that first tab here, that's the "Store." The second tab is your "Actors" tab. Now, there are two ways to get actors into Appify. The first is you can use one of these store actors, as I just mentioned. And the second is you can build your own actor. We're actually going to be building a very simple actor using ChatGPT and some basic Node.js JavaScript later on in this video. But as you can see, we could either browse actors in store or build our own.

What happens when you browse an actor in the store? So I'm just going to go over here, I'm just going to click "Save." I'm going to go back to our actors. Is you'll see this has now been populated with the actor that I was just using. Google Maps Extractor is now actually in our little dashboard here. The reason why they do this is it just lets you save your configuration settings, and I'll talk about the specific configuration settings in a second. So this tab is just used to list all of the actors that in your account. Nothing special.

Next up is your "Development" tab. This allows you to see the actors that you are creating. So you can actually create an actor. They'll guide you through a big list of different programming languages, libraries, and approaches. I'll show you guys what to do with this momentarily, but you just give this a click and then work your way through, and then you'll have some stored actors in this tab.

Then you have your "Saved Tasks." Essentially, as I mentioned, a task is a specific instantiation or specific run of an actor. And so what you can do is you actually save all of the configuration details of a specific run to this tab, and then you can just fire them off one by one anytime that you want.

You then have a "List of Runs." Every time you run this, it's sort of like your log here. You'll see their outputs, you'll be able to click on specific ones and then.

And then finally, you have an "Integrations" tab. The Integrations tab is pretty sweet. You can basically take actors and then you can connect them with other services in order to automate things. You can put things on a schedule. You can set it so that every time this thing finishes, you get a Slack notification. There's really almost an infinite number of different things that you can do with this.

Then we have a "Schedules" tab. This just allows you to schedule and run actors at specific times. So you can create schedules for daily, minute, hour, weekly, monthly, or even yearly. You can set specific configs here where you want to do it on specific days, last days, you want to do it Monday, Tuesday, and Thursday. There's a variety of I want to say granularity here with which you can use to schedule things.

Then you have "Storage," which has data sets, key-value storage, and request queues. A data set is just that simple store that I mentioned earlier where we have JSON, JSONL, CSV, Excel, HTML. You have all these formats basically that Appify will come up with and allow you to build things in. Then you have some key-value stores. This is basically, you can think of it as, I don't know, like a like a NoSQL database essentially, just a big list of key-value items. Then you also have a request queue. This is specifically used in Appify's web crawling feature, where basically when you run one of the actors or a variety of actors, they'll scrape all the links available on the page, and then you can just add them to a big request queue, and then it'll call this request queue constantly to figure out what that next link is to access. So more on all of this detail later. I just wanted to give you guys a top-level overview of what all these freaking tabs are on the left-hand side, because personally, when I start a new program or a new platform, I always just like not to feel super overwhelmed. I always like to be able to like look at something and be like, okay, I think I know what that does.

Next is "Proxy." If you're unfamiliar with proxy, essentially what this is, is just a way to mask where the request is coming from. As I'm sure you guys can imagine, because Appify does a lot of things that aren't necessarily allowed by the terms of service of the platforms that we are requesting, a lot of the time these platforms have it's sort of like a cat and mouse game between the platform and Appify or scraping tools in general to try and figure out, well, where is this request coming from? Is this request a legitimate user of our service, or is it just somebody trying to scrape us? What proxies allow you to do is they basically allow you to mask or hide your request, pretend that it's coming from some different place or resource. In our case, a different IP address or different home or a different mobile phone. And then, as a result, you know, we can access the data a lot easier.

Then you have a "Settings" tab here. This is just pretty self-explanatory. You can change your email, username, you can add a little profile here, add a bunch of your programming stuff like your GitHub homepage. You can change the theme. I'm a big fan of the dark theme personally. Hell yeah. Two-factor authentication, session stuff. You have access to your API tokens over here in "API and Integrations." This is easily one of the simplest ways to connect to a service that I personally found. I mean, I use Make.com and N8N and Zapier, and all you need to do in order to like send a request to Appify is just click on the Appify module and just paste in this token. So it's really easy.

"Organizations" - you can actually join a bigger company here, just as a user, and then you can set your notification and privacy settings over here.

The last thing that I'll touch on is just this "Billing" tab. Now, Appify bills in four different ways. It bills you based off of actor compute units, which is the amount of like compute, the amount of CPU or or or runtime basically on the serverless cloud program that you are instantiating. You have data transfer, which is data sent to and from Appify. You have proxies. So proxies are billed through Appify if you use their Appify proxy tool, which I recommend for simplicity. So they'll charge you based off that. Then you have storage. You can see that they store, they charge you based off of your data sets, your key-value stores, and your request queues. So this is sort of how everything is managed price-wise under the hood. But as you can see, our free usage is currently $0 out of $5, meaning we have five whole bucks of free usage, which is pretty great. The $5 is usually more than enough to do whatever you want to do with this.

Now, we also have access to historical usage dashboards. You can actually see on a daily basis where all of the dollars that you are spending on this platform are coming from, which I really appreciate. A "Subscription" tab that allows you to obviously sign up to a variety of different packages, right? Free, Starter, Scale, Business, Enterprise. Some information about pricing. Some actors are priced at different dollars per compute unit, I believe. You also have proxy costs. All these things change over time, and so this is just a way to see them at any given point. "Invoices," and then you can also set some limits so that you know you don't go over spending $10 a month or something, or Appify notify you if you're approaching your limit.

All right, so I think now by this point, you guys probably know everything you need to know in order to make Appify at least make sense to you from a bird's-eye view. Let's actually go through now and let's run an actual scraper. Let's get real data from a social media platform. Let's just let's just take a look at how all this stuff works. Let's actually run through a practical use case, an example of using Appify to do something for business.

All right, so let's actually build out a real scraping application. What I'm going to do in this case is I'm just at the store here. I just clicked "View All." I'm just going to find something that I could use. I think in my case, let's just start with Instagram because this is built by Appify, it's probably the most supported and is very simple per-unit based pricing, which I think everybody here will appreciate.

So, in order to use this, when I click on it, you'll see there's some additional information. First of all, we have the title. Then we have the rate. Now, not all actors are paid in the same way. This one is a specific type of payment plan called "Pay Per Result." As you see here, it says "$2.30 per 1,000 results." That means basically every result that comes in is going to be, I don't think my math is super good, but it's going to be what, two cents? I think 2.3 cents or something like that. I think 2.3 cents. Yeah, no, it won't be 2.3 cents, it'll be 0.0023 cents. I think. The idea there being that, uh, I'm so bad at this, I'm actually going to run the numbers. Okay, it's 0.0023 cents. So it is 0.23 cents. As I mentioned, 0.023. So it is 0.023 cents. As I mentioned. So you can tell I clearly need my coffee. Anywho, what that means is every result that you get, you'll have to pay 23 cents on, which I consider very fair and reasonable pricing. You're getting a thousand results, pay two bucks for that. That's pretty sweet, considering that a single result can do great things for it, right? Even one result can make you thousands of dollars if you're doing this for some sales reason. But anyway, so you get the pricing up here, you get the number of monthly users here, you get the number of stars, and then you get the person that it's crafted by and maintained by. I usually look for things that have more than maybe a hundred or so monthly users. If you have fewer than a hundred monthly users, a lot of the time it's just like a new scraper that was developed by somebody that's just getting started in this space, and it's a little, a little bit less reliable, a little bit less trustworthy.

Regardless, let's actually go through how to how to use this stuff. The first thing you'll see on the left-hand side is this big "Input" tab. So you'll see here we have a manual mode and JSON mode. So I'm going to use manual just for the purposes of this demonstration, but I want you to know that you could also just like feed in JSON or JavaScript objects directly into Appify as input for any one of these scrapers, and it'll work. That's actually how you communicate with them through API, which I'll share to you later as well.

So you can see here, right, "Instagram URLs" you want to scrape. This is actually just this element here, "Direct URLs," and it's an array of URL objects, actually just URLs in strings. So you just list a bunch of Instagram URLs here in this tab, and then it will go, it'll scrape a bunch of information from that for you. You then have a few additional settings. What do you want to scrape from each page? Do you want to scrape posts, comments, details, or profile? Post, hashtag, replace, profile, mentions, profile, Reels. In my case, I'm just going to stick with "Posts." What are the max number of results per URL? 200. Let's just do 10 for the purposes of demonstration. Then you also have a little filter here that allows you to go "Newer than." So you can select only things that are newer than, I don't know, 2024-03-10 or something like that. Let's do 2023-03 and then let's do 10. So now we're only going to be scraping results that are newer than March 10th, 2023, from this Humans of New York Instagram page.

The purpose of this demonstration, let's actually jump in here and let's just see when sort of posts we have. Most of these look to be photos, which are quite nice. And yeah, looks like we have a bunch that are going to be before or after, rather, December or March the 10th, 2023. So we should have access to a lot of these. As you can see, we're going to have some photos here, we're going to have, obviously, like a big caption, we're going to have some additional things like comments, posts, and so on and so forth.

So let's actually pump this in and let's see what sort of data we get. In order to do so, just click "Save and Start" down here. This will now run the actor. And what's happening when we do this, and this isn't going to happen instantly because we're actually literally spinning up a server and sending it, but this is going to be scraping that resource that we called. It's then going to be using whatever the logic or the programming, whatever the code in that specific scraper is for, or or or was built for, it'll be using that to extract data, and then it will add that data to a data set that Appify just created for us that is now linked to this actor as well. And as a result, we now have a link or a list, I should say, of various posts here with all of these columns.

So what I'm going to do, just so we could see everything, so I'm just going to go over to JSON here, although you can also view in tabular, but I'm just going to jump it into JSON and then show you guys the data that we just received from Instagram. Keep in mind that Instagram does not have an API that allows you to do this stuff, right? We just created our own API essentially.

So this first one is, you know, we have an ID of the Instagram post, we have the video type, we have the short code, we have a caption. I kind of miss mail attention, interesting hashtags, mentions. We have a URL, like an actual link to the post. If I just copy this and then paste this in here, you'll see that you actually have the whole video. This looks to be one of the more recent ones. You see that there's a comment. We actually have access to the first comment on the post. Then we have a big array with a list of other comments. I mean, I think you guys are already starting to appreciate, but we could do a ton with this data. Like you could theoretically, and this is just literally off the top of my head, you could scrape pages that have above a certain number of followers. So find a big list of pages that have over a million followers or something, and then get all of their data, scrape the most, I don't know, five most recent posts, get like the caption, get the photo, maybe feed the photo through AI, get a couple of the comments and stuff like that, and just use that to recreate the same sort of content and then repurpose it and use it to grow your own Instagram channel. This is something a lot of people do right now manually. There are whole teams that are just tasked with scrolling through famous people on Instagram or TikTok or various social media platforms and using them as idea farms. You can just do this completely autonomously now through Appify. You can also scrape posts from people. You can scrape comments from people. You could look for comments that say certain things and then respond to people, send them DMs or something like that, work through other automations and platforms. I think you guys can appreciate just how unlimited the options are here. And we received quite a ton of data for something that really just took like two seconds. As you can see, we got 10 results. The price was 2.3 cents, so 23 cents times 10. Not bad, if I do say so myself.

Okay, so that's the Instagram scraper, and I'm going to show you guys how to use these to actually go out there and then do cool stuff with it. So I'm going to show you how to connect this to like a no-code platform like Make.com to run one of those AI flows that I talked about. Show you guys how to use webhooks to pump this into anything you want, whether it's N8N or Power Automate or whatnot. But before then, let me just run through a couple of other ones and let's just see if we can get some more interesting data.

So now, maybe instead of trying to scrape Instagram, what I'm trying to do is I'm trying to scrape Twitter instead. As you see here, we have this "Tweet Scraper V2," "Pay Per Result," "X Twitter Scraper." And then you can see we have "40 cents per thousand tweets." Instead of being $2.30 cents per thousand records, we're now only spending 40 cents, which is pretty sweet. And what I want to do is I just want to read through this thing top to bottom. It looks like what we're doing is we are adding search terms here. So jumping over to Twitter, we are adding whatever search term that we want. So let's say "automation." And then we're getting all of the posts that show up for automation. From that, we're also going to get a ton of data on each post. We're going to receive, I imagine, probably like the post text itself, the person that like made it, you know, probably some comments, stuff like that, and we'll be able to do some cool stuff with it.

So it looks like all these fields are optional. Looks like you can do one or the other. You could scrape URLs, so we could just feed in a direct URL. We could feed in a search term like "automation." That's probably what I'm going to do for simplicity. You could also feed in Twitter handles if you wanted to. So maybe scrape all the posts from Elon Musk or Taylor Swift. Then even you have the ability to pass in conversation IDs. So the only thing I want here is I'm only going to want, um, "automation" just for the purpose of this demonstration. But I think you guys can see, you know, you have a lot of latitude here. If you wanted to like try and repurpose Elon Musk's tweets or something, or have some sort of like Twitter watcher or something that takes, you know, Elon Musk's tweets every five minutes and then tries and and and does something cool with them, you absolutely could. You can do the same thing for a variety of different approaches: start URLs, search terms, Twitter handles, conversation IDs, and so on.

We for a maximum number of items on output. I don't actually want to pay 40 cents, so why don't we just do 10 again? And then what we're going to do is we're going to sort this by "Latest." Then we have access to a variety of other filters. So we could filter tweets by their language. We can only make verified users pop up, only Twitter Blue, only image, video, or quotes. Maybe you wanted to run something that extracted images and then edited those images slightly, maybe reversed them, and then uploaded them to some other service. Maybe you wanted to only do quotes so that you could extract the text and then try and repurpose that text for some other purpose. You can also do what looks like a "Query Wizard," which is their advanced sort of tweet finder tool, where you can filter by author, reply to, mentioning, geo tags, near, and so on and so forth. So they just filled all the stuff in just so we could see what this looks like. But I'm just going to exit out of all this because I don't want to constrain our search with any unnecessary filters.

Then finally, we have some "Advanced Options." And this is what I was talking about earlier, where you can get 80% of the way there just using a pre-built scraper and then just modifying a little bit in code. We're not going to do any advanced options here. What you can do is you could take the output object and then you could map it, you could do something different with it, you could add strings, you could filter things. If you know a little bit of JavaScript or Node, those skills go a long way. There are a variety of pieces of documentation that sort of teach you how to do this. So I'm not going to dive into that aspect on my own. But at the end of this video, I'm going to cover some places that you can go to learn more about how to do things like write little code snippets using Node and so on and so forth to do stuff like that.

Okay, great. Now that we have all of our settings, let's actually pump this up to 20. I'm happy to spend, I don't know, 4 cents on this, however little it is, and let's see what sort of data that we get. As you can imagine, what's happening? Well, bottom left-hand corner, you'll see this memory tab just shot up from 0 to 256 megabytes. The reason why is because we actually just spun up a server, a 256 megabyte of memory server, that actually went through Twitter and then scraped, you know, all of those posts. Again, we didn't need to know how to build a scraper, we didn't need to know any of that stuff. All we needed to do was set this up with a search term "automation," and it handled all of the BS for us, and it did so for 0.04 cents in total.

Now, one thing that always confused me when I started with Appify was this "Output" tab. You'll see here there's "Summary Data," "Check Storage Tab for All Fields," and then there's "All Fields." One thing that I always used to confuse was, I used to think that all the fields that I'm seeing here were all the fields that I got. But in reality, if we click "All Fields," you'll see there's actually way more data here. We basically doubled the amount of data that we managed to get from this and included a bunch of additional columns. So, for instance, we have 29 fields buried under "Author." So we have type, user, username, we have URL, Twitter URL, ID, name, is verified, profile picture. If I just check this profile pic really quickly, let me show you guys that you can actually do basically anything with this. Granted, this is a very small profile picture, but it looks like two stickmen, one walking up this little pillar that says GC. You have descriptions. You could use this to copy profiles. You could use this to run some competitor analysis in a space. I mean, like the ideas are virtually limitless. It looks like buried under a card, you have a full photo images, which is pretty sweet. You have various entities, and so on and so on and so forth. This is pretty cool, right? You know, because you have access to the links to. You can run some sort of QA thing where actually jump back in, make sure it works. Looks like this was some hiring post for a company called GCMC that was hiring for a cyber security automation engineer. Speaking of hiring, you can totally run campaigns that like just look for the term "hiring" and then get all of the posts, find, you know, add them to a big spreadsheet, then maybe you or somebody else just goes through that spreadsheet once a day, clicks the links, and then submits a CV. So very, very many creative ways to get up and running with Appify. And I think the purpose of me doing this brainstorming live is just to show you guys how many possibilities there are, how many ways you can augment your own workflow with the results of an automation like this.

Let's see if we could scrape something else. So we just scraped Instagram and then we scraped Twitter. Why don't we try scraping Google Maps? Let's say we're looking for, I don't know, we're running some campaign and we want to send a bunch of people emails. Specifically, we're running a campaign to sell some project management system for restaurants, maybe some order management system. Well, the search term we're going to supply here for "Google Maps Extractor" is "restaurant." We're then going to look for all restaurants within a relatively populated place, New York, USA. Say, let's say we only want to search for 10 places per search term or URL, because I don't want to bump that usage up too high. It's already two whole cents. The variety of filters I can do here, and I'm not going to cover the filters because, I think as you guys are probably appreciating, basically every scraper has different filters because you, as a developer, get to choose what you want to add. But I'm just going to click "Save and Start." Let's just see what pops up here.

In the bottom left-hand corner, you guys just take a peek here. You'll see that we are now spinning up a server. As opposed to 256 megabytes, the server is now 1 gigabyte. The reason why some servers are bigger and other servers are smaller, or the reason why they have more or less memory, is just because some platforms have stricter scraping controls, and so in order to scrape a resource, you need to do a little bit more work, you need to have a little bit more running under the hood. Anyway, it took us something like 18 seconds from start to finish to get all of this data. And let's, let's take a look at what we actually have. First thing we have is we have the image URL of the restaurant. That looks pretty tasty. Looks to me like some diner or something. We have the total score, we have the reviews count, we have the street address, the city, we have the state, country code. Um, we even have the website for all restaurants with websites. We have the phone number, we have the category name, we have URL, and so on and so forth. And keep in mind that I'm just using the "Overview" field here, right? There's more. There's "All Fields." And as you can see, we now have substantially more data on this.

So what could you do with this? You could just very quickly and easily write up a little program, or in no code, do some dragging and dropping of modules and find only places that have a review score, let's say under four. And then maybe you pump them into some cold call campaign and you call them and say, "Hey, you know, I noticed your reviews are under four. I wanted to solve that for you with my special thing." Or maybe you look for restaurants that have above a certain number of reviews or something, use that as your filter to say, "Well, this is probably a pretty big restaurant. They probably have a fair amount of money if they have that many reviews." Let's see if we could go to their website and then get their email address, get their phone number. A lot of the time, you are going to have their phone number, but a lot of the time this is just like the main restaurant phone number. If you want to contact, let's say the general manager or something, you need to go a little bit deeper. But I'll show you guys how to do all that, um, momentarily, using this data source, because this is one of my most commonly used data sources to find phone numbers and email addresses of small to medium-sized businesses, and I want you guys to appreciate just how valuable Appify can be for that purpose.

Okay, great. So we did Google Maps, Instagram, we did Twitter. But there's a variety of other things that you can do here as well. Let's do one more. Let's see Facebook Ad Library scraper, Google Search Result scraper, TikTok video, TikTok scraper. Hmm, so many options. Facebook Ads, Booking.com, Indeed. I like the Indeed scraper, just on that same note of us looking for jobs. You can look for positions, let's say "automation." Or actually, let's just type in, I like "automation." We'll do "automation" in the United States, specifically, why don't we do Austin? And then we'll just return 10. As you can see, this is five bucks per thousand results, so we should be, this should cost us another five cents or so, I believe. Um, we can "Follow Redirects for Apply Link." This is interesting. It looks like the scraper actually has the ability to follow any redirects to find the link to apply directly to the job. So if you were a smart, enterprising individual looking for work and you had, you know, a couple of minutes available and $5 free Appify usage, you could totally search for jobs in automation and apply to them in basically an automated way.

Okay, so let's click "Save and Start" and let's just see what happens. See what sort of data comes out the other side. As you guys can see, this particular scraper has a memory of 4 GB. Why? Well, because this one probably needs substantially more resources in order to, you know, start and finish. A lot of the time, when you scrape these resources that really aren't meant to be scraped, like Indeed, like Facebook, like Google, they have very strict controls and protections in place that prevent you from actually going that extra mile. And the only way to get around them is by spinning up, you know, a server with much larger memory, doing a lot more, essentially. And that's basically what's going on here. That's why this is 4 gigabytes, whereas the other one was 1 gigabyte, and the one before that was only 256 megabytes.

Okay, great. You'll see that our free usage just jumped up to nine whole cents. Whoop be. Let's look at some of the data that we got from this job scraper. The first thing that we see is the position name, then we see the salary, we see the job type, company rating, number of reviews, even see a URL. We get company info here. If I go to "All Fields," you see we get substantially more. We actually get descriptions. Wow, so much you could do with that. You could pump that into artificial intelligence. You could use that artificial intelligence to write up or customize your CV based on these descriptions. That sounds like a pretty fun thing. Let's actually do that as an example. Different job types, full-time, contract, and so on. Position names, locations, obviously all of them are in Austin because those the parameters that we set. Posting name, posting date, rating, reviews, salary, scrape at, search input, URL. So I'm not going to make this the whole focus of the video, obviously. I don't want you guys have to necessarily just run through every single one of these. That would take me probably several hundred, if not a thousand hours. Um, but I want you, I wanted you guys just appreciate the diversity of scrapers available and the various things that you could do with them on Appify.

All right, now you know everything about Appify that you need to know in order to actually get out there and make real money with it. So that's what I want to cover next. What do you do with this data? How do you take this data and then use it for some actual business-capable purpose? So what I'm going to do is I'm going to be walking you through a couple of quick and easy ways to connect Appify with a few third-party applications. I'm going to show you how to connect it specifically to one called Make.com, very simple no-code builder that I use a lot. I'm also going to show you how to get the outputs as a webhook, which technically allows you to take that data and then put it anywhere on any platform, any server, any script that you're running, and so on and so forth.

For the purpose of demonstration, I'm going to be using two simple examples. One is our restaurant example from earlier. So we got a bunch of restaurant data in New York, right? I'm going to pretend that we're actually running a campaign where we're cold emailing restaurants with some copy, right, with with some pitch. I don't know, just made this new cool order management software. We're just trying to get

It out to as many restaurants as possible. We show you how to use Appify to scrape that data and then use the various fields that are returned to enrich the website to get an email address. And once you have the email address, I'm going to show you how you can use AI along with those Appify data fields to customize the cold email and then actually draft one in your email inbox.

The second is going to be, um, based off the Indeed data that we got with the job listings. We're just going to pretend that you are looking for a job right now and you want to build a system to help you. So this automated system will scrape jobs using Appify. If the job has a description, like a detailed write-up of what the job is looking for, we're going to feed that into AI and have it customize a resume template for us, then create a Google Doc, and then ultimately queue that email up as well.

Um, I bring these up to you not because I think that these are the most perfect examples of the coolest and highest ROI systems. I've built thousands of way cooler systems, well, hundreds of way cooler systems in this, probably at this point. Feel free to check my channel if you want examples. Um, but just to show you guys using the actual examples, like using the actual runs of the data that I just showed you a moment ago, how many cool things you can do. I came up with these in five minutes. If you spend a lot more time and energy on this, you'll probably be able to come up with substantially cooler use cases and ultimately do things that make a ton of money.

Okay, so let's start with our very first example, which is to get a restaurant, check to see if the website exists. If it does, enrich it, um, AKA get the email address. If you're familiar with the parlance, customize the cold email and draft the cold email. So I'm going to head back over here and then what I'm looking for is our Google Maps extractor, right? We're looking for restaurants in New York. And instead of, you know, what, 10 or something, I'm going to search 100. I'm going to click save and start.

Now, what I'm going to do is I'm going to go to my no-code tool. It's called, and I've actually already built this out just a second ago. Um, the second time I'm recording it, unfortunately, my first recording did not include any audio. And then what I'm going to do is, and I realized that I'm actually going to board this because I don't want to run it yet. Um, what I'm going to do is I'm going to show you how to take Appify and then connect it with Make.com, which is a third-party no-code tool, one that I use very often. I teach people how to use. So go Appify restaurant, see email system. And the very first thing we have to do is click on this purple plus button. Like, in order to get here, you need to go to, let me just show you guys for the purposes of illustration. You just log in to Make.com and click create a new scenario up here. And then you'll be tasked with this, uh, very intimidating purple button. Just type in Appify here. What you're looking for is Watch Actor Runs. Give that a click.

What you need to do is you'll need to create a connection. So I'm just going to go restaurant example under webhook name. Then for connection, I'll click add. And I'm just going to enter my email address here and then have API token. Because remember back when we went inside of Appify settings, then API and Integrations, we saw this personal API token section. Well, all I need to do is copy this, then head back over to the scenario in question, which is this one, paste that in, and click save.

From here, this will now be connected to our account. So Appify and Make.com, this third-party no-code platform, these are literally now connected together. You can shuttle data to and from them. One piece of data that we've gotten is we've gotten a list of actors that have been used on the account. So Google Maps extractor, Indeed scraper, Instagram scraper, Tweet scraper. What we want is Google Maps extractor. So I'm going to click save here, then I'm click okay.

And now what I'm going to do is I will go back to my actor, Google Maps extractor here. Now I'm going to click start. And then inside of Make, I'm just going to click run once. And what this is now going to be doing is this is currently, as the title suggests, watching the actor run. So as the actor runs, as the memory gets spun up, the free usage goes up alongside it. We're now going to be sending a completed status update to Make.com with all of this information. So it's currently scraping. I see that it's just about done. And that data, if you guys were very, very astute here, you'll have seen that we got this little check mark in the tab before we got the data in Make. So there's about a second delay or so. But we have a status message along with a bunch of other data columns, your ID, actor ID, user ID, started at, finished at, blah, blah, blah. We got the status message, say the scraping is finished. You can view all scraped places laid down on a map here. You can take some time to load for fully large data sets. That's cool.

Okay, great. So we've now determined that we can send data from Appify to Make. Well, can we use Make to get data from Appify? Can we kind of go in the other direction? And that's actually sort of the, the default way that Appify likes you to communicate with its API. Let's go back to Appify and what we need next is we need this Get Dataset Items module. So I'm just going to drag that over and connect that. And what we do is, if you click on this, you'll see it says Dataset ID. This is the ID of the dataset that we just dumped the data to. And in Make.com, this is just an empty field. We basically need to fill it with something.

Okay, so really, what we're looking for is we're looking for this variable right over here. It's called Default Dataset ID. You can see the last run, the value is ZB012ZT, blah, blah, blah. Now, the issue is, if I just click on this, if I just leave, leave everything else, then click okay, and then I run, well, now Make.com is just waiting for the actor to finish running again. So we don't actually want to do that. What we want to do is we just want to, we just want to grab the default dataset ID and then we just want to hard code it. So the way that you do this is you head to Storage and then there'll be this Dataset ID variable here, ZB012ZT. And then in our case, we're just going to hard code this directly in here so that every time we run this module, it's just going to grab the data from the dataset ID. This is the difference between testing and like production-ready flow. And this, in our case, we're just testing. So I just want to grab the same data every time.

Okay, so now I'm going to click run once and let's see what happens. What you'll see is we have the search string, restaurant search page URL, we have the title of the restaurant, category name, address, neighborhood, street, blah, blah, blah. We even have, uh, yeah, it looks like we have a ton of data on this, honestly. We even have stuff like the, um, reserve table URL. You can literally reserve a table at the restaurant. If I scroll through here a little further, you'll see that some of these have descriptions. Comfy diner with counter stools, serving American eats from all-day breakfast to sandwiches and steaks. There's a lot of value here, basically. And this is just shuttling the data directly from Appify into our Make.com instance.

Okay, great. So what's the next step? Well, now that we have a bunch of data on this, the question is, like, hypothetically, if I wanted to whip out this cold email campaign, there are a couple things I would need. Like, I would need, for one, I would need an email, right? So let's tackle that next. In Make.com, it's pretty easy to get emails, um, as long as you have, you know, some loose change. Uh, very simple platform that I like using to get emails is one called Anymail Finder. So you'd have to sign up to Anymail Finder for this, but basically, what you do is you pump in the domain name, and then it goes and it scrolls the internet and looks, it trolls the internet and looks for an email address at this domain name. And if it finds one, then it returns one to you.

So I'm going to use the Search for Company's Emails module to show you guys how easy it is to get an email address. It's so straightforward, unbelievable. I'm click Search by Domain, Email Verification Filter. I'm just going to use all found emails. And then for domain, I'm just going to type in the website here. Now, I don't want to return all 100 of these because I don't actually want to consume a 100 of my credit. So I'm just going to set the limit to five. And then I need to add a filter in between these two modules, which is just going to be, does the website exist? So I'm going to go over here, Find Website, and then under this filter operator, I'm going to say basic operators, exists. So now I'm checking to see, does this website exist? If so, continue. If not, don't.

All right, sweet. Now I have everything necessary for me to actually test this. Looks like I had, uh, some error with authentication. I imagine that's probably because I was using the wrong connection there. My bad. So now we're actually running this. We're actually going out, we're finding email addresses, or we're attempting to, anyway. This system can take however long, you know, the number of domains that you're feeding into could take. So in my case, I'm feeding in three. We'll see how long that this takes. On the first run, looks like it took like 5 seconds or so. Looks like we are mostly receiving true results, which is nice. And sorry, I fed in five. So it looks like out of the five, there were four with websites. It looks like one of them was a Facebook website, so I don't think this is going to have the right email, but that's cool. And of the rest of these, we actually found four, four out of four, so three out of four, I should say, three out of three with websites, have results. So we go to emails, you'll see there's info at Lena tutoria. [Music] Wonderful. There's probably going to be a bunch of Facebook ones. No. Anyway, doesn't look like we got anything from the Facebook. Looks like we didn't get any. Oh, my bad. Actually, we didn't actually get these emails. I'm not entirely sure why success was true for some of these. But anyway, it does look like we got one for lor.com, which is sufficient for me to think that the system is working. Realistically, in my experience, you usually get somewhere between like 30 to 50% of the domains that you pump in there. So we fed in, uh, four, and we got one. Just sort of like luck of the draw. I bet if you fed in 100, we'd probably get 50 or so.

But okay, great. Why don't we do the next step to this? Once we find a website, once we find, um, an email address, if we have an email address, why don't we now pump this into AI? So I'm going to go GPT, then what I want is Create a Completion, right over here. This Create a Completion will just allow us to very quickly and easily, um, basically create a system that uses AI on the backend. So you'll have to set up a connection. Now, I've already set up my connection. Under Select a method, Create a Chat Completion. And under Model, I'm just going to use whatever I like, GPT-4o. Now, this is entirely up to your personal preference. You need to play around these models a little bit to understand the limitations of each and so on and so forth. Under Messages, my rule of thumb is always to create a new system message that says something like, "You are a helpful, intelligent email assistant." I find that this typically just makes it perform a tiny bit better. And then next, you just give it the actual prompt that you want. This is the system prompt. This is sort of how it identifies from here on out. You actually just say, "Hey AI, can you do the thing that I want?" In our case, "Can you customize an email template based off of information about this website?" So obviously, we need that prompt now. I've already written that prompt before. So just for the purpose of this video, not to make us watch me write a big prompt all over again, I'm just going to copy this from my previous run. And let me explain it to you now.

Uh, let's go here, paste this in. "Your task is to customize a provided email template based on information about a restaurant. Here's your template: Hi, I found you when looking for restaurant type in location. I love [thing about your restaurant]. We make AI-powered order management systems specifically for [restaurant type], and I wanted to see if you we'd be interested in seeing what that might look like. Our system increases restaurant revenues by 10 to 50%, and we just finished raising a $10 million Series B round. I'm 100% confident it's better than what we already have. Would this be worth a call? Let me know and I'll set one up. Thanks, Nick." At the bottom, it says, "Return only the email in plain text."

So this is a pretty good email. It's not the best email. It's pretty reasonable. I mean, I don't know. I think it's probably like a seven out of 10 or so. Um, let's just say checking and out instead of seeing what that might look like. Once we're done with this, we just need to actually provide, like, the data. So I'm going to go Provided Data and then I'm just going to feed in some stuff. So Category Name, Location, um, not getting a description here, but I'll just say Description. Okay, so now we're feeding the description into AI. Is there anything else we could feed in? I don't think so. That seems pretty relevant to me. IDE, like theoretically, you can feed in whatever the hell you want, right?

Okay, great. And now what I'm going to do is I'm going to make sure that emails basically has, like, there are records inside of it. So let me go Array Length is greater than zero. So if the emails array is greater than zero, then continue. And just because I already know what I'm doing here, I'm going to go to Email and I'm just going to go Create a Draft. I'm actually going to create a draft of the emails that I'm generating. The email I'm sending this to will be the email that we found here. Um, sorry, I need to change my connection settings again. So let me just make sure this is with the right account. I have so many accounts because I've done so many connections, so many different clients and different people and different projects, my own, that it's gotten a little annoying to keep track of these days. I think realistically, what I should do is I should go back and delete all of them that aren't, uh, relevant. So I'll say, "Quick question" will be my subject line. The content will just be whatever the email body is that we just sent.

Okay, great. Why don't we do this? Uh, I don't know. I want to, I want to queue up a couple. So I'm going to go eight. I'm run once. Let's see what happens. Looks like we are working our way through it. Looks like we got a second email. So we're now queuing up a second email. And what I'm going to do here is I'm just going to go back, um, to this and I'm just going to see what sort of email drafts I have set up here. Awesome. That's what I got for Trattoria 35. Pretty cool. "Hi, if anyone looking for an Italian restaurant in Bayside, New York. I love your intimate atmosphere and the delicious wood-fired pizzas. We make AI-powered order management systems specifically for Italian restaurants. I want to see if you'd be interested in seeing what that might look like. Our system increases revenue, our revenues to 10 to 50%. We just finished raising a $10 million Series B round." This must have been from a previous one, actually. Now, I don't really like how it's putting the exact address here. So I might have to tweak the prompt a little bit. Maybe instead of, um, this, we do the neighborhood instead of the actual address. That's probably going to be a little cleaner, right?

Why don't we do that? So I'm just going to go back to my prompt here and instead of me pumping in the actual whole address under location, I'm going to [Music] return, um, we'll go in neighborhood. That way it's, I'm looking for restaurants in East Bronx, just to be a little more conversational. But yeah, that's, that's about that. That's system number one, good to go. And I don't think we had to spend more than 15 minutes, maybe 10 minutes, really, from start to finish to get an actual, like, email customization system you can use to actually talk with real customers. The only thing we have to do is we have to the dataset ID now. So instead of us hardcoding it, it's the actual dataset ID brought from the Watch Actor Runs module. So just going to map that in here. And then instead of that limit, I might do like 100 or something at a time. Looks good to me.

Okay, so that's system number one. Let's make system number two now, where instead of functioning off restaurant data, what we're doing now is we're going off that Indeed data. So just to clarify, the system is going to get the job listing. It's then going to read through the description. If it exists, pump that into an AI, have AI, um, tell us some things about this job. And then we're going to use AI to customize a resume template that I've pre-prepared for us so that it is more in tune with what the job requirements are like. As I'm sure you guys can imagine, this massively adds leverage if you are looking for a job because instead of you having to manually customize every resume, this just does it automatically based on the description, which is pretty sweet. A lot of people are doing this sort of thing right now, although they're just copying and pasting it from ChatGPT back and forth. If you have an Appify scraping system just scraping Indeed jobs for you, as I'm sure you guys can imagine, this is much more valuable and much more scalable as well.

So first thing I'm going to do is I'm going to go back to our actors and then this time I'm going to go to Indeed scraper. Okay, now in terms of the runs, I've actually already ran a couple times. You see this one cost me 8.5 cents, got 70 results here. Uh, I I only want a few items. Let's just do 10. And yeah, let's just search for automation in the United States. Um, let's set this up in Satam Make.com as well, though. This is one, this is our, our restaurant scenario. So let me just copy this over to a new scenario here and let's just call this Job Listing CV Customizer or something. Job Listing or Resume Customizer. Let's do that. And then I'm going to take this and then we're going to do the same thing we did before with Appify. So Watch Actor Runs right over here. But we have to create our own, like, another hook. So this will be, um, Indeed scraper. I have to add, sorry, I can actually use the same connection. I don't have to add a new connection here. I just have to add one of the accounts that I was using for Nick J Wells. And then here I'm going to go, uh, Indeed scraper. And now this is going to be watching the run of the Indeed scraper for me, right? It's pretty sweet. I think we already know what we need to do. We need to get the dataset items from this. So I'm just going to feed in the dataset ID, same thing we did last time. And I'm only going to return 10. And then after this, let's just run this puppy and see what happens.

So I'm going to go back over to Appo. We have unsaved changes. Let's save them and let's actually start this run. Sty is going to be pumping in automation into Indeed and it's going to be doing a bunch of magic hijinks behind the scenes to find listings. Then it's going to extract the HTML from the listings. And then from there, we're going to have everything that we need, which is pretty sweet. So I'm pretty stoked about this. Um, I'm just finding that we didn't actually get anything yet. Odd. This might be because I didn't provide a specific location for the search. Yeah, I might need to provide a location. Why don't we do automation in San Francisco or something? And then why don't we do 100 items? That's probably better. Sorry, let's do, let's do Seattle. I think that's, yeah, you might need a location in order to make this work. In which case, I, uh, I've taken it off. So just use whatever parameters are necessary for your specific search. You may need a location, as I mentioned.

Looks like it's now starting the crawler and it should show us results here live while it's doing the scraping. So let's just see what we got. Okay, it's crawled two out of 21 pages, 18 out of 41 pages. Cool. So it's actually doing the crawling. I don't actually really want any more results than this, but whatever, we'll just wait for it to finish. Obviously, when it finishes, we're going to be getting the result inside of Make.com. So I'm just going to be watching for that. Second that's done, we've already hooked it up to the Get Dataset Items module. So, um, because of this, we'll just have access to all the data that we need. We don't need to do anything else fancy. And then from there, we're going to need to find a resume template. Now, I've gotten a resume template. Looks like this. Pretty cool, if I do say so myself. It's for an unidentified kangaroo. I'm going to go like Peter Schmidt. We go Peter@gmail.com. Maybe we'll do that. And what we want to do is we basically want to take this resume template for this guy who looks like he's done some administrative assistant work, and we just want to customize it to the jobs that we get.

Okay, damn. Looks like this puppy's still running. A lot of automation jobs in Seattle. I put 100 results in there, so I think we're going to have to wait until we get to 100, unfortunately. Um, because I did 100 results here, we're going to end up spending nearly 50 cents on this request. So, you know, this is sort of like an order of magnitude type thing. If this allows you to apply to even a couple of jobs better than you would have otherwise, oh, did I say a thousand or did I say 100? Hopefully I said 100. Well, anyway, I guess we'll find out sooner or later. Um, you know, I'm sure you can imagine if you were down in your luck and looking for a job or something like that, and somebody gave the ability to spend an additional 50 cents and then for 50 cents just have all of your automation, all of your resume customization done automatically, you probably would say yes, right? This is basically the sorts of economies of scale that we're dealing with.

Anyway, we got the actor run completed. And as you see, we have UX Researchers at Amazon, Maintenance Mechanic at the State of Washington Department of Health, Oracle Delivery Lead at Accenture, right? Seattle is a big tech hub, so we obviously get a ton of tech jobs. The question is, what do we do with all this stuff? Well, um, what I'm really interested in is this description key. You see how this description key includes a bunch of information about the job? What I want to do is I want to take this whole description and I want to use it to customize a resume template, the one that I was showing you guys earlier. So how do we actually do this? Well, um, you see how this is nicely formatted and there's like headings and stuff like that? This isn't really going to work for us, unfortunately, the way that it is, cuz we can't just generate and using AI nicely formatted text like this. What we have to do instead is we have to convert this into markdown, which is like a third, it's sort of like a, like an intermediate step, which is similar in nature formatting-wise, but we can't embed stuff like font and and stuff like that like we can here. But I'm going to show you guys how to do it anyway.

We're going to take this resume example, then I'm just going to pump this into AI. I'll say, "Convert this to markdown." I'm going to paste, excuse me, I'm going to paste this in and click go. And I'm just going to have this write all this stuff using a markdown format for me, which is just a format that AI tends to understand really well. You see how it has these little number signs here? This is just going to allow us to take this data and use it to create, like, a Google Doc, just like this. So that's what I'm going to do next. Um, I'm going to click on this module, go OpenAI, and while my other OpenAI call is running, I'm going to create a completion. Thank you very much. ChatGPT just finished. I'm going to copy this. And what I'm going to do is I'm going to go GPT-4o and then I'm going to add a similar message to last time, just this time it'll say, "You are a helpful, intelligent resume assistant." Then underneath here, I'll say, "Resume template." Right over here, I'll say, "Your task is to customize a resume template based on information about a job. Only return the resume. Take information from the description and use it to cater the language on the resume to said job. Don't make anything up, but rewrite sections as needed to make them more palatable to the hiring manager we're sending this to. Only return the customized resume." Cool.

Now, let's actually provide the data. We'll go Description, then I'll stick into the description. And that should be pretty good. Yeah, I think that is basically everything we're looking for. I guess we have the company name as well, right? Eh, most of the time that'll probably be included in the description. Cool. Um, now from this, we need to make a Google Doc. So I'm just going to go Create a Document. I'm going to take the output of this. I'm going to say Resume for Company. Resume for Amazon Web Services. The content I'm seeing here, it needs to be an HTML. Some modules are like this. I'm actually going to go in, let me adjust this. I'm going to go in and actually add an additional module in between these two, but I'll show you why in a second. And then the content, um, okay, the content is being output of this, uh, AI module as markdown. So we need to convert that markdown into HTML, which is just the format that's used in Google Docs. So we have, luckily, in Make, a bunch of modules that allow us to do this quick conversion. You can basically convert any file type or formatting into another one. Uh, this is a simple module called Markdown to HTML. So it's just going to take the input as markdown and then it's going to convert it into HTML, which I can then feed into the content field. Awesome.

We should have basically everything that we need now. For testing purposes, I'm going to unlink this. I'm going to grab the default dataset ID and then I'm just going to feed it in here so we can actually run and sort of see what this looks like on real data. Let's just do two jobs to start. And the only thing we need to make sure of is that there is a description. So I'll say, "Does the description exist?" If the description does exist, proceed with the flow. Make a Google Doc for us. Okay, looks like on our first job, the description existed. That's why we got this little one filter. Looking pretty solid. And looks like we are now AI generating. Now we're creating the document. And looks like the description exists in the second one as well. Let's actually go in and see what this looks like. Resume for Amazon Web Services, Inc. You could see that we have now catered our resume basically entirely to the job. The questions on the job. You can see we even specifically, we specified that we're looking to leverage my strong organizational communication skills to contribute to AWS Kumo's innovative team environment. Right, it adjusted the job, uh, details. It adjusted basically everything that we need, basically, in order to make this happen. I am looking here and I'm seeing that there might be some additional output from the AI. We don't want this. So we could adjust our prompt to make it so that it doesn't give us any output aside from the actual resume body itself. Obviously, if it says this resume does X, Y, and Z. Ah, okay, cool. This looks pretty good to me. Yeah. And now, you know, instead of, previously I talked all about like our operations experience for Amazon AWS Kumo. Now it's all about detail-oriented professionalism with four years of administrative experience, a strong background in managing tasks, maintaining systems. Why? Because that's exactly what it asked for in the job description.

So hopefully those are pretty cool. And I think you guys can see the value here when we can combine Appify to a third-party no-code platform, drag and drop a couple modules, run, and literally in less than 30 minutes have functioning systems that can drive a ton of revenue. I mean, you can use the approach that I just did to help you land $150 to $200,000 jobs. And I think a lot of people have probably done this. It's just not really being talked about too much because, uh, you know, it's being done intelligently. It's being done subtly. You're not just copying and pasting stuff from ChatGPT anymore. You're actually building out integrated flows that allow you to manage your life. These sorts of, um, more like agent-enabled flows, I want to say, they're, they're only going to grow more and more popular with time. So the quicker you learn how this stuff works, obviously, the more powerful your you're going to be. Um, as we, as we go into, you know, like this AI-enabled future.

But it's not all about Make.com. What I want to do next is show you how you can use Appify with a variety of other platforms as well. Um, let me jump into Appify again, one second here. We'll jump into the console and let's go over to Integrations. And then what we want to do is go to, I don't know, any one of these. Let's just go Google Maps extractor. That seemed pretty good. Um, what you can do after you're done with this is, you know, how like Make.com we had our own little flow here with HTTP requests and webhooks and stuff like that? Well, all that is really doing, like, all the, all the add connection button that I was clicking before is really doing under the hood is just adding a webhook using this modal diagram. It's just sort of like a, they just had like a little snippet of code setup that just does what I'm about to show you. But you don't actually need to hook it up to Make.com. Like, this is a Make.com URL, right? What you can actually do is you can just hook this up to anything that you want. So if you go Add Integration, you'll see there's actually a variety of different ones here. And they have some suggested. These images are broken, which is why they're not showing up. But you can, you don't just have to do webhooks and stuff. You can actually send emails. So you could send an email to my email when the actor is completed. That's sweet. You could upload the result file to a Google Drive when the actor is completed. You could create a new GitHub issue when the actor run fails. You have a variety of these no-code platforms, right? Like Make was one of them. Zapier is another one. If you want to connect, this is Zapier. You literally just click this button and it'll walk you through it. You have some AI stuff with vector databases and whatnot. You have, um, an HTTP webhook. I'm just going to do an HTTP webhook just because that's the simplest and easiest for us to understand. But basically, what we could do is we could set an event and we could say, when the run is created, succeeded, failed, timed out, aborted, resurrected, created, succeeded, failed, timed out, or aborted. Sorry, the first half is the run and the second half is the build. Um, you know, you can, you can do something with this. And so what I want to do in this case is I want to send it to a URL. Let's just do like webhook.site or something. Or maybe it's called webhook site. Basically, this is just a third-party tool that allows you to get a webhook request and then just view it. So I'm just going to grab this. This is my URL. I'm going to paste that in here. I'm going to save and test. And now what this is going to do is it's just going to wait for a request. Oh, I think I'm doing a POST request, actually. Hold on a second. I know if this is okay. Okay, yeah, it looks like it's okay. And you see that we just sent the, we just sent the, the request over. And I guess Appify, so they have a resource with a joke called Chuck Norris can watch it episode of 60 Minutes and 22 seconds. Oh, man, man, these guys are great. So that's such a dad joke. That's exactly what I would do. Speaking of which, this whole video has been light on dad jokes recently. What the hell? Uh, definitely not to form. Okay, but anyway, so we actually want to give this puppy a run. So let's click start here and let's actually do this on like a sample of data. I hope I didn't get like a 100 results. Anyway, whatever, even if we get 100 results, it's fine. So, uh, what we're doing now is we, we basically just created a webhook in webhook.site. Same thing as the Make webhook. I'm just using this as an example to show you can connect this with any service. You could pump this into a Zapier, or you could pump this into an Nady, whatever the hell you want, you can do. Um, and then now we're just waiting basically for this run to finish. When this run finishes, we're going to receive a little POST request here with a bunch of data, which is pretty sweet.

So looks like it just did finish and it looks like it just sent it to that fourth webhook, which is us. Um, we're just going to wait for this UI to update. We need some Jey music. Wonderful. And here we have the HTTP request. Pretty simple, right? We got tons of data. We actually looks like we got a bunch of data as well. Oh, never mind, they're just like advertising to us through the JSON, man. These guys crack me up. Yeah, so, um, from here, you know, you can connect this to whatever the hell you want. Like, you could make your Nady 10 webhook, you could do your Zapier webhook, you could connect this to your own server and Heroku, or I don't even know if Heroku is still a thing. Am I showing my age here? Yeah, no, it's definitely still a thing. Um, and then you won't just be beholden to the Make.com Appify flow that I showed you. You could really just pull this data out and connect it with anything that you want.

All right, next up, I want to cover how to build custom scrapers in Appify. A disclaimer here, you don't need to be a programming professional in order to make this work. Uh, what I'm going to show you how to do here is take maybe two hours worth of JavaScript knowledge and then combine that with ChatGPT to write a script that does something. And I'm not saying that everybody here will be able to do this. Obviously, there's only so far you can get with ChatGPT and other AI aids. But I think you'll see that it's actually a lot easier to build a custom scraper for advanced purposes than you guys think, as long as you combine it with some of the rationale and some of the logic that I showed you guys in the previous module on connecting with no-code platforms.

So why would you want to build your own custom scraper? Well, there are a couple of reasons. The first is you might just have some advanced requirement or use case that isn't covered by the Appify store. The second is that maybe there's some web components on a website that you want to interact with. For instance, maybe instead of you just scraping a resource, you actually want to do something. You want to click a button, you want to update a field. This is venturing into a slightly different area of, um, web technologies. It's not scraping, it's actually browser automation. But it's still something that Appify allows you to do. There might be also a bunch of instances where other scrapers or pre-built ones just don't work. Maybe there's some very particular edge case. The website monitors your network connection activity, it takes a look at your cookies and and, you know, boots you off of it if you're not verified or validated, and you just need to go a little bit deeper than otherwise.

What we're going to cover next is how to build a simple scraper that allows you to do these things using Appify IDE. IDE stands for Integrated Development Environment. Basically, it just means that instead of you having to download a bunch of stuff on your own computer and then muck around with libraries and development configurations and whatnot, you can actually just do it directly in the cloud. If anybody here has used Replit, it's quite similar in nature. So all I'm going to do is put together a simple example using some very elementary JavaScript that just lists website contents. I'm going to use a template that Appify provides you for this purpose, and then do some modifications, and I'm going to have ChatGPT help us build this. The idea being that I'll just show you an accessible way to build scrapers despite not having too much programming experience. I do recommend that everybody here at least understand what JSON is, JavaScript Object Notation. You know, they take a couple of hours of Codecademy for JavaScript or Node. Just makes everything that I talk about and everything else on the web scraping world a lot easier. But I don't necessarily want to preclude people that don't have that from, you know, being able to get up and running. So without further ado, let's actually jump right into it.

I'm going to head back over to Appify itself. You guys remember at the very beginning of this video, I covered development? Well, this is where you can actually go and create an actor. So that's what I want to talk about next. Let's click on this Create Actor button. Going to see that you have a bunch of different options of how to create set actor. You could start with a code template, for instance. You can also start completely custom, or you can link a Git repository. GitHub is just like a place where people put, um, scripts and then you can also push your code from the command line. All this stuff sort of just puts the cart before the horse. I'm just going to try and get you guys up and running as simply as humanly possible. So obviously that's Appify Goole. As well, you could use JavaScript, TypeScript, or Python. We're going to be using JavaScript in this case because it's just simplest. TypeScript is just a little bit more involved than JavaScript. They do a bunch of like, uh, strict typing and stuff. We don't need to concern ourselves with any of that. What we're going to be doing is we're just going to go Start with JavaScript, scrape single page with provided URL with Axios, which is a scraping or a web request library, then extract data from page's HTML with Cheerio. You'll also see that there's Crawly plus Cheerio. These are different libraries. Crawly plus Puppeteer plus Chrome. This is like, uh, sort of like the, the grandpa stack or, well, maybe not grandpa, but the daddy stack where you can get out there and then actually, like, do browser automation, clicking buttons and stuff. But I'll leave that for you guys to check out on your own.

When you click the button, you'll see there's a bunch of JavaScript already inside of the page, inside of your integrated development environment. And, you know, there's a bunch more information about what you do, how it works, so on and so forth. And then there's even some guides on how to kind of dive a little bit deeper than I'm going to dive into, um, and actually build out, you know, more complicated libraries. But the few things, there are a few things for us of note. So the first is that we're importing this library, we're importing another one called Cheerio, and we're reporting this like Appify library as well. Anytime you see an import statement in JavaScript, just means that you're pulling data from from somewhere else and you bring it in your flow. So, uh, we're basically importing these three libraries. So just good to keep in mind. We are then initializing our actor. That's what this await actor.init bracket does. We're then defining the input. So input equals get input. The reason for this is because sometimes you provide the actor input. This is just sort of like a way to handle, um, the Appify platform where you can actually just provide your own actor input through, like, a structured data field thing. In our case, what we're doing is we're just making, uh, the input that's provided will just be a URL, a single URL. And then this is really the thing that matters. We are getting the URL. It's going to give us a bunch of page HTML, and then we're just going to convert that into, um, uh, data that is parsable using a parsing library called Cheerio.

Okay, so if that sounded like French to you, let's actually click use this template. Let me just show you how this works. The simplest example possible. So this looks pretty solid. We now have the code in our own integrated development environment. It's pretty sweet. I'm going to head over to Input Schema and just show you guys what this looks like. We have, um, properties, URL, and then we have the URL that we're providing. So I'm going to show you guys how to actually provide a URL to this and give it a quick example. What we do first is we need to build this actor. And once we're done, we're going to go to this input tab. With building an actor is, is it's basically, see how this, uh, server used to just bumped up to 4 GB? It's basically running this code in the cloud for us.

Okay, now we're going to go to input. And you see here how there's now just some fields where we just put our own data in? Well, what I want to do is I want to scrap down my own website. Left click. A. Right over here. Okay. Seems simple enough. I'm just going to pump this in, save, start. And let's just see what happens. I think seeing what happens is probably a better and simpler way for us to scrape data from the website than kind of building up the thing ahead of time.

Okay, so the output of this scraper was pretty straightforward, right? Looks like it's logging a bunch of stuff. Extracted heading level H1 text. This extracted heading H1 text. This H1. What this is basically doing, to simplify it, is it's just going through my website, it's extracting all of the headings, and then it's just listing what the level of the heading was, H1, H2, H3, H4, H5, and then the text as well. So that's pretty cool. We're actually already doing something with the scraper. Not bad. You see the output here in a more structured format where we actually now logged this to a dataset, right? If I wanted, I could go to storage, grab this dataset ID, and then do a

Cool automation with make.com, like I just did. Um, so we're already using custom code, which is sweet. Anyway, the file that we want to manipulate is this main.js over here. Now, I don't actually necessarily just want to do this. Um, maybe instead, I want to do something different. You see here where it says headings? Basically, what we're doing, if I scroll and zoom in a little bit, hopefully you guys can still see this. Uh, basically, what's happening is we're doing this H1, H2, H3, H4, H5, H6. Um, these are just the various things on the page that we're cycling through. But there are a variety of things in HTML that you could cycle through. For instance, if I check out the code on this website, this is an H1, right? So that's one thing that we just saw in the app. If I out, but but there are other interesting snippets of code that I want to know about. For instance, how about this P? I like this P tag. Why don't I add P tags in there? Um, maybe in addition, I want, let's see, obviously we're going to have a ton of P tags. Maybe we'll have spans as well. So maybe we want to add some additional tags to this so that it doesn't just return, um, you know, all of this. Maybe in addition, we want to return P, and we want to return span. Then maybe instead of extracting extracted element, sorry, extracted heading, we want to go extracted element. And then let me just see if there's anything else we'd have to do. Maybe instead of headings, we just want elements. And then push data, we'll just do elements. And then elements. That looks pretty good to me. So I'm just going to save this, build it, and start it. And now, instead of going through a list of headings, it's actually going to go through just all of the elements that I listed, six of which include headings, but two of which are not actually pings, which is pretty interesting. So now we're running, and as you see, we're getting way more elements. Why? Well, because this isn't now just returning the, um, headings. It's also returning all of the paragraph tags as well.

Now, how cool would it be if we just took all of the text on this page, now all of the things that we've just returned, and then we fed that into AI and did something cool with it? Well, I can actually show you how to do it, right? I'm just going to grab this data set ID and I'm going to go back to make.com. I'm just going to open up a new sort of flow here, create a new scenario. I'm going to go to Appify, get data set items. I'll just paste in the data set ID here. Then I'm going to return this. Now, we already have the text that we were looking for before. Pretty cool, huh? Now, what we're going to do, and this is, um, just something that's make.com specific, we're going to aggregate all of this. What I want is I just want a bunch of text, basically. And then what I want after this is I just want to take all this text and I just want to join the resultant array with a space. I'm going to run this. The output of this, oh, sorry, actually, we have, oh, my bad, there's a little text parameter. We actually have to map this array, and the value that I'm looking for is text. Then we have to join this with a space. Now, if we run this, you'll see that we now just have all of the text on this website, basically. And now, let's say we want to feed this into AI and just have AI tell us something about this website. We can pretty easily. Let's go GPT-4. And let's just omit the system prompt for brevity. We'll just say, "Extract a one-line and paragraph description of this website. Return in JSON. Use one line and paragraph as key names." We're going to go show output down here, response format JSON object, parse response. Then I'm just going to add all of said text on the website. And let's see what it returns.

We've now essentially just built a simple system in less than 2 minutes or so that takes a custom scraper in Appify. It then outputs, or in our case, we converted a custom scrape where just outputted headings to one that outputted both span tags and, um, P tags as well. We then took all of that data and then just concatenated it together into one long string just to get all of the plain text on the page. And then we fed that into AI to have it extract two things. One was a one-line summary of what this website is. Maybe you need this for some database that you're creating when you're scraping a big list of websites. Another is a paragraph, which might be a little bit more in-depth. Maybe use it for some report or document that you're creating later. Um, this took just a few minutes to put together. Obviously, there was some annoying string manipulation that I had to do that most people probably wouldn't know how to do as quickly. Um, but rest assured, there are multiple ways to manipulate this, and you can definitely use ChatGPT to assist you in the, uh, creation process.

Now, hypothetically, what I'm going to do is I'm just going to go back to my code and then go to builds here. Sorry, I'm going to go back to development, go back to my actor, and then what I want to do is go back to code. And now, you know, I just did this because of my knowledge of code. But let's hypothetically say I want to convert this code so instead of just getting, um, I don't know, maybe instead of just getting the H1 tags, we just get all the text on the page. What I'm going to do is I'm just going to take this. I'm going to feed this into ChatGPT. I have it on a MacBook, which is why I can just click a little hotkey and then spin it up, but you can also just like paste it into the browser. So this code currently gets all H1s, H6s on the page, plus P and span tags. But I just want to get anything with text or H. Maybe instead of that, why don't we ask it to do something different? This code currently gets all the H1s, H6s on a page, plus P and span tags. But I just want to get the meta description of the page instead. Convert this code into one that pushes that gets the meta description. This might be good for some SEO purpose.

Okay, I just ran this. Just going to drag this over here and let's see. It's currently writing the code. Wonderful, wonderful. Um, and now we have just a way to very quickly get the meta description. We get the meta with the dollar sign instead attribute content. What I'm going to do is just copy this over, go back here, I'm just going to paste this here. Then we'll just go meta description, sorry, meta description. Then I'm going to build and start. Let's just see how this goes. Just going to make sure that there was no erroneous value here. Looks pretty good to me. I just copy and pasted some pretty simple code, so I don't necessarily think there's going to be any problem. Um, instead, we're just going to push the meta description. Oh, the one problem might be that it's on JSON. We're just pushing the meta description as is. Yes, there might be a problem with that. Let's take a peek. Yeah, it looks like it, it was expecting data of type object but received data of type string. That makes sense. Uh, okay, let's, oh, actually, let me show you guys what I would do in this case. I would actually just copy all of this and then I'd go back to ChatGPT and I'd say, "Output needs to be an object. I just got this error." I'd actually paste this whole thing in and just have it fix it for me. So it's going through this code now, it's fixing it. And now it's saying, "Okay, paste this in instead." So I'm going to go back to the code and then this section here, the previously erroneous one, I'm just going to repaste it. And then I'm going to run it as an as an object instead of whatever the hell it was before, which is, I guess, just a string. So let's see if we could do a little bit of debugging with ChatGPT.

Okay, looks like we're still, for whatever reason, um, not getting the right data. So I'm just going to pump that in one more time. Okay, great. Now we're just going to give this a little copy. Then I'm going to go back to, um, the code section up here, then paste this in. Okay. And then now we're not going to have a null entry, which should work fine. Cool. And we now have the extracted meta description. U, meta description on a website is, uh, if you guys are unfamiliar, it's just like the snippet that you do that shows up on Google. So this is the meta description on the website. This is the meta description on GitHub. This is the meta description on YouTube. Meta description on Crawley, so on and so on and so forth. So yeah, we've now just changed our scraper to deal with meta descriptions instead of just the headings on the page. Obviously, you can do whatever the hell you want. Um, my recommendation to you, if you don't know any code, would just be to copy this whole thing into ChatGPT and say, "Hey, currently this is doing this, but I wanted to do this instead. Could you help me out?" If it gives you an output that works, fantastic. If it gives you an output that doesn't work on the first run, don't actually be dis disincentivized. Don't actually just call it quits. Just paste the output error back into ChatGPT and just say, "Hey, I just got this error. What do you think we should fix?" And just do that three or four times. And if you can't finish it by, I don't know, your fifth round or something like that, you probably don't know enough about this to really be screwing around with it. But you'll be surprised. A lot of the time, at least at the current level of technology, you can totally get away with this without actually knowing basically anything about code. Um, then from this, you can bridge that knowledge gap and build systems that actually make tons of money.

So yeah, that's how you build a custom scraper in Appify. As you see here, you do all of the actual development operations, the devops stuff, just directly on like on Appify itself, just in this little development tab. You don't need to worry about, you know, mucking around with, um, complicated, uh, scripts or libraries or anything like that. You don't need to download Python onto your computer and muck around with the 20 different versions. You can instead just do it all in Appify, and all the libraries are supported. Just cuts a massive learning curve down, which is great.

All right, now I want to talk a little bit about scheduling. I want to talk about storage, and I also want to talk a little bit about proxies. So let me run you through some advanced configuration and settings in Appify, just to walk you through some of the cooler additional features. And these all, well, schedules are simple enough that I can cover it, but storage and proxies are such complex topics that they basically deserve their whole own videos. So depending on how this video goes, I may just record some follow-up videos specifically on storage, uh, and proxies as well. But first, let's just cover schedules. You saw how back there, um, sorry, under actors, we had that Google Maps extractor, right? And it just ran queries looking for restaurants in New York, USA, 100 restaurants searched every scrape. Well, hypothetically, what if I wanted to run this every week? Well, we can actually do this really simply in Appify. We could say, run this every week. Oh, sorry, and then, my bad, I was going to add this without any, um, uh, any scrapers. What you do is you select the time up here. It'll tell you what time and what time zone. My case, I don't want that. I want us mountain. And then down here, under actors, your tasks, what you want to do is you want to add an actor. The actor that you want to run is, in our case, the Google Maps extractor. So basically, you then have the choice to enter in your own search terms. And so we could say, I want you to repeat the restaurant search term once a week. And maybe from this, you're just like, looking to see whether or not any new restaurants have been added on Google Maps or something. I'm sure you guys could see there, lots of use cases. When you save this, then just go save and enable. And now you have a schedule that's running. Now, the value in something like this, let's just rename this to weekly restaurant scraper. The value in doing something like this is, you guys remember how earlier our restaurant cold email system would watch the actor runs, right? Well, now what we can do is we can actually just turn this on, and basically, once a week, whenever this thing runs, it'll actually just go through this whole sequence, create the drafts, and maybe send them out. So maybe, um, as an intermediary, we had some Google Sheet step where we check to see if any new entries have been made in this Google Sheet, and if the new entries have been made, then it continues. If not, then it doesn't. I'm sure you guys can see, but there's, there's a lot of value in doing scheduling, and it's one thing that a lot of people don't really understand about Appify, which is unfortunate.

Next up, I want to cover storage. Storage is pretty intense. As you see here, we have a bunch of data set IDs on the left-hand side, but I don't want you guys necessarily to be discouraged by this. Um, basically, every time that you run an actor or change the configuration of an actor and give it a run, we create a new data set ID, um, in in the storage tab. And so that's why we have you check out the created dates on this. Basically, this is, I've been testing this for quite a while today, as you can see. So over the course of the last two and a half hours or so, I've created, um, I guess 15 unique data sets. If you give any one of these a click, you'll actually be able to go in and see all of the historical data of that run, which is nice. So as you can see, this one is from our custom scraper, has spans, H1s, Ps, H2s, so on and so forth. If we go back to maybe this one here, you can see this has a bunch of jobs from our Indeed scraper. Go back all the way over here, you can see this one has a bunch of restaurants, Tutoria 35, the guys that I sent that draft email to. And, you know, so on and so on and so forth. The unfortunate thing here is that if you don't name a data set, it'll be automatically deleted after 7 days. So if you want this not to be deleted, what you have to do is you have to go up here and then rename it. Going to say, "Don't delete me, please." Now that we've done this renaming, um, this data will last substantially longer, and we're also, if you think about it, going to be billed for this, because now this is like in Appify's database essentially. I think, uh, actually, previously it was being billed for as well. It's just that's why they delete it once a week because they don't want you to be billed for stuff that you're not really using. But it's just a quick and easy way to like have access to this data for as long as you want, which is nice. So you know, if you have any runs that you know you're going to be doing testing on over and over and over again, then just make sure to rename that specific record. Um, and then, you know, Appify will automatically delete them. But otherwise, it's actually super valuable and a very easy way, um, to keep track of just all the data that you've generated. Same thing with the key value stores. You see that every time that we've run this, um, we basically created a key value store key, in this case being input, and then, um, in this case, uh, the value was URL left click.a. This is 31 bytes. Let's see if we could find something bigger. This one was 10.2 kilobytes. So this looks like it was automatically created when we ran the Google Maps extractor, and the reason why is because it looks like there were a bunch of, um, in our case, like export URLs that are de-duplicated. I think these are the IDs of the specific places in Google Maps. So just the way that Google Maps stores this data, I could be mistaken, they store a bunch of like additional programming stuff, session pool state. I'm not really sure what this is, to be honest. But as you can see, there's a wealth of data hidden in Appify that the built-in scrapers or community scrapers, um, kind of track for you, and there's a lot that you can do with it. So keep an eye on that.

And then last but not least, there's a request queue. Remember how earlier I said that crawlers tended to create these queues, um, where, you know, they would scan, let's say, a website, and then it would look for all of the links on the website, then it would add that to something, and then just go off of that queue, basically, to find the next link and the next link? Well, that's basically what this puppy does. Um, this is also created automatically by Appify, and not all, all, uh, scrapers that you will use will have a request queue. But just to show you guys a quick example of one that would, if we go to the store here, see it says website content crawler. Basically, what happens with this is this will, you enter some start URLs, a website to crawl, and then it will find all of the additional links on that page and then crawl those. That's the difference between a scraper and a crawler. A scraper sort of just does one page at a time. A crawler will actually crawl around using links to navigate. Now, this one takes a lot of memory, as you can see. We just maxed out at 8, and I imagine that's because it's more computationally intensive to open God knows how many new tabs. But let's actually just watch and see what, what it does. I'm just going to open up the log here, and it is queuing up links. So it just went through that first link, and it queued up 22 new links on this link. What it's really doing under the hood is it's finding every single URL here on all these blue sections, and then it is adding them to that queue. And you can see, you know, that it's, it's doing said queuing process here. That's that's what crawlers do. So, um, yeah, that's more or less what's happening underneath the hood when we run this. If I check the request queue now, you'll see that there's just a big queue of URLs that this is set to access. But there's nothing really special about a request queue. A request queue is still just like a data set. It's just a data set with specific columns like URL and and whatnot.

Okay, great. So the TLDR behind storage, obviously being just a place where you store the data. If you don't want your records to be deleted after seven days, then just make sure to name them. And different actors will use different types of stores, but all sorts of storages are reasonably similar to one another. Data sets are just like the common place where data is dumped after any actor, uh, well, most store actors are completed. Request queues are specific to crawlers. They will create long queues of different URLs and queue them and then just go through them one by one. And then key value stores, um, typically store screenshots. They store metadata surrounding the specific actor run, like we saw earlier with that 10.2 kilobyte thing, um, with the URL exports and IDs and whatnot. Um, in reality, you're probably mostly going to be using data sets and request queues.

Last thing I wanted to chat about was proxies. Proxies are pretty straightforward. They're basically just tunnels that you route requests through. Um, you do this because you don't want your actor or whatever service that you're attempting to scrape data with to be blocked by the service that you are getting it from. So like, let's say you're attempting to get a bunch of data from Google, for instance. Well, you can only really ping Google so many times so quickly before they start being like, "Is this guy a real human being or is it a robot?" And generally speaking, services don't really like robots. So proxies allow you to to just add an additional layer of subterfuge to your scraping, basically an additional layer of complexity. And instead of your IP address being the same one that calls it every time, instead of your IP address just being like the problem IP address that is now blacklisted on all of these social media platforms, um, you can you can purchase temporary usage from a rotating network of different IP addresses that help you escape this.

So yeah, they have a, they have a service called Appify Proxy that's divided into three sections. There's data center, residential, and then special. Data center just means that there are these big server warehouses with IP addresses that they are tunneling your requests through. These are typically the cheapest, but they tend not to be the most secure, the most reliable, just because so many servers are now wise up to the data center proxies and they're saying, "Huh, I recognize that IP address. I think 50 people used that IP address yesterday. Hm, we should block list it." Um, residential IPs are sort of like the next level up. They are same IP addresses used by people like you or me. I don't exactly know how Appify does this. I don't know if they have some sort of like relationship with a bunch of locations to like rent out their IP addresses or or something like that. But residential IPs tend to be a lot more trusted by all of the major providers that you would want to scrape from. So as a result, they also tend to be a lot more expensive. And then special proxies are just proxies that are like purpose-built for, um, handling requests to specific services, like Google Search Engine Results Page scraper, the Google SERP scraper, for instance. In Appify, they, they kind of made their own little IP address that's just like super fine-tuned and and, you know, does exactly what Google wants, um, in order to like minimize your rate limit and give you as high throughput or request volume as possible.

So you can see the usage over here. Just by clicking on the type, you see I use 22.4 megabytes today. I did so without really knowing because a lot of Appify actors use them, um, built in. So you can see I accessed appify.com 19 times through my data center proxies. I downloaded 14 megabytes. I accessed claritydata.ims 11 times. Um, I accessed intercom.io 24 times. Probably because if I just drag this over here, see this little blue thing? That's Intercom. So basically, there's, every time you load a page, so probably the, the crawler that I just ran was doing this. Every time I loaded a page with with a crawler, probably like sent a request over to Intercom, and I had to pay for it. 175 kilobyte, not a lot, mind you. But yeah, you can check out the billing rate on this by going to billing and going down to proxy, and then there should be some, uh, data on this on the subscription. Yeah, so proxy SERPs, $3 per 1,000 SERPs for the search engine results proxies, which I believe is the special proxy that they talk about. It's $10 per gigabyte for the residential. And then I don't know what the pricing is for the data center. Maybe it's not even, maybe maybe they don't even charge you for it anymore. I don't know. But you know, as you can imagine, $10 per gigabyte can really add up, considering I've already used 22.4 megabytes and all I was doing was just some really simple tests. You had systems that were running an actual volume like maybe 50x what I just did, you would have already spent an additional $10, um, just on the proxy stuff, which, yeah, as I mentioned, can add up. So be careful.

You can learn more about this proxy stuff if you want, um, by going over to this HTTP setting thing and then, sorry, I just go top right hand corner, how to use. U, basically, if you wanted to use a proxy in your, your own Appify actor, not just one that's like automatically used, uh, you'd have to get this connection string here with a username, a password, hostname, and a port. Appify will actually automatically provide you the hostname, the port, and the username. Um, the password is sort of like your API key to do the proxy request, and you just have to add that to your actor when you make a request to a specific resource. Basically, you need to, you need to tunnel the request through this. The, you know, you doing this is is beyond the scope of this video because it's very much more programming. But feel free to just jump into Appify docs. It's one of the reasons why they're so great, which I'll cover in a minute. And they'll run you through how to do it. You just need to add them to like your environment variables in the actor that you set up. So yeah, that's that on the scraping, sorry, on the scheduling, storage, and proxy side of things.

Um, why don't we cover one more thing, which is using Appify as a business model? Now, I've talked a lot about how to use the outputs of Appify as a business model, like in the restaurant scraper cold email system, and then the Indeed job listing resume customization system, which, you know, took me 30 minutes to set up. There are many, many cooler systems that you can do if you spend a little bit of time thinking about it. But I want to, I want to show you guys like a new business model, and it's basically a way where instead of you, you know, providing services for people or whatever, Appify actually allows you to make money using their platform by putting together your own scrapers and then just charging people per usage, just like the scrapers that I was using a moment ago, um, as examples. So, you know, logically, if there's an Appify store, some people are going to be making money on it, right? You can absolutely put your own actors into the store and charge for it, and people are making a lot of money with this. And I'm going to find you some examples right now. So let's just jump over to the Appify store. Let's actually just get a feel for how much money people are making with this. Uh, I'm just going to go view all and then pricing models. Like, there are a couple of different ones. Why don't I just do rental for now? Rental is the simplest one to extrapolate cost. Let's just do one that's not from Appify. Let's do traffic generator. Okay, so you see this one is $20 a month, right? Well, this one has 4,600 monthly users. So just do the math on that. This is making $92,000 a month. This one scraper right here, this is a pretty involved scraper, not going to lie, and it's maintained by this company, epcttx. But you can kind of like think about it like, what this is, is this is a distribution mechanism by which you can make a really cool scraper that scrapes some data source and delivers it in a nice format for people. And then Appify will allow you to build $20 a month times 4,600 users for that. Now, not all this money is going to you. Appify takes a fee. And, um, I think Appify store fees, I believe there's some cut or something. How much does Appify take? Oh, geez, I don't entirely remember exactly how much money this was. How to monetize an actor? Oh, monetize your code. Okay, there we go. Uh, so yeah, it looks like there's cost around, uh, there used to be a flat percentage. Ah, okay, okay. So there is a flat percentage still. It's 20%. So 80% of the total goes to you, and then 20% of the total goes to Appify. So of the $92,000 that that guy's making, um, they're making $73,800. It's pretty sweet, right?

So you can obviously make a lot more money than just this. I mean, just run through and find the same one that I was showing you guys a moment ago. Uh, boy, there are a lot of these. We'll go back to the store. And yeah, so that one was $92,000 bucks. Um, why don't we go back to pricing models? Check got some other rental ones. You can see that Appify as well charges you for stuff like Facebook post scraper. There's 1.4k monthly users, $35 a month. They take 100% of it. But I'm just interested in like non-Appify ones. So maintained by community. There we go. Guys, this LinkedIn job scraper, $29.99, so it's say $30 a month. And you, how many people are using it? 64. That's still making you, uh, $188,000. So even with Appify cuts, Bey is printing. I just type in Bey. Let's check this out. They're all rentals here. So let's just add up the total amount of money that these scrapers are making. 15 times 4, so $60 off that. $18,000 off. That's just say $18 grand. Let's say $19 grand. Now, 30 times this is another $15,000. So $34,000. This is 25 times 80, $2,000. $36,000. This is 15 times 3,000, 45. So 39 plus, did I say 39? I think I said 39. Anyway, 39 plus 45 is what? 74? No, 84, I think. Is that 84,000? These guys are making basically zero. That's basically zero. So Bey, this one company is making $889,000 a month with this. So anyway, I'm not, I mean, I'm obviously interested in this. I was interested in this as a business model myself. I'm not telling you you have to go and do this, but I just wanted to run you guys through what it looks like if you, if you wanted to, you know, try and monetize this. Um, you can make a ton of money. A lot of people have whole businesses that are built off of Appify at this point, which is pretty sweet. Um, so yeah, so anyway, there are a variety of ways that you could share this actor. Oh, my bad. You want to go over here to publication on the right hand side here. They'll rate your actor based off of kind of whether or not you have all the fields filled out. You got to have like the title filled out. Ideally, you should have like a profile pic and stuff. They're just trying to help you. Um, you need to have a public account. So you need to like add a bunch of information here underneath, uh, your profile with like your name and your bio and stuff like that. And then once you have monetized, once you have your billing details and payment method set, you can set up a monetization. And then basically, um, you can tell people how much money they need to spend in order to do this. Like, are you going to have people pay you on a monthly basis? Are you going to have people pay you on a per unit basis, like we saw before? Are you going to have people pay you on a platform usage basis? There are a variety of different, um, you know, monetization mechanisms. Um, like the Google Maps scrapers and the Indeed scrapers, they all did this on a per-use basis. But there were a bunch that were also rental-based. So you kind of just need to figure out what works for you cost-wise. Um, yeah, you can either pay per usage, you can either do rentals, or you can do that, uh, pay per result thing. Nowadays, I would say just anecdotally, based off my experience using this a lot, people like pay per result just because it's a lot more linearly scaling. Like, in their heads, they'll be like, "Okay, I'm scraping 10,000 leads today. 10,000 leads means that I'm going to spend, I don't know, $20 scraping it from, uh, Indeed or maybe Google Maps, and then I'm going to have to spend another $15 doing some enrichment, and then I'm going to have to spend $15 in the email platform." And in that way, like the costs are easy for them to calculate. You can probably also, as a developer, get away with charging a little bit more money relative to a monthly rental pricing model. But, you know, pricing sort of like beyond the purposes of this. Then you're going to have like a sample input and a sample output. And the second that all that's done, you have your monetization. You can just publish this to the store, and your actor will literally be seen just like, just like all of these. I mean, you know, yours is probably not going to be seen here under the most relevant tab. Yours will probably be seen under here as the newest tab, cuz you'll have like one user. But yeah, you can very quickly and easily monetize this stuff. And by the way, also, if you want something really funny, just head over to Appify newest and then just scroll through them. Some of these are really, really funny. Um, you know, like there's like joke generators, Ron Swanson quotes scraper generator API. There's, there's a bunch of really cool here, uh, to be honest. So I'm a big fan of their ecosystem. I think Appify is doing pretty great. Awesome.

So, um, that's, yeah, that's Appify as a business model. Next question, and probably the most logical last question to have is, "K Nick, I love everything you told me about so far. You're great, and I should definitely like and subscribe. But, uh, where do I go for more?" Where do you go for more? Um, Appify gives you literally everything you need within the platform. And no, they're not paying me to say this, but they give you literally everything you need within the said platform in order to go out there and crush it. My main reason why I love Appify so much is just because of how incredibly well documented everything is. Like, it is beyond well documented. It's amazingly documented. I cannot stop speaking praises about the, the level and the quality of the documentation of this company. It's insane. I'll show you two things. The first will be a web scraping guide from zero to hero, and then the next is going to be, um, just like a general walkthrough, um, that gets you started with, uh, like basically everything you need to know. But Appify versus like a specific web scraping one. So if you just go like Appify Docs, Google it, you'll see that they have some great documentation over here. But what we're curious about is Web Scraping Academy. They give you free practical courses on web scraping and browser automation. And these courses are fantastic. Now, what you want is, if you're a total beginner, you don't want any of the advanced web scraping ones down here. What you want is this Web Scraping for Beginners course, top to bottom, which will teach you how to do everything you need in order to build a custom scraper, just like I did a second ago, with minimal coding knowledge. You're going to learn all of the principles behind scraping protections, um, JavaScript versus TypeScript, some web scraping for Python versus Node. You're going to learn a couple of the number one libraries, Puppeteer and Playwright. You're going to learn how to like click buttons on pages and do cool stuff with them for browser automation. Basically, if you just make it from start to finish to the end of this, like, web scraping for beginner guide, you will know more about web scraping than like 99% of web scrapers. And that's for beginners. These guys are crazy. If you wanted to monetize Appify and actually get out there and start making money with the platform as well, this is one of the simplest and straightest line paths to do so because they will literally show you what you need to do in order to make money on their platform. Why do they do this? Because they want you to win. Because when you win, they win, because they get paid after you're done with that. Check out the Appify Platform Guide, which walks you through everything you need to know, essentially about, like, now you know how to do web scraping. Well, how do you actually interact with Appify? How do you actually interact with, like, what, what, what is inputs? What are, uh, tasks and runs? And how do you query them via, you know, programming and code? From there, you can learn how to do like some API scraping, including, I believe, hidden API scraping, although I don't know because I don't, I made it to the end of this. Um, you can learn all about anti-scraping protections. So, you know, we were just talking about different types of proxies a moment ago. We can also learn about entirely different things like fingerprinting, for instance, and a variety of ways to overcome it. Um, adding headers and authorization snippets to various parts of the request that you send over. And then down here, you also have an expert scraping part. So you need to basically be like a programmer in order to fully understand all this stuff. But if you get to the point where you can fully understand this stuff, then not only will you be the top 99% or the top 1% of web scrapers, you'll be the top 1% of the 1%. You'll basically be like one of the most dangerous people on planet Earth, to be honest. So yeah, highly recommend you guys check out Appify documentation. I'll make sure to link them all down below. Um, as I mentioned, like, they, they're really, they're not paying me to say all this stuff, believe me. Uh, I just really, really like companies with good documentation, and Appify has by far the best documentation.

That takes us to the end of the tutorial. If you have any questions about anything that I mentioned, feel free to drop them down below. I hope you guys now see just how easy and simple it is to use a service like Appify to go and get real actionable data from websites that for the most part, people thought were never scrapable. Um, we built a couple of high ROI simple flows that enable us to automate things like applying to jobs and sending cold emails to restaurants. You could take these same ideas and you could apply them to any sort of data on any sort of platform to do basically anything. Really appreciate your time. I'd love it if you guys did me a solid and liked, subscribed, and commented on my videos for engagement purposes. I take most of my requests from viewers at this point. So if you have anything in particular that you want me to talk about, maybe something I didn't talk about related to Appify or just web scraping more generally, just leave a comment down below and I'll add it to my queue. And yeah, I will see y'all in the next one. Thanks for your time. Cheers.