Transcription
Hi there. So, in this video, I'll show you how you can scrape any website, including all social media platforms with Claude and MCPs in just 15 minutes and without using code. This setup can be used for lead generation, marketing analytics, content ideation, but the possibilities with this setup are really endless. And in this video, I'll show you a demo, a step-by-step breakdown of how you can set it up for yourself. And the templates will be available in my community.
So, here we are in my Claude desktop. And as you can see, I've connected Claude with more than 30 MCP tools including Air Table to basically save any relevant data to a database. In this case, Air Table. Then I've connected it with more than uh 20 scrapers. As you can see, I have the tweet scraper, Facebook group scraper, Instagram scraper, web scraper, etc., etc. And I did this by using the Appify MCP. And lastly, I've also connected it with the Perplexity MCP to be able to do some deep research.
Now, I'll first show you a demo of some of the use cases for this setup, and then I'll show you step by step how you can connect all of these scrapers and these tools uh to Claude in probably less than 10 minutes. Now, of course, you can imagine with these amounts of uh scrapers right here in Claude, the the use cases are really endless, but I've just um set up three quick use cases that I think are interesting.
Uh so first I set up a competitor analytics use case that can basically do uh analytics on the social media activity of a competitor and creates a slideshow out of the analysis. Then second I've set up a YouTube content ideation workflow where it basically scrapes uh YouTube comments from my past videos to come up with new ideas and it also uses some Perplexity to come up with new ideas. And then lastly I maybe can show you another example with a lead generation and personalized outreach workflow.
The way I've set up these use cases is through Claude projects. And if you're unfamiliar with Claude projects, I do have a full tutorial on how to set up these Claude workflows in my last video, which I'll also make sure to link up here. But basically, what we can do here in these Claude projects is to give specific instruction or like a system prompt to this project on the specific use case we want Claude to execute in this case. So in this case, I've given it instructions on how to uh do competitor analytics on their social media and then create a slideshow right inside Claude. So that's what's in the instructions here. And because I've already pre-instructed Claude what to do with this specific use case, I can just tell it uh for example, hi. And it will already know what to do to get to its final outcome, which is the slideshow. Right? So you can see it first asks me uh which company I want to investigate and to get a marketing analytics report for. Right? So let's just take a random example uh let's say uh Canva, and you can see now it's first using Perplexity and it's using Perplexity to find all of the um social media platforms for Canva. So you can see it tries to get the TikTok account, the Instagram account, the YouTube account, the X account, etc. We get some more information through Perplexity and now it starts scraping the Instagram profile. Now it scrapes TikTok, the Twitter, LinkedIn profile, YouTube uh YouTube channel. But you can see the power of this setup. uh by giving Claude access to all of these scrapers, you can basically access the entire internet, including all social media platforms. And lastly, it's scraping Canva's website. And lastly, it's getting some uh reviews from review platforms to get sentiment analysis data.
Now it's got all the information and as I already pre-instructed it to make a slideshow, it's now going ahead and creating a slideshow right here inside a Claude artifact uh with a marketing or competitor analytics uh report. Basically, as you can see, I've spun up an entire slideshow on Canva marketing analytics and it's pretty impressive with bar charts as you can see and data points, company content strategy overview, multiplatform approach, easy to use design tools and templates, consistent brand voice balancing, professional expertise with friendly, approachable tone. I'll make sure to put the I'll copy this this link in the description below too if you want to read it in more detail. But as you can see it puts in pie charts top performing content across LinkedIn Twitter approach analysis and you can see the pie chart is the types of posts. So product updates is 30%, social media tips 35%, company news 15%, and the design inspiration 20%. And this it does for for all the channels also for the website. YouTube top videos, TikTok, same thing. Instagram even puts in these icons. It's uh it's pretty impressive.
So you can imagine the amount of use cases for these uh scrapers and using the Appify MCP and also using these Claude projects, right? By having these sort of common use cases, let's say you are a CMO or whatever and you have to do these kinds of reports on a monthly basis, you can just spin up a Claude project for this specific use case and uh do it on a on a regular basis and let Claude do it instead of mapping out an entire workflow in in n8n for example, if you're used to working with automation tools. So I really do believe that for some of these use cases, sometimes Claude can really be a superior setup because especially let's say we want to add some information here now. We want to iterate on this presentation. We can just go here and together with Claude iterate on this presentation and let him adjust things, right?
So, let me show you another quick example. So, I can show you the YouTube content ideation one which I thought was pretty good. Uh, this one's for myself. So, to get started, could you please share your YouTube channel name or URL? So, I'll just share my URL and now uses uh uses the as you can see the YouTube scraper here first to get my last videos and then it should use another scraper to scrape the comments and then it will also use Perplexity uh to do uh some research on the latest trends and together with all of that it will basically give me some tips or ideas on uh the future videos I could could make. And then lastly, it uses Perplexity to do some research on the current trends. So you can see here it first scraped my last YouTube videos basically to find my last YouTube videos and then each for each video it scraped all the comments. So you can see it's generated a a YouTube comment analysis first right the most engaged videos key themes and topics of interest Claude MCP integration platform and tool preferences AI agent implementation social media integration business models and monetization and it's already updating my uh my Air Table too with the new ideas which is the second document so most common questions requests right tool selection any relevance AI positive feedback highlights, right? Audience demographics and then in the second document, it gives me video ideas. So, 20 impact video ideas for Benai channel, MCP, the ultimate Claude MCP setup guide for beginners. That's actually pretty good idea. I think let me know if that's interesting to you in the comments below. And then it gives me title variations, too. And basically what I did is besides the system instructions, I also gave it a Claude knowledge base with all of my past titles. So it can also I also instructed Claude to follow sort of the types of copy I use for my titles. So he did it pretty well. How to build your first Claude MCP agent in 30 minutes. Then it explains why it would be a good uh good title. Building an AI research agent. That's also a good one. Connecting Claude MCP to your database. What's that business API plus Claude MCP integration? Tool comparison. n8n versus Make versus Relevance AI guide for AI entrepreneurs agentic RAG business development and it's based on these comments right now of the prompts and the knowledge bases for these use cases that I'm showing I'll also make sure to add in my community right for example the marketing analytics the the project instructions is just how to execute this specific workflow and then I usually have a knowledge base reference in this case on how to make the slideshow.
Now last example I will not uh go over it in detail I'll just show you a past example uh because of course I've also connected it to scrapers of lead generation. So for example Google map scraper and Apollo scraper and I think there's some really interesting use cases with this too because we can scrape leads uh directly personalize emails or outreach uh inside of Claude iterate on Claude with Claude on the emails and then actually send them out on email because of course we can also connect our email or instantly or uh even LinkedIn DMs if we want to do outreach on LinkedIn. And I set this one up really quick, but you can see get me three dental practice leads in Miami. And you can see it used Google Maps to scrape some dental practices in Miami. Then it created uh a spreadsheet with uh possible uh leads here, email addresses, then an email outreach template, which is really bad because I didn't properly instruct it how to do an email outreach template, but of course you can tweak this and iterate together with Claude. Then he actually used um web scraping to find more details on these specific leads and then uh personalizes these templates based on the information he found on the website for each of these companies. And then in this case I added it directly to Air Table as you can see right he added it to Air Table but of course you can also connect it to Gmail etc. Now again just a quick example you can imagine the use cases for this are are really endless but I think this lead generation and outreach use case can definitely be really interesting here inside Claude too. Again I'll make sure to share the system prompt of this one to in uh in my community.
So how do you connect these MCPs to Claude? They're basically three main ways uh to do it. First you can do it through no-code automation platforms. You can also do it to some third-party providers who make it really easy for you to connect uh some popular softwares. And the third option is to do it manually or yourself. Now, if you've watched one of my previous videos, you've seen me use uh mostly the no-code automation platform, specifically n8n to connect MCPs. But for this use case, I set it up manually and through a third-party provider because actually it's not rocket science either and can actually save you a lot of time if you do it manually.
So, I'll walk you through all three MCPs that I used for this specific setup. First, I'll show you Appify, which of course we use to scrape all of these platforms, which I actually set up manually. Then I'll show you Air Table, which I set up through a third-party provider. And then lastly, I'll show you Perplexity, which I also set up manually.
Now, if you don't know Appify yet, Appify is basically a marketplace full of scrapers. They literally have thousands of scrapers available on this marketplace uh to basically scrape anything on the internet. As you can see, we have the TikTok, Google Maps, etc. You can literally imagine anything. And you can see we have more than 4,500 uh scrapers. Now this is the big advantage with MCPs. If we would have to set up each of these scrapers separately through an API, it would cost us a lot of time if we want to implement a lot of these scrapers. Now with MCPs, the huge advantage is if we use Appify MCP, we can instantly access hundreds of these scrapers literally in minutes. So how do we do it? They have a full step-by-step walkthrough on their website. I'll make sure to put the link uh also in the description below. But here we have the actors MCP server. And here we uh because we're going to set it up through Claude. You can see here how it works, right? We have the Claude desktop. Then we have the MCP server. And then we can connect it to all of these scrapers. Now we have to go down here because we're going to connect it to Claude. We have to go here to MCP server at a local host. Now before starting, you see there are a few prerequisites. First, of course, you have to use Mac or Windows. Then you can only use MCPs with the Claude desktop. So if you don't have it yet, you have to download Claude uh onto your desktop. You don't need the paid version, but in my experience, it does work better, right? Uh so you need to Claude desktop app. Then thirdly, you need Node.js installed on your computer. Now, if you're not sure if you have NodeJS installed on your computer, if you're not sure, you probably don't have it, but you can also check it quickly uh by just copying this here and going to your terminal. So, you can on Mac, you can just search for terminal. It's probably similar similar on Windows. Then, you can add that in. And then you can see here with me it has version 23, meaning it's installed. If it's not installed, you'll probably see something like not found. Now, if you don't have it installed, you can just go to the Node.js website and download it from there. It's really, really easy and you can do it in in a few minutes, right? It's also free. So, you can just go to Node.js and go download. And then lastly, you need an Appify API token, which I'll show you in a second how you set that up. And then we go down here and we go to this second piece of code. This is what we're going to need uh to connect uh our Appify MCP to our own Claude. So, we can copy this. I usually just save it in a note section. And as you can see here, we have the actors. And the actors are basically all the different types of scrapers we want to add to this MCP. So, you can see in this case there are two scrapers and we also have to fill in our Appify API token. So, how do you get these things? First, of course, you have to make an account if you don't have an account yet. Once you have an account, you'll arrive to a screen like this. And here you have the Appify store. And here you can find all of the scrapers that you want to add. Right now, for pricing, you have to check because some of these scrapers uh they have a pay for usage type cost and some have subscription cost. So, it's hard to say how much it exactly costs because it highly depends on the exact scraper you're using. But in my case, I use uh I think I use the $40 plan and you can do quite a lot of scraping across many different softwares. So here you can choose your scrapers. For example, the LinkedIn job scraper is does have a subscription, right? But usually do have a trial period too. And then here you can also try it out inside of Appify if you click start. And when you try it out, it will also automatically appear into your actors, right? And here you find all the actors or all the ones that you've used. Now, let's say I want to add all of these to the MCP, all of these scrapers. Uh, first, of course, we need our API key, right? Our token. So, we go here to settings, but then we go to API integrations. And here we have our personal API token. Here we can just copy this. We go back to that little code script that we got the JSON. We add in our Appify API token. And then, of course, we want to add in all of these all of these uh these scrapers. So the way I do it to do it fast because of course you can do it manually just by opening this and then copying this part and adding it into the scraper and then you can separate it here in the actors right and you can separate it uh by comma and you can add in as many as you want. So usually what I just do is I copy this. I go into Claude or whatever GPT, right? And then I add in an image of all of the actors that I want. Please add all the actors to the actors key of the image. Good. So now ChatGPT has added them all here as you can see. And now I just copy this. Now how do I connect it to Claude? I just go to open the Claude desktop. There in the top bar I go to settings. There I go to developer and here you go into edit config. There you get uh the configuration JSON of Claude. You open that. Now you can see I already have my MCP servers, but yours will probably be empty if you haven't done this before. And all you do is you paste in uh that JSON. You save it. You can open this in any code editor or just in the text editor like I'm doing here. Right? Then you save it. You close it. And you have to restart Claude. And now usually it takes a little bit before it appears. Yeah, you can see now I got um the scrapers here connected, right? Tweet scraper, Facebook group scraper, etc.
Now let me show you how to set up the Air Table MCP. Now my last video actually showed how to set up Air Table through n8n, but it's actually far easier through another platform which I just discovered which is called Composio. And Composio is basically a third-party provider that makes it really easy to connect uh MCP servers to some popular softwares, right? So, as you can see, instantly connect to 100 plus managed MCP servers with built-in authentication. So, we don't even need an API key from Air Table. They already make this easy for us. So, you can see we have lots of sort of popular softwares here. Now, unfortunately, they don't all work in my experience, but the Air Table one does. So if you go to Air Table and you open this and by the way this is completely for free uh I don't know how they make money but here you can go to Claude then you can generate here uh the command and all you do is you run this command in your terminal and basically will install it. Let me just delete this. So you open the terminal you can install it right here and you can see all done. Please restart Claude for changes to take effect. So if I restart Claude, I should now have the Air Table MCP already there. And you can see I got Air Table here. Uh and what uh Composio does for you, it can now set up because we of course still need to authenticate connected to our to our account. We can now directly do that from Claude. Uh you can see now the the Appify ones are also loaded. But now I can just say uh please connect uh my account my Air Table account. You can see it's using the tools from the Air Table MCP to check the active connection. It looked like there's no active connection. So let's set one up for you. So it's setting up a connection for us. And now it gives us a URL here which we can just open. Then we can make the connection and authorize this MCP basically. So we can uh share a specific base here or our entire workspace. Uh so let's say I want to share this one. I click on grant access. Authentication successful. And now it should be set up. And the nice thing here is we already have 13 different uh Air Table tools inside of this MCP. And the last time I showed it in n8n, we have to set up all of these API calls separately. So to set up 13 API calls in n8n can take quite a while while this takes you literally 2 minutes, right? And we have all the actions we need to take inside of Air Table. So please check uh the last Air Table record just to see if it works. So here you go. Right, last record uh in ideation table.
Now lastly for Perplexity uh you have a full step by step on how to do it with Perplexity but it's a very similar process. Also make sure to add this link of how to connect Perplexity in the link in the description below. Uh but you'll basically end up uh with this one. You have to go with the npx, right? which is a very similar JSON again, right? You can copy that again. And then you can see you'll need your API key, right? So, we only need that. Now, to get your uh Perplexity API key, you do need a paid account. So, you can go here to settings uh to API and then you do have to uh add some credits uh to be able to use this, right? Then you can generate the API key there. Uh you add it in here. Right? And then you do the same process again. Right? Now in this case, let's say you want to add two. The way I usually do it is I just tell ChatGPT. Right? I just like add this one, this MCP to uh the previous one and then it will format the JSON correctly. You can do this of course yourself too if you know how to work with JSONs, but this might be faster, right? So it adds them both together. Then you copy that one. You go to Claude again. You go to Claude settings again. You do the same process, right? Edit configuration, you add the JSON and you save, you reopen Claude and you should be good. Now we can see we have Perplexity here.
So that's it for the setup to the MCPs. I think very straightforward and especially with these MCPs like the the Appify one for example where we want to implement many API calls inside of one uh MCP or Air Table even where we have 13 different actions then it can be a lot faster using the manual setup uh or the third-party software sort of provider than doing it uh through n8n because in n8n we still have to set up all of these separate nodes for these different API calls uh ourselves which can be quite time-consuming for softwares with many API calls. Anyway, that's it for this video. I think there are many really interesting use cases with these Claude workflows and I think we will see more and more adoption uh because now everyday people can start using uh automations right inside Claude and probably very soon inside ChatGPT. Anyway, if you got any value out of it, I highly appreciate a like and a subscribe. Also, uh I did do have some other tutorials on Claude workflows and many other tutorials. Uh I'll make sure to link it up here too if you're interested. And uh you can also check out my community if you're interested in learning how to build no-code automations here inside Claude in n8n, Relevance AI etc. And we also have uh one-on-one unlimited uh AI consulting and tech help available in the community. So if that's interesting to you uh you can also check out my community. Thank you so much for watching and I hope to see you in the next one. Bye.