The Bug Hunter’s Methodology Jason Haddix @jhaddix

Awesome. Hey everyone. Uh, my name is Jason, and uh, thanks for joining us this morning. Uh, super big thanks to the Red Team Village for having me and all the other speakers on, and the staff for facilitating everything that's going on with the conference. For DEF CON for hosting the village—uh, big place in my heart for DEF CON.

So today I'm going to talk a little bit about uh my methodology for recon. I've been doing this talk called "The Bug Hunter's Methodology" for five years now, and I basically update it every year. Um, and I talk about new tools and techniques um in different spaces of the bug bounty scene and red teaming. So today we're going to go over the recon stuff.

Now, the Bug Hunter's Methodology, TBHM, is a presentation, like I said, running for a long time, and basically it's too big for any one conference slot. So I split it into two sections: one is recon, which we're going over today, and the other is application analysis, which is on-site hacking and and stuff like that. And so today we're gonna do recon, and um, you know, I'm working on this year's version of application analysis; haven't finished it yet. I'm a giant slacker, sorry. So uh, so yeah, today we're gonna go over recon.

A little bit about me: so um, I'm a husband, uh, father, hacker, gamer, and I stream sometimes. These are my socials, so if you want to reach out and ask a question after the talk or something like that, just hit me up. Um, I was a bug hunter, and then I worked at Bugcrowd for uh many years, and now I'm the head of um security at Ubisoft, and uh, so I lead the security team there. Awesome team, great people. And I'm also a gamer at heart, so I play some video games. These are uh, these are my kids, Arcadian and Avalon in the top picture. I took them to DEF CON for the first year last year, did DEF CON Kids or Roots; it was amazing. They had a great time. And then that's my son Arlen uh, him and I being pirates in the bottom bottom right-hand corner.

So it's a little bit about me. I've been doing pen testing for about uh, well, a long time. I'm old, and uh, yeah, so I have a lot of uh, I have a lot of related experience to this talk.

So the first thing I want to talk about is: today we're gonna, we're gonna go over uh recon methodology. And we do recon in red teaming engagements, and we do recon in bug bounties primarily, and you can do it in some wide-scope pen tests as well. Uh, and one of the first things I talk about is: is when you're doing an assessment, some sort of assessment, whether it's one of those three, you have to have a way that jives with you to record your work; otherwise, you're just throwing darts and forgetting what you're doing. And um, this is uh, this is counterproductive to to basically, you know, keeping organized.

So um, the first thing that I talk about here is how I keep notes, and um, and I use a tool called XMind, which is a mind-mapping tool. And uh, basically I create nodes, and I create methodologies as checklists inside these nodes, and I track all my domains and which ones I've worked on by color codes. And um, so this is something I do. I've seen other people do a lot of project tracking for assessments in, you know, just Notepad or Vim, or there's some specialized pen test tools. You know, if you're really fancy and you have a subscription, you could probably do it in something fancy like Dradis or something like that. So um, so there's a lot of tools, but it's just really important to track your work when you're doing recon. Recon is is the art of finding as many assets as possible related to a target, and it can get, it can get pretty data-heavy and dense as you do it. So having a a way that works for you to track your data is is the first thing that's going to guide you to success.

So this is an example of how I track my data when I'm, I'm on a target, right? So Tesla, Tesla Motors has an open scope, or pretty open scope bounty on Bugcrowd, and this is how I would start off my project with Tesla, right? So in the middle I have a node in this mind-mapping program I use called XMind, and then on the right-hand side, on the left-hand side, I have their autonomous system numbers, their ASNs. I have any acquisitions they've made. I have some other notes, LinkedIn discovery, and a link to their reverse whois information. And the right-hand side, I'm starting to build out um their root domains or their seeds, or some roots or seeds or um, our terms, you know, for things like something.com or teslamotors.com or tesla.com or solarcity, right? Um, and so I've started to build up, I've started to build up their seeds on the right-hand side. So you can see Tesla Motors, Tesla, Solar City, et cetera, et cetera. And as I do this, I'm gonna start collecting a lot of data. As I do recon on Tesla, I'm going to collect a lot of data, so that turns into this, which on the left-hand side you can see for tesla.com, that seed, I've enumerated all their subdomains and all of the live links, right? So these are all the live links for tesla.com for their subdomains, so things like, you know, pages.tesla.com, shop.tesla.com, etc., etc. And then you can drill down into any one of these. And so if I drill into www.tesla.com, then I have my methodology notes for that single site, which, you know, uh, inside of that node, I, I keep all of the questions I ask myself when I'm, when I'm basically looking to hack a website, right? Like, does the site have multiple user roles? Does it, how does it reference users? How does it handle special characters? You know, what are dynamic parameters? Does it have an API component? What kind of errors am I seeing? You know, does it have file uploads? Have I done JavaScript analysis for past? Have I done content discovery, uh, you know, all these questions that you're normally going to, you know, kind of keep in your methodology, um, I apply, I apply these all to each one of these subnotes, and then I color code my work based on where I am, right? And this is one of the most important things I think is that um, is that everything that is not filled in hasn't been worked on yet. So most of this hasn't been worked on yet, but everything in orange means I'm currently working on something, and everything in green, which is there's nothing green on here, I've already done. And so just the, you know, the way I can do this, and then I can add check marks and other stuff in XMind to these mind maps helps me track where I am, so I can easily put down a project and pick it back up if I want.

Okay, so today's mission: we're going to talk about uh wide recon. This is the art of finding as many assets related to a target as possible. So you're a red teamer or you're a bug bounty hunter, and you have a scope of X company, and uh, you know, one of the things you need to do is identify all the websites that they own, because the more websites you identify or the more infrastructure you identify, the better your chances of getting in are, finding a bounty, right? And this is, this is a core component in, you know, both of those skill sets. And I break down what we call recon into a couple of domains: so um, finding first the in-scope domains via the program or your project brief; uh, your finding acquisitions for the company or the target; doing ASN enumeration; doing reverse whois; doing a whole bunch of subdomain enumeration; and then doing port analysis. And then we're going to go into some related topics uh, like uh some vulnerability scanning and some um, and some automation information related to recon, because you'll probably do it in this phase of your workflow.

Okay, so you've decided you're gonna do an assessment, or even handed a red team assessment, or you've been handed a bug bounty that's wide scope, right? That's, that's where we are right now. So the first thing we need to do is is parse the program page. So I'm going to use a bounty here, a couple bounties to illustrate, you know, where you would start, right? So here is uh the brief page for the aforementioned Tesla um for Bugcrowd, right? So in their brief they have star.tesla.com, start.tesla.cn, Tesla Motors, and Tesla Services. And then they also have this catch-all sentence here that says any host verified or owned by Tesla Motors that you can find, you know, is also included in scope. So this is what we consider a wide-scope bug bounty. There's lots of stuff that you can do with this. Um, so for your first four seed domains are already listed here: tesla.services, teslamotors.com, tesla.cn, and tesla.com. So we're going to focus on these four and add them to our list, and then anything else we can find that is Tesla's is also fair game. So that's great. If you look at a platform like HackerOne, they also have a pretty wide-scope bounty, and probably the, the most prolific bounty available on the internet, which is Verizon Media. This is like the granddaddy, the biggest scope program I think I know of in the bug bounty scenes. Verizon Media has gobbled up so many brands and had, they have so much infrastructure related to these brands and sites and subdomains—it's giant. So like a lot of hackers make their whole career uh hacking on Verizon Media, which is uh, which is awesome, and they're a great team, and they support the bug bounty community. They also have this catch-all kind of phrase in their scope that says: if you found a vulnerability that affects an asset belonging to them but it's not in scope, report it to the program. And so um, it's a really advanced team to work with. You know, you couldn't even fit all of the domains that you have for this program on one page, which I didn't even try. So uh, so you will parse your seed domains and your in-scope domains from the program page to start off, and this will give you a starting place to to work from.

The next thing that I do uh when I'm scoping out a company is I want to find all their acquisitions. And this is a couple of, couple reasons I want to do this. Um, I want to basically uh see if any of the infrastructure for those acquired companies has not been taken off the line, um, or any of the subdomains or websites or integrations that were too hard to port over to new infrastructure are still online, right? Um, this gives me more and more, you know, infrastructure and sites to hack. So to find the acquisitions, uh, I use a tool called Crunchbase. And Crunchbase is a business intelligence portal. Basically, you add any business name to this search box on Crunchbase, and you'll be able to look at that company and find out what are their employees, you know, are there investor rounds? What acquisitions have they had? You know, what are their socials, et cetera. So here I've used the example of twitch.com. We're streaming on Twitch today, which is awesome. Um, and Twitch, you know, Twitch as an organization here has an entry, and if I click on that, I get a lot of information about Twitch, and I can see that under their acquisitions tab they have acquired four companies in the uh last eight years: Revlo, Clip Mine, Curse, and Good Game. And then also they were acquired themselves on the left-hand side by Amazon. So this gives me a good idea of uh, you know, what they used to be, where I can also look for some infrastructure. If it's a recent acquisition, like Revlo for instance, you know, if I click on Revlo in this page, it'll take me to Revlo's page, which will give me their main domain, which is revlon.com. So I might put that in scope of, you know, a wide-scope, you know, assessment or a red team assessment, right? Acquisitions are important to keep track of. Now, the one thing that you want to make sure here is that on some of those brief pages that we looked at in the last couple pages, some of these are explicitly out of scope. So they'll say in the brief, especially for Tesla, Solar City is not in scope of their wide scope, uh, so even though we found that as an acquisition of Twitch when I looked on this page, you know, when I've done it many times, um, it's, it's explicitly out of scope. So remember to refer to your project page or understand what is okay and what is not okay to hack or do recon on.

Okay, so after I find the acquisitions, which has given me, you know, seed domains like revlon.com and, you know, new, new seed domains other than Tesla Motors and Tesla and stuff like that, um, or any of these, you know, top-level seed domains, what I want to do is I want to find this company's autonomous system numbers, right? So uh, every company that gets large enough ends up, you know, having a collection of networks uh, basically applied an AS number to them. And an AS number, autonomous system number, is just a collection of your known IP ranges for your networks, um, for your infrastructure. And so you can find someone's autonomous system number in many, in many ways. There's many search engines to do this kind of lookup, but I use one called Hurricane Electric or bgp.he.net. I use this uh when I'm doing manual searching because it has a freeform text box at the top, and I can just search the keyword Twitch. And here you can see I found Twitch's two autonomous system numbers: AS46489 and AS397153, and then their associated IP ranges, and IPv4 ranges and IPv6 ranges. Um, so this is a good representation of the infrastructure that, or the IP space that they own, and I can assume that it belongs to—I can only guarantee that it belongs to them—I don't have to worry about scope really here because I know that, you know, all this is owned by them. So if they have that catch-all phrase in there in their bounty project, or you know you've got the go-ahead in your red team engagement to, you know, open scope, then you know this as your target. Um, one thing that this doesn't represent is their cloud assets. So um, this is all owned IP space; it doesn't represent things like AWS and Azure and GCP ranges, so you won't find stuff that they host in those environments inside of these ranges. So yeah, you just want to be cognizant of that.

So here I've searched Twitch, I got their ASNs. With every tool or method here uh in my methodology, I like to give you a manual way to do it because context is important and getting used to the idea is important. Then I like to give you an automated way to do it in case you're going to script up your own recon framework, which is all the fad these days, and I have one of my own and everything. So um, there are two tools you can use to parse ASNs and IP ranges from companies that exist on the command line, right? If you're going to script this stuff up yourself. So one is called Metabigor—I'm still not sure if I'm saying that name of the tool right—by Jessejjjj, and this one utilizes uh the site, the site that we just saw, bgp.he.net, and another site called ASN Lookup, and it will grab the information from those websites, scrape it off, and give you your IP ranges for an organization. The other one is called ASN Lookup by essene; same idea, it uses a different data source called the MaxMind database; it'll pull off their IPv4 ranges from an ASN, and you can give it just a term, you can give these both tools a term instead of um an AS number or anything like that, so they, they will pull it off based on uh, you know, that keyword that we're looking for, in this case Tesla or something like that. Now, um, one thing to know about these tools is they are looking off your keyword here, which we put on the command line, Tesla, and um, you know, there are multiple companies with Tesla in the name; there's like Tesla Research Lab or something like that in Sweden or something like that. So um, you know, just be sure that when you get these ranges and you start seeing these sites, you can identify, oh no, I might have gotten one that's actually not Tesla Motors. And so um, both these tools I verified, you know, pull up the right data for Tesla, so, but uh, yeah, just be cognizant of, you know, since you're using a keyword here to search that you don't get other IP, IP ranges that are not your targets. You'll have to verify somewhat manually there.

Okay, so we've gotten some IP space from the ASNs, we've gotten some seed domains, we want to continue gathering as many seed domains as possible. So um, we have the AS number, and and for here we're going to use Twitch as an example again. Twitch, for their autonomous system number is 46489, and in here what we wanna do is um, we're going to get our first introduction to a Mass. And a Mass is a framework for uh domain intelligence, I would call it I guess, and it is written by Jeff Foley. There is a workshop you should absolutely sign up for; he's going to do a full workshop on Mass and how to use it. Um, Jeff is associated to the Red Team Village; that's super cool. And here we're going to use Mass and feed it our autonomous system number, 46489, and what it's going to do is it's going to go out to all of the IP ranges that are represented by that autonomous system number, and it's going to scan the certificates for all HTTPS sites, and then it's going to parse the certificates and tell us, hey, here are the seed domains I found that are owned by them. And here you can see that we found some seed domains that we didn't know about before: justin.tv, which is the company Twitch used to be; ttvnw.net, which is I think associated to Twitch's streaming protocol; twitch.tv obviously, which is their main site; TwitchCon, which is their conference; and Socialcam, which I have no idea what is. Um, so yeah, and all the links for these tools and the methods are, you know, posted in the slides, so you can click through when the slides get released and go grab the tools and play with them on your own. So here we've already expanded our uh, we've already expanded our assessment a lot, right? We have a whole bunch of IP space, we have a whole bunch of top-level domain, C domains, and we have the C domains already given to us by our brief page.

Okay, so the next method is reverse whois. And what reverse whois does: it takes the whois entry of, you can, you know, search your domain twitch.tv here, and and you can do this with a lot of sites. I use hwoxie.com because it's uh pretty cheap, and you can get a free API key for so many uses, and uh, you basically supply it your domain, and it will give you the, you know, whois data, and then you can use that whois data to correlate uh, you know, who else has registered stuff. An example here is: we gave this site hwoxie.com twitch.tv, and then it said that over the course of the years there have been several companies names related to the whois data and also uh registered uh emails. And so here you can see that the company uh as of the, you know, March 26, 2015 was justin.tv, and the domain and the email, the domain was domainmaster.justin.tv, and then later in 2015 it changed to Twitch Interactive. And if you click the button right next to those entries, you can see there's 20 other domains still registered to justin.tv, which could be interesting to us, and there's 575 domains registered under Twitch Interactive. And so we can start pulling back more data, more subdomains, more root domains from um, from these links when we click on them. Now, one thing to notice to know here is that whois data is is registered data, and so a lot of these companies will park domains um to combat phishing and um, and to hold stuff for marketing campaigns and stuff like that. So they might not be live sites; they might just be parked. So this is a medium-fidelity type of technique here, but I have found some really good stuff using reverse whois uh and the registrant data of these sites. There is a tool to to do this type of analysis; it's called DomLink, written by Vincent Newby Security on Twitter, and it uses that exact site, hwoxie.com, and you can basically give it a domain, and it will find every other associated domain by both registered email and organization name or company name, um, and it'll do it on the command line and return it to you in a script, and it's recursive. So a really cool tool here. You can use this to automate some of that reverse whois lookup; it's called DomLink, the tool's called DomLink.

So the next method that we have, now that we've maybe gotten some more seeds and subdomains from reverse whois and we're building out, you know, our list of things that we can hack, is we want to look at the ad and analytics relationships of our target site. We want to see what other sites are using the same ad and analytics codes as our main site, and this will give us a good understanding of maybe what are their most popular domains, and maybe what are some of the less popular domains but are still using the codes and we didn't know about before. So every page usually embeds a Google Analytics code or a New Relic, you know, code or some kind of, you know, ad or analytics code in the main page. And so here if we use a site called BuiltWith.com, we can search twitch.tv, and twitch.tv, it'll give us a list under the Relationship Profile tab of all of the keys that are on twitch.com or twitch.tv, excuse me. And then you can click on those and see what other sites are using the same ad analytics codes, and maybe we didn't know about those before. So here we can—it's hard to see on the bottom—but we have a couple entries that we didn't know about before: manversusgame.tv, twitc.tv. We knew about twitchcon.com from the previous method, but we're starting to build up more uh domains that we can see that are related to our, our target. So uh, ad analytics relationship profiles is a pretty powerful technique; um, I have used it for success a couple times. You can also do this directly from Firefox or Chrome; they have an extension for this site called BuiltWith, and you can just install it, you make a free account, and then you visit your site in the browser, click the Chrome extension button, and it'll get you the relationship information right in your browser, and you can start clicking around there. So uh, you know, pretty easy to institute while you're, you know, doing your recon in a browser. So you can also use a tool to do this in the command line. So uh, Malik, after my first iteration this talk, uh, quickly scripted up an awesome tool to do this; um, it's called getrelationship.py, and you just pass it your target; here he's targeted uber.com, which also has a bounty program, and pass it to this Python script, and it'll pull out on the command line all of the related domains, and you'll have to manually kind of verify these, but the ones I would look at first, the ones that actually have Uber in the, you know, in the domain somewhere, like uh, you know, daftuber or, you know, driveuber.cod.nz or things like that. So um, you'll have to sift through this, but you can do it on the command line now too.

Then you can just use some general Google hacking or Google Fu to try to find uh associated sites to your target. Um, you can use the copyright text, the terms of service, and the privacy policy, and you can just copy those from the bottom of any of the pages. And here you can see Twitch Interactive Inc would be something to Google for, and then you can see any other page that is hosting, you know, that text at the bottom of their page, which could be something that you didn't know about before in your, your recon so far. So you can just do this manually, and it, you know, you can use other search operators like `inurl:twitch`, or you can just look for these policies. One tip here is that uh you don't just want to target the certain

You know a certain year, like here I have 2019, but you also want to check 2018 and 2017. In fact, the more stuff you find that goes back uh farther in age, the more likely it is to be vulnerable and less likely to have a lot of security assessment associated to it. So probably more vulnerable for your red team engagements or your bug bounty. So all right, the next method that we're going to use to find some seed domains or related infrastructure is showdown. I have to admit I'm not a showdown ninja, but I know a lot of red teamers are, and I actually love showdown; it's just a failing of mine, I need to get better at it. But there's a ton of search operators we could use here on our domain twitch.tv. And what showdown is? It's this site that is a is a site that hosts information from an infrastructure-based spider. And a spider goes to every site on the internet and basically captures um its HTTP responses, its technology stack, its certificate data, and cross-links it. So you can click on any of these pieces of information and find out other things it's seen related and and what organizations it's related to and what, you know, what common tech that they're using, what web servers, what, you know, JavaScript frameworks. You could do anything. So showdown is a really powerful tool, and you can query our domain here and try to find uh try to find related infrastructure. For instance, in in one of the entries when I just didn't even use a great operator here, I just searched my domain twitch.tv; I ended up seeing in the SSL certificate data uh of this search there was twitch.amazon.eu. And uh now I need to ask myself a question like, is that in scope? Right? Like that's not verbati verbatim a twitch domain, but in a red team assessment, if that's related to twitch infrastructure, well maybe uh maybe it's a target that I can go after. So showdown can give you a lot of good information.

Okay, so we found seed domains, stuff to start with. Now we're going to do is we're going to try to find subdomains for those seed domains. Um, and so these are these are things like uh dub dub you know www is a subdomain, but also admin.tesla.com or uh, you know, forums.tesla.com or something like that or forums.twitch.tv or whatever. We're going to try to find subdomains. Uh, and so this idea of starting with seeds and then moving to subs really is going to multiply. The more seeds you find, the more subdomain enumeration you can do, the more subdomains you find, or the more is related to how many sites you find, the more sites you find, the more successful you'll be in time inside of your red team engagement or bug mounting program.

So for subdomain enumeration, um I use, you know, three-ish different methods to get subdomain information. One is linked in JavaScript discovery. Uh, another one is subdomain scraping techniques, and then subdomain brute force. And then there's some auxiliary stuff I use as well. So we're gonna run into those right now. This is the bulk of what a lot of tools are doing right now, and and so you'll see a lot of favorites in these slides. So the first area we're going to use to find subdomain data is our first technique is linked discovery and JavaScript discovery. So link discovery, what is it? It's basically using a web spider to visit a site like twitch.tv, um land on it and spider all the the HTML links. Right? Pretty simple, right? And any of those that come back with um, you know, a sub-domain, uh we just add to our list of in-scope targets. Right? And so this is pretty self-explanatory. It happens naturally when you're using tools like Burp Suite and you're spidering a site, but I'm going to walk you through how I do it. I'm going to use Burp 1.7 because I like the UI better, but it works as well with the crawler or the scanner in Burp 2.0. So uh, you know, use whatever you want. And then I'm going to give you some other methods to do it.

So the first thing we do is uh we load up Burp Suite, which is on the right here, it's side by side to our site, and we just visit our site through Burp Suite the proxy. And Burp Suite the proxy will capture all this data. We've only visited one page. If you visit twitch.tv ever, you see that it has crosslinked and it does request a lot of stuff because it's a streaming media site, so there's a lot of stuff going in the background when you go to twitch.tv. Um, so you can see all that data on the right-hand side. This is everything that's either seen link seen been linked on this page or it's actually been requested. Actually been requested as the stuff in black, seen as the stuff that's in gray. And so what you do is you visit your page, and then you have to set up a rule, uh some type of rule, and you you can do this by going into the target tab and the scope tab, and then you click this box that says use advanced scope control. And what I do is I enter in just a keyword here in advanced scope control, which I just say twitch. Right? So I want to see in Burp anything that has twitch in the URL at all, no matter where it is, it has that word twitch. And you could also add some more here because we've seen in our previous analysis twt uh is also part of their domain naming nomenclature. Right? And so you can add a couple of these scope rules, and then you go back to your site map and you click on the ribbon bar up at top or the filter bar and then you say show only in scope items. So now this will just show in our site map all the twitch.tv links. Right? So we already have a good link, a good list of subdomains here. Right? sentinel.twitchservice.net, api.twitch.tv, you know, so we have a lot of stuff with twitch in the name here. We know that most of this is probably related to twitch. This gives us exponential uh areas to hack. Right? And this is pretty good, but if we select all these and then we spider them with Burp using Burp spider, it will then go to all of them and find their links and find subdomains references in their pages. And we can do this recursively until there's nothing left to find. So here I'm going to select all of these, and then let's spider them. And now you can see since I had them selected, everything in orange on the right-hand side was stuff I knew about before I spidered, and then after I spidered with Burp, everything in white is new stuff. And you can see I've found a combination of a lot of stuff here. I found subdomains for twitch.tv, and I've also found some new root and seed domains like twitchapp.net and twitchservice.net and xt-twitch.tv. So this is a hybrid technique. You can find your new seed domains or roots and subdomains. And so now I have all of this, you know, domain data. I can now select all of these and spider them again and spider those pages for links, and eventually you can do this until you get kind of spider fatigue, and eventually you'll have a large sitemap buildup of targets, which is awesome.

Now, how do you get this data out of Burp Suite? There's not a great way to do this. Pro has a thing called uh engagement tools, and in engagement tools you can do this thing called analyze target, and then in analyze target you can create HTML port an HTML report of all the targets in your site tree that are selected. So that's what I do. I select everything in the site tree, I go engagement tools, analyze target, generate HTML report, and then in the HTML report I have a parsable list of the domains that it's seen at the beginning of that HTML report, and then I take that and I put in a text file for later analysis, and then I dump it into my mind map as well.

Okay, so um the whole the whole linked discovery thing counts on Burp spider. Right? A lot of times I hadn't seen, you know, before last couple years, great options to do this in the command line because, you know, it will require a lot of custom coding, um, you know, bash scripting or Python to create my own spider and do that same process. Well, now there does exist some tools that are command-line spiders with bug hunters and red teamers in mind. There's two; there's one called go spider written by Jesse JJ, and it also designates the types of things it's parsing when it's visiting a URL, so it'll give you okay, hopefully you guys can hear me. Headphone problems, gonna plug it in. You can hear you? Okay, great, great, great. I was getting beeping in my ear, so maybe it's just running batteries. My bad. Okay, so um so you have two spiders here that you can use, the fan like ghost fighter uh and hack crawler by hack luke, and uh both these are awesome. Um, they both have some functionality uh that will parse out the types of things you're getting like JavaScript files, uh subdomains, URL endpoints. Um, some of them will give you I think parameter names and stuff like that. So uh you could institute the whole process of link discovery by scripting up um ghost spider or hack crawler if you wanted. I still use Burp, but these are invaluable tools to have at your disposal, just like a crawler that can do some analysis. So keep them bookmarked for when you might need them.

Okay, so the next place we're going to get subdomain information is by analyzing some JavaScript. And one of the tools I like here just because it has this kind of little added benefit that I think that I haven't seen many places before, but maybe some other tools are starting to implement it now, is a tool called subdomainizer by Niraj Edwards. And what it'll do is it'll take a JavaScript file. You have to point it to a JavaScript file that will parse out all the cloud services, these subdomains, um and it'll do this little extra thing where it uses the Shannon entropy formula or algorithm to identify things that look like API keys hard-coded in JavaScript, which is already kind of a vulnerability if you find a private API key or credential hard-coded in JavaScript, which I know sounds crazy to a lot of people, like you would just find a hard-coded private API key, but this happens all the time, like all the time it happens. Um, and so this uh this is what I like to call like a forward-thinking type of thing is using like an algorithm like this to identify keys. Sometimes it's a little noisy, um but a lot of times it finds you good stuff. So uh so I like this tool to point at all the JavaScript files I found already, you know, on you know on some of these sites. Um, if you're just looking for subdomain information, there's another tool called subscraper by Celian Collins, which has recursion built into it, which can do this method as well, but it doesn't do the API key part. So point this at JavaScript files you find on your site, and you can get back a whole bunch of um of uh subdomains and cloud cloud services that the site might use or the target might use.

All right, so that's link discovery and JavaScript analysis for subdomains. Now uh we're going to get into the big meats of what like a lot of people do is subdomain scraping. Now the idea of subdomain scraping um is going out to these websites on the internet uh that have search boxes basically. So um, you know, there's all these projects on the internet, search engines, security websites, certificate-like projects and stuff like that. They all do different stuff. Right? So census or like rob tex gives you infrastructure information. The Wayback Machine, you know, houses information about domains and their responses, you know, years past, and you know everyone's probably used the Wayback Machine. Um, the certificate sources down here, some of them are certificate projects to provide certificate transparency. Search engines obviously, you know what search engines are. And then there's a whole bunch of security sites that, you know, do different things like give you a rating on how malicious a URL or a site might be. Right? The common thing that these all have is either have an API or search box where you can put in a domain, and they will search that domain, and they will tell you anything they've seen related to that domain. And if you put in a domain like tesla.com, the information that comes back is parsable, and you could possibly find out that they know about some subdomains of tesla.com that we don't know about. So this is the process of subdomain scraping. We're going to go to all of these sources, all of these sources and ask them, hey, what do you know about tesla.com or twitch.tv? Do you know of any domains that maybe I don't know of? Okay, cool, let's do that now.

Um, there are many more sources that are on this page. Right? There's uh these are only a subset, and new sources to parse are coming out as fast as new websites are coming out. So the tools have to keep up um adding new uh novel sources to find subdomain data or URL data um, you know, as they as they mature. Now this is the example of using a search engine like Google to do it. Right? So here what we're doing is we're searching uh with the search opera site:twitch.tv, and we're saying I already know about twitch.www.twitch.tv, so we're saying minus www.twitch.tv, and we already know about watch.twitch.tv, so minus watch.twitch.tv and minus dev.which because we already know about that. So now Google is only showing us things that are not those. And so we can do this process, keep on minusing out domains until twit until Google doesn't know anything more about subdomains related to twitch.tv. And and that way we can basically get a full inventory of what Google knows and what Google knows about subdomains for twitch.tv. So this is the an example of doing it manually with Google, but luckily you don't have to do this yourself. There's tools out there that will do this type of analysis, this scraping for you. The first one is amass by Jeff Foley, and the amass team is a whole team behind on this. And there's two tools I use here at amass and something, but we're going to look at amass first. Amass has probably the most sources for any subdomain scraping tool in existence in its enum section. Right? Amass is a framework for domain information, but the one that actually pulls this method's uh results, subdomain scraping, is called amass enum. And here on the right-hand side, you could say we we gave it twitch.tv, and it went out to a whole bunch of sources it's had in its databases, and it reached out to those web pages on the, you know, using curl or, you know, whatever, I don't know if they use like headless chromium, I don't know exactly what they're using, um and then they parse those pages for subdomains for twitch.tv, and they give us a list of all the subdomains for twitch.tv. Um, and if we do this on all of our seed domains that we've gathered, we have now started to exponentially build out the amount of sites that we have to attack. So this tool is invaluable. Amass is becoming the go-to tool for all subdomain enumeration, uh especially this this thing, subdomain scraping. The other tool I use here is oh, also this is still amass. What amass does at the end of a run, not only does it give you all the subdomains, but it also gives you this great table, which I feel like is underutilized a little bit. And this table tells you, okay, I discovered 439 subdomains, and here is where they were inside of these ASN's. And um you can see that most of twitch's were in Amazon's ranges, which made sense because they were acquired by Amazon, but you can see some of them were in other ASN's. Right? And I've had instances where not on twitch but on on other projects where there was a whole ASN related to the company I didn't know about that I found out because amass built this table. They're like, hey, you know, 70 of these subdomains we discovered appeared in this ASN or these IP ranges, and then I look at that and I'm like, oh, I I wasn't initially looking at those ranges. Weird, let's go back to the beginning of my workflow and start enumerating those ranges. So this table is super cool. It also gives you information on on their third-party kind of tools they're using like uh here you can see that uh, you know, twitch twitch is using bitly and sendgrid and some other stuff, so it also gives you information there and fastly. So yeah. Okay. The other tool I use for subdomain scraping is subfinder, originally written by Iceman and Michael Skelton. I think now man navigated to the project discovery.io team, which is a group of bug hunters releasing some stellar tools, and they also have multiple sources, extensible output. Um, also a really great tool. Uh, both of these tools are really good. They have some each one of them have different sources. So what I end up doing is in my automation I run both of them, and then I cat the output together and sort it and unique it from both the tools. So they both have different, you know, a couple of different sources and a lot of the same sources. So that's what I do for my stuff is I just use both of them.

Okay, so this one is somewhat new and it has been integrated into amass a little bit, but uh I've had inconsistency with run between this standalone tool, uh github-subdomains.py, and the output of amass. So I still use this independently inside of my automation when I'm when I'm looking for subdomains. So this tool is called github-subdomains.py, and it's still scraping. And what this is doing is it's going out to GitHub as a source, and basically you're providing GitHub, and if you've ever been on GitHub, they have the search box up top, and you're saying uh search for twitch.tv, and then anything that comes back with twitch.tv as a piece of source code, it will parse up the subdomains from that piece of source code. Now this uh this was written by uh Gwendolykoo Kwik, and um I still don't know if I'm saying Wendell's name right, and that's okay, uh and um Gwendole uh wrote a whole suite of tools for GitHub enumeration, how to find secret keys in GitHub related to an organization, how to pull email addresses out. He has a wonderful blog, and it's linked here in this slide with a whole bunch of GitHub tools. Now github-subdomains is just looking for subdomains. Um, the thing about using this tool is that the GitHub API, not the tool, um is just kind of instable and and doesn't give you like somewhat returns, rate-limited results sometimes. So what I do inside of my op automation, and this has given me subdomains I haven't found anywhere else using this method, is I run GitHub search or github-subdomains.py um like five times, and I sleep in between each run so that I give it a little while for the rate limiting to die down. Um, and then I give it a big sleep at the end before I run another one, and then that seems and then I cat all those results together and unique them, and that seems to give me more consistency um when parsing the API, and it has nothing to do with the tool that Wendell wrote, it's all about what GitHub does with their search API. So this is an awesome tool to find lesser-known subdomains, and I found some great stuff using using this tool.

Okay, the next one is shoved as a show sub go by incogbyte, and this one will parse using a number of search operators with your API key, it'll parse um showdown. So I include this in my script as well. Again, this is one of those tools that some of the subdomain frameworks like amass and um and subfinder might do for you. I find using the standalone tool for some reason just works better for me, so I don't know exactly why that is, but I've run them side by side and gotten different outputs. So I still use this verbatim, and it's also it's it's a fast-running script. It's not like I'm waiting for it to complete for five, ten minutes or something like that. It usually runs in a minute or two, so it doesn't skid off my nose if I have to wait an extra couple minutes to ensure that I get coverage out of showdown. So show sub go will take a domain and your showdown API key and parse showdown using some search operators and give you back all the subdomains related to here twitch.tv.

Okay, so the last, well, not the last, but one of the other methods to look for subdomain enumeration or subdomain scraping is the cloud ranges. Right? So we talked about this a little bit earlier is that we have like all this IP space, and now we've started to identify subdomains, and you'll notice that some of the subdomains are resolving to infrastructure or sites in the cloud. Now there's this idea that not a lot of people had been doing is just going out uh recently or not recently, but uh people have been doing it for a while, but it hasn't been much public is just scanning the entire ranges for AWS, GCP, and Azure for SSL sites. Right? Anything that responds to um anything that responds to uh, you know, or responds to a connection on 443, and you basically scan the cloud ranges, and you you scan it by IP IP address, and then when it responds it'll give you its SSL certificate, and then you parse the organization name or the domain name out of the certificate data, and then you look at that data and you say, does it match my target twitch.tv or does it have the keyword twitch in it? And you're like, cool, I found some stuff that these people have put in the cloud that wasn't part of their ASM that I probably didn't know about before. Now that's a tremendous amount of scanning. Right? Those ranges are huge, and you can do it yourself with some tools like masscan, and you could script it up yourself to scan um, but um and there's a guide here by Dahi Park which outlines doing it yourself, scanning those ranges, but it's going to cost you a little bit of money on your VPS that you're using, et cetera. There's a service by Sam Erb, and a DEFCON talk he did two years ago where he created a service called bufferover.run. It's an API, and you can give it a domain, and it will go out, and he does this scanning every once in a while. And so you basically take that data, you parse it, and then you get a list of of subdomains that were in the cloud ranges. Now I think he runs his scans every two weeks and updates the service, so it's not exactly live data, and this is one of the sources included in amass. Again, I just wanted to outline the single-shot tool here. You probably could feel safe using amass to get most of this data back, um but uh yeah, it's uh you can use the service as well. So um this method also finds some really good stuff. Right? A lot of people are putting shadow IT infrastructure and registering websites on a credit card or, you know, just not paying attention or thinking that anybody will find their cloud infrastructure because they've never published, you know, those domains anywhere other than internally. And so you can find a lot of things, you know, for your target organization that are just sitting in the cloud pretty much unsecured. In fact, it's been a big part of my research lately is scanning the cloud ranges and just finding wickedly undersecured stuff because people just don't think you'll ever find it. So this is a good method.

All right, so we've scraped a lot of stuff to find subdomains. Now we want to brute force for subdomains. This is just a, you know, tried-and-true method that I'm sure every red team or pentester has done before, usually with a tool like Fierce or, you know, one of the other tools like that in the past 10 years. You just try to give a word out of a dictionary and add it before your company.company.com and see if it resolves. Right? It's pretty pretty simple method. Now iteration in this field has come along in the last four years where, you know, a lot of our the tools that we were using to do this were great, but they were using one DNS resolver, one uh one DNS server to resolve, and it took a long time. I remember running Fierce as a pen tester, you know, 10 years ago, and it just taken so long to finish a large dictionary to do subdomain

Brute forcing, and uh, in the last, you know, I think four years, uh, this, uh, the idea of using multiple resolvers to speed up the process was pioneered by massdns. Massdns was the king for a little while. Now, mass also includes, uh, the idea of using multiple resolvers. So mass uses eight DNS resolvers, um, uh, to to parse DNS data when it brute forces by default, um, and you can add even more by using, um, some flags.

So, uh, here we're just running a mass over, you know, Twitch again, and we're doing brute forcing this time, uh, with the with the -brute option, and then we're adding the source so we can see where some of these came from with the scraping part. And then you can specify the number of resolvers with -rf. So rf means resolver file, I think, and if you basically give it a list of DNS resolvers or DNS servers that you trust, it will use more than one to do your your brute forcing. So this speeds up your brute forcing significantly. I've also heard that another alternative to using mass for this is asdns-brute. I haven't used it yet, but I've heard it's also wicked fast and has multiple resolvers.

So if you wanted an alternative, shuffle-dns by the Project Discovery team also does this. It's, I think it's a wrapper around massdns; it is; it is a wrap around massdns. And if you prefer to break out that type of brute force from mass for whatever reason, whether it's stability or mass, you know, is taking too long for you, I I haven't benchmarked the tools side by side, but a lot of people like shuffle-dns as well to do subdomain brute forcing.

A subdomain brute force tool is only as good as the dictionary or word list you give it, right, because it's just trying to resolve a whole bunch of words in front of your target, right, twitch.tv. And, uh, so in this mind, there's there's two kind of ways that you can approach what word list you feed to these tools. One is a tailored word list where you can build one based off of words that appear, brand names that appear, word, you know, you can build like a contextual base word list to, uh, to your target, um, which Tom Nom Nom, who's a prolific bug hunter, awesome human, have a lot of respect for this guy and the tools he makes and the contributions he makes to the community, he did a talk at NahamCon, which is a conference, uh, a little while ago now, a couple months ago, where he did a whole talk on word lists and how to generate contextual word lists for your target. So I suggest going to watch that if you want to make some tailored word lists. It's called "Who What When Where" word list.

Then you can also use a massive word list. And so over the years, we've had many, many tools that do DNS brute forcing. I went out, and I took the word list for all of them and put them into, uh, one, and it's called all.txt. It's linked in this presentation; people use it on my GitHub, um, and it basically sort and uniques all of the DNS, uh, names that we've seen forever. Now, it has a lot of crap in it, it's true; it's a lot of lines; it's like a million or two million, I can't remember; it's a ton. With using, you know, the multiple resolvers, it actually doesn't take too long to do the brute forcing. Um, I don't necessarily care if I'm sending crap to a DNS resolver, like, uh, you know, like 0.0.0.1.twitch.tv, like obviously that's usually not going to resolve, but, uh, uh, the list has some gems in it that I just can't get past, you you know, like, um, it's efficacy. So, uh, I use all.txt when I'm doing subdomain brute forcing, and I think it's pretty good.

There are some newer school pieces of research to pull out, uh, subdomain enumeration, right, those that file that I made parsed out all of the kind of subdomain brute force and tools that had existed for the last, you know, 10 years or something like that and put them into one file. But there's some new research by the team at Assetnote, which is a which is an attack surface mapping company, and they did some cool research using, uh, Google BigQuery to, uh, generate and discover subdomains that were used on like the Alexa top 10 or Alexa top 1000 or 10,000 or something like that. Or Reddit; or anytime someone referenced it, you were on Reddit, what was the subdomain in that link; or, um, Stack Overflow; anytime someone referenced a URL on Stack Overflow, they parsed these sites with BigQuery, and then they made these, your are these, uh, subdomain lists that you can use. So they called this project the CommonSpeak project. I've integrated CommonSpeak one into the list all.txt, but CommonSpeak 2 came out a couple years ago; it's not in all.txt; it's got some other sites that they decided to target. I recommend checking it out for kind of newer school, you know, research on what subdomain names look like.

Then there's this idea of alteration scanning; this is a type of brute forcing. So you have dev.company.com, but you could also have dev1, dev2, or dev-one, or dev.one.company.com. And so this technique was pioneered by Shubham and Nappy when they wrote a tool called alt-dns. It's now been this permutation or alteration scanning, whatever you want to call it, um, has been built into mass. So, uh, you can you can use mass, and it will try to find these naming conventions for you, um, and, uh, and it will sometimes give you gold because people name stuff predictably in their subdomains. Some of the things I've done with permutation scanning is is bypassing web application firewalls, so where I've had SQL injection on a main target, um, and getting blocked by like a web application firewall, I've managed to find a permutation like w2 and then managed to bypass the firewall because it wasn't applied to ww2. Um, I've also managed to bypass things like Akamai by finding their origin via predictably named alterations like origin-sub or origin.subdomain to bypass filtering to go to the source. So these are some things you can use that origin scanning to, to do as well as just find, you know, new surface to attack. All right, so now we're going to go into some other stuff that's related to wide-scope recon. We have seed domains; we've got a lot of seed domains, a lot of subdomains right now, and we should have a pretty good map of, um, of the infrastructure that belongs to this company.

All right, so one that I didn't talk about, and this is new to this specific talk, that's why I called it 4.002, is favicon analysis. So there is this idea that every page in the tab at the top of your browser has a favicon, right? You see that little Tesla in the bottom left-hand corner of my Chrome tab; that's a favicon. Now, um, favicons are little images, and what you can do is you can take a hash of that favicon, and then you can search for the hash of the favicon on Shodan, and Shodan will then show you every other site that has that favicon, which will find you some gems because people tend to reuse favicons on a lot of their sites of domains you might not have seen before. There are some newer tools to do this; I used to do this a lot in my recon testing; I then took it out for a little while, and I recently put it back in. So favicon analysis is kind of a fringe technique, um, but it's pretty cool. So there's a new tool called `fab-freakout` by, uh, Davinash Batham, um, there at Twitter at @asmodius, and, um, basically he does a couple things here; he parses from Shodan; he'll also he'll also look for hashes in different places, but he also uses the hash; he also has hashes for common infrastructure like a scanner would to look for different types of infrastructure. So on the bottom right-hand side, you can see that he has some, uh, fingerprints on different types of things like Spring Boot pages or BIG-IP pages or Slack instances; they all have common favicons. And so when you scan a list of URLs or a set of domains on port 80 or 443 and you retrieve their favicon and their hash matches one of those hashes, you know, oh shoot, I've stumbled upon a Slack instance for this organization, and you may not have, you know, found that doing anything else. You can also do, um, the same thing if you scan the cloud and you you correlate one of the previous techniques with this one; let me find things that have my domain and the certificate data and have these favicon hashes associated to them. So this is a fringe kind of analysis technique to find even more kind of esoteric related sites.

Okay, so then we're going to go back to a tried and true method: port scanning. Right, we have a lot of domains now; we have a lot of seed domains and subdomains and sites that we can work with, um, we want to port scan them because they may have services that are not 80 or 443 on these pieces of infrastructure. And so, um, the, you know, the tried and true hacker education, hacker education will tell you to use Nmap here, um, but I use masscan because I believe it's faster; it has a, uh, rewritten TCP stack, true multi-threading; it's written in C; you know, directly calls, you know, a lot of stuff. So, um, masscan, in my experience, has been faster than using Nmap, even with flags like `-min-parallelism`, which there was a huge debate on Twitter the other day of which is faster than parallelism or using masscan, and people are like, whatever. When I run when I run a scanning tool, a port scanning tool across, you know, 400,000 hosts, uh, I've always found masscan and it's it's syntax, it's, uh, its advantages to beat out Nmap. So that's just my personal experience. If you really like Nmap to do this, you totally can. And this is strictly for finding ports; it's not to do service scanning, right? Nmap obviously wins in those areas. If you're going to do banner analysis, service scanning, script scanning, like that, all that, all hands down; masscan only does one thing: finds open ports; that's it.

So what I do, uh, also if you want to learn how to use masscan, Daniel Miessler, one of my best friends in the whole world, um, he wrote a study guide on masscan and all of its syntax; it's one of the best ones I've seen, even better than the main page. So go check that out and how to use masscan, um, and how to set the flags correctly and, you know, how to scan. Um, so so what I do is I'll take masscan, and then the problem about masscan is it only scans IP addresses; it won't scan a domain name. Um, so you can use a tool called `dn-masscan`, which will convert your domains that we have, right, we have subdomains and seed domains, and they'll convert them into IP addresses and then scan them with masscan and tell you all the open ports on each IP address. So what I do is I take masscan, and I scan that over all of my subdomains, and then I feed that output, because I know it's open, to Nmap to do service scanning, and service scanning will start to give me more information on the services that are open on those boxes. And then what I do is I do a quick default credential spray across everything that has certain services open, which is FTP, SMTP, SSH, Telnet, any type of SQL database, basic authorization, and some other stuff. And so this is a tool that I used to do that; that's called `brute-spray`. `brute-spray` will take the output of your Nmap scan, so you feed masscan to Nmap, and Nmap does the full service scan and outputs an XML file, and then you feed that XML file to `brute-spray` to do a quick credential, default credential brute force against all of the services that allow you to do that. And so I found some some good wins, uh, doing this, logging straight into SQL databases and SSH and just just some horrible stuff that people leave unsecured on the internet, so on services. All right, um, so while I'm doing all of this stuff, right, we're in kind of the other category right now. So while I'm doing all of this analysis and I have most of this automated in a giant ugly shell script that I use, and we'll talk about recon frameworks in a second, but while I'm doing all this, it takes a little while for these tools to run, you know, the conglomeration of tools and techniques we've already talked about takes anywhere between 5 and 15 minutes, maybe a little bit more if I have a lot of subdomains or that's a big project, it'll take to run all of these tools and give me output. So while I'm doing this, I do a technique called GitHub dorking, and this has found me many, many great things. And what what you do is you basically just go to GitHub and type in your domain as a search operator, so twitch.tv, right? And you go to GitHub and you type in twitch.tv, and then you just start browsing source code that has reference twitch.tv or tesla.com or whatever teslamotors.com, and you can find all kinds of sensitive data that former employees or current employees have accidentally put on GitHub. And so, uh, I have built a script just to build these search queries for me, these dorks I call them, and it's right here in a gist; you can grab it, um, and it looks for things like my domain, twitch.tv, that I'm looking at right now and password or, uh, .npmrc, auth, or docker config or pem private for certificates or s3 config or .htpasswd or credentials or .bashrc profiles or ssh configs. And so it searches these key terms along with the domain, and if it comes up that someone has accidentally, um, you know, put this on GitHub, it's usually automatically a finding; it'll help me get into other systems. Let me give you an example here of something that happened in the real world, um, I found an admin page for a site, uh, I couldn't do anything with it, you know, I fuzzed the, uh, the form, great, didn't work, et cetera, et cetera. Then I found some dude who had, uh, basically posted a password, not for that specific site, but it was related to my domain on GitHub for some other system; that system was internal, so I couldn't access it, but then I tried to use that password and username on this admin portal I had found that I had no luck before; bam, got in, stole credit cards, millions of, you know, records of data; it was game over at that point. So, um, so this can can help. There's a whole awesome, uh, talk here called "GitHub Recon and Sensitive Data Exposure" on Bugcrowd University by the gentleman; it's probably the best primer on doing this type of dorking to find sensitive stuff on GitHub that I've ever seen; I highly recommend you check that out. And then Gwendole also rolled wrote a GitHub search tool that allows you to do this in the command line and not in the browser, which is okay.

So, uh, now you have a bunch of, uh, subdomains, and I think I'm running close to time, so I'm gonna try to hurry it up here. Um, you have a bunch of subdomains here, and now we want to prioritize which ones we test, right? So what you can do is feed all of your subdomains to a tool that does screenshotting, and there are several tools that do this currently. I use `eyewitness`; I was using `aquatone`; I go back and forth, but there's four tools here: `aquatone`, `http-screenshot`, `eyewitness`, and `witness.me`. All are tools that help you take screenshots, and then some do like additional analysis, um, on your domains that you feed it, but more normally I just use them for screenshotting; doesn't matter which one you use; try them out; see which one you like. I like `eyewitness`. Um, I feed it a list of domains; it will take, uh, it will take those domains, visit them with the headless browser, take a screenshot of the page, and then I will just look at that folder and see, okay, which one of these, you know, looks like I want to prioritize first? Is it, you know, does it look like it redirects to the main site? Well, obviously I'm not looking for the main site right now; is, you know, it's some kind of back-end admin portal? Okay, I want to prioritize that. So, uh, screenshots can help you prioritize your work, um, when you're when you're you're given a large list of subdomains like you have now. You can start to look for some vulnerabilities; one is subdomain takeover. Um, there's a repo called `can-i-take-over-xyz` by EdOverflow, and it gives you a list of all of these services and their fingerprints that might indicate that you can take over that subdomain. I'm not going to go crazy into subdomain takeover because short on time, obviously, but you can use a tool to check for subdomain takeovers; the best one right now is `nuclei`, um, and `subover`, I think, are my two favorites right now. `subover` was an independent tool; has been since ported over into the nuclei framework that Project Discovery is making, but, um, go check out `nuclei`; it has the most, uh, the most subdomain takeover, uh, checks that I've seen for any tool, and, um, I would check that out, and you can run it just across a list of large domains, and it'll tell you, hey, possible subdomain takeover at this address. So this is another wide-scope tool that I use in in the end part of my recon when I have all these subdomains, and you can find some vulns just straight off of using this.

Okay, so we're gonna blow through automation real quick. Okay, so when you when you have some of the tools and you know you have a long methodology like I have, you end up automating it, right? I write a horrible bash script to do my stuff, um, it it works for me, uh, but it could be better. Some of the tools that I use don't do certain things, like they're not threaded, they don't take certain types of inputs like list inputs or, um, glob notation like Nmap does or range notation like Nmap does, and sometimes I need to feed a tool with another tool, and so I can't I can't do that. So Michael Skelton, also known as Kadingo, wrote a tool called `interlace`, which basically wraps around other tools and lets them do those things; it will thread them; it'll allow you to take different sources of input and allow you to distribute tools. Here is an example of a blog written by Hackluke, which talks about basically threading Nikto, which Nikto doesn't support inter or inherent threading, so you can use `interlace` to, um, basically glue together a lot of stuff that your other tools don't do. Any tool written by Tom Nom Nom is awesome in automation, right? `http-probe`, `waybackurls`, `meg`. Um, Tom Nom Nom has several talks out there talking about his tools and how they work. Um, I use `http-probe` a lot for the glue between me finding subdomains, uh, and you feed it a lot of subdomains that you have found after your initial analysis, and then it tells you which ones have actual live listening web servers associated to them. So that's `http-probe`. `waybackurls` will find you URLs associated to any old URLs associated to any site that you're currently looking at. `meg` is like a directory brute forcer, but for many hosts, and I use this to find some some stuff too. So all of Tom Nom Nom's tools are just amazing.

All right, last part, I promise. Omar, are we okay to to finish this last little bit? Absolutely. Okay, cool, cool, awesome. Okay, so maybe recon is not your thing, right? I definitely run into hackers who are like, this is the most boring part of assessment to me, and then I run into other people who are like, yeah, recon is awesome; it's like my favorite part of the assessment. So it could be that recon is not really your thing; finding all these sites is not your thing; hacking the sites is more your thing, and that's cool. There are a lot of new tools out here these days that basically automate all this for you, and they're called recon frameworks.

Um, so if you've ever looked at a video game before, uh, you know, like, um, like Diablo, right? Every once in a while, like content creators will make these tiers of builds for Diablo or something like that. So I put recon frameworks into a couple of tiers: C tier, B tier, A tier, and S tier. C tier are recon frameworks that are built around scripting up other tools in Bash or Python; they're step-by-step; they don't really have a workflow; they only have a few techniques, and they're not really super extensible, but they work really well. Let me put out a big disclaimer here: my tools that I use personally on all bug hunts are C tier tools, and they work for what I want and they do, and they're great. Uh, so there's nothing that's bad to say about a C-tier tool. Uh, a B-tier tool, in my mind, and this is all very rough, right, like, you know, these classifications are not in any way, um, like super serious, but a B-tier tool in my mind, um, has, you know, some of its own modules; it's doing some of its own sources; maybe has a GUI; maybe has some workflow in it where they're bringing data in from the end of the recon process back to the beginning if they find new stuff; um, it has more techniques than a C tier; it runs, but it still runs at that point in time, and it's still working on flat files to track the data, um, for recon. A tier is, uh, maybe writing all of their own modules; has these tools start to have some GUIs; some of them, they run, uh, iteratively, so they're like, you know, like cron or something like that; they'll run on a schedule, and they start to manage things via database, so you can compare scans and recon scans over time. And then S tier are kind of the highest level of what's out right now; they write a lot of their own modules; they have a GUI; they run iteratively; they manage all their data via database; they scale across multiple boxes to make the scanning faster; they send alerts back to the user via email and text and Slack and whatever when they find new things; they have some novel techniques that not a lot of other people are doing. So these are how I classify some of the frameworks; I'm going to show some of them to you now, and you can pick one that works for you if you're not into this recon stuff.

Warning: I had to put this in here; I don't want to hurt anyone's feelings, right? Like, I am scaling some tools and giving them grades; it's not serious; it's my rough experience; my gut feel, um, all these tools are wonderful, and I respect the authors of them so darn much for just putting code out there; it's, uh, yeah, I can't say enough about the people that open source their code. And even C tier, like I said, my tool is a C tier tool, so and it works for me, so this is just subjective and based off my own experience. Okay, so see some C-tier tools that you could look into that are wrapping around a lot of, uh, a lot of other existing tools, um, the one of the ones, recon, one of the ones called Ultimate Recon here I like is listed here, and the reason I like Ultimate Recon is because it's using the new nuclei scanner that's out by Project Discovery, and it will, uh, it implements finding subdomains using a couple different tools; it will then take all of those; it will port scan them; and then it will run nuclei templates on them. So it's, uh, it's pretty good; I like it. All of these different seed tier tools can, you know, they're all choosing their own tools to wrap around that they like, so there's there's no best one really, but, um, I I tend to like Ultimate Recon in this, you know, one. There's also there's like other good ones in here too, so, uh, check these out. B-tier frameworks; this is LazyRecon by CaptainMilo, um, and here you can see his workflow, uh, very similar to what we talked about in this presentation; mass and subfinder; combine those two; you get a final subdomain count; then you start to do some vulnerability analysis with subjack and CORScanner to find some vulnerabilities related around subdomain takeover and, uh, CORS vulnerabilities; then you start brute forcing; then you start port scanning; then you do some screenshots; and then you

Have a final, uh, you have a final output. Um, so a lot of stuff we talked about today is implemented inside of this workflow. So lazy recon is one I like to point out inside of the BTR here. Uh, eights here, there's a couple. Um, they all teeter on being S here, right? Like I, I, I wonder if I should even really have like an S here and an eight tier. Uh, usually the thing that's separating some of these is gooey from the other ones. So the one I have referenced here on this slide is called Find Domain, and Find Domain does pretty much everything. It does subdomain discovery and scraping and brute force and all of that, um, via Mass and Subfinder and Asset Finder and everything, and then it stores it in a database. It does iterative scanning; it'll text you if it finds new domains from its last scan; it'll email you, which you see in the bottom left-hand corner, which is great. The only thing it lacks is a GUI. Um, it's it's a command line-based tool, so you know, it kind of teeters on doing a lot of the stuff we want to do, but um, and in the enterprise version it does port scanning and screenshots, and um, it's a that version is a 4K version, but it's affordable to bounty hunters, and some some of those are really good subscriptions to pay for. So Find Domain, pretty cool.

Okay, here's our S-tier framework. Some of them you'll never get to use, uh, because they're commercial products, but I include them here to give people ideas as to like what they should be working towards in their frameworks. Intrigue.io is an asset discovery tool written by Jonathan Crane, who's a friend of mine. Um, he, uh, he builds Intrigue, and he has a GUI for it, and it's a SaaS platform, and businesses buy it to map their attack surface, but it's doing the same techniques that we've talked about today. Um, the cool thing about Intrigue is, even if you don't pay for the subscription of, um, you know, like the SaaS version that he's selling to businesses, he open-sources almost all of the code for Intrigue.io, so you can run out, you know, your own hosted version or at least see what he's doing and his kind of analysis. So Intrigue is pretty cool. Asset Note is another one that's a B2B type play for recon, right? It's a team of former bug hunters who made this awesome platform. This is probably the gold standard of, uh, of asset management or attack surface management tools. You can see that it's it's pretty; it breaks things up by, you know, assets that need attention; it's got a lot of graphing; um, it does a lot of custom vulnerability checking. Uh, so Asset Note is is probably it's asked here, but you know, it's a high price tag, so you're probably not going to use it as a bug hunter or red teamer, but it is kind of the gold standard that a lot of the tool makers should probably work towards.

SpiderFoot is another one, um, that does a lot of OSINT type domain correlation information; does some of the techniques we've talked about, um, in this presentation; they're adding more all the time. I know the author is really strictly, uh, interested in adding more bug bounty focus features in it, so um, and it's got a great graphing library and monitoring and everything. So SpiderFoot is pretty cool. The unreleased Project Discovery--we've talked about Project Discovery a couple of times in the presentation--that team, their unreleased framework looks to be a killer; looks to be awesome; I'm really excited about it when it comes out. So yeah, you can see here a dashboard of a project, all your discovered domains and seeds, um, what the scan process is, and then if you dig into any of those, you get screenshots, port scans, technology identification of the site, its activity, its changes over time. So the Discovery framework, when it comes out, will be pretty cool.

Jals is a vulnerability scanner written by Jesse JJ and team. It is also really good. Um, it is a GUI flow that wraps around some command-line stuff; it does vulnerability scanning, um, which, you know, if you're working with Nessus or, you know, not a lot of red teams are verbatim scanning with Nessus, but if you want to look for some very pointed stuff, you can use Jals and look at their kind of CVE checks that they use, and you can see signatures there for the Zoho management page and Fuel CMS RCE, and so they're looking for some cool volume. Jals, I really like it. OSmedeus is awesome as well; it's very similar to some of the previous pages we've seen; it'll take all of your domains; it will do; it'll take all your subdomains; it'll resolve them to IP; will do port scanning on them; it'll tell you the technologies; um, it will do screenshots; it'll do all kinds of stuff. So OSmedeus is also pretty easy to stand up and really good. HunterSuite.io, also same kind of idea; you can see technology parsing, services, uh, domain discovery, etc. Bounty.offensive.ai, same type of deal, right? Includes some vulnerability scanning, technology fingerprinting, subdomain finding, and then attack refing. Re-Engine is the one I used last week or week before. Re-Engine is pretty sick. You, you can have projects in it that are separate; it'll do the subdomain scanning; it basically takes the methodology that I outlined in this presentation and turns it into a tool, um, a hosted tool, um, and they're they're making some improvements, you know, every week. So lots of, uh, potential on Re-Engine, recon engine, which I highly suggest checking out. And then Scout, PDP on the Sec Apps team, they've been working on tools for well over a decade, and um, they have built out Scout, which is a paid offering, but it's affordable enough for bounty hunters. Same idea, right? Projects, subdomain enumeration, related domains, reverse DNS; it supports screenshotting; it's awesome. So, um, awesome tool, and uh, yeah. And then lastly, Nuclei. I talked about it very briefly, but anyone who's not using Nuclei in their bounty scanning right now is at a disadvantage. They're coming out with some epic templates for CVE identification, and and you can basically build your own via a YAML file, and it's a scanner that will go out and scan all your domains for, uh, vulnerabilities, subdomain takeovers, all kinds of stuff. So I really have been using Nuclei to great success lately, and that's it; that's all I got. So that's the bug hunters methodology, and uh, thanks for giving me the time today. I really appreciate it. Sorry, I went over a little bit. No worries. Thank you so much for the presentation and amazing support, and once again, you know, from the bottom of my heart, thank you for supporting DEF CON and the DEF CON Retin Village, and thank you for everyone here. You know, please take a look at all the talks and activities that we have in our website. We have the CTF, of course, the Cyber Wraith, um, contest as well; tons and tons of activities throughout this weekend. The link should be in the description at in the bottom of your stream. I know that some of you are in Twitch, some of you are in YouTube and other platforms, and also please join the conversation in Discord as well, right? So we're gonna go in a break right now, and the next presentation will just start in a couple of minutes. So thank you again.