📱

Get Our Mobile App

Take your business learning on the go!

Download on the App StoreGet it on Google Play

DEF CON 30 Blue Team Village - Project Obsidian - Kill Chain 3 Endpoint Forensics Walkthrough

DEFCONConference29:17

Transcription

Hello and welcome to the Projects Obsidian uh endpoint forensics kill chain three walkthrough. Uh, before we go through the actual walkthrough, uh, I'd like to draw a little context around what this presentation is. So, first of all, what is Project Obsidian? Obsidian is a project that the Blue Team Village has been working on for a few years now. It is designed, we created a fictitious company, we staffed that fictitious company with employees, and then we had the Red Team, the Project Obsidian Red Team, attack that, that network. And from there, we did incident response, forensic examination, malware reverse engineering, cyber threat intelligence, and cyber threat hunting.

This presentation today is really the endpoint forensics part of it. So, the forensics team uh will have several presentations. This one is kill chain three, and it covers only looking at the endpoints from a forensics perspective. To do that, we did collections on the machines in the environment after the environment was attacked. I'll show a little bit of a timeline. We're not going to focus on the timeline because mostly what I want to show in this presentation is the artifacts, how we would look at the artifacts, how we would uh draw conclusions from those artifacts, and then a small demo of what it looks like when you would actually do the analysis.

Uh, here we're seeing the timeline. We'll see several slides, and I'm not going to really cover these, but I just wanted to cover the kinds of things that we'll see in today's presentation. We'll see some uh, some BloodHound being run in the environment, SMB scanning, connections from a machine called RDPO1 to domain controllers. We're going to focus mostly on three machines: uh, RDPO1, which is the source of where the attacks came from, and then uh, some activity on the domain controllers. We'll see some malicious admins being added via automation. We'll see some evidence that the actor tried to exfil some data, some lateral movement, dumping of credentials via dumping LSASS, and then NTLM downgrade to allow less secure hashes to be dumped, which could be cracked offline, and then security logs being cleared. Excuse me.

So, let's see the first piece of evidence that we notice in this environment. One of the things that we look at very often when we're looking at a possibly compromised environment is the, the 7045 message. And this one does not disappoint. On the domain controller Magnum Tempest.financial, we see Powers, PsExec being run on that machine via the PsExec service. Um, and, and furthermore, uh, we don't see any additional uh, 7045 messages, but, but this is uh, something that we'll keep in mind as we, as we look more at the data.

One of the things that we do when we analyze a machine is we look for for files that look like they're hostile. Now, actors don't always name their files obvious, uh, this is a hostile file, you know, .txt, but in this case, we did see uh, several files that that looked like they could be uh, created by the attacker. One of them was this BloodHound zip file on Patrice's desktop. We saw several other files indicating that PowerSploit had been run and the Invoke-PortScan module had been run on the machine. In addition, we saw some files that indicated credential dumping, uh, and evidence indicates that the first one was using the Task Manager, LSASS memory dump, and the second using a technique to use rundll32.exe and execute comservices.dll to dump LSASS.

In addition, we find some other very interesting on the Brett Social desktop, four files called computers.txt, 1.txt, 2.txt, 3.txt. And diving into those files a little further, we identify that there are a list of IP addresses. Now, this is important because when we look at these files as an investigator, now we have some evidence to decide the next set of machines we should look at. Now, those four files were used by PsExec on the RDPO1 machine to automate adding uh, hostile users to the local, local administrators group. Right, we see the addition of Combo Security, a user ID called Jimbo, the user ID called Hass, and a user ID called Andy using PsExec and automating that using these four files.

So, diving into some of the evidence, the first place we're looking here is that Patrice's uh, PowerShell console history. And again, we see that they used the BloodHound tool to scan the environment, find vulnerabilities in Active Directory. We also see, just as an aside, this LSASS dump that we had talked about earlier. This is coming from running, running Chainsaw against the Sysmon operational event log. Uh, and Chainsaw identified this activity, event ID 11, as LSASS memory dump file creation.

Looking further into some of the evidence, we find in uh, Patrice's uh, console history, PowerShell console history, uh, execution of port scan, uh, running against uh, the subnet 172.16, looking for evident, looking for uh, port 445 being active. Again, looking further at this evidence, we see evidence that uh, the Invoke-ShareFinder module was used. And, looking a little bit further down at the log, we see evidence of Mimikatz being run in the environment to uh, dump the, to dump the hashes, LSASS hashes.

Next, looking further at again, Patrice's console history, PowerShell console history, we see this dump file that we had identified earlier. Console history doesn't contain dates and times in it, but going back to the creation time that we saw earlier of this dump file, we can get an idea in our timeline of when this activity took place.

Looking further at browser history, we would identify, uh, we collected browser history using Velociraptor, and then taking that browser history, uh, converted it to a CSV file, and then took that CSV file and put it into a reporting mechanism that I use called ACH Report. Uh, using this, and I'll show that in a little bit, we see that uh, this machine reached out to file.pizza and transfer.sh, indicating that the actor wanted to take the data from the machine and exfiltrate it out. We also also see it reaching out to interact.sh. Now, this one might say that this could be evidence of exfiltration. Interact.sh is actually an out-of-band detection mechanism. It's used uh, when you would, when a hostile actor would execute an attack, and then this interact.sh is used to report uh, on whether the attack was successful or not. In this case, we don't see the typical URL we would see from interact.sh, so the actor reached out to interact.sh and was likely just looking at it, probably not being used for exfil.

Looking further into the evidence again, we see in the Sysmon logs uh, event ID 13. Uh, and PowerShell, I mean uh, uh, Chainsaw identified that this was likely an NTLM downgrade. So, this is at 1909, and at 2054, we saw the actor uh, execute an NTLM downgrade. The idea in an NTLM downgrade is to force the machine to allow insecure uh, hashing, uh, and then those credentials can be dumped, those hashes can be dumped, and they're easier to crack offline. Uh, so we see evidence and of that activity. And if we wanted to, we could actually look in the user's uh, registry, actually not in the user's registry, in the machine registry, HKEY_LOCAL_MACHINE, and we would find the evidence uh, in LMCompatibilityLevel, NTLMMinClientSec, and uh, RestrictSendingNTLMTraffic. We would find evidence in those keys uh, that the uh, NTLM had been set to to allow downgrading.

Looking at this activity in uh, Chainsaw, we also see further evidence of PowerShell being executed in the environment. Uh, kind of just an added thing. We were really looking at the NTLM downgrade, but this is what you'll find often when looking at evidence is other pieces will pop out. You'll take note of them and then add them into your timeline.

Here we're seeing uh, evidence of PsExec. Now, this is important because what we see here is the two machines that are being attacked, right? We see DC02 and RDPO1. So, we see RDPO1 authenticating to DC02, and then we see DC02 running PsExec, running the service that allows PsExec. And you'll notice that all of the timestamps are 8:59. So, this is a strong correlation that RDPO1 uh, authenticated to DC02 and ran PowerShell. I'm, I keep saying PowerShell, ran PsExec uh, on that machine.

Uh, again, we mentioned earlier that uh, these files found on the desktop, this desktop computers.txt, 1.txt, 2.txt, and 3.txt, were used to automate PowerShell to add hostile actors to the administrator, hostile user IDs to the administrators group. And here we see that evidence in the Sysmon logs, again, extracted using uh, Chainsaw and formatted using ACH Report. So, these are uh, event ID one that we saw Chainsaw identify as Hurricane Panda, but really what we're looking at here is PowerShell being used to automate adding hostile administrators to several machines that are located in those TXT files, IP addresses located in those TXT files. Uh, on the machines themselves.

Here we're seeing on DC, the commands being executed, and we see Combo Security being added with a password of b4byMetal. An important piece of information because now we can go to that machine and we know both the hostile user that was added and its password. Also, we see here later on in this log, evidence that the event log, the event logs were cleared: Application, Security, and System.

Another piece of added information that we can add to what we know about this machine is we saw, we see now Patrice's in his console history that uh, Mimikatz was run, Invoke-Mimikatz from PowerShell Empire. Uh, and these hashes were used. You'll see the hash being used to escalate this machine to administrator. As mentioned earlier, we see now, uh, using Chainsaw, Chainsaw identified that the event logs had been cleared. Uh, this would be an important time in our timeline because it would define the point in time where we can really, uh, we really won't know uh, about what happened. So, once the event logs were cleared, uh, data previous to this would not be available to us in the event logs. We would have to look to other pieces of evidence to identify activity on this machine.

Okay, that was a real quick run-through of, you know, the evidence that we looked at. Uh, actually, um, not a lot of pieces, uh, not a lot of artifacts, but a lot of information being gathered. So, let's take a look at some of that data in its, in its native form, or in the form that, you know, we, we would review as we gathered information about this machine.

First, on the DC system, uh, I mentioned that we were looking at the 7045 messages. So, here we're looking at uh, the system event log. In this case, probably the best way to look at this data would be to filter the log looking for 7045. And sure enough, as we mentioned in our, our walkthrough earlier, we see the evidence of PowerShell, the PowerShell service piece being run on the, on the domain controller.

We also mentioned on RDPO1. Let's see, we're looking at DC here. RDPO1. This is the, the, the report tool that I use. Uh, this report tool does not do a lot of the uh, of the parsing of the data. It does some parsing, but mostly it's used to put all of the data, to run the parsing utilities, uh, and put them all in one place. So, what we're going to look at is we identified that BloodHound was running in the environment. And here we see that Velociraptor collected the console history, PowerShell console history of Patrice. And this reporting tool simply shows the, the activity uh, that that was uh, that was in that console history. And here we see the execution of uh, a BloodHound, and then the file name that it was saved under. And this gives us uh, information on, if we look at that file, when that was run. Now, that's not an indication of the first time it was run. It was, it's really an indication of of what is available to us. So, BloodHound could have been run five times, four times, it could have been run once, uh, but what we have in this evidence is that BloodHound was run, and that it was run at a specific time.

We also identified um, that a port scan was run. So, again, looking in this console history, we identify that the port, that port scan was run from uh, from PowerSploit. One of the things we previously identified was the dump file uh, created by running com, running comservices.dll through the through rundll32.exe, creating this dump file, which would be a dump of LSASS. Now, if we look at that, search for C colon backslash dump, we also find here is when that was run. So, looking at RDPO1, we see uh, that dump file being created. And uh, in the uh, in the uh, Sysmon event log, uh, Chainsaw identified this activity and reported it, and we just, we just take that and put it into this reporting format.

Another thing that we saw earlier was evidence of at least attempts to exfiltrate data. So, looking at the activity around that time, we see the actor reaching out to file.pizza, transfer.sh. And if we look a little bit further down, we also see the actor reaching out to webwormhole.io. These are all evidence that the actor was at least interested in exfiltrating the data out of the environment.

One of the other things we saw, we mentioned earlier, was an NTLM downgrade attack. So, here looking at the Chainsaw output, we see what we had seen earlier, which is the NTLM. Chainsaw identified this as NTLM Net NTLM downgrade. And again, we could go to the registry itself if we wanted to have some corroborating evidence. We could go to the registry itself and show that those registry keys had indeed been modified. Uh, furthermore, on DC02, uh, we also identified that PowerShell, the PsExec, had been run on that machine. Also, we see that in a event ID of 11. So, let's look for event ID 11. And we saw that activity at about at 20:59. So, as we look through this data, go down to 20:59, and we see the evidence that PsExec had been run on this machine at uh, 20:59:57.

All right. So, looking further at our reporting tool, we see on RDPO1, this evidence on RDPO1 that PsExec, and again, this is, you know, just what we had seen uh, in the screenshot, but here we're looking at the evidence itself. Event ID one identified by Chainsaw as PowerShell executing, automating the addition of Combo Security to the administrators group on that set of IP addresses in the computers.txt, 1.txt, 2.txt, and 3.txt files. As we look at this data, we could then look around it, and see that the actor had also done some uh, discovery of the environment, running whoami and other activity. Again, we see the, the dump of LSASS. So, uh, looking at this data in context, uh, we can identify the times that this activity happened, and we get a better idea of sort of what the actor was doing on this machine, and that can help us draw conclusions about what the actor was after, what they were interested in, uh, and, uh, what they uh, accomplished, or, and sometimes what they didn't accomplish.

So, uh, looking at uh, the domain controller, right? We'll see the evidence that that activity was successful on the domain controller. So, let's do a find and let's look for Combo Security. And here we see that uh, on the actual machine, that net user Combo Security was run. And, and again, uh, we're able to draw some context around around times and activity uh, on this machine.

Moving back to RDPO1, we also saw the Pass-the-Hash escalation or elevation activity. Uh, and again, looking at Patrice's console history, PowerShell console history, we don't have times, but we do have a good picture of what the actor did uh, using PowerShell. Not entirely, not all the things the actor did, but, but the activity that the actor did on this machine, uh, when using PowerShell.

The other, the last thing we're going to look at, of course, is this evidence that um, the actor cleared the event logs. We can see not only the execution of that command, oops, that the event logs were cleared, but we can get an idea of exactly when that happened, right? By looking at uh, what is left in the event logs after they're cleared.

Okay, so recapping uh, what we did on this machine to analyze it and to draw some conclusions. We had an environment that was compromised. We ran Velociraptor on the machine to gather telemetry and artifacts. Sysmon was running on the machine, so we used that as the basis for a lot of our conclusions. We ran Chainsaw on the machine to look at the event logs and identify hostile activity. Chainsaw uses Sigma rules to identify hostile activity. We took that data and we put it in a reporting tool, ACH Report, to put all the data in one place to make it easier to kind of draw context around times and activities. Uh, and we also used a tool called MFT Dump, which takes the MFT uh, which Velociraptor collected, turns that into a comma-separated values file, a CSV file, and then ACH Report can ask that file questions about files that are identified on that machine. And this is where we get, we got some timestamps of when those hostile files were created, and how they were created, what tools were used to create them.

This concludes uh, our walkthrough of kill chain three. To learn further about Project Obsidian, about Blue Team Village, please join us on our Discord channel. Join the conversation. We'll be releasing all of this data as open source. We'll be providing it to the community for lots of reasons. Um, one is so that you can walk through the data using your own tools or, or the tools we used, uh, to learn more, and to gain some hands-on experience of, you know, how your workflow would, would, would go when you're analyzing data, you're learning to analyze data. The second thing is that we hope that this will be used by people with with new tools. So, as new tools come out, new blue team tools are always coming out, this data can be used to test those tools to see if the same conclusions are drawn, or if those tools find other information that we may not have uncovered. And also, you can join the conversation on Twitter. We're at Blue Team Village, or if you want to learn more about the Blue Team Village, you can go to our website, blueteamvillage.org.