📱

Get Our Mobile App

Take your business learning on the go!

Download on the App StoreGet it on Google Play

How to Actually Run Your Coding Agent Safely (And Avoid the Horror Stories)

Cole Medin17:33

Transcription

Claude dangerously skip permissions, otherwise known as a Yolo mode. It's what makes it so your coding agent can run any command on your computer without ever asking for your permission.

Every single coding agent has some version of this Yolo mode, and you're probably using it, maybe even every single day like me. It makes your coding agent very capable, but also there are some big risks that come with this. You cannot just let your coding agent run wild on your computer without having some real guardrails in place.

You probably heard those horror stories of Claude, Code, Cursor, Codex wiping entire databases, deleting directories. The chance is low because coding agents are smart, and you might think it'll never happen to you until it does. It just has to happen once for there to be pretty drastic consequences.

So, I want to cover all the risks with you here because even if you think you know all of them, you probably don't. There are so many issues that can come up when you run in Yolo mode. So, we'll hit on all of those, and then we'll get into our primary solution with sandboxing. I'll show you how this works. We'll be using Docker sandboxes cuz it's free, super easy to set up, and very capable. So, that'll guide all of our demonstrations here.

But really, the idea with a sandbox is it's an isolated environment for our coding agent to work in. So, we can give it full autonomy without all of the risks that come with having it run directly on our machine. And this is important because let me tell you, the solution here is definitely not to avoid using Yolo mode. You don't want to sit here spending the time approving every single action the coding agent wants to make because there are going to be hundreds of these. It's a waste of time. Your coding agent isn't even autonomous anymore. And so, that's why we want it to run in Yolo mode, but we just need to do it in a safe way. And it's really not that difficult to do that anymore thanks to tools like Docker sandboxes. I'll even show you in a little bit how easy it is to get started with this so your agent is safe doing anything it needs to to get the job done.

All right. So, jumping back over to this diagram here in the Docker sandbox documentation, we can talk about the risks and then what we're isolating in the VM and the sandbox at the exact same time. And I just love the way that Docker has explained the different levels of isolation, how that works in their sandbox specifically. I actually worked with them on this video to make sure that I'm really explaining all the different levels of isolation here because you can go pretty deep and it's worth doing so so you can really be secure.

And so, anyway, the first and most obvious risk of running a coding agent directly on your computer is it has access to your full file system. And so, yeah, there are some caveats to this with permissioning you have set up on your computer, but for all intents and purposes, you should assume that if your coding agent is not running in an isolated environment, if there is a file or a folder on your computer, your agent can access it. It can edit it. It can even delete it.

And along the same lines as the file system, that also applies to our entire environment and all of the applications that we have running on our computer. And so, your coding agent, like Claude Code, Codex, whatever, it can kill processes, edit your environment variables, it can do anything. And a lot of times coding agents get tripped up when they're dealing with things like port conflicts and so they'll kill applications in an attempt to get things working in the current session. A lot of risk with that. We need isolation where we have our own set of processes, even our own Docker engine. So, it's managing its own set of containers, not destroying containers for other things we have running. And even having network isolation, so we're only allowing requests to go out to certain websites. And that is important because if you have a prompt injection attack hit your coding agent, it can grab an API key and then send that to some URL where an attacker is waiting for that information on the other side. And so, having an allow list of URLs that it can reach out to as it's doing research and testing our APIs and things like that, that is incredibly important. And of course, So, you an example of all these things in a little bit, but I'm just trying to give you a high-level overview of if your agent can access your entire system, there are so many things that can go wrong.

Okay, so if you were wondering earlier why I have so many Claude sessions open, you're going to find out now. We're going to go through some examples here of what can go wrong with a coding agent running in Yolo mode directly on your computer. Now, all the examples I'm going to show you here are very fabricated just for the sake of our conversation right now, but I will show you some real things that went wrong with people using Claude code. It's funny and unfortunate at the same time going through those.

But anyway, the first one right here, I'm explicitly asking it to blow away the node modules and the lock file, reinstall all of my dependencies. This is going to be slow, token heavy, definitely some negative consequences of this. And yes, I'm asking it to do it explicitly here, but this is something that coding agents will do by themselves once in a while where you're down this long rabbit trail of debugging, and it exhausts all these other options, and it's like, "Okay, I'm just going to reinstall the dependencies. There's some dependency issue here." I've seen this happen myself. It's super annoying when it does.

And in this conversation here, there's that intelligence talking about earlier where it actually protests at first. Like, it understands that this is a risky operation, but the scary thing is I literally just give one follow-up prompt like, "No, actually just go ahead and continue." And then it flips automatically to just wiping everything. Like, that's kind of crazy how easy a coding agent goes from like, "No, that's too risky." to "Okay, let's just go ahead and do it." right? Like, when you have those longer conversations with a coding agent and it resorts to those last couple of options, that's when you have things like this happen.

And then another thing that happens way too often with coding agents is it'll read private keys or environment variables even if you asked it earlier in a conversation not to because it figures out for debugging or whatever like, "I got to read this key because something's not right here." It'll just read that file. Like, I asked it to read this file, and it did it right away without protesting at all. And what really makes this rough is the fact that it reached out to my root.ssh folder. So, not even reading a key in my code base, going out to a folder that is totally separate from my current working environment. Reading a key that I would never want a coding agent to look at. And this kind of thing I've I have seen it happen when it's debugging an SSH connection.

And another really classic example here is your coding agent messing with your database. Something I've seen before is you have a bug in your code, it can't figure out that bug, and so it just assumes that because the code is accessing your database, there's something wrong with the database schema. So, it'll roll back a migration, make a new migration, and it'll totally mess with your database, and it can totally break your application. And so, right here, just like before, it does protest at first, right? Like it understands this is risky what I'm asking it to do, but then one more prompt, and it just does the entire thing. Now, I do appreciate that it did make a backup of my database. And so, it understands the risk, it protests, it even makes a backup. It's good, but we can't guarantee that the coding agent is always going to take these precautions. That's the main thing that I want to communicate here is we can see the intelligence coming through, but especially when you start to have longer conversations, coding agents will lose that intelligence, and they will do those really stupid things that hurt you.

In fact, if you've ever had a coding agent do something really bad, it's probably towards the end of the context window limit for the LLM, because every single large language model has a dumb zone. The first couple hundred thousand tokens, it's operating at peak performance, but if you have that longer conversation, especially in those longer troubleshooting sessions, you're going to get past that core 200-300,000 tokens into that rest of the window where you start to totally overwhelm the LLM, and it makes dumb decisions, because it forgets the instructions you had at the start of the conversation, even including its system prompt. And so, all the guardrails that you have built up in the prompting, here's what you want it to not do, here are the risks you want it to keep in mind, it forgets those things. That's when it's going to potentially wipe your database without a backup, delete folders, that kind of thing. And so, the main thing I'm trying to say here is we can't rely on guard rails in the prompt. Just because the agent generally seems like it understands what it should and shouldn't do, doesn't mean it always will. And so, that's why we want to use a sandbox to enforce these protections instead of asking the agent to implement or keep them in mind.

And of course, I promised I'd show you some real examples of this happening. And you can look no further than the issue list in the Claude code GitHub because people love sharing when Claude code is screws them over. Like this person, they had their Claude code execute the all infamous rm-rf, deleting their entire home directory. You do not want your coding agent really ever executing this command. R is short for recursive, as in delete every folder and all the folders within it, and then F for force. And so, yeah, just they wiped everything for them. And then it happened again for another person just a month later. And yes, this is almost a year ago now. Coding agents are less likely to do these things, but it's definitely still a possibility, right? Like this GitHub issue was closed as not planned because it's not a problem with Claude code itself, it's just sometimes large language models decide they need to do this. When they get to their last resort, you're really far along in a conversation with that context rot I was talking about earlier, that's when these things can happen. Like running a destructive database command, wiping a production database, which, my friend, you should not have even given Claude code the ability to connect to this production database. But who knows, maybe it figured it out somehow looking at configuration elsewhere on their computer. That's why you need sandbox isolation.

And then this is another really common one here, dropping an entire Git stash, so a bunch of uncommitted work they had in Git. Sometimes coding agents get really confused trying to deal with different get branches or merge conflicts and they'll do this kind of thing. So, a bunch of examples here just to show like why we need that isolation. It shouldn't be able to reach to the production database. It shouldn't even be able to edit or delete anything in the home directory at all. That is why we need the sandboxes.

And I've specifically waited until now to make this video because Docker sandboxes in my mind is the first solution that's really made sandboxes accessible. It is a single command to install this now. Very easy to configure things and they work with coding agents like Claude Code right out of the box. And of course, you can also just give this link to your coding agent and have it install everything for you. That's what I did. So, I'll link to this in the description. Very easy to get started. Totally free as well. So, this runs just locally on your machine just like Docker Desktop and all your Docker containers do as well. And Docker is something I've been using for 8 years now, even before I was doing anything with AI. So, I absolutely adore all the products they release and yeah, sandboxes has been fantastic for me.

So, once you run this command for either Mac or Windows to install Docker sandboxes, getting it up and running with a coding agent is a single command. SBX, short for sandbox, run and then the name of your coding agent like I'm going to use Claude. And everything seems normal here. It's just a regular Claude Code terminal, which is great, but under the hood it's running in an isolated sandbox VM. So, now I have all of the security measures and things that I can configure as well that I'll talk about in a little bit.

And so, for example, I can do pound or exclamation LS to get the current directory. So, you can see that we have our current code base that I'm working on here mounted into our VM. But then if I do an exclamation LS and then I do the .ssh folder I showed in the demo earlier, you can see it's not found. We don't have access to the rest of our machine. So, we have true isolation. But at the same time, at least with this code base here, we can work on everything just as we normally would. We're not losing out on anything being in our sandbox, but we are fully protected.

So, I have a couple of conversations just loaded here to show you what I did ahead of time. But, within my sandbox, I can install dependencies. I can start the dev server, build this into a Docker image using the isolated Docker engine. Right? Like, all of my work goes on as normal. I just have to maybe set up some policy so I can reach out of the container when it needs. And so, going to another conversation here, you can see that it is unable to read from my .ssh folder. So, we are protected there. Another conversation I'm having it try to reach a URL that is not in the allow list, so we get a 403. Not from the server, but the request never left the box because the sandbox proxy blocked it. So, we have our configuration set up defining the different websites we can go to, so we don't have to worry about our agent sending requests to things if there's ever a prompt injection attack. And then, we can set the policies here for the different websites that are allowed. So, I'm checking, like, can we reach this site or not within our VM? And then, I'm allowing it, and we can see now that we are allowed to reach this website. So, anytime we set up a sandbox going forward, we can access this site.

And if this looks a little intimidating to you, I know we're covering specific commands here, don't worry. You can just feed the Docker sandbox documentation to your coding agent and have it set these things up with your help. And even spinning up the sandbox and testing things. Like, when I created everything for this video here, I just had Claude code set up everything for me. So, you can have it configure the sandbox and get everything perfect for you to then do all your development.

And so, take a look at this. This last conversation I have to show you here. I know there's so many tabs I have open. I literally told it, "Uh before I trust the sandbox with real work, I want to verify the isolation is set up right. So, I want you to run these checks. Can you see any of my host files, no matter how hard do try? Can you reach a service on my host machine? Right? Like do we not have that process isolation? Can you touch the host Docker socket? Can you mess with my containers on my computer? And it went through everything here. It ran a lot of commands. And you can see at the end, host files outside the workspace, not visible. Host services, not reachable. Docker socket, it's the nested daemon, not our host, right? We have that separate Docker engine. Everything is completely isolated here, but we still have the full environment we need to get real work done. And we can also work on an isolated copy of the code base in our sandbox VM as well.

And so going back to this command right here, SBX run the name of our coding agent. This is going to mount our current directory into that session in the VM. So we're editing these files directly on our host machine, but we can only access these files. But if you want literally complete isolation, you can add the --clone here. So it's going to create a copy of the code base and then that is what is brought into the sandbox. And so that way we're not messing with our Git history and stuff. We don't have to worry about the agent running those malicious Git commands. Like everything is totally protected here. And so yeah, literally no matter the level of isolation and control you want, you can do it with Docker sandboxes.

I know it can seem like there's a lot going on here, but I want you to understand the risks and how we're solving for these things with the VM. And so the last thing I want to show you here, just to like really bring everything home, is the different isolation layers that we have operating with our sandbox. In this case, Docker sandboxes. But really you're going to see this everywhere. Docker just does it best.

And so the first layer is our hypervisor isolation. This is what gives us the process and file system isolation, so we can't accidentally close down processes or delete files on our host machine. And the cool thing here, I didn't show you this command, but it has full cleanup as well. So when you remove a sandbox with SBXRM, the VM and everything inside it is deleted. So, we don't have a bunch of space being taken up on our machine as we're creating all of these tiny little instances where we are running our sandboxes. And also within the VM, I don't know if you realize this, but the agent runs as a user with pseudo privileges. So, it's able to do everything because we don't care about what it does within the VM. That's the point of having that isolation.

And so then we have our network isolation. This is what I showed earlier where we're able to make requests to certain sites and we get to manage the policies and the allow list there. And then we have the Docker engine isolation. This is probably the one that's unique to Docker, which is why I appreciate it so much. But having that separate Docker engine, so we maintain a separate set of containers as we're building up and testing the application we're working on in the sandbox right now. And I love this because my personal Docker desktop, that's like actually on my host machine, it gets very bloated over time as I'm spinning up containers and testing different applications. I really love the idea of having an isolated environment per app that I'm working on. And then the main Docker desktop on my host machine is reserved for the applications I actually have running for myself, not for the sake of development.

And then we also have the workspace isolation, which again, that is we can do a direct mount where within the VM we are editing the files for just that code base locally, or we can have the clone mode, so we have a full copy ready to edit there, so we aren't anything at all on our own computer.

And so with that, we've covered all the risks of running in Yolo mode and how we're solving for it with a sandbox. It is so important to implement the guardrails here because of prompt injection attacks and coding agents wiping databases and files and get state. There's so many things that can go wrong, especially when those coding agents get in those longer debugging sessions and they start to hallucinate more and more. We need to protect with a sandbox. So, I encourage you, just try Docker sandboxes today. So easy to install, free to get started.

And so, with that, if you appreciate this video, you're looking forward to more things on AI coding, I would really appreciate a like and a subscribe. And with that, I will see you in the next video.