Transcription
Hey everybody, how is it going? This is Christian and in today's video I'm going to show you how to simply set up the Proxmox backup server, which is, as the name already suggests, a backup system for the Proxmox virtual environment. And guys, if you're running Proxmox in your home labs or even your production environments and you're not using the backup server, let me tell you, you've been doing it all wrong. This is going to be slow, it's going to take up a lot of space on your NAS, and you don't have the ability for easy file recovery.
Now, with a Proxmox backup server, you can do all of this. It supports incremental backups, deduplication. By the way, I later explain in the video what that actually is. [music] And it's not really a big deal to install and set it up. You don't even need a separate device or server for it. So, you can install this directly on your Proxmox virtual environment, and it's going to make your backups so much better. So, if you want to know how to do backups the right way on Proxmox, then keep watching this video.
But before we jump right into the backup server, I have also got to show you the new cable management system from Patchbox, the sponsor of today's video, because Patchbox just sent me their brand new Patchbox One. And guys, if you ever struggled with a tangled mess of cables in your server rack, this is also a true game changer. You just mount this solid Patchbox frame into your server rack and then slide in these modular cassettes that neatly hold your Ethernet or optical fiber cables. So when you need to connect something, you just unlock the integrated cable stopper, pull out exactly the length of cable you want, lock it back in place, and now you've got a clean and well-organized cable management. And what I absolutely love about the new Patchbox One is that the new cassettes seem way lighter and easier to handle. Now, they've also upgraded the cable stopper and pullback mechanism to be even more snappier and smoother. Of course, you can still get these cassettes [music] in different colors and cut standards to customize your setup exactly how you want it. And both the frame and the cassettes are even [music] compatible with the older Patchbox versions. So, if you want your server rack to look professional and you want to stay organized, and by the way, also save tons of time during setups or maintenance, then the Patchbox One is definitely the way to go. It's perfect for both professionals and home labs as well. So, of course, I'll drop you a link to their website in the description so you can check it out yourself.
All right, guys. So, now let's get started with the Proxmox backup server. And first of all, I want you to go to the official homepage of Proxmox, proxmox.com, because there you can learn more about the Proxmox virtual environment and their backup server. And as you can see, they just released a brand new version for both the Proxmox VE and the backup server, which I haven't upgraded to yet because it is always the same. Once I just finished recording or planning a video, if they release a new version, that's going to change a few things here and there. But I think, um, most likely the system will be kind of the same. They just introduced a few interesting smaller features here and there, but I think the tutorial will be still valid for the new version as well.
Okay. So, uh, first of all, go to products and go to the backup server and then go to requirements because I want you to take a look at this. But first of all, click on the recommended server system requirements because the Proxmox backup server is a system that Proxmox says you should better install on a separate physical device. So this is their official recommendation. You can find some minimum server requirements that you can use for testing, but ideally you should install this on a separate physical machine with at least four CPU cores and 4 GB of memory. And the reason for this is, if you're storing critical backups for your virtual machines, you should definitely do this outside of your regular system. So in case something goes wrong and you have to restore the entire system, you could do this from a separate physical device with separate discs and a redundant storage pool. So that is definitely the official recommendation.
However, as you might know me, I don't really care so much about this because, um, if I would have to buy a separate physical device for the backup server, it would cost me a lot of money because I need to buy equipment. But also, in terms of power consumption, the device has to be running 24/7 most likely. But the good news is that you can still install the Proxmox backup server in many different ways. You can install this on a small virtual machine. You could even install this in a container, or you could use the Debian packages to install this directly on the Proxmox VE. Now I personally have seen many tutorials on the internet that recommend you to install the Proxmox backup server in a virtual machine. But honestly, I don't think this is the ideal setup because it would potentially cause some problems and create this inception problem that you might have heard of. So in the context of the Proxmox backup server, this refers to a situation where the backup server attempts to back up or interact with itself in a way that creates some recursive or self-referential condition, which is not really a good idea. So in simpler language, if you try to back up the backup server itself, it's going to create a problem. And that's why I thought it might be better to run this in a small container. I recently did a video about LXC containers on Proxmox, and I think this is really the perfect scenario for installing the backup server.
Of course, I'm going to show you one of my favorite community projects, the Proxmox VE helper scripts. They help you to set up the Proxmox backup server and other containers or settings on the Proxmox virtual environment in a super easy and simple way. So just go to this website, go to view scripts, and there you will find the Proxmox and virtualization category, a Proxmox backup server script that will automatically create a small LXC container on your desired Proxmox instance. And there is even a post-install script that you can run to do some maintenance tasks, update the system, disable the subscription window, and stuff like that. Of course, I've already done this on my Proxmox backup server. So, I've just copied this small command and then I went to my Proxmox server. Then you need to select your Proxmox node where you want to install the LXC container with a backup server. Go to the shell, log in with your root user, and just paste the command from the Proxmox VE helper scripts. That will launch the installation wizard. Usually, I would recommend you to go with the advanced settings because we have to change one particular setting in the installation wizard, which will set the container type from unprivileged to privileged.
Now [snorts] this is kind of important if you later in the Proxmox backup server want to connect to remote storage locations on an NFS because only a privileged LXC container will have the correct permissions that are required to mount an NFS share. So make sure to set this to privileged because you cannot really change this setting later. So you would have to recreate the backup server. Then you can set up a root password. You can also leave it blank for automatic login. But I personally always would recommend you to set up a password. And you can set your container ID. You can set up your hostname. In my case, I'm just going to name this to PBS-PRO-1 if I want to follow my usual naming convention. And then you can set up a disk size. And now this is mainly used for the main operating system of the Proxmox backup server. You can later add more storage locations. So I think it's just fine to use the default 10 GB and then of course give it some CPU cores and memory. So just follow their recommendation on the requirements and then configure your other settings like giving it a static IP address is always a good idea in my opinion. And if you want to authenticate with SSH to your Proxmox backup server, which we by the way later in this tutorial have to do. So you definitely should upload your public SSH key and enable root SSH access. But for the other settings, you can just leave the defaults. I think that's totally fine. And then the Proxmox VE helper script will do its job and install the Proxmox backup server in a small container. So there's nothing else you need to do. You just need to answer this small question here, if you automatically want to mount all available VAPI devices. Honestly, I didn't really know what that was. And it says something with GPU hardware acceleration. So, I just ignored this. I don't think we need this on the backup server. And [snorts] then just wait until the setup is complete.
All right. So, once that is done, it will show you the address of the IP address that you have configured. You can just copy the URL, uh, paste it in your browser, and this will take you to the Proxmox backup server web interface. So, this nearly looks identical to the Proxmox uh web interface. Of course, you just have a few different menus, but it's really kind of like the same style and look. I like this. So, Proxmox users should already be familiar with this. And to finish the installation part, let's quickly return to the Proxmox VE helper scripts and also do the post-installation script. Now, that of course, you don't have to run on your Proxmox shell. You now have to select your Proxmox backup server's LXC container and then go to that container shell and there you have to execute the post-installation scripts. But that just goes through a few simple steps, updating the packages, disabling the subscription notification, and so on. Then the Proxmox backup server will reboot once more, and then you're done with a basic setup. That's all really super simple, and it's now ready to be used.
Okay guys, so now that we've installed the Proxmox backup server, we now can think about where we actually want to store our backups. And depending on if you followed the official recommendation to install the Proxmox backup server on a separate physical device or you followed my guide to install it on an LXC container, you have different options here. So either if you've got a separate physical device where you can configure your storage discs in the administration settings, or if you're running an LXC container and you mount some redundant storage pool that you've already configured on your Proxmox server, that would be the best way to do it. But also, if you're like me and you're a crazy dude just having one simple NVMe drive in your physical Proxmox server, you can still use this storage for the Proxmox backup server as well. And I later want to synchronize the local backups and send them to a remote storage location using NFS, then I can benefit from both. I can have super fast, quick, and small uh backups on my local Proxmox server discs and still have a separate physical location where I could do some disaster recovery if the entire server goes down or anything is damaged with the discs. That is what I definitely would recommend you to do. So if you don't have a separate physical server where you install the backup server, store it at least once a day to a remote location, but I will show you how to do this step by step. First of all, let's take a look how to configure a storage locally.
Now before we can configure this on the LXC container, we of course have to stop the container first and then we just need to go to the resource settings and add a new mount point. Of course, if you've got a separate device like a storage controller with separate discs that you didn't configure on Proxmox already, so they are just there for backups, you can forward the entire storage controller. But I think a mount point is also fine because there you can select a storage disc. So either the local LVM if you just got one storage disc in a Proxmox, or you select your ZFS storage pool or whatever that you're using to store your virtual discs as well. Then configure a separate path for this. So in my case, I always just use `/mnt` and then `/backup-local` so that I know that this location is stored on my local Proxmox server, and then I just give it a disk size. In my case, 200 GB. Actually, it doesn't sound like a lot, but it's enough to store all my discs. Even if the virtual discs of the VMs are much larger, but because of the deduplication job in the Proxmox backup server, it's enough to store everything for me. What you probably should deselect is this small backup checkbox because then this mount point will automatically be excluded from any backups. And once you created this mount point, you should see it under your storage location under CT volumes, that there is a new virtual disc created that the Proxmox backup server can use to store the backup files on.
All right. So now we need to switch to the Proxmox backup server's UI and go to data store and add a new data store. So here we just give it a name. In my case, I just picked the same name as the backing path. So the `/mnt/backup-local` and then you can select two settings here. The GC schedule, which stands for garbage collection. So this is a process that helps to manage and reclaim storage space by removing some data of the backup system that is no longer needed. And the prune schedule, which applies retention policies. So how long or how many you want to keep your backup files. These jobs you can run daily at midnight by default. You can also adjust the time if you want to, but I think once a day at midnight is fine. And then in the prune options, you can define if you want to keep the last X backups or if you want to keep one daily, weekly, monthly. I personally tend to keep one daily backup, one weekly backup, and one monthly backup so that I got enough flexibility. But adjust this according to your needs. Because it's an incremental backup, you can really keep many different backups. So you have more options to go back in time because you might not immediately recognize if you have deleted or broken something the next day. So better keep more backups than less. So you got more options to go back in time and revert to an older state if that's needed. Especially when doing file-level recovery, that is becoming really handy because then you can restore just a specific file that you have deleted 20 days ago. So adjust this according to your needs and how frequently you want to keep these backups.
All right. So now the data store is created. When you go to the data store summary, you can get more details and insights about the data store. You can see the usage. By the way, here you should see the maximum disk capacity of your mount point. If you don't see that and you still just see 10 GB or so of the root disk of the Proxmox backup system, you should double-check if you've used the correct path. And here you can also see how many container backups or host backups you stored, what is the deduplication factor. So the higher the deduplication factor, the more disk space you're saving. And here you can also inspect the content. So what are the different backups currently stored? Of course, this is uh still empty. The prune jobs and GC configuration, you can add synchronization jobs and also verification jobs. And a verification is quite important when it comes to backups because when you're just taking backups and you're seeing, oh, the backups are arriving and everything is fine, and then you want to restore something and then you only notice there is something wrong with it, well, you are in big trouble. So therefore, a verification job is always, always important when taking backups. So therefore, I would definitely switch to the options and enable the verify new snapshots. So this will always verify any new backups immediately after completion. This is definitely is super useful. Then you don't need a separate verify job.
All right. So now we can configure the Proxmox virtual environment to connect to the backup server. And before we do that, I want you to create a separate user account on the Proxmox backup server. So the Proxmox VE can securely connect to that, but it doesn't have all the root privileges. So just go to the configuration, access control, create a new user. And then when you added the user, go to permissions. Here you can add a new user permission to the path `/datastore`. So then the user will only have access to that data store location. Select the Proxmox user you have created and select the role Data Store Admin.
Now we can switch back to the Proxmox virtual environment and add the Proxmox backup server as a separate storage location. Just give it an ID, for example, in my case, it's just PBS-backup-local. The server will be the IP address of the backup server or the DNS name, and then use the credentials you have just created. Also, enter the data store that we have created, in my case, backup-local. And now we need to grab the certificates fingerprint. That might be a bit weird, but just go to the Proxmox backup server under configuration, certificates. Select the self-signed certificate that should have been automatically created, and then copy the fingerprint. You just need to put that into the field when you're creating the Proxmox backup server. Also, you could configure backup retention here, but usually, um, this is configured on the Proxmox backup server. So, just leave that default. And then if you want, you can create an encryption job. Now, of course, if you do this, you should never lose your encryption key, otherwise, you won't have access to the backups anymore. So this is an option that I personally, I didn't need to do, but of course, it is recommended if you're storing this on a separate location and for security reasons, you could create a client encryption key, might be useful, but again, in a small home lab, I don't know if you need that.
Okay. [snorts] So once you added the storage location, you can now, um, create your backup jobs, or what you can do is you can edit your existing backup jobs if you stored that previously on an NFS share. Just go to your backup job, click on add or edit your existing one, and then change the storage location from local to your Proxmox backup server. Just select the virtual machines that you want to back up. Select your time schedule, and then usually there is nothing else you need to change or configure. Then this should already be working. You can test if the backups are working. If you click on run now, this will immediately start a new backup job. And as you can see here, that is a backup job that is currently running.
Now you might have noticed that this is kind of like the same speed as the regular backup jobs. And you might be wondering why the heck, Christian, you just said the Proxmox backup server was so much faster and so on. So why I'm not seeing a difference? Because the incremental backups at first will require one full backup of the entire virtual machine discs at day one. But the next backups you're taking, they will only store the changes. And if your retention policy applies, so the prune job runs on the Proxmox backup server and the last day one with a full backup needs to be removed, it [snorts] will merge the full backup with the next incremental backup so that it always has at least one full backup of the entire virtual machine, and then it only stores the changes of the files or the file system of that virtual machine. And this alone is saving so much disk space, especially if you're keeping a high number of backups. I didn't really realize that the usual backup job on Proxmox doesn't support incremental backups. Now, with Proxmox's backup server, I don't really need to care about this and I can keep, I don't know, 30, 40, 50 different backup timestamps, and it's not really making any huge difference on my disk. So, this is really, really incredible.
Now, incremental backups are one thing where the Proxmox backup server really shines. Uh, but there's also the next feature that is called deduplication. This will be automatically done on the Proxmox backup server and it works the following. Without deduplication, the backup server would have to store each chunk of the data. Like if you represent chunks with A, B, C, D, E, F, and so on, these chunks from time to time repeat. And that regularly happens depending on the type of the data. So when a chunk contains exactly the same data as the other chunks, without deduplication, that would be stored as a separate chunk on the disk. Now with deduplication, the Proxmox backup server will know that the chunks are repeating. It just stores a small reference. So here, for example, you can see A, B, and D were chunks that were already saved on the disk, and the small reference doesn't take as much disk space as the entire data. Now I think it's worth noting that deduplication requires CPU power because this reference and calculation stuff takes some resources. So not only when you're storing the data, also when you're restoring data. However, the deduplication job is running in the background as far as I know. So it's done after you've taken the backup, and the disk savings can vary from 20% to 90% of disk space depending on what is the type of data. And the good news is that virtual machine images or backup data, they have a huge duplicate data, which is leading to higher savings when using deduplication. So you can really benefit a lot from this technique.
Just to show you an example, this is my Proxmox server, and you can see I'm storing a few virtual machines, one that has a disk size of 400 GB of data. That would already be double the amount of disk space that my Proxmox backup server has. And then I'm also storing some 60, 68, 68 GB of discs on the backup server. So that alone is like 600 GB of data. But the Proxmox backup server can still store this on its backup-local disk that only takes 163 GB of data, which is in my case a deduplication factor of 4.24. So this is indeed quite impressive. And if you go to the content, you can still see all these backups here. You can see the last timestamp that they were verified. And if you click on that drop-down button here, you could see the individual backups here as well. So don't let yourself be confused when more backups appear. They stack up, and they always show you the maximum size of the disk of the VM, not what actually is taken on the Proxmox backup server disk. So just how large the disk is on its own. But I can show you how, uh, it works when I go back to my Proxmox server and, for example, let's run a backup job now. And you can see how fast it is. So just go to data center, go to backup, and let's say run now. So here you can see it's using fast incremental mode, storing 384 GB of data in just 14 seconds, and it's even faster for the other virtual machines that have much lower disk volumes. Yeah.
All right. So, that was it. It didn't take a minute to store all my virtual machines on the Proxmox backup server. So if we switch back here and reload, you can see the backup appears. And, uh, here you can see what I mean. It's the same disk size, but it's indeed not taking much more space on the, uh, data store. If we go back to the summary, you can see it just increased a little, and I still have enough space to store all the other increments. And when it's applying the retention policies and the prune jobs and the garbage collection jobs, it even compresses this more. The deduplication factor might increase, and I even have more disk savings on the Proxmox backup server.
The cool thing is you can even do file-level restore. Now, one thing that might be a bit confusing here, you cannot really do this from the content of the backup server. So here you still see the blobs of data. That doesn't help you much. But if you go back to the Proxmox server and select the backups here, you have an option, file restore. And if you click on that, you can see the individual files on the discs. And you could really restore a specific directory or just one specific file that you need, like a database or, I don't know, some other files that you might have lost. Now, one thing that is a bit unfortunate, you cannot easily restore them in the virtual machine itself because the Proxmox backup system is an agentless, um, backup system. It's not like something like Veeam, so where you can put that back in the same location. So you have to download this as a zip file, extract it, then shut down the service on your server, and restore that particular database. So it might not be as intuitive and simple as a professional backup system. But honestly, uh, I think it's still great because it's, it's an open-source solution. So why would you argue about that? And I think this already is such a big plus and such a strong argument to use the Proxmox backup server. This is why I never would go back to storing backups on a separate NFS share.
However, as I said in the beginning of this video, it is still a good idea to use NFS or a remote storage location to synchronize these local backups too because if something might happen with this local device, you're having a problem. Let's [laughter] let's say it this way. But the good news is you can still do this with the Proxmox backup server, even when it's running in an LXC container. But, uh, if you want to follow this tutorial and use a remote NFS location for synchronizing the backups, you definitely need to run the LXC container in privileged mode. Otherwise, it won't work. I've tried it a bunch of times, and it's not as intuitive as the other settings because, first of all, we need to open the terminal and log in via SSH to our Proxmox backup server. That's why I said you should enable remote access. Then it's going to be much easier. And then first of all, you should create a separate folder where you want to mount the NFS share to. Because I've already created a mount point for the LXC container under `/mnt/backup-local`, I thought it might be a good idea to just call this `/mnt/backup-nfs`. And then you should change the ownership to set this to the backup user. That is what the backup server will use internally. And adjust the permissions to 775 so that the Proxmox backup server will have the correct privileges to save to this location.
Now, the next thing that we need is the NFS common packages on the Debian system of the Proxmox backup server. Usually, by default, they are not installed. So just run `apt install nfs-common`. Also, you can install Vim at the same time because to mount the NFS volume automatically so that it always is mounted when you reboot the backup container, you have to edit the file system table by using `vim /etc/fstab`. So here we need to enter the IP address of our NAS or the remote location where the NFS server is running. In my case, I want to store this on my Unraid NAS. By the way, if you don't know what that is, you can check out my Unraid video. That's pretty cool. So, this is my bigger storage server with a redundant, uh, storage array where I can better store backups for a longer time. So, the strategy that I'm following here is I'm just keeping a daily backup on my local Proxmox backup server, and on the remote storage location, I'm keeping more backups so that I can go back in time. The recovery probably will not be as fast if I would do it from the local device. But anyways, this is more for an archive backup restore option. Here on the Unraid server, I had one issue that I needed to solve because currently, I only have a rule created that anything from my production local area network from with the 10.20/16/24 network can connect to this server. But the flex in the parenthesis are wrong. So I needed to add another one that is the `no_root_squash`. So this is an option that allows the root user on a client machine to have root privileges on the NFS mounted directory as well. This is needed for the backup server to correctly initialize the data store. Without this option, it will not work. So this is what I needed to adjust.
But when we go back to the Proxmox backup server's shell in the `fstab` file, I just needed to enter the IP address. Then the mount point on the remote NFS server. So my Unraid NAS, this is usually under `/mnt/user` because it's a user share, `/nfs-backups`. And then I mounted this to the local folder that we have just created on the backup server, `/mnt/backup-nfs`. Of course, we're using the NFS protocol with a version 4. And then I needed to add the correct flags. I just copied this from my tutorial. So you can probably just use the same settings as well. They just work. And then you need to reload systemd with `systemctl daemon-reload`. This should automatically mount the remote NFS location. You can check, um, if you do an `ls` on this mount point if there are any files that were coming from the remote system. As you can see here, this is currently full with the old backups of my old NFS, uh, backup location that I used before. I needed to remove them all so that when I created the data store on the Proxmox backup server, it was initialized successfully. What was a bit weird is this error message in the background and that it took a bit longer to create this data store, but once that was complete, it indeed worked. And then you can create a sync job. Now you have to do this on the backup NFS, uh, data store. Create a sync job and select the location, local. So then it will pull the backups from the local data store and upload it onto the remote NFS store. And I've also configured this to run daily, maybe not at the same time as the regular backup jobs run. So most likely you will have to configure this once these are complete. So one hour later should be enough. And then it's running automatically and copying over everything from your local data store to the remote NFS location. You can still restore your virtual machines from both locations if you need to.
All right, guys. So, that's it about the Proxmox backup server. [music] I think this is really a fantastic solution which takes much less resources than just pure NFS backups on Proxmox. It's much faster, and you can even do a file-level restore. I think this is really great. But no, tell me what do you think about the Proxmox backup server? Is there anything I missed or something that you would change or adjust? Then please let me know in the comments. And as always, thank you so much for watching. A big thanks goes all to all of the supporters and members of my channel. You guys are really amazing. And I'm going to catch you in the next [music] video. Take care. Bye-bye.