Transcription
What is this and how to use it? In one video, I will give you all the necessary theory and practice for a basic understanding of how it works, so that you can perfectly answer an interview question about how it works and what it has, for example, directives. Today we will find out what it is. We will look at the basic features of Nginx. We will practice more, there will be less tedious theory, and at the very end, we will even deploy Nginx on our own server. Let's see how it will be. It will be very cool. Hello everyone, friends. My name is Artem. I am a Python Backend developer, and on this channel, I tell you how to improve your hard skills and advance your career. Be sure to subscribe to the channel and give it a like if you enjoy this kind of content. Well, let's go. So, first, I want to draw your attention to a small diagram. On the left, it's you, the user, who is currently watching the video on a phone, tablet, or computer, and you are using either a browser or some application. On the right is the Webserver Nginx. I want to convey to you the idea that absolutely all requests you make on the internet, well, okay, I'm probably lying, about 50% of requests are processed by Nginx. This is really true. Nginx is used by half or even more than half of the websites on the internet. And there is a way to check this. Even if on this site where I am now, Miro, where you can draw diagrams, right-click and view the page source. Like this. Right-click, view page source. Here, however, there is no such thing. Therefore, I will use a hotkey and open the developer panel like this. You can do this on absolutely any website. There is a tab called "Network". If I start moving anything here, bam-bam, bam-bam, back and forth, back and forth, requests will fly. Let's click on any of them and see what's happening here. We have some address. It's not particularly important to us what this address is. What we are interested in is what response we received. We have response headers. And here there is a very important field "Server:", you see. So we can really verify what responded to us here. It accepted the request, passed it somewhere to the backend of the site and the Miro company. The request was processed there somehow, this data, we can see the data itself here. And we received a response from this backend. Nginx carefully passed it to us. Yes, we received some response. What is important to clarify right away. It does not contain any business logic of your application. What to do with the data, how to save it to the database, in what format to give it to you. No, it only proxies requests. This sounds terrible. By the end of the video, you will understand what this means and how it happens. But in fact, Nginx takes the data you sent, headers, your IP address, all the information you transmitted through the browser, or as a programmer, if you directly access some API, some site, and then gives it to the service responsible for data processing. We will even write such a service today, actually. Let's move on to the key features, the key characteristics of Nginx, for which it is so loved and constantly used. The first, perhaps the simplest, by the way, yes, we will use Visual Studio Code today. You can choose any other and write code wherever you are comfortable, even in Notepad. And before starting, of course, you need to install Nginx. This can be done on all operating systems. I will share a link on how to install it on Windows in the description, because it's a bit more complicated. But if we are talking about Unix systems, i.e., macOS and Linux, it's not difficult. For example, on macOS, Brew Install Nginx, and on Linux, it's SUO UPTGET install. Again, all commands and instructions will be in the description. We have installed it, and we can run it with this command. We already have Nginx running. Well, great. Now, if we go to Visual Studio Code, here I have a special folder with Nginx already built-in, its path is already reflected here. And here is the single most important configuration file that you will most likely work with, or at least have seen at least once. And it's called nginx.conf. It looks very scary, I agree with that. It looks very unclear, but now we will figure out what's what. We won't pay attention to other files for now. It's important to clarify that today is not some super detailed guide on how to configure it in a production system. No, it's clear that there are many nuances in configuring Nginx. And today we will skip most of them. That is, we will use Nginx basically just so that you get an understanding of what it is and what it's used for. It's important to understand that Nginx always has such a configuration file, and it usually lives at some address within our server, within our computer. In this case, if you hover your mouse here somewhere, yes, it's the address opt/nginx/nginx.conf on Ubuntu. It's just the address nginx.conf. It's important to understand this because if you use it in a Docker container, as we'll see at the end of the video, then you definitely need to know where the config file is located. But we continue to study Nginx. What do we need here? Well, the first thing I suggest doing is deleting all comments. We don't need them, they only spoil the first impression. The less code, the easier, the calmer it is to perceive all this. Let's delete everything unnecessary here and leave only what is needed for our work. I have deleted all comments. Let's go to our browser. And if we go to the address localhost:8080, which is the port specified here in our file, we will see the standard welcome page of Nginx. Welcome to Nginx. What will we do? Well, first, let's try to serve static content. This is one of the features for which it is very loved, because it does it super fast. In order for us to have some static content, some kind of website, let's create it. I have a folder here, nginx_video. And here I will create a simple file and name it index.html. And here, with a simple manipulation, I will create some "Hello, friends" body. So, we will have just plain text. So, we want our file to be displayed instead of the basic welcome page of Nginx. "Hello, friends." What can be done for this? In Nginx, there are so-called directives in the config. There are directives, for example, events, there is the http directive, the server directive, and so on. And we are interested in location now. In this part of the configuration, it is written that if someone accesses our web server at the address /, i.e., simply localhost and port, then we serve the index.html file from the HTML folder. We want to serve our wonderful file now. What do we do for this? Well, first, we need to understand where our file is located. For this, you can simply type inside the pwd folder, i.e., get the working directory, the current one, or hover your mouse, and the address will also be shown. In general, all methods are suitable. And here you can simply specify root and our folder. Now, if we refresh the page, absolutely nothing will change. The fact is that when you change the Nginx configuration, you always need to reload it. But it happens that you change the configuration, for example, forget to put a semicolon or put an extra slash somewhere, anything can happen. First, you need to check that the config is okay. For this, we write nginx -t in any place in the code, in any directory. And it tells us that the syntax is okay, meaning everything is fine, there are no syntax errors. It also tells us, of course, where the config file is located. So you can always find out where it is. And we can now restart Nginx. How is this done? The command is nginx -s reload. We reload Nginx. Now, if we refresh this page, bam, "Hello, friends" is displayed. If we go to view the source, yes, right-click, network, look here, refresh the page again, then here is our document. Here it is, wonderful. And we will see in the headers that the server here is the latest, the newest version. This is how, friends, serving static content works. That is, Nginx can simply take static content and transfer it. Very simple. It's clear that this example is simplified. It's clear that static content is not always located on the same server. The configuration doesn't always look exactly like this. It certainly looks a bit more complex in production, but for understanding, this will be enough. Now let's look at the second feature of Nginx, which is redirects, or redirection. Imagine that you, your company, has a website and for 10 years the site lived on the domain example.ru. Suddenly you decided to move to example.com. And a billion links with the old domain name were scattered across the internet, and they cannot be changed. And you want any request to example.ru to be redirected to example.com, so that a person doesn't get a page like "this site does not exist," but gets the example.com page, is redirected where it should be. Redirects exist for this. How is this done? It's actually very simple. Again, if we return to the same location, let's comment out our "Hello, friends" file for now. And instead of displaying this file to the person, we will redirect them to another site, for example, to Google. Google.com. What needs to be done for this? We use the word return, as in returning, yes, as if we are in some function in Python or anywhere else in JavaScript. Then we write the status code 301. It must definitely be any 3xx status. Exactly. And in this case, it's 301. That is, we are simply redirecting the person, and then we specify where we are redirecting them. We can redirect them not only to another site, we can redirect them to our site, just to another address, by address. But let's redirect them to google.com. And be sure to put a semicolon at the end, never forget it. Before reloading Nginx, let's check that everything is okay. nginx -t, yes, everything is great. And now we can reload it. Great. Let's go back. Refresh the page now, and bam, I'm on google.com. Absolutely imperceptibly for me, as a user, this happened. Moreover, if we are cunning again, we know that we have the developer console, let's hide this story, then we will see that there is localhost, and we get not a 200 status, but a 301. This is exactly the number, the figure that we specified here. Moved permanently, meaning a permanent address change. And notice, we have the server, and Nginx added a header called "Location" and google.com for the browser. So, all this we just typed in, and it sent it to the browser, and the browser itself understands that it needs to redirect the user to this address. We can also, you know, if we go to our site, or rather, not even go, but try to call our site in the terminal, i.e., localhost:8000, sorry, 8080, then we will get this wonderful story. So we will see what it responds to us, and it responds with a 301 status. Let's remove the redirect. Of course, there are a huge number of redirects. They can be more complex, they can be based on regular expressions. We are not considering all of that, we just want to understand what features Nginx has. And the third feature is compression. Nginx allows very large files to be compressed into very small ones. What does this mean? This means that the client, well, a person who uses a browser, who looks at our pictures, receives video materials, static content, they will receive them faster because fewer bytes, fewer kilobytes will fly over the network. It's very simple. How does this happen? It compresses the response, sends it to the browser, and the browser decompresses this binary data into some JSON format. Let's look at a very simple example. Besides the HTML file, let's create a JavaScript file here, script.js, and just make it very large. I don't know, let's write some function that will return something. And let's replicate it to an incredible number of times. Let's make it 124 lines. And this file, of course, we will load here. Load script.js. So now, every time we visit our site, let's bring back our "Hello, friends," reload, and let's go back to localhost. Of course, we will see. Oh, by the way, notice, it's still redirecting us because the cache was used. Let's disable the cache. So browsers really like to cache everything, everything they need and don't need. So you can disable some incomprehensible browser behavior like this, disable the cache. Look, now every time we refresh the page, script.js is also loaded. It's also served, of course, yes, you can check it here. And notice, the file weighs 2 KB. This is a lot for such a small file. What does it offer in this case? We can use compression. For this, we will need a new directive. The directive is called gzip. First of all, we need to enable it, yes, gzip on. Then we need to specify that files in JavaScript format should be compressed or compressed. You can specify application/javascript here. There can be a huge number of formats, and you don't want to compress all of them. Here we will specify this specific file format. And then you can specify what our compression level will be, how much it will be compressed. You can imagine an analogy with archiving, yes, like RAR format or ZIP format. Our files are compressed very strongly. We specify comp_level, i.e., compression level, for example, six. And we added these three lines. Let's reload Nginx, check that the configuration is okay. Yes, the configuration is okay. Reloading. And now, attention. It was 2 KB. Refreshing. 348 bytes. We compressed our file more than five times. Yes, you can check a few more times, refresh, but nothing will change. The file will weigh exactly the same. Moreover, if you go to preview or response, you will see the entire file. And you might think, well, it's unclear at what stage the compression happens at all, where it is, why the file weighs so little. In fact, as I said earlier, Nginx sends binary data, so we can't really read it properly for the browser, and the browser deciphers it. How can this be seen as a developer? Again, in the developer console, look, in the headers, there is a header "Content-Encoding", i.e., gzip encoding. Gzip is a utility for compressing, in fact, any file format. The browser understands gzip, and that's it. And it unpacks it, and beautifully, perfectly presents it for us, for frontend developers mainly. So, usually, the developer console is used by frontend developers, of course, or very advanced backend developers. Now let's move on to the fourth feature that we will consider - this is load balancing. Everyone talks about some large, complex systems, distributed systems, about super high loads, RPS, like 100,000 RPS per second. Let's see how to balance the load in the simplest way possible. And for this, we will need a little knowledge of Python. I have a whole separate large playlist on my channel about writing backend in Python. In fact, you don't need to know any of it. I'll quickly do it for you in a minute. Let's create, rather, let's go to the terminal, to our Nginx_Video folder. Here, through Python, I will create some virtual environment. After that, I will enter it and install the library. Let's go in here. `source venv/bin/activate` and install, install two libraries. The first is FastAPI. I have a separate large course on it, both free and paid. And Uvicorn is some web server that interacts more with Python. You don't need to think about what a Python-side web server is now. Both are web servers. In general, backend is not a simple topic. We are installing two libraries. FastAPI is the most popular framework for writing APIs. And we will see how simple it is. Let's create a file main.py, and it will be very, very short. This is, of course, some simplification of the backend by hundreds, or even thousands of times. We create some class and say that if someone accesses the address /, we will give them, for example, some data "hello world". Like this. And let's run two. Actually, you can practice, run any number of application instances. I will run exactly two. For this, I need two terminals. One will be here on the left, the other on the right. Let's do it like this. Here we write `uvicorn main:app --reload --port 1111`. This is our Python web server. `main:app` means the app object in the main file. And specify, for example, port 1111. Bam. And this application, this API, will start on port 1111. And let's copy the same command, but run it on port 2222. So we will have, oh, sorry, yes, again. `uvicorn main:app --port 2222`. Like this. And we have two applications running. We can simply write localhost:1111, get the response "Hello World", access the second server. Oh, sorry. So they both work. If you look at the consoles, then each processed the request. But we are not interested in accessing them directly. We want to do this through Nginx, through proxying. So now we will look at both proxying and load balancing simultaneously. It will be cool. What do we do for this? So, first, we need a new directive. The directive is called upstream. Usually, let's call it, for example, api, or backend, in general, call it whatever you want. Here will be our addresses, yes, these can be IP addresses, these can be local application addresses, yes, or running projects. These can be Docker container names. And it can actually be many things. We have, as I said, localhost:1111. And we have localhost:2222. And here we must not forget to specify server. And here too, server. So we indicate that we have two servers now. And let's specify that if, for example, someone accesses the address /api, i.e., our API will live at some other address. Very often it is removed not just under some path like /api, but to a separate subdomain, i.e., api.example.com. And here we will do that we will use proxy_pass. This is a very scary word, very often incomprehensible to beginners, but essentially it means that we have some internal application running on the server, right now on our computer. So, I launched two Python applications. Each can process requests and return "Hello World". And it is not available to external users from, say, the internet. But Nginx is available to them. So, Nginx will accept the request and proxy it, i.e., pass it to one of my services. Either 1111 or 2222, to one of them, or if there are more, to a larger number of servers, a larger number of applications. And this is called proxying. Simply passing all headers, data, addresses, IP addresses, in general, all the data that is in the request, to another service, another application. Here we write http and /, yes, so that this /api is equal to this api, just for understanding. And Nginx will either access this server or that one. Moreover, here now there will be a very simple concept of how the server is calculated, who to give the request to. Simply in a circle. Round robin. So, first to the first, then to the second, then again to the first, then to the second. This is not very good. In production, you almost never need to use this, but again, we have a simplified example. Let's reload Nginx and see what we will have. nginx -t, yes, the configuration is good. nginx -s reload. Reloading. Reloaded. And let's see what we have here. We had two requests here, there was one request here. Let's go to the browser. On our main page, if we refresh it, nothing happens. So no requests are added, because we have /api. Let's go to /api. Bam. And we get "Not Found". The reason is that in Nginx, I forgot to specify the slash here. And for an unprepared user, for someone who has not worked with Nginx, it might seem like just a slash at the end. Does it affect anything? And in fact, this is a huge headache for everyone who has ever worked with Nginx. You can spend hours looking for a problem, and then it turns out that you forgot a slash. Let's reload again. We added a slash here. And now everything works. We get "Hello World". Moreover, let's look. In Let's, let's, you know what, let's run both our servers again, so as not to count how many requests there were. Running. And let's do two page refreshes. 1, 2. The server received two requests. Great. Now again. 1 time, 2 times. Our first one received a request. Let's refresh a few more times. And now, well, they are receiving requests approximately evenly. Let's do more. 1, 2, 3, 4, 5, and more requests have been added. There are three here and one there. By the way, it's curious that more requests go to the second server. When I experimented, an equal number of requests went to both servers. Well, in principle, in the long run, it will be exactly like that. So, we have looked at load balancing. We can run a huge number of such application instances. If we are talking about production, development, about some large projects, large companies, then it's often hundreds, or even thousands of such application instances, to be able to handle huge loads. But if you have a small project, a small startup, then, of course, this method will be more than enough for such load balancing. And as I promised at the beginning of the video, friends, let's see how to launch it on your own server, so that it works 24/7, so that your website, your startup, your future commercial project works constantly, without interruptions, without errors. For this task, we will use the servers of Selectel company. Selectel is one of the leading IT infrastructure and cloud providers in Russia. And we will rent a cloud server here. It's enough to register in the control panel my.selectel.ru and you can create your first server. Let's go to infrastructure, go to cloud servers, and here you can create a server. It takes literally 20-30 seconds. We will have Saint Petersburg, you can choose Moscow, you can choose, in principle, what is more convenient for you, what is closer to you. I will have Ubuntu version 20.04, yes, so it's Linux. And I will choose some configuration here. You can choose a standard one. We don't need super high loads. You can choose a shared configuration, where we will have slightly less CPU, where we will have slightly less RAM. We don't need it now. So, I will take one CPU, one 1 GB of RAM, and let's say 10 GB of storage. We will have a public IP address. And in general, that will be enough. You can click "Create Server". And literally in a few moments, we will have an IP address, and we can connect to it and configure it there. We have an IP address. Great. Let's copy it. Let's go to our terminal and here, using the SSH command, space, root, at sign, and our IP address, yes? So, root is the username. Press Enter, let's see if the server is ready. It might still be preparing. Yes, it's still preparing. You can try in a few seconds. In the meantime, let's go, get the password. Let's go to the server, go to the console. And here is the password. Let's copy it. By the way, you can see that Oh, everything is ready. You can also use this terminal. I use this one. I'm more familiar, more comfortable with this terminal. I press "yes" here. Yes, add this IP address to the list of known hosts. I enter the password, press Enter. And, something is wrong with what I copied. Let's copy it again, 10 times. And yes, we are in. Great, we are on the server. What do we need? We need to call Nginx. We don't have it. We can try to do `apt install nginx`. And most likely, nothing will work, right? If you are on Linux, then of course, you need to first run the `apt update` command, and all packages will be updated, all dependencies will be pulled, and after that, we can install our wonderful Nginx. Let's run our `apt install nginx` command again. And yes, now they are offering to allocate 12 MB of memory. So be it, let's allocate and see what's happening. By the way, our IP address is here. Let's go to it, see what's happening on it now. Here is the welcome to Nginx, the welcome page. Since Nginx is running, you can see, you can check `nginx -t`, see where the configuration file is located, you can go into it, yes, for example, using Nano and the address itself. And we see that all the settings are here with comments, of course, which we don't really need. And our Nginx is configured basically. It works, it processes. You can view the source, you can view the Nginx version, if you are interested. The Nginx version can also be viewed. By the way, if you type `nginx -V` or `nginx -v`, you can understand that the version here is 1.18. And here you can place your files, your website, your application, backend, frontend on your own server. And it will work 24/7 on your cloud server. A cloud server, I remind you again, we rented from Selectel. That's all for today, friends. Thank you very much for subscribing. Don't forget to subscribe to my Telegram channel, the link is in the description. And see you next time. Bye. Yeah.