Transcription
Greetings to you in the video, during which we will break down how the Internet and web applications work at the level of networks, protocols, and infrastructure from a developer's perspective. We will step by step analyze the basic principles and logic of operation, but without delving, for example, into signal coding or data transmission physics. We will start with the basics of computer networks, understand how devices interact with each other, what role clients and servers play, and why network interaction is impossible without protocols. Then we will move on to IP addressing, routing, analyze how packets are transmitted over the Internet, what NAT is, and what physical limitations it imposes on applications. Next, we will analyze DNS in detail, how domain names turn into IP addresses, and how resolving works. After that, we will move on to the transport layer TCP and UDP, connection establishment, and modern protocols such as Quick. A separate block will be dedicated to HTTP, its structure, methods, statuses, as well as how cookies, sessions, and tokens are implemented on top of it. Then we will analyze HTTPS, TLS, encryption principles, and certificate trust. We will also touch upon real-time communication, server infrastructure, load balancing, and CDNs. And in addition, we will talk about DPI and modern methods of analyzing and controlling network traffic. Don't forget to subscribe to my Telegram channel, where interesting posts about development, updates to my libraries, and spoilers for videos are published. And also, if you like this kind of content, please like and subscribe to the channel. This greatly supports my work and helps me create new videos. Well, we will start with a small prologue. It is not mandatory to watch, but in it, we will briefly touch upon the physical structure of the Internet. The Internet is a vast network consisting of many interconnected networks. Data centers, server racks, routers, and switches are real devices that receive, process, and transmit data around the clock. Data does not move as a whole, but in small fragments that travel independently of each other, strictly along routes defined by equipment and protocols. Every website opening is a physical process. A request travels through cables, reaches a server where it is processed, and a response is returned. Server operation requires energy, cooling, and a reliable connection. They work constantly, and systems are built so that even if part of the equipment fails, the Internet remains accessible. If you think about it, the Internet is the largest machine ever built by humanity. It has no center, does not belong to a single owner, and cannot be turned off with a single button. Everything relies on standards, agreements, and strict adherence to rules. Protocols and infrastructure ensure the predictability and reliability of this system's operation. And now we will analyze everything in order, starting with cables. There are several main types of cables used for data transmission: copper twisted pairs, coaxial cables, and fiber optics. They differ not only in construction but also in physical principles of signal transmission, range, speed, and application. Let's start with copper cables, the simplest and most visual form of network connection. The most common type is Ethernet, used in home and office networks. Inside such a cable, there is not one, but several copper conductors. Usually eight, combined into four pairs. Each conductor is insulated, and the pairs themselves are twisted together. Twisting performs several tasks at once. Firstly, it reduces the influence of external electromagnetic fields when electrical noise from another cable, a power supply, or an electric motor passes nearby. It affects both conductors of the pair almost equally. The receiving device compares the signals within the pair and discards the common noise, leaving the useful information. Secondly, twisting reduces the influence of adjacent pairs on each other. Without it, the signal from one pair would induce interference in another, which would sharply reduce the speed and reliability of transmission. But there is another type of copper cable: coaxial. It was used in television networks, cable Internet, and early computer networks. Its construction is fundamentally different. In the center, there is a single solid copper core through which the signal travels. Around it is a layer of dielectric, then a metal braid or shield connected to ground, and an outer protective sheath. The shield isolates the central core from external interference and simultaneously creates a stable electrical environment for signal propagation. This construction makes coaxial cable more resistant to noise and allows the signal to be transmitted further without distortion. But it is less flexible, thicker, and scales poorly for modern speeds. Now, briefly, about how data travels through copper cables. I will not delve into signal physics and electronics here. Here we will just understand the general idea. Data is transmitted using an electrical signal. Digital information is converted into voltage changes over time, as well as into pulse shapes. The source of this signal is active network equipment. In a computer, it is the network card. In infrastructure, in network infrastructure, it is the ports of switches and routers. The signal has several key characteristics: voltage level, rate of change, and pulse duration. It is their combination that encodes zeros and ones. On the receiving side, network equipment restores the original data. It is important to understand here that the signal is analog, and it is subject to distortion. The main reasons are: copper resistance, conductor heating, external electromagnetic fields, and interference from adjacent pairs. As the cable length increases, interference accumulates, and data begins to be lost. Repeaters and amplifiers are used to compensate for these losses, which restore the signal shape. But even with them, copper cables are limited. Their speed and range are only suitable for short connections at home, in offices, or data centers. They are unsuitable for long trunk lines and intercontinental links. Fiber optics are used for the global Internet. Unlike copper lines, data transmission here is based not on electric current, but on light propagation. The physical basis of fiber optic cable is glass fiber. In the very center is the core, a thin thread of ultra-pure glass or quartz. Its diameter is comparable to the thickness of a human hair. Around this core is a cladding of glass with a different refractive index. It is the difference in these coefficients that causes light to reflect repeatedly from the material boundary and propagate strictly along the fiber, practically not escaping. And on the outside, it is surrounded by several protective layers. These include coating, strength members, and outer insulation. These layers do not participate in data transmission, but they provide the cable's mechanical strength and protect it from the same bends, moisture, pressure, and temperature fluctuations. Data transmission begins in active equipment. The so-called transceiver receives digital data and converts it into a sequence of light pulses using a laser or LED. Each pulse encodes information through the presence of light, duration, or phase. At the other end, photodiodes capture the light and convert it back into an electrical signal for processing by network equipment. The main difference between fiber optics and copper is that it does not interact with the external environment. There is no conductor heating, no electromagnetic interference, no influence of external fields. Signal loss is minimal, so amplifiers and regenerators are placed tens, and sometimes hundreds, of kilometers apart. This allows data to be transmitted over 1,000 km without significant distortion. In addition, fiber optics has a colossal bandwidth. Modern technologies allow transmitting multiple independent data streams over the same fiber using different wavelengths of light. This makes fiber optic lines the basis of the entire global network infrastructure. This is why all modern trunk and intercontinental connections are built on fiber optics. For such distances and data volumes, there is simply no alternative. After we have analyzed copper and fiber optic cables, as well as their structure, role in the Internet infrastructure, and, in principle, their differences, let's move on to the next topic. How the Internet crosses oceans and connects entire continents. Contrary to a common myth, the global Internet relies very little on satellites. More than 95% of global traffic travels through cables. The reason is simple: only they provide the necessary speed, stability, and bandwidth. Submarine cables are laid by specialized cable-laying vessels. On board these ships are giant spools of cable hundreds and thousands of kilometers long. The laying route is calculated in advance with high precision. The seabed topography, seismic activity, shipping lanes, and fishing zones are taken into account. The cable is carefully laid on the seabed, and in shallow waters, it is additionally buried in the ground. By the way, the construction of a submarine cable is significantly more complex than ordinary fiber optics. Inside, of course, is the same glass core that transmits light, but on the outside, there are several layers of protection. These are waterproof sheaths, reinforcement with additional wires, insulation from moisture and corrosion. This allows the cable to lie on the seabed for years at a depth of several kilometers, withstanding water pressure, currents, and the aggressive marine environment. Nevertheless, it is impossible to completely eliminate risks, of course. Submarine cables are sometimes damaged by ship anchors, fishing trawls, or underwater earthquakes. This is precisely why the global infrastructure is built with fault tolerance in mind. Trunk routes are always duplicated, and network protocols automatically redirect traffic along alternative paths. We will talk about this in this video. In most cases, the user, in principle, does not even notice that a break has occurred somewhere on the ocean floor. Within countries, trunk lines are usually built by national telecommunications operators. In Russia, for example, Rostelecom and other major communication operators, which connect cities, regions, and data centers within the country, primarily perform this role. And intercontinental routes are the responsibility of international consortuns and technology giants. Companies like Google, Amazon, and Microsoft invest billions of dollars in building their own submarine lines. It is thanks to these cables that the Internet has become truly global. Without them, the modern world as we know it simply would not exist. In principle, our story about cables is coming to an end. Let's move on. Look, cables themselves, of course, do not make the Internet the Internet. They are just physical roads. A logical question arises: where exactly do all these data streams come from, and how do different networks exchange traffic with each other? The Internet is a huge number of independent networks belonging to different providers, companies, and organizations. For them to interact, there are special connection points. They are called Internet Exchange Points, or IXP for short. IXP are physical locations. Most often, they are large data halls where powerful switches and routers are installed. Internet providers, hosting companies, cloud platforms, and large services connect directly here. Terabytes of data pass through these nodes every second. When you open a website, your request may pass through several such points. It leaves your provider's network, enters an IXP, then enters another operator's network, then possibly another exchange node, and only after that reaches the desired data center. CDNs, content delivery networks, also start working at this level. They store copies of data in different parts of the world and connect directly to exchange nodes. Thanks to this, requests do not always go to the original server but are served at the nearest point. The user simply sees that the site loads faster. How exactly this works and why it is needed, we will analyze in detail later. That is, in essence, IXPs are Internet intersections through which a huge part of global traffic passes. Every request, every page load, and every data stream almost certainly passes through one or more such nodes. It is they that turn a set of separate networks into what we call the global Internet. Now let's talk about data centers. This is where requests are directed after passing through the network infrastructure. And this is where the main data processing work is done. A data center is a complex of premises with server equipment. Inside are rows of server racks, each containing dozens of physical servers. In total, several thousand such machines can operate in one data center. They receive incoming connections, process requests, work with databases, and generate responses, which are then sent back to the network. The operation of such a volume of equipment requires significant energy resources. Servers generate a large amount of heat, so data centers use constant cooling. Air or liquid cooling systems maintain a stable equipment temperature and prevent overheating under prolonged load. Power supply in data centers is organized with redundancy. Several independent power lines, uninterruptible power supplies, and diesel generators are used. This allows maintaining equipment operability during external power grid failures without connection interruptions. From the perspective of network interaction, a server is not a specific device, but a process running on data center equipment. This process listens on a specific port, accepts network connections, and processes requests. It can be restarted or moved to different physical equipment without changing the service's behavior for the user. Data centers are connected directly to backbone networks and traffic exchange points. This allows requests that have passed through the Internet infrastructure to reach the computing environment with minimal latency. Now that we have more or less covered cables, traffic exchange points, and data centers, we can assemble this infrastructure into a single chain and trace the complete physical path of data from the user's device to the server and back. It all starts at your home. When you open a website and press Enter, the browser forms a network request, a set of bytes in the computer's memory. The operating system transfers these data to the network stack, where they are broken down into packets and sent to the network card. The network card converts digital data into a physical signal. If the connection is wired, it is an electrical signal over a twisted pair. If fiber optics are used, the signal is formed directly as light pulses. And in the case of Wi-Fi, data is first transmitted over a radio channel to the router, and then goes further via cable. Next, the signal reaches the Internet provider's equipment. In this segment, data passes through the provider's switches and routers, which direct the packets further through the network. Within a city and between cities, fiber optics are almost always used, so the electrical signal at the input is converted into an optical one and propagates along communication trunk lines. If the server is in another country or on another continent, the data goes to international trunk lines, respectively. On the other side of the trunk line, the data enters the network of another operator and passes through traffic exchange points. These are physical nodes where different companies' networks connect directly, as we have already discussed. Here, routers make decisions about which network to transmit the packet to next so that they reach the desired data center. After that, the request reaches the data center itself. Backbone fiber optics are connected to the data center's network equipment, from where traffic is distributed across internal networks. Packets reach a specific server rack, then to the network port of the desired server. On the server, the physical signal is converted back into digital data. The operating system transfers it to the server application, which processes the request, reads data from memory or a database, and then generates a response. And the response itself follows exactly the same path, but in the opposite direction. It is converted back into a physical signal, leaves the data center, passes through backbone networks, traffic exchange points, and the provider's network, after which it returns to your device. The network card receives the signal, restores the data, and the browser displays the result. Now we have some picture of the physical layer of the Internet, from the user's device to the data center and back. Next, we move up to the next level. We will talk about networks and addressing, how devices find each other, how routes are chosen, how DNS works, and why requests always arrive where they should. Well, we will start with the most basic, computer networks. A computer network is a way by which devices can exchange data. In its most basic form, a network consists of at least two devices and a communication channel between them. One device can send data, and another can receive it. And for a network at this level, nothing else is required. It does not matter what these devices are. It can be a computer, phone, server, or any other equipment capable of transmitting data. The distance between them also does not matter. Devices can be in the same room or on different sides of the world. And the communication channel is the medium through which data is transmitted. It can be a physical cable, Wi-Fi, or a mobile network. For a basic understanding of a network, there is no difference between these options. Data simply moves from one device to another. And here it is important to understand that the network itself knows nothing about websites, browsers, or applications. It does not understand what a page, button, or form on a website is. At this level, the network is only concerned with transmitting bytes, a sequence of data without meaning or context. Well, simple byte transmission is not yet complete communication. As many know, bytes are just sequences of zeros and ones. One computer sends them, another receives them, but these numbers themselves mean nothing. Without rules, devices will not be able to understand what each byte represents, and the exchange will simply be meaningless. So that the data has meaning for the recipient, protocols are used – a set of strict rules and agreements. A protocol defines how data is divided into packets, how their integrity is checked, what the transmission order is, what headers to use, and how devices respond to errors. In other words, a protocol is a common language that devices speak. Without it, the network cannot function, regardless of the communication channel used or where the devices are located. And the most popular protocols are HTTP, TCP, and UDP. HTTP allows exchanging web pages. TCP ensures reliable delivery and packet order, and UDP provides fast but unreliable transmission, for example, for video games. All these rules allow different devices to exchange data correctly and predictably. We will analyze the protocols themselves during this video. In the meantime, let's understand how data physically moves through the network, especially when it comes to long distances. Let's start with the network scale. There is a local area network, or LAN. This is a network within a small area, for example, an apartment or an office. That is, computers, phones, servers, and other devices are located nearby and are connected directly through a minimal amount of intermediate equipment. In a local network, data travels a very short distance. Usually, there is only one router between the sender and receiver, or perhaps a direct connection, so the delays here are minimal, and the transmission speed is very high. Well, we also have a wide area network, or WAN, which stands for Wide Area Network. This is a completely different scale. This is a network that, in fact, connects many local networks together. As I already mentioned in the prologue, when you open a website, your device does not communicate directly with the server. There are dozens or even hundreds of intermediate nodes between you and the server: cables, racks, routers, and data centers. And here a logical question arises: if data passes through such a number of devices and communication channels, how can information be transmitted reliably and efficiently? The answer: packets. In computer networks, data is never transmitted as a single large block. Any information, text, images, video, HTTP request, or server response is first broken down into small parts. These parts are called packets. Let's consider a small example. You go to Telegram and perform some action. For example, you type a message and press the send button. Your device forms a stream of bytes. Then the operating system's network stack breaks this stream into packets of fixed or limited size. And each packet is an independent unit of transmission and can be processed by the network independently of the others. Any network packet consists of two parts: a header and a payload. Look, the header contains service information necessary for delivering this packet and processing it. It includes data such as the sender's IP address, the recipient's IP address, protocol information, sequence number or packet identifier, technical flags, and service fields. Technical flags are control parameters that determine transmission behavior. They can indicate, for example, the start or end of transmission, the need for acknowledgment of receipt, the connection status, and packet processing specifics. And service fields are used for data integrity control, flow control, and correct packet assembly on the receiving side. The payload itself is the actual part of the transmitted data. Each packet contains only a small fragment of the overall stream. I just said that after formation, packets are sent to the network separately, and in a global network, there is no single predefined route for them. We already know that the path between the sender and receiver is quite long. Providers, backbone networks, and so on. And each intermediate node, say a router, makes a decision only about the next transmission step, without knowing the packet's full path. As a result, packets of the same stream can take different routes, arrive with different delays, be received out of order, and be temporarily lost. On the receiving side, packets are not interpreted immediately as application data. First, the headers are analyzed, i.e., integrity is checked, belonging to the stream is determined, and the correct sequence is restored. When using reliable protocols, lost packets are re-requested. When using unreliable protocols, missed data is ignored. And only after that, the restored byte stream is passed to the application layer, where the logic of the specific service is applied. In this case, it is Telegram. Exactly the same process occurs in the reverse direction when the server sends data to your device. Now we move on to the next part of the first chapter, the first module. And let's look a little closer at network types. As we discussed, we have local area networks (LAN) and wide area networks (WAN). But besides these two, there are several other interesting types of networks that occupy an intermediate position between local and global systems. These are MAN (Metropolitan Area Network) and PAN (Personal Area Network). Let's start with MAN, a metropolitan network. This is a network that covers one city or a large campus. Imagine that several office buildings in one district are connected to each other with high bandwidth via fiber optic lines. They can exchange data faster than if it were, for example, a regular WAN, because the distance is shorter, the infrastructure is optimized, and the routes are carefully designed. Unlike WAN, MAN usually does not connect cities to each other, and the management of such a network is often centralized within the city or a specific organization. This is why it cannot be considered a WAN, because its scale is smaller, control is accordingly higher, and the network does not require global routing. As for PAN, Personal Area Network, it is the smallest network of all that you can encounter around you every day. It connects a person's devices, such as a laptop, phone, tablet, headphones, or, say, smartwatches. Such networks use technologies like Bluetooth, USB, or sometimes Wi-Fi over short distances. PAN is very limited in distance, bandwidth, and the number of devices. And its main task is to ensure convenient synchronization and data exchange between personal devices. This is a completely different level than WAN, because PAN does not connect multiple users, it does not work through providers, and, in principle, does not require global routing. Let's move on and discuss the next, let's say, concept that defines the structure of the global Internet – Autonomous Systems, or AS. An Autonomous System is a large network or a collection of networks managed by a single organization. This can be an Internet provider, a large company, or a cloud service. Each such autonomous system has a unique identifier. It is called an Autonomous System Number (ASN), which allows other networks to understand who they are exchanging traffic with. And within an autonomous system, data can be transmitted through any internal routes optimized for speed, reliability, and load. However, for traffic exchange between different autonomous systems, a special protocol called BGP, or Border Gateway Protocol, is used. It determines the paths by which packets will be transmitted between networks on a global Internet scale. And thanks to autonomous systems, the Internet is not a chaotic mess of networks. Each AS manages its part of the infrastructure but interacts with others according to clearly defined rules. For example, when you open YouTube, traffic first passes through the autonomous system of your Internet provider, say MTS or Rostelecom, then through one or more backbone ASs, and only after that reaches the autonomous system of Google, where YouTube servers are located. And each of these stages is controlled by its own autonomous system, with its own routing rules. Let's return to protocols. As we already know, these are the rules on which device communication in a network is built. And now let's consider where protocols come from and who develops them. This will be a small digression. We will analyze the transport protocols TCP and UDP later. Look, each protocol is born out of a specific need. When the first networks appeared, it became clear that without unified rules, devices simply could not understand each other. And it was necessary to define how to structure data, divide it into packets, check for errors, and restore order. That is, these are real communication tasks between devices that become more complex with the network. With the growth of the Internet, it became clear that one device or program cannot set rules for the whole world. Protocols must be unified, open, and understandable to everyone. Therefore, there are special organizations involved in the development and standardization of protocols so that new technologies work with existing ones. And thanks to standardization, your browser can interact with a server on the other side of the planet, regardless of the manufacturer or equipment. This is why we still use TCP, IP, and HTTP. Their principles remain the same, and new versions only adapt them to modern requirements. Let's also briefly discuss who is involved in their development, protocol development. There are several, perhaps, key organizations. The most famous of them is the IETF, or Internet Engineering Task Force. This is a community of engineers, developers, and experts who voluntarily discuss, propose, and approve new protocols and standards. It, of course, does not manage the Internet, but it creates and sets the rules of the game that all network participants then follow. Besides this organization, there are others. For example, ISO is the International Organization for Standardization, which forms general standards for network technologies, including the OSI model, which we will discuss now. And besides it, there is also an organization like IEEE, or the Institute of Electrical and Electronics Engineers. This institute develops standards for local networks, such as Ethernet and Wi-Fi. All these organizations work independently of each other. Well, as we can see, their efforts complement each other and create a common language for absolutely all devices. So, it turns out that protocols...
This is the result of the work of thousands of engineers around the world. After such a digression about the protocols themselves and who is involved in their development, let's move on to the next concept. This is the OSI model, which I just mentioned. Look, OSI is an abbreviation for Open Systems Interconnection. And in fact, it is not some specific technology or protocol. It is a conceptual model that helps to understand how data moves from one device to another. As we already know, information on the network is broken down into packets, passes through many devices and intermediate nodes. OSI gives us a tool to structure this process and explain what happens at each stage of data transmission. This model divides network interaction into seven logical layers. Each layer is responsible for its part of the work, and the layers interact with each other strictly in order. That is, if some layer encounters a problem, it is easier to diagnose or fix it, because we know exactly which part of the chain is responsible for a particular function. Let's break down these levels. The first level is called physical. Here we are already familiar with this topic from the prologue. This level is responsible for the physical transmission of signals: electrical impulses in copper cables, light impulses in fiber optics, radio signals in Wi-Fi or mobile networks. The second level, Datalink, is the data link layer. Here, information is organized into so-called frames, which include the addresses of local network devices, MAC addresses. It ensures the correct transmission of data between devices within the same local network and checks for errors that may occur at the physical level, for example, due to interference in the cable. The third level is called network. This is the network layer. This is where IP addresses are used and routing between networks occurs. Remember, we talked about WAN, MAN, and packets. It is the network layer that decides how packets get from one node to another through many intermediate autonomous systems. The fourth level is called transport. The transport layer. This level manages the delivery and order of packets. It is this level that guarantees that the packets we broke down when sending a message or video arrive in the correct sequence and that lost packets can be retransmitted. TCP and UDP work precisely at this level. The fifth level is called session. It is responsible for establishing, maintaining, and terminating connections between applications on different devices. For example, when you open a video on YouTube, the session layer ensures that the connection with the server is maintained during viewing, or rather, for the duration of viewing, and then it is correctly closed. Following this is the sixth level, presentation. It deals with data format, encoding, and encryption. This is the level that ensures that text, images, or videos sent by one device are displayed correctly on another. And finally, the seventh level is application. This is everything that we see as end-users: browsers, messengers, email, and streaming services. This level interacts with users and uses all the lower levels to transmit data and receive responses. It is this division into layers that makes the OSI model so powerful for understanding networks. And although in practice the Internet does not directly use OSI, and modern networks are built on the TCP/IP model, OSI remains an incredibly useful concept. In essence, it is a kind of convenient network map that shows what happens at each stage of data transmission from a physical signal to an application. Well, and now I suggest we move on to practical models, such as TCP/IP, and compare them with this conceptual scheme. If OSI, as we found out, is a convenient theoretical scheme, then TCP/IP is a set of protocols and rules by which data is transmitted on the network every day. It is this model that all websites, messengers, streaming services, and cloud applications work by. And the TCP/IP model is more compact and practical than OSI. Here, instead of seven, there are only four levels, but despite the less detail, it covers everything we have discussed in OSI. It simply combines some functions and makes them more applicable. Let's break down these four levels. The first level is called Network Interface or Network Access Layer. This is the physical and data link part of what is divided into Physical and Datalink in OSI. At this level, the device knows how to physically transmit packets through cables, fiber optics, Wi-Fi, or mobile networks, and how to address them within the local network. Everything we talked about regarding cables, repeaters, routers, and local networks is fully used here in practice. The second level is Internet. It is responsible for delivering packets between networks, i.e., for routing. It is here that the IP protocol is used, which allows each packet to find its way through many autonomous systems. The third level is called transport. Here, TCP and UDP work. This level directly ensures reliability, sequence control, and data integrity, which we examined in detail using the example of Telegram. And the last, fourth level is called Application. This level combines everything that is at the upper levels of OSI: session, presentation, and application. HTTP, HTTPS, FTP, SMTP, DNS, and other protocols that interact with the user and ensure real data exchange between applications work here. Well, in principle, we have covered the four levels of the TCP/IP model. Again, I want to note that we are not delving into the details of each protocol here. We will cover them a little later. It is important to simply understand that the application level is where digital signals and packets are transformed into human-understandable actions and services. Well, and in principle, let's summarize the main feature of TCP/IP. It is that it is as close as possible to the real world and solves specific tasks. How to deliver a packet from your device to the server, how to guarantee their order and integrity, and how to interact with network applications. In principle, this is the end. In principle, the second, or rather, the first module is coming to an end. Let's summarize some small conclusions. We have covered what a network is and how it works. That is, data is always transmitted in packets, and protocols ensure their correct delivery. We also talked about network types, got acquainted with autonomous systems that manage routing. And finally, we examined the OSI network models with seven levels and the practical TCP/IP with four levels. And now, with this foundation, we can delve further into routing. And move on, accordingly, to the second module. And to begin with, let's figure out what IP is, as well as its versions. IP stands for Internet Protocol - it is a digital address on the network. It allows you to precisely determine where a packet came from and where it should go. It can be represented, well, IP can be represented as a kind of postal address. To send a letter, you need the recipient's address. To receive a reply, you need to provide, accordingly, a return address. And, as we already know, in every network packet, the IP addresses of the sender and recipient are recorded in the header. Now let's break down the key things in IP addresses. That is, how they are structured, why they are needed, how they are generated, and how old IP versions differ from the new IP version 6. Let's start with version 4, because it was the first standard and is still actively used. Look, IPv4 is a 32-bit numerical identifier. In its usual form, it is written with dots. You see an example on your screen, but behind this notation hides a sequence of 32 zeros and ones. And to make such a long string of numbers easier to read and use, the address is divided into four blocks of 8 bits, octets. Each such octet can take a value from 0 to 255. If you look deeper, each block is a binary number. You see the decoding on your screen, and you can see what 192, 168, 1, and 10 turn into. And by combining these blocks together, we get exactly 32 bits, a full IPv4 address. But an IP address is not just a set of random numbers. It has a structure that helps the network understand where to direct data. One part of the address indicates the network, and the other indicates a specific device within it. This is similar to a postal address, as I gave an example above. That is, first the letter is delivered to the correct city, and then to the street, and only then to the correct house. And similarly, the packet first reaches the correct network, and within it, it finds a specific device. Well, there is one problem. Look, IPv4 was created at a time when the Internet was not yet the global infrastructure that we know it as today. 32 bits allow for the creation of just over 4 billion addresses. And when this standard appeared, this was more than enough. However, with the growth in the number of computers, smartphones, servers, and other devices, it became clear that the address space was running out. In addition, some addresses are reserved for special tasks, such as internal networks, testing, and so on. And all this accelerated the industry's transition to the new IPv6 standard. Version 6 of IP became an evolutionary continuation of IPv4 and solves the key problem of address shortage. The main difference here is in the address length. Instead of 32 bits, 128 bits are used. This number is so large that it allows assigning a unique address to every device on the planet with a huge reserve for the future. IPv6 addresses look different. They are written in hexadecimal and are divided into eight blocks of 16 bits, separated by colons. You see an example on your screen. And to make the notation less cumbersome, sequences of zeros can also be shortened. As a result, the same address can look different. You see on your screen how it can look. The structure of version 6 of IP is also divided into two parts. The first part defines the network in which the device is located. The second is, as it were, a unique identifier of the interface within that network. The main idea of IPv6 is that now it is possible to build networks without worrying about address shortages. Each device can have a unique public IP and be directly accessible. This simplifies interaction between systems, makes networks more flexible, and prepares the Internet for further growth. However, to effectively manage the address space and understand which devices are in the same network, addresses alone are not enough, of course. A mechanism is needed that clearly shows the boundary between the network and the devices within it. And therefore, we will now talk about CIDR. CIDR stands for Classless Inter-Domain Routing. It is a way of writing IP addresses that shows which part of the address belongs to the network and which part belongs to the devices within it. Today, this approach is used everywhere: from home routers to the global Internet infrastructure. Previously, IP addresses were distributed according to a rigid class system. Let's break them down. We had large networks, medium, and small. Well, there's a problem here. This led to huge address wastage. For example, if a company needed about 500 addresses, a small network was not enough, and the next category provided tens of thousands of addresses, most of which remained unused. And as the Internet began to grow rapidly, it became clear that this approach was not scalable. And that's when a system was needed that would allow allocating just as many addresses as needed. Thus, CIDR appeared. You now see a notation on your screen. This is the IP address 192.168.1.0/24. This notation indicates that the first 24 bits of the address are used to designate the network. Since IPv4 contains 32 bits, there are 8 bits left for devices within it. This part determines how many devices can be connected. And, accordingly, if 8 bits are available, this gives 256 possible addresses. However, one address designates the network itself, and the last one is used for broadcast, or as it is called, broadcast. This is a special address to which, when sent, all devices within the subnet receive the packet. Well, in short, in the end, 254 addresses remain for devices. This is more than enough for an apartment or a small office. And it turns out that CIDR allows creating networks of the required size, reduces routing overhead, and prevents wasteful loss of addresses. Without it, IPv4 would have run out much faster, and routing would have become excessively complex. Okay, we've covered CIDR. That is, when we see a notation like /24, we understand which part of the address belongs to the network. But how do devices themselves determine this? For this, a subnet mask is used. This is a special value indicating where the network ends and the host begins. In binary form, the mask is a sequence of ones and zeros. Ones denote the network, zeros denote the device. And for a /24 network, the mask looks like this. You see it on your screen. And in its usual form, it looks like this. It turns out that everything that falls under the ones belongs to the network, and everything that falls under the zeros belongs to the devices within it. Now you see a breakdown of an address of some device on your screen. That is, we understood that CIDR and subnet mask are two forms of writing the same idea. CIDR is simply convenient for people and routing, while masks are used in device settings and network protocols. Now that we have more or less figured out how devices get addresses and how the network determines their location, it is logical to move on to the next level of identification. If an IP address can be compared to a house address, then a port, which we will now discuss, is an apartment number or a door within that house. It is the port that determines which application on the device should receive the data. A port is a logical number that allows multiple programs to use the network simultaneously without interfering with each other. When data arrives at a device, the operating system looks not only at the IP address but also at the port number to send it to the correct application. It turns out that a complete network address always consists of three elements: the transport protocol, the IP address, and the port number. The IP address indicates where to deliver the data, and the port indicates which program is expecting it. From a technical point of view, a port is a number from 0 to 65,535, transmitted within the TCP or UDP header. Historically, ports are divided by purpose. The lower range is reserved for standard services. For example, web servers use port 80, secure HTTPS connections use 443, and remote SSH management uses port 22. These numbers are known and standardized so that clients can connect to services without additional configuration. Higher up are ports used by applications and services, and even higher is the dynamic range used by client devices for outgoing connections. That is, when a browser opens a web page, it temporarily selects a free port, uses it for the connection, and releases it after the exchange is complete. This is why it is said that a server listens on a port. This means that the program is constantly waiting for incoming connections on a specific number. Clients connect to this port, and the server processes requests and sends responses. After we have covered IP addressing and routing, we move on to routing. I think I've already mentioned this in the video, but I'll repeat it just in case: every packet passing through the Internet does not know the entire path to the recipient in advance. Imagine traveling through a huge maze where every intersection decides where to send you next. On the network, routers perform this role. Each router looks only at the next destination and decides where to send the packet next. This principle is called hop-by-hop. Step by step, the packet moves through the network from one router to another, and each step depends only on the local information of the current device. Routers use so-called routing tables, which resemble a road map. These tables specify which interfaces are available for different networks. When a packet arrives, the router compares the recipient's address with the map and sends the packet to where it will bring it closer to its destination. That is, if part of the network is overloaded or unavailable, packets bypass the problematic section. On the recipient's side, the system collects them in the correct order, checks their integrity, and eliminates losses. Let's move on to the next topic. And here, let's talk about a problem, so to speak. Look, every device on the network needs a unique address for data exchange. However, it is understandable that global IPv4 addresses are insufficient, and connecting every device directly to the Internet is simply impossible. And here NAT appears, which stands for Network Address Translation. This technology allows multiple devices on a local network to operate through a single public IP, hiding the internal structure of the network from the outside world. For example, you have a router to which your computers, smartphones, tablets, and so on are connected. But only one address is visible from the outside. When any device sends a request to the Internet, NAT replaces its internal address with the external one and remembers who sent the request. Thus, the response from the external server will return to the exact device that was expecting it. To manage a large number of devices, PAT (Port Address Translation) is often used. PAT assigns a unique port number to each connection, allowing the router to distinguish tens, hundreds, or even thousands of requests and direct responses correctly. Thanks to this, all devices can use one IP. At the same time, the Internet does not get confused about who needs which response. However, NAT creates problems when external users need to connect directly to an internal device. If you want to run your own server or online service, standard NAT is insufficient. The Internet sees only one address, and the connection cannot reach inside your local network. But there is a solution: port forwarding. The router receives a request on a specific port and forwards it to a specific device. Modern providers also often use CGNAT, where hundreds of users share one public IP. This saves addresses but complicates direct connection to devices. In such networks, port forwarding may not work, and running your own server becomes a rather complex task. However, NAT remains a crucial tool. It not only saves addresses but also enhances network security by hiding internal devices from direct external attacks. It turns out that NAT and PAT help create a manageable network where each device has Internet access, but internal details are hidden. And if NAT takes care of how devices see the Internet, then within the local network, it is important to manage addresses so that everything works automatically and without conflicts. This is where DHCP comes in. DHCP stands for Dynamic Host Configuration Protocol. It is a protocol that assigns IP addresses and all necessary connection information: subnet mask, gateway address, DNS. When a device connects to the network, it sends a request: "I need an address." In turn, the DHCP server responds by assigning a free IP for a specific period. This is called leasing. And upon expiration of the lease term, the device can receive the same address or a new one. Leasing plays a key role here. It prevents conflicts when two devices accidentally receive the same address and simultaneously allows IP addresses to be reused if a device disconnects. This makes the network dynamic and resilient even with a constant connection and disconnection of a large number of devices. DHCP is especially convenient for users. Connecting to the network happens instantly, without manual configuration, and the network remains organized. But for servers, dynamic address assignment is impractical. Servers must be accessible at a constant address, otherwise clients will simply not find them. Well, in principle, module 2 is coming to an end. And we are moving on to module 3, in which we will talk about DNS. Every day we open a browser and enter website addresses like google.com, youtube.com, or github.com. We easily remember these names, share them, and save bookmarks. But computers do not understand these words. To open a website, a device needs the IP address of the server where it is located. The question arises: how does the Internet know which numerical address is hidden behind a familiar domain name? This is precisely the task that DNS, which stands for Domain Name System, solves. DNS translates human-readable domain names into network-readable IP addresses. DNS translates human-readable domain names into IP addresses. You can think of it as the Internet's phone book. You enter a name, and the system converts it into the required number. But DNS is not some single registry or server. It is a system built on a hierarchical principle, thanks to which the Internet remains scalable and resilient. At the very top level are the root DNS servers. These are Root Servers. They do not know the address of every website, but they know where to look for information further. If you imagine DNS as a tree, the root servers are its base. The next level is the next level. Top-level domains or TLDs: .com, .org, .net, .ru, and many others. These servers are responsible for information about domains within their zone and indicate which servers know the final answer. And at the last level are the authoritative DNS nodes, which contain the final data about the domain: the website's IP address, mail servers, and service parameters. When a request reaches them, the search is completed, and the client receives an exact answer. It turns out that the request moves down the hierarchy from top to bottom until the source of reliable information is found. But a domain name can return not only an IP address. DNS can transmit different types of data that describe the domain's infrastructure and service operation rules. When a DNS server responds to a query, it returns a record of a specific type. The record type indicates what specific information needs to be obtained. And let's break down all these types. Let's start with the most basic record - the A record. An A record associates a domain name with an IP address. When a browser opens a website, it most often receives an A record that specifies the server's IP address. You see an example on your screen. This means that to open the site, you need to connect to the server with this address. The A record remains the fundamental basis of the web's operation because IPv4 is still widely used. Moving on. The next record type is the AAAA record. It does the same, but only for IPv6. You see an example on your screen again. Modern systems often request both A and AAAA records simultaneously. If IPv6 is available, the connection can be established through it. This increases routing efficiency. So, it turns out that A and AAAA records are fundamental records that allow the client to find the server. But DNS can work not only with them. Let's therefore examine the next type called CNAME. These are domain aliases. CNAME, which stands for Canonical Name, allows one domain name to point to another. You see an example on your screen. This means that www is just an alias for the main domain. When a client requests www.example.com, DNS redirects it to example.com, after which an A or AAAA record is returned. CNAMEs are used to connect CDNs, cloud services, and external platforms. There is also a type called MX. MX records determine which servers accept email for a domain. For example, you send an email to support@example.com. The mail system requests the MX records for the domain example.com to determine where to deliver the message. An example of a response might look like this. You see it on your screen. The priority number indicates the order of use. That is, a server with a lower value has a higher priority. And if the primary server is unavailable, then the mail will be sent to the backup. This makes the mail system more resilient. There are also TXT records. These are service verification records. They allow storing arbitrary text that is used by various services. Although they were originally created for comments. Today, TXT plays an important role in security and verification. For example, domain ownership confirmation is implemented through TXT records, for example, for cloud services or SPF, allowed mail servers. That is, in principle, various service verifications. An example of an SPF record might look like this. You see this example on your screen now. What does it do? It informs receiving servers which sources are authorized to send mail on behalf of the domain. Moving on. Next, let's quickly look at the NS record type. NS records indicate which DNS servers are authoritative for a domain. That is, when a domain is delegated to a hosting provider or a DNS platform, NS records tell the world where to look for official information. You see an example on your screen. This means that these specific servers contain the authoritative DNS data for the domain. Without NS records, DNS would simply not know where the domain information is stored. And the last record type we will cover is the SRV record type. SRV records are used to specify specific services. Within a domain, SRV is used in Voice over IP systems, corporate systems, gaming services, and so on. An example of an SRV record can indicate where a voice communication server is located and on which port it operates. What does this provide? It allows client applications to automatically find the necessary services without any manual configuration. In principle, we have briefly covered DNS, well, the types of DNS records. I think this is sufficient for understanding. Therefore, let's move on and discuss domain resolving. Look, suppose you enter a website address, say, cloudflare.com. The browser cannot immediately send a request to the server. First, it needs to know the IP address associated with this name, as we have already understood. And this process is called domain resolving, converting a domain name into a network address. At first glance, everything seems simple, but in fractions of a second, a whole chain of checks occurs. What exactly happens? In short, the system always tries to get the answer in the fastest way possible, without contacting external servers unless necessary. Let's break it down in more detail. First, the browser checks its own cache. If you have already opened this site recently, the address may be stored directly in the browser's memory, and a repeated request is not required. If there is no record, the request is passed to the operating system. The operating system also maintains its own DNS cache, where recently used domains may be stored. If there is no answer there either, the system contacts the local resolver. Usually, this is the DNS server of your router or provider. It is this server that performs the full search, going through the DNS hierarchy, from root servers to top-level domain servers, and then to the authoritative server.
Domain. Once an answer is found, it is returned back through the chain of the Resolver, OS, and browser. After this, the browser already knows the IP address and can establish a connection with the server. However, the cache is not stored forever. Each DNS record has a TTL parameter, Time to Leave, its lifespan. It indicates how many seconds the record can be stored in the cache before re-checking with the authoritative server. A small TTL allows for faster data updates, while a large one reduces the load on the infrastructure and speeds up user performance. And it is precisely caching and the lifespan of records that lead us to the next important effect that developers and administrators encounter when changing domain settings. Look, when you change DNS records, for example, move a website to a new server or change an IP address, the update does not become, apparently, instantaneous. This phenomenon is called DNS propagation or the spread of DNS changes. The reason lies in caching. Since records are stored at different levels in user browsers, operating systems, DNS resolvers, providers, and intermediate servers, old information may continue to be used until the TTL expires. This means that some users will reach the new server, and some will reach the old one. This state can persist from several minutes to 24 or even 48 hours, depending on the set record lifespan and the caching policy of the providers. In production, this leads to typical situations that often cause confusion. After updating DNS, the site may open for some users and not for others. Mail may temporarily be delivered to the old server, or, for example, HTTPS certificates may appear invalid if the user still reaches the previous host. Therefore, DNS changes require careful planning. Before migrating infrastructure, TTL is often reduced to speed up cache updates. After the migration is complete, TTL can be returned to higher values to improve stability and performance. Now that we understand how domain names are converted to addresses, how caching works, how it speeds up performance, and why changes do not propagate instantly, we can conclude the third module, in which we have just covered DNS, and we can, ah, smoothly transition to the next module, the fourth module. In the fourth module, we will talk in more detail about TCP and UDP. I have mentioned them several times when we discussed how packets are structured and who develops these protocols. But up to this point, we have not particularly focused on these protocols in detail. In this module, we will dedicate ourselves entirely to transport protocols. And we will begin with TCP, Transmission Control Protocol, one of the key building blocks of the internet. TCP is a connection-oriented protocol, meaning it is a stateful protocol. Statefulness here means that before sending data, two devices agree that they are ready to exchange information, and a kind of transmission contract is established, which guarantees that all bytes will be received and in the correct order. The basis of connection establishment here is the famous three-way handshake. This process begins with the client sending a special packet to the server with the SYN flag, short for Synchronize, signaling the desire to establish a connection. The server, in response, sends its SYN packet, confirming its readiness to accept the connection, and simultaneously sends an ACK, acknowledgment that it has received the request from the client. Finally, the client sends the final ACK, closing the communication establishment cycle. This sequential exchange of signals ensures a reliable start to communication and allows both parties to synchronize their sequence numbers, which will be used to track transmitted data. After the connection is established, TCP ensures reliable data delivery. Reliability is achieved through several key mechanisms. The protocol monitors the byte order, checks the correctness of each segment using a checksum. And if any packet is lost or arrives with an error, TCP initiates retransmission of only that segment. And thus, a guaranteed complete data stream is obtained. This function is especially important for protocols where order and integrity are critical. For example, for HTTP, SMTP, FTP, and many others. The next important aspect is the streaming nature of TCP. Unlike connectionless protocols, where each packet is independent, TCP treats data as a continuous stream of bytes. This means that applications can send large amounts of information without dividing them into separate messages at the transport protocol level. TCP itself breaks the stream into segments, optimally packages them, and ensures their delivery over the network. The receiver reassembles these segments into the original stream, restoring the order and providing the application with data in precisely the order it was sent. All that I have described now – byte order, checksums, segmentation, and retransmissions – makes TCP a complex and powerful tool for network interaction. It can regulate transmission speed depending on network conditions, avoid congestion, dynamically manage receive and send windows, and adapt to losses and delays. After we have thoroughly examined TCP and understood how the protocol establishes a connection, it is logical to ask: what happens when a connection does not require such strict reliability, when speed is critical, and not absolute delivery guarantee? It is precisely for such tasks that UDP exists, which stands for User Datagram Protocol. UDP fundamentally differs from TCP in that it is connectionless. There is no three-way handshake, no delivery confirmation, and no byte ordering. Each packet, or datagram, is sent as a separate entity, and the sender receives no confirmation that the packet has reached the recipient. This approach makes UDP lightweight and fast because the protocol spends virtually no resources on control and coordination, allowing for minimized delays and overhead for data transmission. Despite the lack of delivery guarantees, UDP is highly in demand, for example, in video and audio streaming, where the loss of a few packets does not destroy the overall stream but is perceived as minor pixelation or brief audio noise, or in online games, where delay is more important than the exact delivery of each packet. The game client cannot wait for confirmation from the server for a second or two, otherwise controlling the character will feel sluggish. Special attention should also be paid here to VoIP, voice over IP, that is, voice transmission over the internet. UDP is also indispensable here because voice packets must arrive as quickly as possible, and even if some of them are lost, the conversation remains understandable. Small gaps are compensated by the human brain. TCP in such scenarios would be too slow due to constant retransmission of lost packets and sequence control, which would lead to unnatural delays and lags in voice communication. Technically, UDP packets contain a minimal header, which specifies the source and destination ports, packet length, and checksum. This compactness makes the protocol easy for network devices to process, reduces the load on routers and servers, and also allows for scaling applications with a large number of parallel connections. However, this simplicity means that the responsibility for error control, integrity checking, and delivery order is entirely shifted to the application. Therefore, developers of VoIP clients, game engines, and streaming services create their own algorithms for loss correction, buffering, and data resynchronization to compensate for the lack of built-in reliability in the protocol. To summarize, UDP is a tool for scenarios where speed is more important than guarantees, and applications can independently handle losses and correct the data stream. To summarize, UDP is a tool for scenarios where speed is more important than guarantees. Unlike TCP, which cares about every byte, UDP trusts applications to decide how to use available traffic. Now that we have examined the specifics of TCP and UDP, we can ask ourselves when and why one protocol is chosen over another. Let's start with TCP. HTTP, the protocol on which most of the web is built, works over TCP not by chance. When you open a website, the browser expects every part of the data to arrive completely and in the correct order. The HTML page, scripts, styles, images. Everything must be correctly assembled into the final stream. TCP here guarantees the delivery of all these elements, ensuring sequence control, retransmission of lost packets, and integrity checking. Without TCP, browsers would encounter broken pages when individual files are lost or arrive in the wrong order, and the user experience would be extremely unpleasant. The reliability of TCP and its streaming model allow HTTP to be stable, predictable, and universal, even if the network is overloaded or unstable. But if we look from another perspective, modern demands for speed and minimal delays have led to the emergence of protocols over UDP, such as QUIC. This abbreviation stands for Quick UDP Internet Connections. We will talk about it in more detail in the next module. QUIC was developed by Google to combine UDP's speed with TCP's necessary reliability, but to do it more efficiently for modern web applications. It uses UDP as transport because UDP does not establish a connection and does not wait for acknowledgments. At this basic level, QUIC implements its own mechanisms for acknowledgments and recovery of lost packets, stream ordering, and encryption, which is built-in by default. This allows for reducing TCP overheads, such as slow connection opening and head-of-line blocking, where one lost packet blocks the entire delivery queue. So, it turns out that TCP and UDP are two tools with different philosophies. One is focused on absolute reliability and ordering, while the other is focused on speed and efficiency, leaving control to the application. And now we move up one more level of the network model and look at the protocol that every developer and user encounters daily. We are talking about HTTP, the language with which browsers and servers communicate with each other. HTTP stands for Hypertext Transfer Protocol. Despite its name, it is used not only for transferring HTML pages but also for sending images, videos, JSON data, files, and any other resources. HTTP belongs to the application layer. It works over transport protocols, most often TCP, utilizing their reliable delivery, but it itself defines the rules of communication between the client and the server. At the core of HTTP lies a simple yet extremely powerful interaction model called Request-Response. The client, for example, a browser, sends a request to the server, specifying the resource address, the operation method, headers, and sometimes the request body. The server receives this request, processes it, and returns a response, which contains the execution status, service headers, and data. After this, the cycle is considered complete. A key concept in HTTP is idempotence. Simply put, an idempotent request can be sent multiple times in a row, and the result will not change. For example, if you open a user page, refreshing the page again does not create new data and does not break anything. You simply get the same result again. This is important for network resilience. That is, if a request is lost or sent again due to a connection failure, the server should not accidentally duplicate the operation. Therefore, some HTTP operations are considered safe to repeat, while others are not. And this directly affects API design and client behavior. HTTP also defines methods, a kind of verbs, that tell the server what exactly needs to be done with the resource. Let's break down these methods. The first is GET. Used to retrieve data. It should not change the server's state, is considered safe and idempotent. Browsers actively cache such requests. Next is POST. Used to create new resources or perform actions that change the system's state. For example, user registration, form submission, or order creation. A repeated POST request can create duplicates, so it is not idempotent. Following is PUT, which means complete replacement of a resource. If you send the same PUT request multiple times, the result will remain the same. The resource will be brought to the transmitted state. PATCH is used for partial updates of a resource, when only individual fields need to be changed. Repeated sending may yield the same result, but it depends on the logic of the changes, so PATCH is usually not considered strictly idempotent. Next is DELETE. It deletes a resource. It is considered idempotent because if the resource is already deleted, a repeated request will not change the server's state. And then there is HEAD. It works the same way as GET, but the server returns only the response headers without the body. This allows checking for the existence of a resource, its size, modification date, or cache metadata without downloading the content. And the last one is OPTIONS. It tells the client which methods and parameters the server supports for a specific resource. This method is used, for example, in the CORS mechanism, when the browser pre-checks if a request from another domain is allowed. In addition to methods, HTTP also uses response statuses. These are three-digit codes that report the result of request processing. They are divided into categories. Codes starting with 200 mean successful request execution. For example, 200. Then, those starting with 300 report redirects. And those starting with 400 indicate a client-side error, for example, a bad request or lack of access. And those starting with 500 signal server-side problems. It turns out that HTTP is a universal protocol for application interaction, defining the structure of requests, operations, and response codes. HTTP, thanks to its simplicity and flexibility, has become the foundation of the web and modern internet services. Now let's look at how HTTP itself has evolved significantly over time. The first versions, they, in principle, handled document transfer well, but the growth of web applications and the emergence of dynamic content, mobile internet, and complex interfaces required increased speed, reduced latency, and more efficient use of connections. Thus appeared HTTP/1.1, then HTTP/2, and finally HTTP/3. Each version solved specific performance problems of the previous one. And now let's examine these versions and what they solved. HTTP/1.1 was standardized in 1997 and became the standard of the web for many years. It introduced a key improvement: persistent connections (Keep-Alive). In earlier versions of HTTP, a new TCP connection was opened for each request. The browser downloaded HTML, closed the connection, then opened a new one for CSS, another for images, and so on. This led to enormous overhead. Each time a TCP handshake was performed, latency increased, and network load grew. Keeping a single TCP connection open and sending multiple requests through it consecutively sharply reduced latency and sped up page loading, especially for pages containing dozens of resources. However, this version still had a serious architectural problem: head-of-line blocking at the application level. This means that the browser could send multiple requests consecutively, but the server was obliged to respond strictly in order. If the first request was processed for a long time, all subsequent responses were delayed, even if they were already ready. In real conditions, this led to queues within the connection and slowed down page loading. To bypass this limitation, browsers began to open multiple parallel TCP connections to a single server, usually from six to ten, which partially sped up loading but created additional load on the network and servers. HTTP/1.1 became the foundation of the modern web, but its text format and sequential request processing model eventually began to limit performance. Now let's look at HTTP/2. It was standardized in 2015 based on the SPDY protocol developed by Google. It was created to solve performance problems without changing HTTP semantics. That is, all methods, statuses, and logic remained the same. Only the data transmission mechanism changed. The main innovation here is multiplexing. Now, multiple requests and responses can be transmitted simultaneously within a single TCP connection. That is, data is broken down into small frames, which can be interleaved and reassembled on the client side. This eliminates queue blocking, meaning a slow resource no longer delays others. The second important change is the transition to binary frames instead of text format. In HTTP/1.1, data was transmitted in text form, which required string parsing. HTTP/2 uses compact binary structures that are processed faster and reduce the amount of transmitted data. HTTP/2 also introduces header compression (HPACK). In real requests, headers are often repeated, such as User-Agent, and compressing them significantly reduces traffic volume. Another feature is server push, meaning the server can send resources to the client in advance, without waiting for an explicit request. For example, along with HTML, the server can immediately send CSS and JavaScript. This reduces latency, although in practice, the mechanism is used cautiously to avoid sending unnecessary data. Despite the huge performance gains, HTTP/2 still works over TCP, meaning it inherits transport-level blocking issues. The loss of a single TCP packet can temporarily halt the transmission of other data. And here we move on to HTTP/3. HTTP/3 is a more radical step in the evolution of the protocol. It was standardized in 2022 and no longer works over TCP, but over UDP, using the QUIC transport protocol, which I mentioned earlier. QUIC was developed by Google and standardized by IETF as a modern transport protocol combining UDP's speed with reliability, loss control, and security. The main advantage of HTTP/3 is the elimination of TCP-related delays. That is, connection establishment is faster due to integrated TLS. The loss of a single packet no longer blocks other streams. Each data stream is independent of others, and reconnection is instantaneous thanks to the connection migration mechanism. Since QUIC natively uses TLS 1.3, HTTP/3 always operates through encryption, which enhances security and speeds up connection establishment. Transmission "over UDP" here allows bypassing TCP limitations, i.e., reducing latency in mobile networks, improving stability with packet loss, and speeding up page loading with unstable connections. It turns out that the evolution of HTTP is a path from a simple text protocol to a highly efficient, multi-threaded, and resilient data transmission system. To summarize, HTTP/1.1 made connections persistent. HTTP/2 eliminated blocking and optimized transmission, and HTTP/3, in general, rethought the transport itself, minimizing delays and increasing internet resilience in modern network conditions. However, the speed and efficiency of data transmission are only part of the picture. Once a connection is established and requests begin to execute, the next important question arises: how does the server know who is sending the request and what happened previously? HTTP is inherently a stateless protocol, meaning each request is independent and does not store information about previous interactions. This makes the protocol simple and scalable, but creates a problem. Without an additional mechanism, the server cannot distinguish one user from another or remember what happened a minute ago. It is precisely to solve this problem that cookies, sessions, and tokens appeared – mechanisms that allow preserving the context of interaction between the client and the server. For example, when the server wants to store small data on the client side, it sends a `Set-Cookie` header. The browser stores the cookie and automatically attaches it to every subsequent request to the same domain. Thus, the server gains the ability to identify the user without needing to store state within the HTTP protocol itself. Cookies, they are small text pairs, ah, key-value type, which can store a session identifier, some interface settings, user language, or some technical parameters. They are automatically managed by the browser and transmitted unnoticed by the user. When working with cookies, security parameters are important. Let's examine a few attributes in a cookie. The `HttpOnly` attribute. It prohibits access to the cookie from JavaScript. This protects it from theft through XSS attacks. Next is the `Secure` attribute. It instructs the browser to transmit cookies only over HTTPS, preventing data interception in unencrypted connections. And additionally, the `SameSite` attribute is often used, which restricts sending during cross-site requests and helps combat CSRF attacks. But a cookie by itself is just a container. In practice, it most often stores a session identifier, and the main user data, they, of course, remain on the server, i.e., email, password, I don't know, their name, and so on. This approach is called session-based authentication, if we are talking specifically about sessions. That is, when a user logs in, the server creates a session, a record in a database or storage, well, for example, Redis, and assigns it a unique ID. This ID is sent to the client in a cookie, and with each subsequent request, the server receives the ID, finds the corresponding session, and restores the user's state, authorization, roles, and settings. We will not delve into authorization and authentication now, just a superficial overview. Sessions have a huge advantage: a high level of control and the ability to instantly terminate access by deleting the record on the server. The disadvantage is the need to store state and scale the session storage with a large number of users. An alternative is also token-based authentication. Here, instead of storing state on the server, the system issues a token to the client, a cryptographically signed set of data containing information about the user and access rights. Such a token, for example, a JWT, is sent to the client in the `Authorization` header with each request. Unlike sessions, the server does not need to store state. It verifies the token's signature and extracts data directly from it. But tokens are harder to revoke before their expiration date, so additional mechanisms are often used. These include short lifespans, refresh tokens, and revocation lists. Well, we will not delve into the topic of authorization now, because it is huge in itself. Since our video is about something completely different, if you want to understand it more deeply, you can watch a separate theoretical lecture on the channel about authorization and authentication, as well as a practical video where we implement authorization using sessions. So, to summarize, HTTP remains a stateless protocol. But thanks to these mechanisms, which we have just discussed, full-fledged systems of authentication, personalization, and access control are built on top of it. In principle, the module on HTTP can be considered complete. We have covered the basic request-response model, methods, statuses, and evolution. And now we have a holistic understanding of how the web interacts with users, transmits data, and maintains real-time work context. But there is one critically important point. Classic HTTP transmits data in plain text. This means that any information sent via HTTP can be read by those who can intercept the traffic. As we already know, data is transmitted over the network in packets through many intermediate devices. And if the connection is not secured, the content of these packets remains available for analysis. For example, imagine a user connecting to public Wi-Fi in a cafe or airport. When using regular HTTP, an attacker on the same network can intercept traffic using special packet analysis tools and see the transmitted data: logins, passwords, form contents, cookies, card numbers, or personal messages. This is called traffic interception or sniffing. Interception is limited to reading data. If the traffic is not encrypted, it can be modified in transit. Since packets pass through many nodes, an attacker can inject malicious code, substitute page content, or redirect the user to fake websites. This type of attack is known as a man-in-the-middle attack. For example, when loading a website without encryption, an attacker can substitute a JavaScript file by embedding malicious code that will silently intercept user data. The user sees a page that looks correct, but is actually interacting with fake logic. In addition to attacker attacks, the absence of encryption allows intermediate systems to analyze and modify traffic. Providers or corporate networks can insert ads, block content, or redirect requests. Since data is transmitted openly, the user has no guarantee of the integrity of the received information. It turns out that unencrypted HTTP provides neither confidentiality nor data integrity. Any information can be read, modified, or substituted in transit. It is precisely these vulnerabilities that led to the appearance of HTTPS, a secure version of HTTP, which adds encryption and guarantees that data remains private and unreadable. And now we will examine how TLS works, the protocol that makes this possible. TLS, which stands for Transport Layer Security, is a mechanism that ensures the confidentiality and integrity of transmission. It is TLS that underlies HTTPS and protects connections through which passwords, bank details, correspondence, in short, any information is transmitted. The main task of TLS is to create a secure communication channel over an already established TCP connection. At the same time, data is encrypted in such a way that only the recipient can read it, and any changes in transit will be detected. To understand how this works, it is important to understand two types of encryption: asymmetric and symmetric. Asymmetric encryption uses a pair of keys: a public key, which can be shared with anyone, and a private key, which is kept secret. Data encrypted with the public key can only be decrypted with the corresponding private key. When a browser connects to a server via HTTPS, the server sends its public key. The browser uses it to securely transmit secret information that only the server can read. However, asymmetric encryption is computationally intensive and slow. Using it for the entire data stream would be inefficient. Therefore, TLS uses it only during the establishment of a secure connection. That is, before starting the secure data exchange, the client and server first perform a process called TLS handshake. This is a series of steps during which the parties agree on the TLS version and a set of cryptographic algorithms. The server sends its digital certificate and public key. The client verifies the authenticity of the certificate, and the parties securely agree on a secret symmetric key. Modern TLS versions, for example, TLS 1.3, use Diffie-Hellman or ECDHE algorithms, which allow creating a shared secret key even over an insecure channel. At the same time, the key is not transmitted directly; it is calculated independently by both parties. This is an important point. That is, even if an attacker intercepts the entire exchange during the handshake, they will not be able to reconstruct the final encryption key. After the handshake is completed, the parties obtain a shared symmetric key. Symmetric encryption means that the same key is used for encryption and decryption. Algorithms work significantly faster and are ideal for transmitting large amounts of data. For traffic protection, TLS uses modern symmetric encryption algorithms, such as AES (Advanced Encryption Standard), as well as data integrity control mechanisms, for example, Authenticated Encryption with Associated Data (AEAD). This provides three levels of protection at once. First is confidentiality, meaning the data cannot be read. Second is integrity, meaning any changes will be detected. And third is authenticity. The client is confident that it is communicating with the correct server. And after the TLS handshake is completed and the symmetric key is established, all further HTTP traffic is transmitted in encrypted form. Even if packets are intercepted, they will look like a random set of bytes. This is why when connecting to an HTTPS site, an attacker can only see the fact of the connection, but not the content of pages, forms, or transmitted data. Now let's examine the role of SSL certificates and who guarantees that the server you are connecting to is indeed who it claims to be. Look, when a browser establishes a secure connection via HTTPS, it must verify that the server actually belongs
to the domain that the user is addressing. SSL certificates solve this task. In fact, it is more correct to talk about TLS certificates lately, since SSL (Secure Sockets Layer) is obsolete, and TLS is used. This is Transport Layer Security. A certificate is a digital document that links a domain name to the server's public cryptographic key and confirms this link using the digital signature of a trusted center. For such a document to have legal and technical value, it must be signed by a trusted participant in the ecosystem. This is precisely why certificates are issued by organizations called Certificate Authorities, or abbreviated as certification centers. They verify that the applicant truly owns the domain, after which they sign the certificate with their digital signature. Among the well-known certification centers are DGERT, GlobalSign, or Let's Encrypt. When a certification center signs a certificate, it confirms that this public key truly belongs to the specified domain. Browsers and operating systems have a pre-installed list of trusted ones, so their signatures are considered reliable. However, trust on the internet is built through a multi-level model, a chain of certificates. At the top of this chain is the root certificate, embedded in operating systems and browsers. Below are intermediate certificates, signed by the root center, which are used to issue certificates to specific websites. That is, when a server sends its certificate, it also transmits the intermediate one so that the browser or OS can build a chain: the website's certificate, the intermediate one, and the root one. If this chain is correct, then the connection can be considered secure. A little later, we will analyze again why the browser trusts the root certificate and how these trust lists are formed. In the meantime, let's move on. Look, not all certificates are signed by trusted centers. Sometimes a certificate is created and signed by the server owner themselves. This is called a self-signed certificate. It provides channel encryption but does not confirm server authenticity, which is why browsers show a warning about an insecure connection. Such certificates are usually used in development, test environments, or internal corporate systems where trust is established manually. And now let's analyze wildcard certificates. Look, when a conditional infrastructure begins to grow, a new task arises. It is necessary to protect not one domain, but many subdomains. And it is precisely for this that wildcard certificates are used. For example, a certificate for subdomains of domain.example.com will work, conditionally, for app.example.com, for shop.example.com, and any other subdomains. This greatly simplifies infrastructure administration and reduces the number of certificates, and also reduces maintenance costs. And it is against the backdrop of such tasks that automation of certificate management has become particularly in demand. Look, previously, obtaining a certificate was paid and required manual configuration, which slowed down the adoption of HTTPS. The entire situation changed in 2015 with the advent of Let's Encrypt, a free certification center created with the support of the Internet Security Research Group. What did it do? It automated certificate issuance through the ACME (Automated Certificate Management Environment) protocol. The server can independently obtain and regularly update a certificate without administrator intervention. This is precisely what became the key factor in the mass transition of the internet to HTTPS. To summarize, a TLS certificate performs two key functions at once. It includes connection encryption and confirms server authenticity. Without certificate verification, an attacker could impersonate a bank's website or an online service and intercept user data. That is, certificates form such a foundation of trust on the internet. And now let's talk about why the browser trusts a certificate. When we talked about the certificate chain, I mentioned that trust on the internet is built not directly, but through a hierarchy of certification centers. But here the question arises, why does the browser trust this chain at all, and who decides which centers can be trusted? And here the answer lies in what are called root trust lists. When you install an operating system or browser, they come with a set of built-in root certificates. This is a list of certification centers that the platform developers have decided to trust. If a website's certificate can be linked through a chain of signatures to one of these root centers, the connection is considered secure. For example, operating systems from Microsoft and Apple, as well as most Linux distributions, support their own trusted root certificate stores. Browsers like Mozilla Firefox use their own independent list, and Google Chrome and Microsoft Edge rely on the operating system's store. That is, when you open a site via HTTPS, the browser receives the server's certificate and intermediate certificates, and then it tries to build a trust chain to the root center. If such a root certificate is present in the trusted list, the signatures are correct, the expiration date has not passed, and the domain matches, then the connection is considered authentic and secure. If at least one condition is violated, for example, the certificate is self-signed, the expiration date has passed, or the certification center is not in the trusted list, then the browser shows a warning about an insecure connection. This is a signal that the server's identity cannot be confirmed. It is also important to understand that these trust lists are not some static mechanism. Certification centers can be excluded from trusted ones if they violate certain security rules or make mistakes when verifying domains. Such cases have actually occurred, and browsers have promptly updated their trust lists to protect users. It turns out that HTTPS trust is built on trust in a limited circle of verified organizations and cryptographic verification of the signature chain. It is this mechanism that allows billions of devices worldwide to securely establish protected connections without knowing each other in advance. Well, with that, we can consider the module on TLS, HTTPS, and connection security complete. Now, I think it is clear to you how data is encrypted, verified, and how the browser decides what to trust. Let's move on. HTTP, as we have already studied, was initially created as a request-response protocol focused on obtaining documents and resources. Each request is initiated by the client, the server responds, and the connection can be closed after data transfer. For most web pages, this was sufficient. The user clicks a link, gets a page, everything is logical. But when applications requiring instant data transfer, chats, or online games appeared, the standard HTTP model showed its limitations. One way to organize almost live data updates via HTTP is polling. The client periodically sends requests to the server, checking if new data has appeared. At first glance, this solution is simple, but in practice, it is extremely inefficient. Constant requests create unnecessary load on the server, increase traffic and latency. Between the moment an event appears on the server and its reception by the client, there is always a pause. The more frequently requests are sent, the higher the load, but even with high frequency, it is impossible to completely eliminate this delay. The attempt to make the connection persistent via HTTP, known as Long Polling, slightly improves the situation. The client keeps the connection open until data appears, after which the connection is closed and the client opens a new one. This approach reduces the number of repeated requests, but it still suffers from the overhead of the HTTP protocol and limitations on the number of simultaneous connections in the browser. Moreover, the server still has to manage a large number of hanging connections, which increases infrastructure complexity and scalability requirements. These limitations showed that a fundamentally different approach is needed for real-time communication. A channel that does not require constant re-establishment of the connection and is capable of providing two-way instant data transfer. It is this need that became the impetus for the creation of protocols like WebSocket and Server-Sent Events, which we will now analyze. Let's start with the WebSocket protocol. Unlike classic HTTP, where the client initiates every request, WebSocket creates a persistent, two-way communication channel between the client and the server. This means that both the client and the server can send data to each other at any time without needing to open new connections for each message. This model is ideal for chats, game servers, or any applications where instant response is important. Technically, WebSocket is built on top of TCP, which we have already considered. TCP, as we know, ensures reliable delivery, byte order, and a streaming nature of data transmission. WebSocket uses these properties, creating a secure and persistent channel over TCP for message transmission. The client and server first perform a so-called HTTP request upgrade to the WebSocket protocol. And after a successful handshake, the connection remains open. And then data can flow freely in both directions without additional overhead for establishing connections. The use of WebSockets is particularly useful where many simultaneous connections need to be maintained. In chats, this allows instant delivery of new messages to each participant, and in game applications, it synchronizes the state of the world and player positions. The protocol also supports binary data, which is important for transmitting images, audio, or any other streaming data. Furthermore, WebSocket integrates with modern web technologies and works easily through standard browsers, proxies, and firewalls, making it a practical tool for developers. It completely solves the problems of polling and long polling, allowing the creation of true real-time applications without unnecessary load on the server and network. In general, WebSocket is the first step towards full two-way communication on the web, providing both speed, reliability, and efficiency that HTTP alone cannot provide. Continuing the topic of real-time communication, SSE, which stands for Server-Sent Events, is also worth considering. This is a technology that solves a slightly different problem, namely, one-way data broadcasting from the server to the client in real-time. SSE works on the principle of a persistent HTTP connection, but unlike WebSocket, the channel is open in only one direction. The server can send new events to the client as they appear, and the client is limited to sending data only through standard HTTP requests. This makes SSE particularly convenient for notification systems, news feeds, statistics streaming, and any applications where the client simply needs to receive a stream of information without needing to send back messages in real-time. Technically, SSE uses the familiar HTTP protocol and supports text formats like EventStream. When a client subscribes to an event stream, the server keeps the connection open and gradually sends new messages. Browsers automatically process these events and trigger corresponding handlers on the client-side. The main difference from WebSocket is, in general, that WebSocket is a two-way channel, allowing both the client and the server to exchange data in both directions with minimal latency. Whereas SSE is a one-way channel, which simplifies implementation and reduces server load, but is not suitable for interactive applications where the client actively transmits data, like some messengers. Finally, what we will consider in this module is WebRTC (Web Real-Time Communication), a technology that allows browsers and applications to connect directly to each other without constant server mediation. WebRTC is based on P2P connections. This means that audio and video data are transmitted directly between session participants. The server is only used for signaling, exchanging information about network addresses, and connection parameters. After the connection is established, the entire data stream goes directly between clients. This reduces latency and decreases server load, making communication as fast and responsive as possible. The technology supports audio and video communication, making it key for video conferencing, online lessons, streaming, and real-time collaboration. Examples of use include Zoom, Google Meet, Discord, and other modern web applications for video communication. Built-in stream encryption using DTLS (Data Transport Layer Security) and SRTP (Secure Real-Time Transport Protocol) ensures data protection from interception, making P2P connections secure. It turns out that WebRTC complements WebSocket and SSE by providing tools for multimedia interaction and direct data exchange between users. With that, the module on HTTP and WebSockets and real-time communications in general can be considered complete. We have analyzed the limitations of HTTP, two-way and one-way channels, as well as P2P technologies, which today form the basis of chats, notifications, and video communication. Now let's move on to a more fundamental part of the infrastructure, namely, we will talk about how servers are structured, through which all this data passes, and what allows them to receive and process requests. What is a server in general? A server is a program that constantly runs on a computer or virtual machine and listens on a specific port, waiting for incoming connections from clients. As I mentioned in previous modules, a port helps determine which application on the device should receive the data. But now we are more interested in the concept of server operation itself. From a technical point of view, a server performs three key actions: bind, listen, and accept. First, the server binds itself to a specific port and network interface. This is Bind. After that, it starts listening for incoming connections, ready to accept a request at any moment. This is Listen. And when a connection from a client arrives, the server performs Accept, that is, it establishes a real channel for data exchange. It is important to understand that a server is not just a passive reception point; it is a process that manages the queue of incoming connections and distributes them among threads or asynchronous handlers. Thanks to this, a server can maintain communication with many clients simultaneously, isolating their requests from each other and ensuring a continuous data exchange cycle. But a natural question arises: what to do if there are too many requests and one server cannot cope? This is where load balancers come to the rescue, systems that distribute traffic among multiple servers so that everything works quickly and without interruption. A load balancer is an intermediate component between clients and servers, whose task is to distribute incoming connections as efficiently as possible. There are several approaches to balancing, and most often they are divided into levels: Level 4 (L4) and Level 7 (L7). Balancing at the fourth level relies on the transport layer of the OSI model, which we also discussed. It looks at IP addresses, ports, and protocols to redirect traffic to a specific server. This method is fast and simple but does not understand the content of requests. It works with data streams like boxes, without looking inside. Balancing at the seventh level works at the application level. It analyzes the request itself, i.e., the URL, HTTP headers, and content type. This allows, for example, directing specific pages or API calls to different servers depending on their purpose. This approach is more flexible and intelligent but requires more resources because each request must be read before distribution. As for load distribution algorithms, one of the simplest and most well-known is Round Robin, or circular system. That is, the load balancer sequentially directs requests in a circle to each server in the pool. This method is simple but does not take into account how much load is already on a particular server. A more advanced approach is Least Connections, where the load balancer directs a new request to the server with the fewest active connections. This distributes the load more evenly, and no server is overloaded. Load balancers, in general, allow systems to scale horizontally. As users grow, more servers can be added, and the load balancer will automatically distribute traffic, ensuring stability and fast response. Thanks to this, web applications remain available even under high load, and users get a consistent, smooth experience without noticeable delays. But now the next question arises: who exactly receives requests first, manages traffic, and decides where to send it next? This is where a proxy, or reverse proxy, comes in. It is an intermediary server that receives requests from clients and forwards them to the internal servers of the infrastructure. For the user, it looks as if they are communicating with a single server, although in reality, a whole cluster of services may be hidden behind it. The main idea is transparent routing. The client sends a request to the domain, a connection is established with the reverse proxy, and it decides where to forward the request next: to an application service, a static file server, or a separate microservice. At the same time, the internal servers remain hidden from the outside world. A reverse proxy often serves as a single entry point to the system. It can terminate encryption, process headers, add service information, compress responses, and cache static resources. This offloads the applications and allows them to focus solely on business logic. Among the most popular solutions in this role are Nginx and HAProxy. The former is widely used as a high-performance web server and reverse proxy, perfectly suited for serving static content and HTTP traffic. The latter was initially created as a load balancer and is particularly effective when working with a large number of connections and high loads. In real infrastructures, they are often used together, complementing each other. Another important function of a reverse proxy is request rate management, or rate limiting. This mechanism allows limiting the number of requests from a single client within a certain time period. It protects the system from overloads, automated attacks, and abuse. For example, if one IP address starts sending thousands of requests per second, the reverse proxy can temporarily block or slow down its traffic, maintaining service stability for other users. In addition, proxies help implement caching, data compression, header filtering, and protection against common web attacks. It becomes a kind of filter between the internet and the internal system, providing control, security, and optimization of data transmission. As a result, a proxy becomes a central element of the infrastructure, simplifying scaling, increasing security, optimizing performance, and in general, allowing flexible management of routing and traffic while remaining completely transparent to the end user. And we move on to the final topic of this module. We will talk about CDNs. When an application starts serving users from different countries, even a well-configured server and load balancers can no longer ensure equally fast loading for everyone. The further the user is, the more time it takes for data to reach them. Well, that's logical, in principle. This is precisely the problem that CDNs (Content Delivery Networks) solve, allowing websites and applications to work quickly regardless of the user's geography. CDN stands for Content Delivery Network. It is a global network of servers located in different regions of the world. When a user opens a website, their request is directed not directly to your main server, but to the nearest network node. If a user is, for example, in Frankfurt, the content will be delivered from the nearest European point of presence. In North America, visitors, say, from New York, Boston, or Toronto, will be served by the nearest nodes on the East Coast. And if a user is in Asia, for example, in Tokyo, Seoul, or Singapore, the request will be directed to the nearest Asian network nodes. By reducing the distance between the user and the server, the data transmission latency is significantly reduced, and page loading is accelerated. One of the key mechanisms of CDN is caching at the edge nodes of the network. When a resource is requested for the first time, the CDN retrieves it from your main server and saves a copy. For subsequent requests, this file will be served directly from the nearest CDN server without contacting the origin server. This approach is particularly effective for static resources: images, styles, fonts, and any media content. So, one of the main reasons for using CDN is to reduce latency. Latency arises due to physical distance, the number of network nodes, nodes on the data transmission path, and request processing time. In general, I think this is all clear. And CDN minimizes these delays precisely due to the geographical proximity of servers. But besides accelerating content delivery, CDN also significantly increases system resilience and security. Many CDN providers filter malicious traffic, provide protection against DDoS attacks, and can temporarily serve cached versions of pages even if the main server is unavailable. CDN becomes especially important for projects with an international audience, high traffic, and a large amount of images and videos. The use of CDN allows reducing the load on the infrastructure, decreasing latency, and ensuring, in general, stable application operation as user numbers grow. In principle, we can conclude this module. In it, we talked about what a server is, discussed load balancing, and CDNs. And now you see how, in principle, modern systems ensure the speed, scalability, and reliability of their web applications. Now let's talk about DPI, a technology that allows looking inside network traffic and analyzing its content. DPI, or Deep Packet Inspection, is a method of network traffic analysis in which the system looks not only at packet headers but also at their payload. In other words, DPI can read what is inside the packet and draw conclusions about the application, content, and intentions of the traffic. That is, it answers questions not only about where and from where, but also about what exactly. Unlike classic routing, which is oriented towards addresses and ports, DPI looks deeper. It matches content with sets of signatures, patterns, recognizes protocols and applications by characteristic patterns, and can make decisions based on content, not just metadata. This makes DPI an effective tool for detecting malicious activity, but it also requires much more computational resources and more complex logic. The practical applications of DPI are diverse: network protection, i.e., detection of attacks, exploits, and leaks; content filtering, blocking unwanted applications or sites; quality of service assurance, i.e., prioritization of critical flows; and analytics, collection of statistics and behavioral patterns. In all these scenarios, DPI turns a stream of bits into events and reaction rules understandable to the operator. Technically, DPI is implemented as stream processing. Packets or connections are copied or captured and analyzed in real-time. The system compares byte sequences with signature databases, analyzes request-response sequences, and tracks session state to correctly classify encrypted and unencrypted traffic, multipacket protocols, and complex application schemes. However, there are important limitations and challenges. TLS traffic encryption significantly reduces the effectiveness of classic DPI because the packet content is hidden. DPI either requires decryption points or is limited to metadata and behavioral analysis. Furthermore, deep inspection introduces latency and requires scalable hardware support, otherwise performance loss becomes critical. At the core of any DPI system is a multi-layered architecture that allows processing thousands and millions of packets simultaneously without loss. At the first level, DPI scans packet headers, extracting key information: IP addresses, ports, used protocols, as well as additional TCP/UDP fields. This information allows determining traffic direction, belonging to specific applications, and transmission routes. The next level is payload analysis. Here, DPI looks inside the packet, matches byte sequences with predefined patterns and signatures, searches for keywords, application or protocol indicators. Thanks to this, the system can distinguish, for example, HTTP traffic, encrypted via TLS, streaming video, or game connections, even if they use non-standard ports or masking. At the rules level, DPI makes decisions based on the analysis. The system can block suspicious packets, prioritize critical traffic, or log events for subsequent auditing and investigations. Rules can be static, predefined signatures, or dynamic, using traffic behavior analysis. Special attention is paid to the interaction of DPI with protocols and tunnels. HTTP and HTTPS, VPNs, and other secure channels create additional complexities, so the system must be able to correctly process encrypted connections, determine traffic type by metadata, packet size, and behavioral patterns, without disrupting application operation and ensuring minimal latency for end users. This is how the principle of DPI operation is formed: multi-layered analysis, combining header and content data, applying rules, and correct integration with modern network protocols. In Russia, as part of the national policy for internet regulation, in addition to standard traffic analysis methods, there is a special level of technical control: TSPU (Technical Means for Threat Counteraction), or sometimes called "black boxes" from Roskomnadzor. This is a software and hardware complex that must be installed in the networks of all telecommunications operators in accordance with the Federal Law on the Sovereign Internet. These are amendments to the Law on Communications and Information that came into force in 2019 and are aimed at ensuring the stability and control of the national segment of the internet. Essentially, TSPU are traffic filtering and analysis nodes located on backbone routes in broadband, mobile, and cross-border communication hubs. Operators are obliged to route traffic through these devices, but they themselves do not have access to them and cannot control their configuration. This is fully controlled by Roskomnadzor through a centralized monitoring and management center. The main difference between TSPU and ordinary DPI is that the DPI within TSPU not only analyzes packets but is integrated with a centralized management system that sets rules for blocking, filtering, and prioritizing traffic at the state level. DPI in a regular network can analyze packet content, recognize protocols and applications, but it does not have the authority to interfere with global policies. TSPU, however, uses DPI to detect VPN protocols, proxies, and other methods of bypassing blocks, recognizing characteristic signatures, behavioral patterns, and metadata, including SNI for TLS, which allows controlling even encrypted traffic. These systems can actively slow down connections, limit speed, and interrupt VPN or proxy operation if the traffic is considered to be violating the rules. By 2023-2026, TSPU equipment is installed at 100% of communication nodes in the country, which means that all user traffic in Russia passes through these complexes and undergoes centralized analysis and classification. In practice, it works like this: traffic can be passed without changes, prioritized, slowed down, or completely blocked. These systems take into account domain names, IPs, protocol types, and even metadata of encrypted connections. As I already said, operators who violate installation requirements or attempt to bypass filters bear administrative or even criminal responsibility. Well, to summarize, TSPU is a state infrastructure for controlling and managing internet traffic, which not only filters and classifies data but also actively interferes with the operation of various bypass technologies, such as VPNs, affecting the speed and stability of user connections. Well, congratulations on reaching the end of this video. Don't forget to subscribe to my Telegram channel, where I share news and spoilers for new videos. If you liked the content, don't forget to give it a like and subscribe to the channel so you don't miss the next lessons. Thanks for watching. See you next time.