Transcription
[music] Yeah. [music] Yeah. [music] Yeah. [music] Yeah. [music] Yeah. [music] Yeah. [music] Hello everyone. This is Yandex Summer School. I am incredibly happy to see all of you who found the strength to join our stream after the first week of learning. It will be interesting. My name is Sergey, and today I would like to talk about how large teams write code together and how version control systems and other developer tools help them with this. I am the technical lead of the Sourcecraft platform – this is a platform that provides these tools, so there will be plenty to talk about and show. I will demonstrate all examples on this platform. And this lecture will be unusual. It's not even because it's happening on a Saturday or because there will be no homework. I will try to avoid two extremes that are very easy to fall into when talking about version control systems. Many lecturers start by going into details because the topic is actually very large and interesting. Git is designed to work quickly and fault-tolerantly. Therefore, the lecturer begins to talk about how hashing works, why our file storage is called context-addressable. Which files are in which folder and why. All of this is undoubtedly interesting and one really wants to talk about it, but after about 15 minutes, the viewer or listener's brain starts to shut down because there is no understanding of why they need all of this. I tried to cut out as much material as possible, and somewhere I will wave my hands and say this is outside the scope of this lecture, because you can read about it here and there. The second extreme is when lecturers start giving practical recipes. What to do if I accidentally commit a secret to the repository? What to do if I lose a commit? What to do if I want to merge? And so on. All of this is very useful, but it is completely not retained in memory. Therefore, what I would like is that after an hour and a half of this lecture, you will have a mental picture in your head of how Git actually works, what entities exist, and how they interact with each other. You can forget the syntax of a specific command, but you will understand what you need to do, because this mental picture will always be with you. The format of the lecture is as follows. In the first half, I will give a basic explanation, I will talk about the fundamentals of Git, I will talk about the problems, what they are, and how we solve them. In the second half, I will give practical advice on how to organize work in a large company, in a large team. Well, even if your team consists of two people, you will still find something useful. Let's go. I would like to start with the problems of growth. It's good to write code alone. You know every line of code. You fully own the codebase. You don't need to agree with anyone about anything. But the startup grows, the number of lines of code in the codebase increases, you get your first clients, and you start thinking about things like how to store code securely. What if my laptop, on which I am currently developing, breaks down, will the development process stop? That is, I need to store the code somewhere so that I can restore the latest current state of the codebase. I want to develop from different devices. For example, I'm going on vacation. I don't want to take a heavy laptop. I'll take a lighter one. The more code there is, the higher the chance of making a mistake. Yesterday's brilliant refactoring seems like a bad idea in the morning. You want to roll back the version. Another situation. The client complains that there is a bug in production, and at the same time I am working on completely different functionality. I want to switch, fix it, and switch back to finish the new functionality. The startup grows, my colleague appears, and with him new problems. First of all, I need to give him the opportunity to get the current version of the codebase. I want to merge changes, since my colleague is not me, he has his own coding style, and I want the codebase to be monolithic. That is, there need to be some rules. And I want to know what he, what he, in fact, committed yesterday, what he did. But real software products are written not even by dozens of developers, but by hundreds. And each has their own background, their own level. They are often not even in the same office geographically, so agreeing becomes even more difficult. Anyone who has tried to agree on a meeting between five adults in one place knows how difficult it is. The more chaos, the more problems arise in development. Each edit individually makes sense. Someone fixed a bug, someone added new functionality. But when we combine them, problems occur. Development stops, tests are constantly breaking. It works on someone's computer, it doesn't work on someone else's. And to curb this chaos, two things are needed. The first thing is established processes. A simple example of a process is: if we are developing in Python, we say we follow PEP. Or, for example, if we have production, we don't release on Friday after 6:30 PM. And tools are needed to support these processes. One of the main tools on which all interaction between different developers is based is version control systems. What is it in a nutshell? Let's look at an example. Suppose I'm starting a new project, and I've created a README. And I can ask this system to take a snapshot of the current state of the codebase. This action, unlike Ctrl+S when we press it in the editor, is meaningful, and we, like in a journal, must say what this snapshot is. For example, that this is the first version of our software. Then I add some functionality and save again. I give, for example, a description that we have created a demo application. I performed a small refactoring, and I saved. From time to time in this lecture, terms will appear. I ask for special attention to these terms, because otherwise we will be speaking different languages. And these snapshots of the state of the codebase, the entire codebase, with a description, with who created it, we will proudly call a commit. So, a commit is a snapshot of the codebase's state. It is not a change, it is a complete snapshot. What can be done with it? Well, it's clear that you can load the previous version, you can calculate the difference between the previous version and the current one. For example, to see what has changed and why everything worked yesterday, and today everything is broken. But, even though our system would be very useful in this form, we add another key element. We move the storage of these snapshots to a remote server. In Russian, "remote" is a loaded term. It has two meanings. The first is that someone deleted it. No, "remote" means "remote." That is, it is located somewhere far away in a data center. And what follows from this? Well, my colleague and I can save snapshots and get snapshots of the codebase's state from this single source of truth. And also, on this single source of truth, we can merge our changes. So, the version control system acts as a source of truth. Multi-versionality. We store a large number of versions. Versions of what? The codebase. Past, present. We can collaborate by merging changes. And this version control system serves as a basis for other tools, which we will briefly discuss today. Today's hero will be Git – it is the most popular version control system today. According to the Stack Overflow survey for 2023, it held a dominant position among developers. It is integrated into all IDEs, is available on any server, and is the de facto standard for development platforms. A key difference, which I want to discuss separately now, is that it is a distributed version control system. What is a distributed version control system? Let's get a little sidetracked. There's Yandex Music. While I was on my way to this lecture, I was listening to music, and suddenly my internet connection dropped. I entered a tunnel. No internet, no music. A typical example of a client-server system where the server is a centralized remote server that we connect to. A typical example of centralized version control systems was SVN. That is, if there is no connection to the server, you cannot view previous commits. Git works a little differently. With this Git icon, I have schematically drawn a repository. This is another term I would like to introduce. It is the storage of all versions of the codebase and metadata. And what Git does first is it allows you to copy part of the repository to your local machine. A typical example. If I mentioned Yandex Music, then there's Yandex Disk, right? You have Yandex Disk, which is on Yandex's remote servers, and on your local laptop. You take your laptop on vacation to the mountains, you can work, save documents. Then you arrive at the hotel, synchronize, everything is great. Git works exactly the same way, somewhere on a remote server. That is, the remote server, it hasn't been deleted, it's somewhere far away. There is a source of truth, and there is a copy of this source of truth, the database, and file storage on your local machine. Plus, there are synchronization mechanisms between the remote server and your machine. If there is no connection, you can develop locally, but theoretically you can connect to your colleague. In this case, we have a truly distributed system. All our agents are equal. But if we talk about how Git is usually used, how it was designed, how it is usually used. We still have a dedicated server that acts as the source of truth. So, we have a distributed system that is used as a centralized one. So, what's good about this? Firstly, every developer has a local copy. I can look at the development history while sitting in a tunnel. This is great. It was developed at a time when there was no broadband internet. Plus, fault tolerance. If everything breaks on my end, I can connect, copy. Great. Let's look. Before this, I waved my hands and said: "Let's take a snapshot. Let's make a commit." How is a commit actually created? Why is creating a commit not just like pressing Ctrl+S in a document? The fact is that when you save a document, the previous version is lost. Git is about history, it's the evolution of your product from one milestone to another. And making a meaningful contribution to this evolution is the developer's task. That is, he must take a meaningful action. What do I want to say with my next commit? What was this commit about? What will I put in it? What description will I enter? Let's see how it works. Here we have a working directory. Some task. I open my laptop in the morning, get the latest version of the codebase, and start working on new functionality. I've finished the new functionality, and I notice there's a typo in the file `merchial.go`. I could make a commit with both the new functionality and the corrected typo, but then my team lead would say I'm not a very good person because a commit should be about one thing. It shouldn't be about everything at once. Firstly, such commits are easier to review. Secondly, it's a milestone in your project's history. Therefore, I say: "Please take only these two files, don't touch anything else." So, we end up with two files being included in the new commit, and the typo not being included. There is a special term for this, called staging. Let's look at this diagram. This is a cheat sheet. You don't need to pay too much attention to it right now. We'll look at it again when we do the demo. And we end up with staging being an intermediate stop between changes and a commit. That is, we can say that these files will be included in the commit, and others will not. If we make a mistake, we can always exclude some files from this intermediate stop. So, staging, an intermediate stop. And then, when we have prepared these files to be uploaded in a commit, we say: "Please make a new commit. The description is as follows." And indeed, in our currently linear history, linear because later we will learn that this history can be non-linear, a new commit appears. Please note that development is never linear; development is evolutionary. We take the previous state and, by changing something, create a new state. Therefore, we have a directed graph. The direction is traditionally drawn from the new commit to the previous one. This can be a bit confusing because time usually goes from past to future, but our new commit points to its parent. Why this is important will become clear later when we look at non-linear graphs. And here's the new commit. And here will be the first time I wave my hand and say that this part goes beyond the scope of this overview lecture. We have a context-addressable storage where files, commits, and trees are placed. All that's important to know from this is that each entity in Git has its unique identifier. A commit that has been added to Git gets a hash. A hash uniquely identifies a commit. Knowing the hash, we can load the commit and verify any operations. The hash is currently 20 characters long. It's usually abbreviated. Well, we will show it like this. Before we get to the demonstration, I want to mention one more important thing. People are very lazy. And telling Git every time, "Please, take these files," well, nobody wants to do that. Meanwhile, the operating system, your development environments, and package managers download some resources that you really wouldn't want to store in the repository. Why? Because these resources are local, they are created, or they are very large. Therefore, Git provides a very convenient feature: a special file with a special format, which I will not explain how to create. You can find special wizards on the internet that will create this file for you. You just say: "I'm working on a Mac, I'm developing in Go or Node.js." And you will get this syntax. In principle, the syntax is very simple. And then these folders and files will be invisible to Git. That is, it will simply ignore them. So, now I've opened the cheat sheet, we've reviewed the theory, and now we'll try to code a bit together. Here I have a laptop simulator, as if there will be a simulator of two laptops, with a split. Where does the code even come from? Well, if someone has already used Git, you know there's GitHub. We have Sourcecraft, so I'll show all the demos on it. Somewhere there's a button that shows how we can get a unique address and protocol, and consequently the address of this repository on the remote server, in order to clone, i.e., make a copy of this repository to our machine. Well, we'll copy it, and then we type the command. Let's make sure it's empty, there's nothing there, and clone this repository to our machine. And in the subdirectory `demo1`, we see, let's see what we see. We see a complete copy of the repository that is on the remote machine, the current version, and all the history that was in it. Let's see if there's history there. I think there's only one commit. No, there are indeed many commits, many different branches. And we've received them all. Now we'll practice a bit of changing files, editing files, staging, unstaging. And this cheat sheet will help us a lot with this. Let's start with a new file. Let's create one, write something in it, and save it. Git is very helpful. There's a special command `status` that we will use very often. This is literally Git, tell me what's going on with my current repository and my current working copy. The working copy is where we are working on the code. The repository, the clone of the repository, is what is linked to this working copy. And we'll ask, and it says we have one new file, it's untracked. Untracked means that for Git, it doesn't exist yet. For Git to become interested in this file and start tracking its history, we need to explicitly ask it to. And it even tells us how to ask. Do `git add`. And on our diagram, it means we can move from this state to this state, immediately making it go into staging via `git add`. Let's ask `git add 1.txt`, and again let's see how we're doing. Yes, indeed, we have a new file. Let's also look at the state from unchanged to changed and from there go to staged. Let's edit the text. For example, let's replace "go" with "Go." And indeed, what do we see? Git tells us that we have a modified file, but we haven't yet placed it in the intermediate state, the intermediate stop between the modified file and the commit. And it even tells us how to do it. Plus, it says something quite useful: `git restore`. If we don't like these changes and want to revert them, we'll get back to that. Let's do `git add README.md`. Everything is ready to make a commit. All that's left is to perform the act itself and write some smart description. For example, "super commit." Why this is not how it should be done and why the description should be meaningful, we will discuss in the second part of the lecture. Let's look at the history. And remember that each commit is assigned a unique identifier. Here it is. This is the unique identifier, which is based on the author, description, and parent commits. And let's look at the graph. Here was the commit from which we started working. And here is the new commit. Before this moment, we discussed a linear history. That is, there was an initial commit, we added new ones, new ones, new ones to it. And this resulted in some picture like this. But real development is never a journey from point A to point B. We always have some experiments that we want to do, and if they don't succeed, we want to discard them. There is simultaneous parallel development on several features, several functionalities. If I have several people, then naturally I will work on a functionality, my colleague will work on a functionality. Therefore, all modern version control systems offer branching. What is branching? Well, our chain of commits turns into a directed graph. Why a directed graph? Well, because, as I said, development is evolutionary. We always take some commit as a basis and create something from it. So, we get a directed acyclic graph. Let's introduce a second term. It might be a bit confusing because, seemingly, in everyday understanding, there is branching, so there should be branches. Like, we branched off, and we got two branches. But the problem is that Git's discourse is a bit unusual. We have a subway line, and by a subway line, we mean the whole thing. In Git, a branch is a pointer, a named pointer. That is, we take a name and point to the last commit in a certain chain. If we have a branching, the last commit points here, and the branch named `feature` points to this commit. If a new commit appears there, the branch lengthens, the pointer moves. So, in essence, a branch can, in some context, be said to be all these commits, but in Git's understanding, it's actually just a pointer. Moreover, a repository always has a default branch. When we create it, we need to create something. It used to be called `master`, now sometimes it's called `main`. In some other systems, it was called `trunk`. In general, there are differences in names, but the meaning is that this is the default branch, we land in it by default when we have a repository, and usually the stable version of the application is placed in it. There are different methodologies, a footnote here, but usually it's done like this. So, let's look at the traditional workflow for working on our new task. We create a new branch and start working on new functionality. That is, initially, the green color in this lecture will always indicate the version that is active for us. We created a new branch, named it `feature`, and made a new commit. As a result, the main branch remained on this commit, while the one we are working on moved. After that, I wrote unit tests. After that, I was asked to fix a bug. I did. We meet another team, called Checkout. There's a more modern name, called Switch, which allows you to move from one branch to another, i.e., activate that branch. What will this lead to? If I commit now, the pointer of the active branch will move. After that, I was asked to return to my task. I switched to the `feature` branch. And then I can take on a new task, branching off from the stable version of our code. At the same time, I wait for my colleague to review my changes. When he reviews them, I make corrections and can return to developing my unique feature. So, branching is very useful. You can develop code together without interfering with each other. You can work on multiple tasks simultaneously. You can experiment and then return to experiments or discard them. You can navigate this graph. This graph is called the commit graph. Well, we already know the first way. We have two commands. We say either `checkout` or `switch` and give the branch name. We did it, we switched. `Switch` is a new group of commands that appeared quite recently. Along with other commands, it is intended to refactor all Git commands. Plus, they provide some convenient shortcuts. For example, to switch to the previous branch, we could type `git switch main`. But whoever knows the command line, there's `cd`, Change Directory, and `cd` with a minus sign. The minus sign means go to the previous directory. So they made a similar mnemonic shortcut `cd` for Git `switch`. And then we will go to the previous branch we were working on. Very convenient. So, we worked, made a bug fix, and returned. So, this is the first way to point to our commits by branch names. As you already remember, Git assigns a unique identifier to each commit. We can simply use it, we can say `git checkout` and say, "Please go there." You can address it by the full 20-character hash, or by a shortened one. If Git understands how, i.e., if this short hash is unique enough that Git can unambiguously determine which commit you mean, it will go there. In addition, there is a very important need to attach some kind of labels, tags, to some commits, tags that will never change. A typical example is what you see on the screen. Here we have some plugin for Go, and it has many different releases, like 0.14.4 and so on. And we would really like the commit from which we built this release to always have this tag. A tag is a tag, so there are concepts like tags. And these tags will always be attached to this commit. That is, always in the sense, of course, you can detach them, but they won't move anywhere. You can address them just like branches, simply by name. There's an interesting question: what happens if we name a branch the same as a tag? How will Git understand what we mean? It won't understand anything. It will say: "I don't know what you mean. Please clarify." So, we now have four ways to address commits in our arsenal. There is also a way to address relative commits – using this tilde symbol. If we are at this commit, we say tilde. Tilde means go to the previous one. Similarly, we can use the tilde for any previous addressing method. And then we will get here twice. These are the main ways to address commits. They can be used in any Git command. Well, checkout, then we will look at diffs, and so on. Let's move on. We have learned to separate branches, but everything that has a beginning must also have an end. Therefore, we must have the ability to merge these branches. That is, I started working on my functionality in a separate branch. I finished, and I want to merge it. So, we should have some small commit that has not one parent, but two parents. And like in genetics, commit father, commit mother, it partially takes changes from the father, and naturally from the mother. We get a commit that combines. Merging is a merge, so while on the `main` branch, we say: "Please merge changes from `feature`." This is where a complex process called three-way merge comes in. What does this mean? Why three? Where is the third place? The fact is that, I don't know, someone, they...
Those who know about functors and so on will understand. And those who don't, I will explain in simple terms. We take a snapshot. This is a snapshot. So, I remind you again, this is very important, which is why I will remind you very often. Commits are not changes. Commits are always a snapshot of the state of the entire codebase. So, we take that commit from which the split occurred, and call it the base. After that, we look at what has changed, meaning what transformations need to be made, applied to the base in order to get this commit. And what transformations need to be applied to the base to get this commit. Transformations are usually called patches. Well, let's call them diffs here for simplicity. So, we have a diff between this and this, this and this. And a simple task needs to be done. Well, a simple task, conditionally, a transformation needs to be made that would combine both these and these. And in the simplest case, if files that are unrelated to each other have changed, meaning, for example, in this branch we changed the file consto, and in this branch we changed testsgo, then this merge is simply a combination of two changes. But what if we suddenly changed the same file? Here, the game begins with the fact that we need to combine changes in one file. And the simplest way is when we made targeted edits at different points in the code. They can be combined. For example, I corrected the header, added a new item, and, for example, deleted something. But often a situation arises where the same fragment is changed. Git will try to do everything it can to merge safely. But what cannot be merged safely, will turn into such a puzzle, right? And from one branch, we have a change in the header. This is the base, this is the branch we are on, this is the branch we want to merge into. And we need to figure out which one to choose, this one or this one. At the same time, the fragment that can be merged uniformly, well, so to speak, painlessly, will be proposed in this way. This is a visual interface. How it will look in the console, we will see a little later. A degenerate case, what will happen if the merge does not occur in this format, from the base to one branch, from the base to another branch, but the base, well, nothing has been committed yet. You can go the previous way, in fact, you can make the same commit here. So, what do we do? We look at what changes need to be made from here to apply them here, and we see that the second changes are empty, right? Meaning, nothing needs to be done there. And it's not hard to guess that the snapshot of the codebase itself will be the same. That is, the commit itself, the file system itself will be the same, but the metadata will differ. That is, since this commit has this one as a parent, this commit will have two parents, and the commit will already have a different identifier, although nothing differs in what we do with checkout here, checkout here. And in general, we can simplify the task by simply moving the branch pointer here. That is, if we were here before, and three or four or five commits have passed, it doesn't matter. And we, roughly speaking, did a fast-forward of these commits, moved the pointer forward, and it is actually called: Fast Forward. The advantages are clear, the history is linear, the disadvantages are also clear. If we merge a branch in which we were developing, there might be, I don't know, 100 commits. Well, I'm exaggerating. But a lot of commits that represent some creative struggles. I wrote the main part, created the interfaces first, then implemented the interfaces, then wrote the tests. And all these commits will end up in the stable version of the application. And the history will be, "added new authorization functionality," "fixed a bug," and then these creative struggles will follow. Therefore, usually this method, it will be useful, we will see this a little later, but usually this method is not used. But when is it useful? When we want to pull changes from a remote repository. And we smoothly transition to synchronization. Before this, we talked about the fact that there is a repository on a remote machine, and there is a clone of the repository on our local machine. And we have processes that we have established. By order, we can pull changes from there to us and push changes back. How does this actually happen? So, we find some repository on some development platform, clone it locally. What will happen? Git is also lazy, actually. And in a large project, for example, in Chrome, there can be tens of thousands of branches. And to save something, it won't pull them all by default. It will only copy the main branch to us. And here another pointer will appear, another one. I highlighted it in blue. This pointer is called, it's also like a branch, but it's called a remote branch. How it differs, we will find out now. But first, I will explain. If, for some reason, we need another branch, we will ask to copy this branch, the same situation will occur. And the local branch is the branch we are developing on. We can activate it, if we make new commits, it will follow, well, the pointer will follow these new commits. And this remote branch is just an informational thing that says that at the moment of the last synchronization, I saw that the head, well, so to speak, the last commit pointed here and was called this branch. Therefore, Origin is usually called the remote repository. In principle, this is also an abstract name. It's just that if we have one remote repository from which we want to synchronize, it will most likely be called Origin. And therefore, the remote branch will consist of two parts. The first is the name of the remote repository in relation to us, again arbitrary, usually it's Origin, and the name of the branch. And if, as I already said, we made changes, committed them, our local branch moved, but the branch we call remote remained in its previous place, because it shows, it is actually used in synchronization processes, and it points to the commit that it pointed to at the moment of synchronization. So, how do we actually get the update? Well, there is such a command, Git Fetch. Let's carefully examine this diagram. So, at the moment of the last synchronization, Origin/main pointed to this commit. That is, in Origin, this branch was located here. During the time between synchronizations, it managed to move two commits forward. Then communication between the two, between the client and the server, begins. Negotiation occurs. The client says, "I know that they were located over there." The server says, "I know about such branches." The server thinks, "So, if they were located here, then I need to give him these branches, these commits, as well as all the content that was associated with them. That is, if new files appeared there, I need to transfer them to him as well. If new files appeared here." And, in general, it forms this response. That is, the packet that needs to be applied to our local repository to update it. So. But Fetch only downloads, synchronizing the state with the remote repository. Our local branches, on which we were developing, remain in the same place as they were. Therefore, we use the second command, which is called Git Pull. This is plus Merge. Why Merge? Where did it come from here? Well, if you remember, Merge is to combine, to combine the content of two branches, right? The first branch is us, on which we were on main, and the changes that arrived, which they don't have, are Origin/main. Well, it turns out that we need to download changes from the remote repository and also merge them. And here, as you can see, this pleasant thing is that you write Git Status and they tell you that your branch is behind by, say, 320 commits, but at the same time, fast-forward is possible. This is exactly about it, right? We downloaded 320 commits, and since we didn't do anything in this branch, and our history is, so to speak, linear, we can just take it and move it forward. This is the case when fast-forward is very useful, right? We took it and applied it. But there is another case, right, when we managed to commit something. What will happen in such a case? From main, as usual, we will calculate what we are missing. We download them, and a very ugly merge commit will appear. Why is it ugly? Firstly, because it, well, so to speak, if our main branch contained some meaningful commits, then this is just a combination of some temporary work. Secondly, it is not in the remote, and we need to push it. Therefore, usually you can turn off the flag and say that in such a case we will just complain and do nothing. In good development methodologies, such cases are generally impossible if you adhere to them. Well, if we know how to synchronize in one direction, let's see how we will synchronize in the reverse direction. Let's assume that we have a local branch, a remote branch. I will no longer draw Origin to avoid cluttering the diagram. Again, negotiation occurs. The server asks, says, "I have these branches, they are at these commits. The client says, 'I have these.'" And then the client-server discuss what needs to be done again. But now the packet goes from the client to the server. And the links are updated. A logical question arises: what if someone managed to commit one state, and we are proceeding with a different state? Well, a conflict will occur. Git will simply refuse to pull such changes that contradict each other, because they cannot be merged painlessly. It will say that your branches have diverged. I have one commit, you have one commit. And maybe, by the way, I have 20 commits, you have 300 commits, and I don't know what to do with it. Like, I won't do anything. Well, you can always tell Git, "I know what I'm doing. Please push these changes to yourself." And indeed, it will push them. What will it do? It will take your new commits, add them to its history. The previous commit that was there, its pointer will move to your commit. That is, we have, in essence, rewritten history by force. And this commit disappears. That is, it will remain in the repository for some time, and you can even revert all these changes. But it is already outside the commit graph, because the commit graph is built from pointers. That is, it exists, but it is actually very difficult to get. So, let's take a short break here and try to look at how two employees work with one version control system. I will prepare now and launch. [Music] Yes. And here we are again, and we will pretend that two employees are working on the same laptop. So, this is the laptop of an employee with the login Tk, and above with the login Leleo. They have cloned the same repository. And they are all up to date. What will happen if I suddenly start working on the repository? Well, first, I want to create a new branch. I say Git Checkout -b feature. Well, for example, I will, yes, our test example is a calculator written in Go. It already has the ability to add. Let's add the ability to subtract. And I make some changes. Well, for example, Go. Here we have some code. I will add to it. Well, also format print f percent d minus percent d equals percent d/n x y. And here we have some package where we will later implement subtraction. Well, let's look at Git Status. Indeed, it says that main.go has changed. No, we make sure that we are on a separate branch. We do add. Okay. And now we need to do a push, that is, to push the changes from our local repository to the remote one. Okay. And what does it write to us? Well, by the way, here I just set it up so as not to get distracted by this. By default, it will say that if such a branch does not exist, you need to create it and so on. You can also set up such a configuration, and it will be much more convenient. Uh, let's do it here. Well, first, let's do Git Fetch. We see that according to the repository data, a new branch has appeared in it. And we have a new local branch Origin/feature/sub. Now we can switch to it. Let's see if the changes have been loaded. It would be surprising if they weren't loaded. Indeed, the changes have appeared. Now let's implement our function. Yes, and, well, I'm tired of writing Git dot, so people are also lazy and usually write Git Commit -a -m or just -am. What does this mean? It means that all the files that Git tracks, please record their changes, put them in staging automatically. But please note, only those that are tracked. Therefore, if you add a new file and do Git Commit -am, it will not be included there. So be careful. Well, and we pushed it. And now let's try to see if the difference between Fetch and Git Pull works. Yes, let's look at the current state of the repository. Yes, here is our new branch feature/sub. And remember the prompt that is highlighted in red here, Origin/feature/sub. Let's do Git Fetch. We see that Origin/feature/sub, that is, the remote, has moved, that is, synchronization has occurred. But at the same time, if we open our file calculator.go, we will not see the function here, because our working copy, our working copy is still on a commit that is one commit behind the future one. And we see this magical phrase that I always love to see. Our branch is behind by one commit and a fast-forward is possible. Let's agree, let's do Git Pull. And indeed, the fast-forward led to the calculator being loaded. Let's look at our graph. And here we have a fully synchronized state, that is, our working copy and the main branch point to the same commit. Let's, for the sake of interest, create some conflict, right? To do this, let's make incompatible edits. We are on the same branch. And let's edit, for example, README.md. Let's write "Hello Git" here. Git Commit -m "Hello". And let's assume that we also edited something here. Now, for the focus to work, I need to see which line I edited. I edited the third line. Let's do the same line, edit it. Let's also edit the third line. Let's write "Buy Git". Git Commit. And let's send these changes there, to the remote server. Yes, everything has been updated. We exit. And let's try to send them ourselves. What do we see? We say, this is exactly the situation when the remote server says, "I don't know what to do with this." It points to a completely different commit. It simply says in plain language, literally, that the update cannot be applied because it contains work that I don't have locally, and suggests doing Git Pull. Let's do Git Pull. And here it says, well, by default, it doesn't know how to do Git Pull in such a case. And it says, "If you want Git Pull to work in case of a difference, only fast-forward," that is, the safe way I talked about, you can specify this in the config. But let's look at the status. And here we have it that our branch is on the commit where "Hello" is written, the remote branch is written on "Buy", and what happened? It managed to do Fetch, but it couldn't do Pull, because these branches are different. Let's ask it to do a Merge. And Git is very conservative about merging conflicts automatically. Therefore, in all places where it has doubts, it suggests that a person work on them. And now you will see how conflicts look without these wrappers of your IDEs. Let's look. Well, actually, it's not as scary as it might seem. It puts these guards and says, we had "hello". And the branch we merged with. How to read this? Head is always a conditional pointer that points to the active branch. That is, the commit we are currently on. We were on the "hello" branch, so we see "hello" here. Here it suggests that we decide what we need to choose. Well, we can, for example, do this and say that actually, it should all be combined into one line. Then let's look at Git Status. Git Status is the most useful command you can see if you are working from the console. And it really says, "Here we have one commit there, one commit here, they are different. And we have something that we haven't merged yet." When we have manually merged, we can say, "Please add." Let's see what happens. What happened? And we say Git Commit -m "I am". Well, the history we got is quite messy. And how to avoid this, we will learn in the next part of our lecture. Why is the history messy? Because we have a commit in the history that makes no sense. It's just that we pulled changes. Let's look at how to do it humanely. Well, as I already said, a typical situation. I'm working on a branch, I'm stabilizing some functions, for example, that I want to call. That is, instead, I made a placeholder, and my friend, ah, sorry, about this later, because at the beginning, editing history is from a very simple case. I'm in a hurry, like right now, and I forgot to commit some file. Since we've already said that Git is about history, I would really like to not have a meaningless commit that says, "I added this file." Therefore, the Git Commit --amend command comes to our aid. --amend means to correct, that is, to make a correction. We make the correction, the commit is successfully amended, rewritten. And here is the example I wanted to tell you about at first. I implemented some code. It calls some functions that my colleague hasn't written yet. To make the code compile, I made them placeholders. And here the colleague happily says, "Everything is merged. Please pull my changes." And I really want to get these changes into my branch. I could have done Git merge, as I showed in the demo. Then I get an ugly commit that doesn't make sense other than that I pulled changes. Such commits are very difficult to read. And I will also show later, in fact, why it is important to have a clean history. For now, just take my word for it. And there is a magical command that everyone fears, called Rebase. What will it do? It will do this. Whoosh. And it seems, how is this even possible? That is, we don't know how to move things. And what does it even mean? In fact, under the hood, it does the following. It takes this commit, looks at this commit, and looks at the transformation that needs to be applied from this commit to get this commit, and tries to apply this transformation to this commit. What is a transformation? Well, just open this file, add a line to it. If this line can be added to this file, it will do it. If there is some conflict and someone has already added something to this line, it will again ask you to resolve it manually. Therefore, there is no magic, you will still have to work. Similarly, it does the rebuilding of the second one and moves the pointer. As you already remember, if there is no pointer to a commit, to a chain of commits, even if it exists in the repository for some time, it is very difficult to get it. And as a result, we get that we have indeed rebased our branch. Well, rebase. Rebase is a very good command. We took the changes, did not create meaningless commits, but the problem is that we got completely new commits. That is, the hashes, what was here and here, are completely different, because, let me remind you, the hash is calculated from the commit, from the description, from the content, and from everything combined. Someone learned to draw in Zoom. Thank you, please don't draw, I kindly ask you not to. Okay. So, this can be potentially dangerous changes, because if someone, my colleague, copied the repository, cloned it, happened to check out this branch, and his local pointer points somewhere here, and suddenly I rewrite history and push this history to the remote server, he will get a lot of headaches when he tries to merge it. Therefore, the rule of thumb, I don't know how to say it in Russian, is like, never rewrite history in shared branches. And we have probably touched about 30% of the entire volume of our material. We have covered the main points. Of course, we could have considered Git Reset, Cherry-pick, and all that, but in my experience, the brain starts to shut down around here. Therefore, I will now start explaining how I build workflows in large teams. I am a technical lead. My role is to unite a large number of people and ensure their work in such a way that they do not interfere with each other, do not elbow each other, but rather synergy occurs. That is, many, many different guys, many talents. If you let them loose, there will be chaos. But if you give them tools, rules of the game, by which they can work together, then synergy will occur. They will be much more effective together. And I will talk about things that have proven themselves well within Yandex. Of course, there are other methodologies. Maybe you even prefer them, and this is not for a holy war. I'm just saying how, over, say, 5 years of my work at Yandex, why we chose this methodology, why it's good, what happens if you follow it, what happens if you don't. And this methodology is called Trunk-Based Development. What does it consist of? It consists of the fact that you have a default branch, and any commit from this default branch can be rolled out, a release can be built from it. I sometimes switch to Russian, sorry. A release can be built and deployed to the production environment, that is, the environment where people can access from the outside. We showed this site Sourcecraft Deft, this is the production environment. What's good about this? In other methodologies, you have separate branches where you build releases, where you test these releases. But the question arises, what is currently in main, can I roll it out now, has it been tested or not? There is no such dilemma here. High culture is needed, so that you merge what has been tested, but at the same time, you can roll out a release at any moment. You can roll out a release manually. You can have automation that rolls out new releases every evening or twice a day. You can set up automation so that every time you merge your working branch with the main one, some integration tests, end-to-end tests are launched, and you would guarantee, without human factors, that everything works. That is, you can automate this. And the second point, Trunk-Based Development is also called Green Trunk, meaning that each release is marked with a separate tag. What's good about this, I'll explain. Suppose my colleagues have prepared a lot of new functionality. It is indeed in the codebase, it is in the main branch. At the same time, we have version 0.3.0 on production. And then we are told that some key scenario is not working as desired. For example, yesterday we had a release, we finally exited the private version. And we have user suggestions. And we had a bug where user suggestions, if no one was found, returned someone completely different. That is, it should return empty, but it returns the first users it finds. And I want to roll out what is called a FIX. That is, I don't want to roll out all the new functionality that has appeared to people yet. Why? Because they are used to the buttons being on the left, and suddenly they will be on the right, you need to announce some release notes and so on. Therefore, I know that version 0.3.0 is on production. I do a checkout on this version, after which I make a commit with a hotfix and roll out a release with the new tag 0.3.1. It's very convenient, believe me. Second point. Branches should live short lives. The shorter a branch lives, the fewer conflicts you will have to resolve. That is, if you make small, as they say, changes, and you make changes, send them, the more people you have, the higher the probability that someone will commit something in a unit of time. And the more changes, the longer your branch is marinated, the more you will have to transfer, rebase, right? And the longer it lives.
The more you put into a branch, the more functionality you will put into that branch. Here is an interesting graph. It may be outdated, but it shows that the human brain is also a lazy thing and can process a certain amount of information very well. If there is more information, it will not be able to find errors as well. So, there is such a graph. The more lines of code you have in changes for pull requests, what are pull requests, we will discuss later, but for now, consider that this is how many changes you have brought into your branches, the harder it is to check. And after a certain limit, it starts to exceed 3,000 lines. Well, everything is probably fine with this PR. Simply, a person physically cannot look at it adequately, because when we write code, we write it evolutionarily, and when we check, we immediately receive such a monolith of this new code. The most we can do is conduct what people call "bikeshedding." That is, we can do "bikeshedding" - this is a term that came about when an atomic power station was being built in the USA. Instead of discussing, I don't know, what measures should be there, zinc or not zinc, a committee spent a very long time discussing what color the bicycle shed should be. This is where the term "bikeshedding" came from. And with large commits, people start arguing about indentation, quotes, anything, instead of checking the essence of the PR. Therefore, the recommendation is to make very small branches, very small commits. If you have a large task, break it down into subtasks accordingly. Well, you branched off and merged it. The second principle is to maintain a clean history. Only merge commits in the main branch. You never pull changes from the main branch into feature branches. Even if it's inconvenient at first, believe me, you will win in the long run. The next principle is a meaningful commit message. I'll show you an example. Imagine your history looks like this. Ticket, what changed? Ticket, what changed? Ticket, what changed? And if you suddenly want to understand where an error occurred, you can do this: if everything worked here, and it doesn't work here, we take this commit in the middle. Okay, ticket 101, number 11. We did some sanitization here. It works. So, the problem is here. This is called bisect, by the way. Bisect means dividing in half. So you can divide the history in half and see where the problems start. Imagine you have a history like this. Work, more work, added a missing file, committed at the end of the day. Well, you have to agree. Therefore, a meaningful commit message should convey. Saving 20 seconds today, you are planting a mine for yourself and your colleagues in the future. There are different methodologies. Choose any. The main thing is that all your developers follow it. For example, you can use conventional commits. You can even build in a check that if the commit description does not comply with conventional commits, such a commit will not be accepted into the repository. For example, here is an example: this is a feature, it relates to the API component, and it is linked to such a ticket. A related principle is to name branches uniformly. For example, agree that all new functionality is done in branches with the prefix "feature." All bugs with the prefix "bugfix," all corrections not related to tickets are in "chore." Chore is when you need to, I don't know, clean up the apartment. Release branches, as we've seen, are branched off from a tag and will have the prefix "release." Moreover, in various development platforms, you can directly enforce policies. That is, you can say, "Please do not allow force pushes to the main branch." Forbid merging merge commits into feature branches. You can say that only branches with names matching a certain mask are available. Everything else will simply be rejected by your remote repository.
Now, finally, we come to pull requests and tickets. Up to this point, we've talked about having many employees who can be remote, distributed, not in the same office, and can live in different time zones. And yet, we want to ensure code quality. That is, any change that enters our main codebase must be reviewed by someone. When we take a branch and want to merge it into the main one, it's called a pull request somewhere, a request to merge, and a merge request elsewhere; it's called differently in different systems. And a pull request in development platforms looks like this beautiful interface. First of all, you can write what this pull request is about. For example, here I added basic GitFS support and explained where to look. I can invite people to review, and I can connect some automation. Let's look at the checks in more detail. First of all, everything that can be automated should be automated. If you say, "Before committing to the repository, you must run, I don't know, tests," there will always be someone who, in a hurry, forgets to run tests, or it will work on their machine but not on yours. Therefore, using CI/CD (Continuous Integration/Continuous Delivery), which is also available in most modern development platforms, you can set up automated checks that will run style checks (linters) and unit/integration tests on any code that you want to merge. They can also run security checks. This is a new, trendy topic that says, and the simplest thing, "Did I commit an API key?" This is the simplest way. There are also "Am I using a vulnerable library?" and other more advanced security tools. All of this can be configured so that we automatically guarantee code quality. But, of course, automation is automation, but the human factor has not been canceled. And there might be something that automation misses. Therefore, we need to involve people in code reviews. If your team is large, you can even say, "Here is our repository with code." Let the front-end developers review the "frontend" folder, and the back-end developers review the "backend" folder. And a senior's approval is definitely needed. That is, someone like a senior developer looks at it and says, "Okay." So that an intern can easily make a change, and a senior reviews it, gives some suggestions. "N" is an abbreviation, called "nitpicking." That is, you pick at the мелочи, or you say that this is usually a matter of taste, but there are things without which I don't want this pull request to be merged. And pull requests, this place is where your team interacts. This is a place for collaboration. You don't need to call Zoom meetings. You shouldn't bicker in chat. Why? Because you have context for the comments. You have a list of all comments. No one will miss anything. Imagine you say in Telegram, "Fix this, this, this, and this." And your employee fixes three items but forgets about two. And your automation will simply not allow such a change to be merged.
Finally, we arrive at the point where we need a single point for managing the project. What does it mean to manage the project? During development, we encounter bugs, i.e., defects, behaviors that are not as we expect. We have functionality that we are developing. We have projects that include new, large-scale functionality. And if you have all this information in one place, your life becomes much easier. Let's consider bugs as an example. All the bugs that we... our platform is based on the principle of openness, and our issue tracker is public. We may have shot ourselves in the foot, but it's very interesting because we want to interact with our audience, and we are interested in feedback. You can create a bug report for us, and we will prioritize it, communicate, and request details. If a bug requires closer interaction with your repositories, which may be private, we make that ticket visible only to you and us. And if you have a single place where you store this, you can prioritize, and then you can use all your development principles like Agile. You prioritize bugs by criticality, you prioritize new features by criticality, and then when a new sprint begins, you simply take the top 10, say, tickets from there. Each of these tasks is a ticket in our slang. And you take as many tickets as your sprint can accommodate in story points, and you start working. You can set a deadline for a ticket, you can assign an executor, and you get a complete link. That is, when you look at the history, you have a link to the ticket, and you can see the details. Additionally, sometimes it's necessary to see why a particular line of code appeared. You simply select the line, and you use a feature called Git blame. Git blame will show you the commit in which that line was changed, and you use the history of your repository as a source of truth and as a protocol. And you understand that this line appeared when we did, for example, "sanitize display name." And then you look at why we did it that way. Your commit should answer the question "why." Whether it answers this question through a full description or in conjunction with a ticket is up to you to decide. But in general, it should answer the question "why." Especially since the codebase written today will live for 5 years, hopefully even 10 years. And perhaps even the employees who wrote this codebase are already working on other projects. This also happens. And there's no one to ask. And this is one of the parts that, in addition to unit tests, makes your codebase healthy and maintainable. Unfortunately, I have to end on this positive note, because there are actually many topics. I have touched upon, well, perhaps the very, very top edge of all the topics related to building development in large teams. How version control systems can be useful. How this is a basis for knowing what to look for. I really hope that some phrases have stuck in your mind, for example, that a commit is a snapshot of your codebase, not a delta. You visualize these commit graphs in your mind. You imagine what is actually happening, that you have a local copy of a large repository. And with that, I would like to conclude, wishing you success in completing the challenging path of summer schools. Furthermore, I would like to express immense gratitude to all the people who made this lecture possible. To everyone who came here on a Saturday. Special thanks to our designers who, overnight, transformed my presentation mock-up into this beautiful "Enter" button with legs, which is very cool. And now, it's probably time for questions. Questions should be sent to my phone, and I will answer them one by one. I probably have about 20 minutes to answer questions from the chat. Please send the most interesting ones. I will voice some of them, and then throughout the day I will answer, because the timing is unfortunately strict. I have to leave soon. And now it's time for questions. So, question number one. Good afternoon. I wanted to clarify, does Git support projects in any language? Correct. I used JavaScript before. Approximately Go. I became interested in using it, and Git is agnostic to what you store in it. Any language, it's primarily text-based, so all merging tools, diff tools, will be the same as long as your content is text. What can be specific to JavaScript? Remember the slide with .gitignore? You just need to configure your project in such a way. Let me scroll back. Somewhere here it was. You need to configure Gitignore so that the artifacts generated by JavaScript do not end up in the repository. This is usually node_modules, here some JSPM, packages, npm, and so on. Otherwise, there is no difference. Now, a nuance about why I talk about text format and digital. For example, if I could just use Git and its merge functionality for a binary file, for example, I was preparing a presentation, I prepared it yesterday on one laptop, then I prepared it on my home computer, and then I was sent a beautiful version of the presentation. And if I could combine them using Git, it would be ideal. But the algorithms we considered, remember the block of text from one branch, block of text from another branch, they are not applicable to digital data. We simply won't understand, for example, a piece of binary code, which one to take, this one or that one. Plus, there might be. Therefore, the only thing it's not very good for is if you have some binaries. Plus, there are limitations on the size of a binary file that you can put in a repository. But this limitation can be overcome with Git LFS (Large File Storage), which connects and simply says, "We don't even look at how these files change. You can store them there." I was making a 3D game, and I stored large textures and models there. So it's agnostic. You can even merge journal articles in it. And this is actually very convenient. So, regarding syntax, Git is exactly the same. It's hard to believe there are no differences in different languages. No, absolutely none. It treats content as text. Do you use any utilities for working with Git, Lazy Git, IDE interface, or do you do everything through the console? Look, I have a principle. I use the tools that are convenient for me. For example, my muscle memory has developed for adding everything, committing, rebasing. But when I resolve merge conflicts, let me open this picture again, I use some kind of interface because usually this is a very simple, trivial example, but imagine that we are applying changes, and some terrible refactoring has occurred, some function names have changed from one to completely different, and everything is red. Therefore, I use the functions that are convenient for me. Why am I using the console now? Because, for example, when you go to some server and want to do something, the console will always be there, somehow colored. Plus, by understanding how it works, understanding how it works under the hood, you will be able to understand where that functionality is located in the GUI. It's in VS Code, it's somewhere down there in PyCharm, or somewhere else. Where are the remote repositories that we push to GitHub physically located? Oh, that's a good question. I like it very much because I can talk about it for hours. Let me briefly explain. There are different approaches to how to store code in remote repositories. The simplest way. We use that Git. Git, in fact, has about 180 console commands. And we simply deploy it on a remote server and trust Git to interact with us via protocols like HTTPS, SSH. And a folder called .git appears directly on the physical server, and so on. This is path number one. It's good, it has its problems. What if we have... and the problem immediately arises that our system must be resilient, it must be fault-tolerant, right? We have one server. If it fails, then that's it, you log in and you can't connect. Therefore, in large production systems, there are multiple servers, often located in different data centers. And if we simply create a .git folder, we need to somehow synchronize this folder between different servers. This is an engineering task, it's very interesting. We took a different path. And in our Sourcecraft service, we don't use Git. We wrote our own file storage based on the open-source library called GoGit, which is stored on S3, and our own metadata storage, which uses a database. So, under the hood, we don't have the actual Git, but we have all the functionality, parity, as it's called. And we get distributed fault tolerance, which is determined by the means, the storage systems we use, i.e., databases that are replicated across multiple data centers. S3 is also Object Storage, which is also across multiple data centers. We decided to go a different way. This is also a very interesting task, because when you implement Git from scratch, you interact with all these internals. If you're interested, I talked about this at the last Infraconf conference. Can you recommend good resources and not simulators for learning Git? Oh, I recommend. First, there is an excellent resource called, if I'm not mistaken, "First Aid Kit for Git." It's called that, a play on words, Git Kit. It has typical problems, what happens if I push a secret. There are recipes there. Then, if you want to study the theory, what I was waving my hands about, about content-addressable storage, I won't talk about it. There is a main website called GitBook, SCM Git, called The Gitbook. It's available in different languages. I highly recommend reading it. It's an engaging read. Plus, you'll learn how this trick actually works. Why is creating a branch so cheap? Why, despite the fact that each snapshot of the repository saves all files, our space doesn't grow exponentially? All of this is explained there, it's all very interesting reading for the weekend. So. And it seems I'm being signaled that my time is running out. Thank you all very much to everyone who stayed from beginning to end and is still awake. And we, unfortunately, say goodbye. Thank you very much. Goodbye. See you at Yandex Summer Schools. [Music] Yeah. [Music] Yeah. [Music] Ta hey [Music]