Transcription
You were a programmer from an early age. You co-started coding at 10. First things you built were a video game at 11 and then eventually, 10 years later at 21, you programmed the initial versions of VK single-handedly. Can you talk to me about your programming journey that led to the creation of VK? What was the VK stack? Is it PHP mostly? How did you figure out how to program websites? All of that.
I wasn't interested in programming websites at first. I didn't even have access to the internet when I was 10 years old. But I liked video games. I didn't have enough of them. And the scarcity forced me to start building them, more computer games just to play myself. Yeah. It's actually an interesting thing that we sometimes don't realize it, but scarcity leads to creativity. And one of the reasons you have so many people who love to code coming from the Soviet Union or other places which didn't have much access to modern technology and more importantly, modern entertainment is that perhaps we were not so much distracted by all this abundance of different entertainment options. Which is not to say it's bad to have those options. It's just a fact that we sometimes don't appreciate.
So I started to build computer games. My brother would sometimes guide me. For example, I would create a turn-based strategy. Of course, two-dimensional. Back then, three-dimensional was too much for me. But it wasn't as sleek in terms of the scrolling FPS frames per second, um, parameter, and I asked my brother how to optimize it. He would guide me, and this kind of learning and training really shaped my coding skills when I was younger.
Then I started to create video games for my classmates when we played, for example, tic-tac-toe on an infinite field in my class during the breaks. You know, not tic-tac-toe that three in a row. This is what five in a row and an infinite field. This is a much more interesting game and it gets quite complicated if you keep playing it. My classmates used to love it, and some of my classmates were really smart, you know, champions of math olympiads, sons and daughters of professors at the university. And I decided, no, I want to win every single time. I don't want to lose even a single time. So, how do I win? I need to practice more. But how do I practice more? I need an opponent stronger than myself. So, I coded this game so that I would play against the computer, and the computer would calculate, I think, four moves in advance to choose the optimal strategy. That wasn't enough. Four moves in advance, I would still win over it. If I tried to calculate five or six, it was too slow. So, I asked my brother to help me out here. So, he made this algorithm. Eventually, I trained myself to win every single time, even with the computer back then. We didn't have, uh, modern CPUs. And, um, I could still retain some self-confidence. We go back to school during breaks, play with my classmates, and soon people started to lose interest. None of my classmates wanted to play this game anymore. I killed the game. There isn't. Yeah.
So after that, when I got into the St. Petersburg State University, it was quite boring just to study because it was too easy. So I thought, what can I do there? I created a website for the students of my faculty. First, I organized the creation of digital answers to all exams and a digitalized version of all lectures, which was something very unique back then. Remember, it was 25 years ago. I would put together a website where I would, uh, publish all these materials, and pretty soon it became super popular. I opened a discussion forum there. In a few years, I expanded to the university, uh, with all of its other departments, and then to other universities. We ended up having tens of thousands of users just as a student's portal. We had all kinds of social features there: friends lists, photo albums, profiles, blogs, all of it was quite successful.
And after I graduated the university, one of my ex-classmates from the school reached out to me after reading about my successes in a newspaper, the main business newspaper of St. Petersburg. And he asked me, "Are you trying to build a Russian Facebook?" I said, "I'm not sure what Facebook is." So we met. He, since he graduated an American university two years before that, he showed me Facebook. I thought, well, I can already have all of this technology, but it's valuable to know which elements I should get rid of in order to scale this thing and have millions of users. This is also something people don't appreciate, that sometimes in order to move forward and have more success, you have to get rid of things, including technology. Getting rid of features is super important. Simplify both for scaling and for making it amenable to, uh, just growing the user base where people get it immediately. Yes. Otherwise, it's just too complicated for the new user. The existing users will be happy. They'll be praising you. They will be asking you to add more stuff to make it even more complicated. So, it's easy to lose track and get disoriented if you're only relying on the feedback of existing users.
So, as a result, I started the website called V Contact, or VK. It means "in touch" in Russian, initially to solve my own personal problem. I graduated the university that same year, and I wanted to be in touch or remain in touch with my ex-classmates from from the university and and the other fellow students. And of course, as a 20-year-old, I wanted to meet other people, including good-looking girls. So I started to build it from scratch. For that one, I thought I'm not going to use any third-party libraries, modules, because I want to make it as efficient as possible. I was obsessing over every line of code. But then, how do you start something that large? Like, I didn't have any prior experience of quitting a project of that scale which would involve everything. Before, I would reuse some existing solutions. Here, I wanted to build from scratch.
Sorry, I called my brother. He was a postdoc student in Germany at that time in the Max Planck University. And I asked him, "What should I start from?" And he told me, "Just build a module to authorize users. Just don't, not just to log in, you know. Yeah. Not even to sign up, just to log in, because you can prepopulate the database with with credentials and emails and passwords. Doesn't really matter. But once you see that you can type in your password and email and you're in, and it tells you hello using your name, then you will have a clear understanding where to go from there." Yeah. I mean, that's true. That's one of the best advice I've ever got in my life. It's, it worked perfectly, by the way. I started to build it, and before I knew it, I would have there on that website, photo albums, private messages, this guest book we used to call the wall back on. And I guess in the early days of Facebook, we'd end up building something even more sophisticated than Facebook at the time with more features.
I had a girlfriend at the time. I asked her, "We need to somehow come up with a database of all Russian schools and universities and the department and subdivisions." She did a great job trying to source all this information online or sometimes writing emails to universities saying which departments you have exactly. At this point, we, we need to know, or reaching out to the department of education both in Russia and then in Ukraine, and then eventually in Belarus and in Kazakhstan and other countries where VK ended up to be the largest and most popular social network. So we did a few things that were quite unique at the time. And for the first almost a year, I was the single employee of the company. I was the backend engineer, the front-end engineer, the designer. I was the customer support officer. I was the marketing guy as well, coming up with all the awardings and the announcements, coming up with the competitions to promote VK, which worked quite well. That was an incredible experience that gave me knowledge of every aspect of a social networking platform. Also, understanding of how much a single person can do. Exactly. It's one of the reasons why I'd like to think I'm an efficient project manager and and product manager inside Telegram, because I will not take anything but ambitious deadlines from my team members. If somebody gives me, "Oh, I needed three weeks to do that," I always reply, "Well, I built the first version of VK in just two weeks. Why would you need three weeks? It seems like something you could make real in three days. Three weeks. What are you going to do the rest of the three weeks apart from this three days?" And, you know, the team knows me, and, uh, that's why we are able today at Telegram to move at a very good pace of innovation. Every month, we're pushing several meaningful features. I think outcompeting everybody else in this industry in terms of what you can do within the short time frame. So yes, that experience was invaluable.
As for the stack, I started from PHP and MySQL, Debian Linux, but very soon I realized I need to optimize this. I started using Memcached. Apache servers were not enough anymore. We had to set up Nginx, and my brother was still living in Germany, so he couldn't help me much for the first year of building VK. Sometimes I would manage to get through to him through a call. I would use an old-school phone to call him with wires. I said, "What do I do? How do I install this thing called Nginx? I'm not a Linux guy." If he felt particularly kind that day and not too busy, he would show me the way to do it or set it up himself. But for the most part, I had to rely on just myself. Having him there, though, helped when we started to grow fast and started to scale it, because at first, you realize not one server is not enough. I need to buy another one, then another one, and another one. The database should be in a different server. Then you have to split the database into tables. Then you have to come up with a way to shard the tables using some criteria that would make sense and wouldn't break your user experience. When we got to over a million users and beyond a dozen of servers, like surviving without the input from my brother in terms of taking care of the scaling aspect of it became impossible. I remember asking him to come back because you need to help me with this thing. It's starting to be really big. What was worse is that since we became popular, somebody started to do attacks on us, as it always happens, right? And then we had people that wanted to buy a share of VK. And interestingly, every time we had a negotiation day, the DoS attacks intensified. So we had to come up with a way to fight it. I remember having many sleepless nights trying to figure it out.
So that was your introduction to all kinds of bad actors. DoS, business, then later you'd find out there's such a thing called politics, and then later geopolitics. But these are the initial stages that it's not just about creating cool stuff. It's having to deal with, as you now have to deal with Telegram, seas of bad actors trying to test the limits of the system, trying to break the system. Unfortunately, if we didn't have bad actors and pressure, it would be the best job ever. You just get to create. Yeah. Yeah.
And so, uh, the help from your brother, like you mentioned Nginx and sharding the tables, some of the scaling issues are algorithmic in nature. It's almost like theoretical computer science. So it's not just about like buying more computers. It's figuring out how to algorithmically make everything work extremely fast. So some of it is mathematics, some of it is pure engineering, but some of it is mathematics. Yeah. So at that stage, I could do the basic stuff. I could understand how I implement scalability into the codebase. How we shard my tables in the database, where I include Memcached instead of direct requests to the database. That was quite easy because it was still PHP back in the day.
When my brother got back from Germany somewhere around 2008, I asked him, "Can we make it even more efficient? Can we make it super fast and at the same time so that we would require even fewer servers to maintain the load?" And he said, "Yes, but PHP is not enough. I'll have to rewrite big part of your data engines in C and C++." I said, "Okay, let's do that." He invited a friend of his to help him, another absolute champion in world's programming contests twice in a row. And they, they put together the first customized data engine, which is far more efficient than just relying on MySQL and Memcached because it was, first of all, more specialized, more low-level. So they rewrote in C++ a large chunk of it, like, for example, the search, the ad engine, because VK had targeted ads, they built that. It was, it was very efficient what they did. Eventually, the private messaging part, the public messages part. At some point, we realized there are very few websites online that load faster than VK. Nice.
I remember in 2009, I went to Silicon Valley and I met Mark Zuckerberg the first time and some of the other core team members of early Facebook. Remember, Facebook was just four or five years old. And and everybody kept asking me, "How come even here in Silicon Valley, VK loads faster than Facebook? Everything seems to appear instantly on your website. What's the secret sauce?" That was one of the things that made them very curious. And that was always important to you to have very low latency, to make sure the thing loads. And cuz that's one of the things Telegram is really known for. Even on crappy connections and all that kind of stuff, it just works extremely fast. Everything is fast. As one of the core technological ideas, we prioritize speed. We think that people can notice the difference, even if it's just like a 50-millisecond difference. The difference is subconscious. It also allows us not just to be faster and more responsive, but also more efficient when it comes to the infrastructure expenses, because if your code executes faster, it means you need fewer computational resources to run it. So there is no way you can lose in making things faster.
And that's why we have always been very careful when hiring people. I would only hire a person if I'm ultimately certain it's the best option. If, if you hire somebody who is maybe a little bit distracted, inexperienced, you may end up with inefficiencies in your codebase that results in tens of millions of dollars of losses. And think about the responsibility. Like, if we jump to today from the VK days, Telegram is used by over a billion people. They open it dozens of times every day. Imagine the app opens with a slight delay, say half a second delay, multiplied by dozens of times by a billion. Because centuries, millennia lost for humanity without any reason other than just being sloppy. That is so important to understand and so wise that it's actually, if you're just a little bit careless as a developer, you can introduce inefficiencies that are going to be very difficult to track down because you don't know that it can be faster. Like the code doesn't scream at you saying this could be much faster. So you have to actually, as a craftsman, be very careful when you're writing the code and always thinking, "Can this be done much more efficiently?" And it can be tiny things because they all propagate throughout the code. And so there's a real cost in having a careless developer anywhere in the company. They can introduce that inefficiency, and all the other developers won't know. They'll just assume it kind of has to be that way. And so there's a real responsibility for every single individual developer that's building any component of an app like Telegram to just always ask, "Okay, can this be done more efficiently? Can this be done more simply?" And that's like one of the most beautiful aspects, the art forms of programming, right? Oh, yes. Because when you manage to discover a way to simplify things, make them more efficient, you feel incredibly happy and proud and accomplished.
And to your point, I can recall a few instances in my career when firing an engineer actually resulted in an increase in productivity. Say you have two Android engineers building their app, and then just they just can't make it. They are not keeping up with the pace of of of of the feature release schedule. And you think, "I probably have to hire a third one." But then you notice that one of them is really weird, falling behind the schedule, complaining some of the time, doesn't assume responsibility. And you ask, "So, what if I just fire this person?" And you fire this person, and a few weeks you realize you actually don't need and you never needed the third engineer. The problem was this guy who created more issues and more problems than he solved. That is so counterintuitive because you know, in developing tech projects, we tend to think that you just throw more people into something and then things get solved miraculously by themselves just because more people means more attention from them. No, that's again extremely powerful. The, you know, Steve Jobs talked about A players and B players, and there's something that happens when you have B players, which is kind of like the folks you're talking about, introduced into a team, they can somehow slow everybody down. They demotivate everybody. And it's very counterintuitive. They basically, part of the work of creating a great team is removing the B players. It's not just hiring more. In generally speaking, is finding the A players, quote unquote, and removing the people that are slowing things down. Oh, yes. Because the other thing that people don't realize is how demotivating working with the B player is. Everybody can tell if the other person, the other engineer they're working with is really competent. And if, if it's very visible, if the person is not competent, they're asking the wrong questions. They keep lagging behind. And at a certain point, if you're an A player, you get this dissatisfaction, this feeling that you are not able to realize your full potential, accomplish what you're really meant to accomplish because of this person working next to you or pretending to work next to you. And by the way, in some cases, it's not because the person is lazy. Some cases is just, you know, their mental, their intellectual ability is not there. It's not about experience. Most often, it's about natural ability and persistence. In 90% of percent of cases, it's just the inability to focus on one task for an extended period of time. Not everybody has this ability. So for people who do have this ability, it's an insult to work alongside someone who is distracted and cannot go deep in the projects that they're responsible for.