Transcription
Designing scalable systems is changing a lot. If you think system design is only about load balancers, scaling your databases, scaling your APIs, then I think that's not enough.
Nowadays, we are already seeing a lot of change in the engineering ecosystem. With the introduction of AI, there is a lot of new and interesting topics that one should keep in mind while designing systems. There is a good chance that the system you might be building will also help some downstream team or some upstream team to probably power their AI ecosystem.
Today, you will see most of the system is not just about storing and retrieving data, but also to get the context out of the data. Also, to run LLMs on top of it. How you can actually introduce it as an MCP server so that LLMs can actually communicate with that. How you can optimize on the tokens. How do you can optimize on the context window. How you can optimize on the overall costing. There are so many important things now you have to keep in mind while you are designing system.
And in case you are actually working on things which are directly related to some kind of like an AI product. For example, you are building something which is an agentic ecosystem for solving a problem, then scaling those systems also require some specific skills and require you to have some very specific knowledge all together.
So, if you are somebody who wants to learn system design right from the traditional path, but also need to incorporate all the new AI engineering stuff, then you are already at the right place. In this particular video, I will talk about how you can start learning system design right from the scratch. Right from making basic APIs to also make scalable databases and scalable APIs, and also then start migrating to the AI engineering ecosystem. How you can actually learn about AI agents. How you can actually learn about harness engineering. What is context window. How you can optimize it. How you can make sure that your LLM API calls are as optimized as possible and whatnot.
After this particular video, you will know what to learn and in what order to learn so that you can design scalable systems which can also help a lot of AI ecosystem work as well. So, without any further ado, let's just start.
But before starting the video, if you have not yet subscribed to the channel, do consider subscribing because we are going to put some really awesome content coming up ahead. So, let's just start.
So, before moving forward in the video, I would like to talk about the new system design 2.0 cohort that we have recently launched. If you're somebody who is actually willing to apply for a lot of product-based companies and you are technically confused on where to actually prepare for low-level design, high-level design, and machine coding rounds, then you are actually at the right place.
In the new system design cohort, we are going to actually include all the relevant concepts around high-level design, low-level design, machine coding, and this time we have kept it kind of like bigger and better. This time we have specifically added a lot of company-specific interview problem solving both in HLD and LLD. We have added interesting concepts around distributed system like Lamport clock, vector clock, consensus algorithms, and whatnot. The complete curriculum of the system design 2.0 cohort is mentioned in the link in the description section below. You can use this coupon code coming on your screen to get massive discounts altogether. There is a dedicated video on the channel where you can find all the details regarding the course syllabus.
But what I can assure you is based on my experience working as a software engineer and all of the interviews that I have given, this is going to be a very comprehensive course where we are going to talk about all the relevant things that is going to be necessary not just for you to crack software engineer interviews, but also work as a software engineer. So, do check out all the links in the description section below. Uh all the course syllabus, all the dates, everything necessary is already mentioned there.
So, now let's come back to the video. So, now let's start talking about the detailed road map. Of course, thanks to the AI tools nowadays it is really easy to actually form these kind of like interactive documents in actually no time. So, in this particular document I have listed down all the important steps in the right order which we are going to technically follow, right?
So, I have actually divided this complete journey into a couple of phases. Phase one is going to be around basics of threading architecture and computer networks. The phase two is going to be around low-level design and design patterns. Phase three is going to be around storage infrastructures and databases. Phase four is going to be around distributed systems and distributed architecture. Phase five is going to be around big data concepts and some miscellaneous concepts around change data captures, streams, and whatnot. And the final phase, phase six, or I would say the pre-final phase, phase six, is going to be around basics of LLMs, RAG, agentic AI, and whatnot. And the final phase seven is going to be all the advanced topics around all of these individual phases that we have actually seen. Right? So, let's start actually exploring and don't worry, I'll list down this particular document in the description section below, so you will be able to technically check it out.
Now, in order to start with your phase one foundations, I believe everybody should start exploring things around basics of threads and concurrency. Of course, I'm expecting here that you know basics of programming. So, I believe you should start understanding what's a process, what's a thread, what's a co-routine, right? What is the general memory model in majority of the languages? You can pick one like Java. Understand the synchronization primitives like mutex, read-write lock, semaphore, condition variables, and whatnot. Understand the concept of lock-free programming, right? Try to understand the concept of lock-free queues, atomic references, etc. Then understand deadlocks, livelocks, understand the concept of blocking versus non-blocking IO. How exactly different different infrastructures supports blocking and non-blocking IO. Understand the concept of producer consumer and task queues along with concurrency models. Here everything that you're going to learn will actually help you understand deep dive aspects of threading and concurrency aspects.
Once you're done with that, start exploring things around computer networks. See what is the OSI and TCP model. Explore the TCP internals, right? Explore topics like DNS, TLS, HTTP, right? Some interesting topics would be around UDP and quick protocol, right? Try to explore things around web sockets, server-side rendering or server sent events, you can say. Long polling, short polling, the concepts of basic envelope calculation and whatnot. All of these things will help you to understand basics of computer networks. And with back of the envelope calculation, it will help you to actually later realize a lot of things around why these calculations will be very important for building up and at least for seeing what will be the requirements in your infrastructure. Right? These two things together, I believe, would be great for your basic foundations to start with.
Then start exploring things around solid principles, design pattern, and low-level design. In solid principle, of course, explore what is single responsibility, what is open-close principle, Liskov substitution principle, interface segregation, and dependency inversion principle. Some other very well-known topics that you have to have to go with is composition versus inheritance, right? Concepts of dry and let's say other important principles like YAGNI and KISS. What is cohesion? What is coupling? These would be some really important topics to explore.
Once that is done, start exploring design patterns. Explore different design patterns and try to explore the situations where things are not going great, you are violating some solid principles, and then these design patterns come into the picture like factory design pattern, builder design pattern, singleton pattern, prototype pattern, and whatnot. I've listed down everything. You can actually go through these important design patterns here.
Now, once you have actually gone through all of these, it would be great to actually try and implement some of the machine coding problems and low-level low-level design problems. Try to solve problems around, let's say, short URL shortener. Maybe try to make your own Splitwise. Try to make a simple BookMyShow, kind of like a booking application. Parking lot problem, try to solve. Try to solve the problem of rate limiter. Try to make some games like chess or snake and ladder. Implement logger-like libraries and whatnot. All of these different different problems are going to actually help you apply these design patterns and solid principles. So, all of these is listed. Just go through that.
Then, I believe everybody should start exploring things around internals and basics of databases. Try to see how exactly indexes actually work. What is the B-tree index? What is the LSM-tree based indexes? What are the compaction strategies that actually go on behind the scenes, right? What are different different index types like primary index, secondary index, hash-based index, geospatial index, bit mapping, and whatnot.
Then, start exploring things around concurrency and transactions and isolation. Try to see what is the asset property. What all databases follow the asset property? How they follow it? Try to understand the concept of isolation levels. Very, very important. See what is the meaning of each isolation level. Read uncommitted, read committed, repeatable read, serializable, and whatnot. All of these isolation levels, and in fact, most of these topics that I'm talking about, everything has been in very detailed covered in uh the system design course of AlgoCamp. You can actually check that out. I'll put the link in the description section below for that. But, for now, let's come back.
And you can see you have to also understand what is uh the concept of MVCC. Try to understand the concept of pessimistic lock and optimistic lock. How exactly and which situations pessimistic locking would be great. In which situation optimistic locking will be great. Try to understand some no-SQL um databases. Explore different no-SQL databases and their internal architecture and their use cases and what situations they are very good for. Like try to explore some key-value stores like Redis, DynamoDB, ZippyDB. These are some really great case studies to actually explore. Try to understand some wide column databases. Cassandra would be great to start with. If you still have time, maybe you can explore HBase and Bigtable. Document stores like MongoDB, DynamoDB, and graph databases like Neo4j, Neptune. Any one of these if you can explore it would be great. Right?
Then try to explore the concepts of scaling these databases with the concept of replication and sharding and caching. Right? Try to understand what are the different different uh sharding strategies you can have, the concept of consistent hashing, how you can do cross-shard queries, what is the concept of resharding. Explore the concept of replication. What is synchronous versus asynchronous replication, different architecture of replication, single leader, multi leader, leaderless. Right? What is the replication lag, the concept of quorums, and then start exploring things around caching. Where to cache, what are the different cache patterns, what is the eviction policy. Right? How do you invalidate a cache? How what what do you do for hot keys? All of these are very interesting concepts to go for.
Then I would say try to explore some interesting time series databases because time series databases help you understand some really cool facts and architectures about databases which works on scale and have a very specific problem to solve. See how exactly time ranges actually work, how they actually compress things all together. What is append-only writes, where else you can see append-only writes all together. Right? And similarly you can see you can try to explore some architecture of some important time series DBs like Prometheus or maybe InfluxDB or TimescaleDB.
Then I believe ZippyDB is a great case study. It's a Meta's distributed key-value store which also provide persistence. Try to explore what is ZippyDB, how exactly it works under the hood, how exactly it depends on RocksDB, what is the consensus going on behind the scenes, what consensus protocol it depends on, sharding, how exactly you use it with respect to tunable consistency, and you can try to have some comparisons of it around with Spanner, Cassandra, and Dynamo DB. If one more database recommendation if you want, you can definitely explore something like Spanner, try to understand the concept of true time in Spanner, and whatnot.
Then, I believe you should start with distributed systems, try to understand the concept of CAP theorem, Paxos theorem, right? What are the different consistency models, sequential, causal, session, eventual, all of these consistency models. Then, I believe try to understand the concept of time, right? What is vector clock, what is Lamport clock? Then, apart from that, try to explore the concept of true time in Spanner, right? How exactly it solves the different different problems that are currently not yet solved with, let's say, your normal vector clock or Lamport clock, right? And, you can actually see it some really interesting case studies with Google Spanner, right? Again, as I mentioned, all of these topics like Google Spanner, Zookeeper, DB, etc., everything is actually covered in the system design course.
So, you can see try to explore the concepts of load balancer, what are different different type of load balancer, L4 load balancer, L7 load balancer, different algorithms for these, try to maybe set up a load balancer for yourself, so that you can get an idea of what goes on behind the scenes. What are API gateways, try to explore the concept of API gateways, try to explore the concept of CDNs, right? These will be some really great things to actually explore.
Then, understand the concept of asynchronous messaging, right? What are different queue infrastructures like, let's say, you can try to explore RabbitMQ, you can try to explore something like AWS SQS. What are different delivery semantics like at most, at least once, exactly once, right? Try to explore things around with Kafka, how Kafka is better than these existing queuing infrastructures, right? How Kafka is so scalable, what are the what is the internal architecture of Kafka, what is topics, what is partitions, all of these things about internals of Kafka would be great to actually explore. Right. So, these would be some really important topics that I believe with respect to distributed systems, everybody should bare minimum explore.
Now, post that, you can try to explore some more advanced topics around distributed transactions. What are distributed transactions? How you can actually resolve them. Uh you how you can resolve them using two-phase commit, orchestration saga, choreography saga, right? If you are implementing saga, what is these compensating actions? How you can implement them? There are some interesting patterns like anti-corruption layer, strangler fig, back end for front end pattern. These are some great patterns to explore. Right.
Some event-driven um patterns are there like CQRS that is command query responsibility segregation, event sourcing, outbox pattern, item potency. Really great things to actually explore. Some other important system patterns can be circuit breaker pattern, right? Retry with exponential backoff and jitter. How you can actually do timeouts and deadlines. What is a dead letter queue? All of these are some important and small pieces which will actually fit in bigger infrastructures that you're going to develop.
And then, I believe a lot of learning will actually come up once you start solving problems and exploring actual designs. Like try to actually see and read about how you can design a simpler system with respect to let's say something like Slack or Google Chat, maybe a social media feed, a streaming platform, maybe a cab matching platform, and nearby proximity service kind of like a platform like which also which is also used in platforms like Tinder or let's say Uber, right? Uh messaging platform like WhatsApp, uh storage platforms like Google Drive, YouTube. All of these are some very interesting problems which have some really interesting case studies, corner cases, and some known solution that big tech giants have actually used which would be great to actually explore.
Now, once you are done with that, I believe try to start exploring things with respect to big data. Right. See some important concepts of big data. For example, uh uh try to explore what is MapReduce, what is Apache Spark, what is Hive, Apache Flink, very, very important stream processing system, Apache Flink. Try to see how Kafka for stream processing fits into the picture, how you can use Flink as a consumer. What is structured streaming, right? And you can actually try to read about the lambda architecture, Kappa architecture. What is data warehouse, what is a data lake, how uh uh what is a data lake. Some good approximation algorithms are there which is very important for system design. HyperLogLog, count-min sketch, right? Bloom filters, all of these are really, really important.
Apart from that, some important concepts uh would be to explore things around change data capture. What is change data capture? Why do you need it? How it is different from triggers and whatnot. With this, a good foundation for even high-level design would be ready for you.
Now would be the time to actually start exploring AI in studying. Understand the fundamentals of LLMs. What is an LLM? What is tokenization? What is embeddings? What is the transformer architecture, right? What is the concept of attention, which actually uh made the whole transformer architecture possible, right? Try to understand the concept of temperature. Try to understand the concept of how exactly long input and short input is actually processed. What are the caveats that LLMs cannot process, right? Understand the concept of context window. How exactly you can optimize these.
Then start exploring things with respect to vector database. What's a vector database? Why do you need a vector database? What are different embedding algorithms that you can actually use in order to prepare uh data for these vector databases, right? And again, maybe it would be great if you can try to set up one and try to play around with it, right?
Then explore things with respect to RAG. What is retrieval augmentation augmented generation, right? What are the different phases in the RAG pipelines like loading, chunking, metadata processing, embedding, right? And then how you actually query the pipeline altogether, you re-rank everything in the pipeline altogether. Right? What are the different type of rag patterns like agentic rag, graph rag? Try to explore all of these.
Then, some really important concept around LLM caching because you don't want to just uh over-bombard your LLM APIs and just get a lot of bills. Caching is going to be a really important part in your infrastructure. How you can do all of that is going to be a really important concept.
Then, understand the concept of prompting and context engineering. What is the system prompt? What is few-shot example, conversation history? All of the things that you need for understanding the concept of prompting and effectively prompting your LLMs is going to be really important. I've listed down all the important topics that one should explore.
Then, I believe everybody should try to set up their own MCP servers, understand what is MCP, how you can set up a new MCP servers, try to integrate with some existing in MCP servers. It would be a great exercise to start with. And of course, how can you forget the agentic infrastructure? Try to understand what is a AI agent, what is the agentic loop? What are the different agent patterns like react pattern, reflection pattern, right? Uh what is plan and execute pattern? These are different different agentic patterns that one should explore. How you can actually make a multi-agent application? Try to explore that. Try to see how uh bigger um uh I would say infrastructures are built around it. How you can actually handle things in production when failures actually happen, right? How exactly you can avoid hallucination for your agents and for your LLMs? All of these are going to be very important concepts.
Then, some advanced concept that I believe, if you still have time, you can try to explore. Like, see what is the concept of evaluation harness, what is the agent harness concept, how you can write your own guardrails, how you can write your own evals to see whether your LLM infrastructure is working at par as your expectation or not. Right? How you can actually fine-tune your um LLM for your use case? Try to explore the concept of uh Laura, Q Laura. Right, these are going to be very, very important to actually fine-tune models on top. And then, of course, when you're deploying LLMs on production, observability and monitoring is going to be very, very important to explore. How you can actually set up observability and monitoring with respect to LLMs is going to be something that you should definitely explore.
So, I've listed down everything in this road map, and this is This has been structured in like sequence, step-by-step manner. All you have to do is just try and go and follow this. I'll put the link for this in the description section below. Just open it up, try to explore it, and I would say don't just go with this. Every topic that I have mentioned here, try to start reading about it, try to find articles on top of it, and explore as much as possible. Explore in-depth all of these things as much as possible. That's what will actually prepare you for making sure that your engineering skills, your system design skills are at par, and everything that is required to build highly scalable system, you know all of that.
So, uh do check out the link for this particular road map in the description section below. I would put all the relevant links there, and if you have any questions, do let me know in the comment section. I would be happy to answer all of them. That being said, let's wrap this particular video here. We're going to meet soon in the next set of videos. Till then, take care. Bye-bye. I'm Sanket Singh, signing off.