Transcription
First, we have to create a virtual machine. Uh, I have already created a virtual machine on AWS's account. So, you can use AWS, GCP, or you can, uh, do this in your local system as well.
So, what I'm going to do here, first, I'll try to connect this machine. So, click on this, click on connect. Okay, my server is connected. So, here I'm going to deploy my Red Panda cluster as a Docker. So, first, what we have to do, we have to install Docker and Docker Compose. So, let's install Docker and Docker Compose. So, let's go to Docker documentation. You can see that if you are using a Ubuntu or Debian-based system, we can use this command. Let's go to Virtual Machine and paste.
Okay, now we successfully installed Docker and Docker Compose. Now, we have to create a Docker Compose file for Red Panda cluster, then we will deploy. So, let's go to Red Panda documentation. I'll paste this documentation URL in description, you can get from there. You can see that Red Panda self-hosted quick start. So, we can either go with a cloud option or we can go with the dedicated. We have multiple options with Red Panda. First of all, if you want to self-host, you can do that as well.
Okay, so if you come down here, you can see that Red Panda documentation for Docker Compose. Either you can go with a single broker, so this will give only one broker with, you know, uh, less availability. Okay, but if your workload is very high, then what you can do, you can select the three broker option, then you can deploy this. So, now need to do any changes. If you want to change the Red Panda volume name, if you want to change any other port number configuration, uh, if you want to change any other configuration for this Red Panda, you can do this manually. So, as of now, I'll go with the default option. So, this is quite enough. Let's copy this. Let's go to my virtual machine. Let's create a file called Docker Compose.
So, now I have created a file called Docker Compose. Next step is we have to run this. To run this, we can execute Docker Compose up -d. I came back to my virtual machine. You can see that all are created. So, this is my volume and this is my broker, and at last, you can see that my Red Panda console. Let's execute Docker Compose PS. So, this will give all the Docker containers which created. So, this is my Docker Compose. Sorry, this is my Red Panda 1, 2, 3, then console.
So, if you want to see the console, you can just copy the public IP and this is running on port number 8080. Let's hit on port number 8080 and this will open our Red Panda console. And here you can see that my Red Panda console. I have one topic, I have a three replicas. So, three brokers are only, and this is my master one. Okay, let's go to topic. As of now, I don't have any topic over here. So, if you want to create a topic, you can create the topic from the console itself. Let me call this as my topic. Okay, all test topic. Let's give a partition, uh, two, and replication factor, we can give three. It should be an odd number. Uh, click on create. So, you can see that topic is created, and we have a two partition, and this is replicated through three brokers. So, fine.
Now, let me do a small testing. I'll create a Python script, and the Python script will create some messages, and this will produce messages to our Kafka or Red Panda. So, let's go to my virtual machine again. I have to create a file over here. Nano, I can call it as producer. I'm just testing this producer.py. It's a Python script. Yeah, this is the script. So, uh, the script will have close to 1000 threads. So, 1000 threads will send one message every second. Okay, so every second, we will get 1000 messages to this broker, and, uh, so I would say within 1 minute, we will have 60,000 messages. Okay, let me save this configuration.
Okay, so now you can see that we have two files, one is Docker Compose, another one is producer.py. So, now I'm executing producer.py. So, just execute Python and producer.py. Here you can see that, uh, so this producer.py, this is sending messages to your topic zero. So, topic we already defined in our code, Python code. So, this will automatically create a topic called your topic, and this will send messages to our Red Panda cluster.
So, let's go to my Red Panda console again. So, now you can see that we have one more topic called your topic with a one partition, one replica. Okay, click on this your topic. As of now, you can see that we have a 22,000 messages already here. So, let's refresh. You can see that 25,000. We can just enable this. This will automatically refresh every 10 seconds. Okay, so that's it.
So, if you want this kind of video, please do subscribe to my channel. So, then see you next video. Bye-bye.