Transcription
[Music] thank you hello I'm Vlada Shriner a product manager of red panda let's talk about red panda console the UI tool that helps you with troubleshooting streaming applications.
My career has been coupled with streaming for a good few years and I have picked the use case or scenario that I have seen many many times.
When talking about troubleshooting streaming applications, let's actually make things a little bit more real by using one. So I have created a streaming application that deals with weather. It would be an app that's decoupled between producers and consumers. So in our case, producers, producers would be weather stations and we would like to use a streaming application that would reason about temperatures in a real time. So we would be getting a temperature measurement from more than 3,000 weather stations and we would be presenting aggregates on the state level. Under the hood, it's going to be streaming aggregates, averages over two-minute long sliding windows.
Now let's actually go to the console to see the application running. This is a weather topic that producers use to store their data records. So each recording, each measurement looks like this. So there is temperature, there is the state, and there is the identification of the weather station. And this is my streaming app presenting the aggregates. For Simplicity, I've already used it to just three states and every 10 seconds is presenting fresh updates aggregating all the data measurements for that specific state over last two minutes. Everything is good so far. The application is performing well and delivery results to the customers, to the end users.
Until this moment, something has happened. There is no chance that there would be 18,000 degrees of Celsius in Texas right now. But the weather application is actually looking quite fine otherwise. There are no issues, there are no logs like error logs, there is no issue with the malformed data records, there are no issues with performance or stability. So the problem is clearly on the producer side. However, there are more than 3,000 producers. I can't inspect them or shut them down all of them, right? So I must understand where is the issue actually happening. What do we know to start a troubleshooting? We know that the issue is isolated to a Texas measurements and we know that it hasn't been observed before. Surely this, this time, right? This is like European notation for 7:36 PM.
Now let's use console. Uh, by default, it was presenting last 20 measurements. Now we need to go deeper. So firstly, instead of showing newest data, we would like a console to play data since the issue actually appears. So once back, it was 7:36. 7:36. Okay. Secondly, we know that this somehow Texas related. Console comes with a filtering feature. It's called push-down filters. It's pushed down because the filters aren't executed in a browser, but in the console back-end, very close to the data cluster. First filter I'm going to apply is I would like to get just the records that are coming from Texas. It's still a little bit noisy. Let's actually ask console to show just the temperature field instead of the whole volume.
All right, now are we seeing anything interesting here? Nothing particular right now. The function is aggregate, right? So if the aggregate is wrong, it can be only related to the temperature data. So let's do another experiment that I have prepared a push-down function here for that. Let's filter down just the measurements that report their temperatures. Okay, now we are seeing something and we can inspect the records manually. So what can I see right now is that it's located to the ktxw station. I've do a few more samples. It seems like that. Let's try to to confirm this hypothesis using a fair filter. So if we ask console to show us everything that's from Texas, that's where temperatures, but isn't ktxw, what are we gonna get? Nothing. Okay, it means that we have located the most probable root cause of our issue. There is a broken temperature sensor at this ktxw station. I mean, it's still machines, right? My car refused to warm start a few days ago. Guess what was the root cause? Wrong temperature sensor.
How to fix this issue? There are many ways how to do that. In my case, I would go and quick fix my application to exclude measurements coming from this station. Then I can do something deeper and more holistic. My application is expressed as a simple streaming SQL query. So I will just extend it. So station ID isn't this apxw? Okay. Oh, I need to do another trick. I need to replay all the data to deliver correctly the results. But I don't want to replay all the data. Data if the topic spans for a few years, that would take ages. And console can help here as well. It comes with a consumer group tools and I can actually rewind the offset of this consumer group to a specific time. So what I can do is to replay just the data since uh the moment where things went wrong. So roughly going back a few minutes ago would do the job. Now my app would replay just a few minutes worth of data. Right? So you can see that it immediately resumes and it delivered corrected data to the consumers.
Now let's recap what we have seen. Right? Console is a great tool for ad hoc data troubleshooting. Your app, your streaming app is usually designed for specific workloads, but when something goes wrong, you need more of a Swiss army knife that helps you slicing and dicing the data to troubleshoot. This is exactly when console comes head hand. Why is it a big deal? Firstly, all the data processing happens where the data lives, right? So there is a very performant link between the data cluster and console. And the single Go process is benchmarked to handle hundreds of thousands of messages per second, which is not always the case when troubleshooting from the local station. Secondly, it comes with this like nice self-guiding UI. When people are troubleshooting, they are stressed anyway. So having one tool that that's in place and does the job is just a big improvement. Moreover, you might argue that you are just handy with expert tools, but you can change console to become a self-service kiosk for your team or for your application teams. So it's when something goes wrong, people don't need to call Kafka experts or Red Panda experts for help, right? Red Panda console is here to provide a helping hand to everyone. It comes with integrations to identity management systems and it has its own uh role RBAC system. So you can easily define that this application team from your Google Google directory basically maps to the read role and then all the people can automatically single sign-on to console and do self-service data troubleshooting.
What comes next? Firstly, it's health monitoring. We would like you like to give you a single pane that shows you the health of your Red Panda platform. So what's working fine, what might need your attention, and what really needs your attention and why. Secondly, we would like to double down on the schema registry, notably adding more management features to add new versions of your schemas. And we always work on more integrations with identity management systems. Do you like a console? Give it a try. Either self-managed, the single Go binary close to your self-managed Red Panda cluster, or just go to Red Panda Cloud since console is integral part of it. I hope you like console and I hope you like this short talk. Thank you for listening. Feel free to reach out. See you later.
[Music] thank you