Transcription
Hello, I'd like to show here an example of some of the Gen features that we've launched in Red Panda Connect. So, this is a small pipeline that connects two Red Panda topics, an input topic and an output topic, to OpenAI's chat completion endpoint. And to show you just kind of how this works and what this does, let me run this inside Red Panda Connect.
So, I'm going to, we'll listen on our output topic, we'll write to our input topic, write a key, and then we'll ask, "What color is the sky?" and we'll see the answer here on our output topic momentarily. So, we've seen here, we've gone to OpenAI, we've sent that data from our locally running Red Panda instance to OpenAI, we've gotten an answer on what that is, and then, um, we got the response, and that was written to the output topic, all on a few lines of YAML.
Um, now to really show you the power of some of the things you can do with Red Panda Connect is that you can swap out running, sending your data to OpenAI, and we're using a locally running model where the data never leaves your network, running on your own GPUs. Um, we can use Ollama, and we're going to use the latest released Llama 3.1 model and do the same thing that we just did. So, we're now running the same pipeline, the only thing we switched out was the model and, um, that we're using Ollama locally now. And we're going to do the same thing here: "What color is the sky?" Okay, we'll see here momentarily, we'll get a new record that pops in with the answer of what color the sky is. Now, that demonstrates the power of things that you can do with Red Panda Connect and how easy it is in just a few lines of YAML.
I want to show a more complex example, um, to give just sort of the full power that you can do with this. So, we have a RAG pipeline here set up. We're going to take data in from Red Panda on a text topic, we're going to summarize that text, we're going to compute embeddings for that text, and then we're going to send that data to Pinecone, all in about 35 lines of YAML. So, I'm going to run this pipeline now.
So, now it's, this pipeline is getting set up, it's running. We're going to produce text into our input topic, we're going to use a key of "my-demo-key," and then we're going to use some text that I've prepared about red pandas to, to compute the embeddings for this. So, I've just put this in my topic. Red Panda Connect is going to suck this out, it's going to run that pipeline where it's going to summarize this text using Llama 3.1. It's also going to compute embeddings for this text using a different embeddings model, all running locally, nothing's going to leave my network, and then it will send that data to Pinecone so that we can do, uh, approximate nearest neighbor search.
So, let me show you what this looks like in Pinecone. All right, so here is my record, and you can see that I've got my "my-demo-key" is shown up. It's got the vector here, and also a summary using that was computed using Llama 3.1 of the Wikipedia text that I pasted in. So, I just, thank you for watching. That's a demo of how simple it is to set up these pipelines, uh, using Red Panda Connect.