Transcription
Hello again! This is Leno Tadros.
Now that we have actually created an Azure AI Foundry Hub and a project, we were able to deploy a GPT-4 and also an embedding model 88002. We are ready to start deploying a Prom Flow in Azure AI Foundry inside of that project and take a look at how we can actually start using it.
All right, let's go back to our project again. We're going to click on Prom Flow here on the left side, and we don't have anything deployed yet. I'm going to click on Create. We will start our first one with a standard flow. We'll just go ahead and say Create. That creates a pretty simple one that has something to tell me a joke, but we're going to delete that and do something more useful.
So, let's give it a name. We'll call it, for instance, "Loo Standard Flow." All righty, that's good enough for me. I'll say Create. Once this gets created, we're going to go ahead and open it up in a designer and take a look.
Once it finishes, I'll be right back after this finishes in a few seconds.
Indeed, it took about maybe 60 seconds or so, but now I have the graph for my Prom Flow, and it has four pieces associated with it completely out of the box. The first one is the input, and you can see here it will say "a topic." It's of type string, which I have a lot of different types that I can actually pass in there, and the value will be entered in here as well.
All right, notice that this is a completion, so I'll have to run it. This is not a chat. I could have actually created a chat so I can converse with memory and history and so on, but we're going to keep it as simple as possible.
I don't even like the word "topic," so I'm going to say, for instance, "question." We'll change it to "question." There you go, and we'll say Save to make sure that this is also propagated everywhere.
In reality, I don't need this LLM for the joke, so I'm going to remove it from here. Let me go ahead and click on Delete right there for the joke. We'll take this out, delete the step, and also for the echo in here, which is the Python code that will actually pass the string, I'm going to delete it as well.
Now I'm only left with the question, which is the input, and the joke. I'm going to change it to be the output in here as well, and that will be the output coming in from there.
All right, I think we're good. Now I need to go ahead and add a couple of things. First of all, I always prefer to start a compute session. It's serverless, so I don't have to create a VM or anything; it will create it for me.
When we click on that, it will take usually between 1 to 3 minutes, depending on the machine and the region and so on. In my case, it usually takes about 2 minutes max, but we will end up getting a compute session. Without this compute session, the more tools will be turned off. You cannot actually use any of the capabilities of the more tools until you get a compute session started for you.
We'll come back after this is done as well.
All right, awesome! It's done. If I put my cursor over or click on it, it will tell you what size of a VM was created for us. E A sv3 was created for me, which is great.
Now, let's go ahead and actually create an LLM. We'll give it a name; we'll call it LLM01, for instance, or you can call it whatever you want. Let me make it 01, not 01. Right, we say Add, and there is the LLM.
In the LLM itself, I'm going to actually set up my connection. Let's go ahead and use this connection that was created for us, and it will make it basic. You can use chat or complete; I'll make it a chat. The deployment name, you know, as GPT-4, the one that I deployed in the previous video, will be in there as well.
I'm going to make this 0.2 temperature, and we'll make this a maximum of 1,000 tokens that can actually be passed. You can choose also the response format, whether you want it to be text or JSON. I'm going to make it text for this one; simple enough.
Notice in here, in the system, you are a helpful assistant. This will be the system prompt, so you can actually add, be polite and concise. All righty, so we'll ask it to do that, and there is my user. The question will come in here.
As long as we know where the question is coming in, that means if I validate and parse the input, it will automatically see what variable for the question is, and it will allow me to change it. Where is the question going to come from? Well, if I go in here, notice the input do inputs.question, which I created all the way at the top. Let's click on that, and there is the input coming in. Excellent!
Now, if I go back to the output, it's not called Echo anymore because I deleted that. So if I go in here, there is the LLM01 output, the one I just created.
Now I've got myself, let me move this to make it look better. So actually, I have a very simple way of asking a question. I'm going to come in here; we say I'm going to give you an input. The LLM is using GPT-4. We'll take the input from the question, and the output will be coming directly from the LLM.
Of course, I can add a lot more things in between, which we will do in a lot of different videos, but at least that's the starting point. To do that, you will not be able to run this, of course, unless you have a compute session running, which we do at this point.
Excel! Let's go ahead and ask the question. There is the question string. We'll say, for instance, "Why is the sky blue?" All righty, and let's go ahead and run it. That will take hopefully just a few seconds, and when it comes back, you will notice there is a green check mark that will be in here that this was successful.
Then we will have access to the output that came in from GPT-4. So we'll give it here another few seconds, and then you'll see all the way at the top it says "View Output." First of all, it was successful; it was completed, and then we'll say "View Outputs."
In the view outputs in here, the sky appears blue because of blah blah blah blah. Exactly the correct answer is one expecting.
The nice thing about this is that I can actually see the logs. I can load exactly what happened. I can see the metrics if I want to. If there were any metrics, I'd be able to see it. The trace is an important piece; I can see the flow, how many tokens were used, how long it took. It took less than a second.
If I open up the flow, I can see the questions to the LLM, the chat OpenAI. Of course, it will get a lot more complicated, so this could be very, very useful for us to see exactly what happened in case you have some orchestrator like Semantic Kernel and L Chain. I want to find out where all the time is being spent and how many times it went back and forth to negotiate all of that stuff.
But at least for this video, you'll see this is done. Remember, I am just testing right now; I'm not deploying. So I'm going to just go ahead and use it right from here, from my Prom Flow itself.
Hopefully, this was useful, and I'll see you again in the next video!