Transcription
Hi again, this is Leno Tadros, and this is a continuation of the previous video.
I've already used GPT-4 Vision to test in my own lab in the prompt flow, and now I'm ready to deploy this and see how I can use it from the outside world, not just from the lab of the prompt flow inside of Azure AI Foundry. Let's go ahead and do that.
Okay, great! Let's go back in here. I tested this, the output was correct, and everything is good. I'm ready to actually deploy this.
So, you notice all the way at the top, it says "Deploy." Let's click on that. There are a couple of choices that I need to make. It will come up with a weird name; that's okay for the endpoint name. It could be "Leno project" and whatever letters and numbers show up in here. You can change it, of course, to something that's more meaningful to you; that's fine.
Also, the endpoint is one thing, and the deployment name is something else. Usually, it will take the endpoint and we put a -1-2-D3 depending on how many deployments you're going to end up making based on the same endpoint in the future.
You also have a lot of choices in different virtual machines in here. You can choose based on how powerful, how many cores, and how much RAM it has. It chose for me a four-core machine with 14 GB of RAM and 28 GB of disk, which is more than enough. It costs $23 per hour or 23 cents per hour to be able to use it.
So, I'm going to also use one instance only for this specific demo, but you can actually scale it however you want as well.
All right, let's go ahead and say "Next." The next one, I'm going to keep everything the same. The authentication type will be API key-based. We use the system assigned; all the stuff is good.
The next one is the advanced setting for the deployment. I'm going to actually use the environment from the current flow definition itself. I'm not going to do anything special. We'll say "Next."
This is the end one, and the connection is the one going against the West US, not the East US one, because that's the one that has GPT-4 Vision available in it, which is important as well.
Okay, I'm going to say "Next." These are all the things that the selling is going to do once you click on "Create." At this point, go ahead and get yourself a cup of coffee. It's going to take between 5 to maybe 7 or 8 minutes to make this deployment.
The way to find out if it's done or not is to go again to the models and endpoints. Let's click on that. Oh, let me save all of the stuff as well, just in case I need it later. We'll say "Models and Endpoints."
There you go! All right, so that is my GPT-4 from the West Coast, and this is the GPT-4 Vision and the text embedding. Once this is actually starting the process, it found the machine to do this on. There should be a third section in here for the new Leno project with the weird letters and numbers after it.
So, I'm going to keep refreshing this until I see that at least it's started. Okay, I'll come back in here once it started. It doesn't mean it's done, but it's just started. I'll be back in a second.
Ah, there you go! The endpoint has been created. That means if I say "Refresh," no, not yet. Okay, it's lying. So, it should be coming up in here any second now. Keep refreshing that, and indeed, after a few seconds, this showed up.
There is the Leno project in here. Keep clicking on "Refresh" until you see this. That doesn't mean it's ready; it means it's in the status of creating. This is the part that will take probably between 5 to 8 minutes, depending on what's going on in Azure at this point.
Once it has succeeded, we got ourselves a brand new endpoint that is ready for deployment. I'll be back in a few minutes when this is done.
All right, and it did take about 5 to 6 minutes or so, but once you see the word "Succeeded," I'm in good shape. Let me go ahead and click on this deployment and let's see what now is available for me.
First of all, I want to make sure this is successful. That's great! But now, all the way at the top, there are different tabs: Test, Consume, Monitoring, Logs. This is great!
Let me first go ahead and test it. If I say "Test" in here, this is something that is requiring inputs. Remember, our only input is the image.
Ah, okay, well that's going to be a problem. How am I going to be supposed to paste an image in here? There is no "Load File."
Well, there is a trick here. The trick is you have to go to the JSON editor. Notice it brings in a sample; it says "topic" and then it passes "Adam." Remember, I did not call it "image"; I called it "photo," and it's lowercase. You have to make sure it's exactly the same name you gave it in the prompt flow itself.
I called it "photo," and inside of here, this is the tricky part. Most people are used to putting a URL where the image is. So usually, you would go to Azure Storage, create a content container, and put a blob in there. That means my receipt.jpeg file will be an image file that is publicly available inside of a container. Great! You can do that and then copy and paste the URL for where the file is in the container.
You can also put it in Amazon S3 or wherever you would like, as long as we have a URL for it. The way I like to do it personally is not to actually include a URL, which I know can change. I want it to be completely self-contained in my application.
So, I usually put the base64 value of that image itself inside of the JSON. So how do we do that? Well, let me open up a free utility. There are a lot of websites out there that you can give a JPEG or a PNG, and it will give you the base64.
But I'm going to bring here to the screen a free utility from Microsoft called DevToys. I definitely recommend for you to have it. It has all the stuff that you use every day as a developer or a geek like me, like certificate decoders, color blindness simulators, cron expressions, date converters—these are the tools of the trade for everything we would like to do.
The one I'm interested in is the one that says "Base64 Image Encoder." Let's click on that and let's browse actually to our image. Let's say "Test." I called it "Test Receipt." There you go! Immediately, the entire base64 will be available in here for the image.
Great! Let's copy that to the clipboard, and we will say thank you to DevToys; it did its job. I'm going to go inside of this and I'm going to be pasting the entire base64 inside of there.
Now, if I test this, I'm actually now making a call against my deployed server that contains that prompt flow that I deployed and passing it a photo. Let me go all the way to the top—a photo key that has the value of this base64 of the image itself.
So, let's go ahead and see if it will work. We'll say "Test." Drums rolling, please! If it comes back and gives me the date, the total, the number of people dining, and all of that, then we would be in pretty good shape.
And it did a pretty good job with the base64! Notice the November 22nd is correct, Dixie Cream Cafe is correct, 5139. It didn't still get the number of guests; it's supposed to be 2x6, but again, the receipt was very crumpled in my pocket when I took a picture of it.
But hopefully, that gives you an idea of how to pass this. This is my preferred way of doing it instead of pointing to a file that might not be available in the future. This is now all self-contained.
The final thing I wanted to show you is the "Consume" part. If you click on "Consume," it has the endpoint where your deployed server is at. It has the primary and secondary keys that you can use for authentication, and then it gives you a lot of examples for consumption.
It gives you JavaScript code, Python code, C code, and even if you'd like to see the JSON itself, maybe you want to use it in curl or something like that.
So, in my case, I want to use Python, for instance. Let's click on that. All the code you need—well, not all, but 99% of the code that you're going to need to run this is available in here.
Let's go ahead and copy that to the clipboard, and I would like to go ahead and open up a file inside of a folder, for instance, in Visual Studio Code to run this code. Let's go ahead and do that.
All right, I went and created a brand new folder called "Test Vision." It has nothing in it at all. I'm going to create a brand new file inside of there. Let's do that, and we'll call it "test.py."
All right, and whatever code I got from the consume page in Python, I'm going to be pasting it in here. That's all the code that was given to me. There are a couple of things that are not done yet.
First of all, the data, and the second one is the API underscore key. Everything else, including the URL for where my endpoint is for the deployed server, is there, and everything. So, I don't need to do any of that stuff. I just need to give you the JSON for the picture, and I need to give you the API key.
So, let's go ahead and get those two. First of all, let me save this file before I forget. I'm going to come in here and we'll say, "Give me the primary key." I'm going to copy that into the clipboard, and we will go into the API key, and I'm going to be pasting it.
And don't worry about you seeing this; I'm going to delete the deployment right after this, so it's not a big deal at all. I just wanted to see how it works.
And then inside of the JSON for the data, remember what's the name of the key? It's called "photo." I called it "photo" in the prompt flow.
All righty, and now I'm going to come in here and we'll do exactly what I did before. I need to pass the base64, so I'm going to open it up in here one more time. We'll say "Copy the base64," and I'm going to be pasting it here one last time.
And there is a lot of the base64 in here, so I'm going to say "Ctrl + S" to save all of this.
All righty! If you close and reopen "test.py," it should be able to format it better. It's a lot of the base64, but I'm going to minimize this or collapse this so you don't actually have to see all of that stuff.
Now I'm ready! Now it's all saved. How do I actually run this? So, I'm going to come in here and say "Terminal," "New Terminal," and we will go into the "Test Vision."
And we say "Python," and we'll say "test.py." There you go! Let's see if it will work. Drums rolling, please!
And it's actually making the call from my machine in here, and look at that! Here are the requested details: the date is November 22nd, Dixie Cream, total amount is incorrect. That's it; it actually got it incorrect.
This is not correct, so it's very, very crumpled. The image, I guess, so but hey, it got the number of guests correctly this time; it's 2, not 6.
So, again, try to get a cleaner image of the receipts that you're working with, otherwise you're going to do this as well.
All righty! So hopefully that was useful to you, and you can understand really what we're doing at this point. This is very, very important to have a pretty good understanding of why you would want to actually deploy.
Now I can actually do the same thing from my web application, from my phone application, from my React, Angular, Vue—whatever you want—making these calls into that server that I deployed.
I can scale that server in AKS or ACA and do whatever I want with it. Hopefully, this was useful to you, and I'll see you again soon in another video.
Thank you! Bye-bye!